10 Years of the PCG workshop: Past and Future Trends

Antonios Liapis

As of 2020, the international workshop on Procedural Content Generation enters its second decade. The annual workshop, hosted by the international conference on the Foundations of Digital Games, has collected a corpus of 95 papers published in its first 10 years. This paper provides an overview of the workshop's activities and surveys the prevalent research topics emerging over the years.

Introduction

While algorithms have been applied in digital games for almost 40 years, in games such as Rogue (Toy and Wichman, 1981) and ELITE (Acornsoft, 1984), their prominence as a field of academic research is fairly recent [100]. The workshop on Procedural Content Generation (PCG) has served an important role in formalizing the field of study, collecting publications, and mobilizing an academic community interested in PCG research. Since its first incarnation in 2010, the PCG workshop has been hosted by the international conference on the Foundations of Digital Games (FDG). At the time of writing, May 2020, the workshop has been running consecutively for ten years. As a celebration of the 11th anniversary of the PCG workshop in 2020, this paper looks back at the workshops' ten past iterations and identifies topics and directions in its 95 published papers. The survey covers all published papers at the ten first PCG workshops, collected from the ACM Digital Library and the workshops' websites in years where the workshop proceedings were not published by ACM.

In the decade since the first PCG workshop, research in artificial intelligence (AI) for generating game content has bloomed. PCG research of all types has been accepted in high-tier conferences and journals, and three special issues on topics directly relevant to PCG [10, 53, 99] were published in the IEEE Transactions on Games (and the preceding IEEE Transactions on Computational Intelligence and AI in Games). A textbook on Procedural Content Generation in games was published [77], and a Dagstuhl seminar took place in 2018 specifically focusing on AI-driven game design [87]. Finally, the annual PROCJAM event1 has been running since 2014 and has fostered a community of developers and researchers interested in "making something that makes something". Despite these developments - or perhaps because of them - the PCG workshop remains a highly visible academic venue for publishing PCG research and for researchers to pitch and discuss ideas, algorithms, and games.

PCG Workshops of the Last Decade

The above table lists the general information regarding the first ten PCG workshops examined in this paper. As noted in the Introduction, the first PCG workshop took place in 2010 during the FDG conference at Monterey, California. In the first four years the organizational committee was fairly stable, and a subset of the 1st and 2nd workshops' organizers helped run the next workshops. Joris Dormans, Julian Togelius and Gillian Smith were organizers in three of these first PCG workshops, each time with a different set of co-organizers. After 2014, a rotation of organizers was established to ensure that the workshop would not become stale and a larger part of the research community would be engaged in organizational tasks.

In terms of output, the PCG workshop has had a fairly stable number of papers published annually, with little fluctuation. An outlier is the 2014 workshop with only five papers published; this may be due to the fact that FDG 2014 took place on a cruise ship and thus the logistics and cost made attendance only for a workshop paper prohibiting. The last five years have been very stable in terms of participation, with a community of 30-40 people usually attending and on average 10 papers published every year.

On a surface-level analysis of the 95 published PCG papers over the course of the workshop's ten years, papers are authored on average by 2.9 co-authors (median: 3). The corpus of 95 papers includes 17 single-author papers and 10 papers with five or more authors. Over the ten years examined, 184 distinct authors have contributed to the published papers. The above figure shows all authors of published PCG papers; it should be noted that 78% of authors contributed only to one paper while Julian Togelius and Rafael Bidarra were the most active authors with 14 and 10 papers respectively.

The keywords provided by authors could also shed some light on the topics favored by the PCG workshop community. However, it should be noted that 20% of papers did not include any keywords. Of the keywords found, The above figure shows their distribution while omitting the generic keywords "procedural content generation" (in 42 instances), "procedural generation" (in 7 instances), "procedural content" (in 2 instances), "PCG" (in 1 instance). Due to the lack of keywords for a fifth of the corpus, and the fairly ad-hoc way in which keywords are chosen by each author, the thematic analysis performed in the following sections contains richer information regarding the topics covered by the 95 PCG papers.

Types of Content Generated

PCG refers to game content in terms of "all aspects of the game that affect gameplay other than non-player character (NPC) behaviour and the game engine itself" [100] and includes "such aspects as terrain, maps, levels, stories, dialogue, quests, characters, rulesets, dynamics and weapons" [100]. While game content is fairly broadly defined and incorporates most facets of games [55], the majority of PCG workshop papers focus on the generation of levels or worlds (51%). The category of level generation is fairly broad, and encompasses the generation of terrain [15, 70, 80], dungeons [2, 24, 31], mazes [49], settlements [39, 74, 75, 86], and many others. Tightly related to but not subsumed by level generation, architecture is also a common type of generated content: six papers published in the first decade of the PCG workshop focus on architecture, e.g. as 3D models of buildings, as interior placement of furniture [7] or decoration [106]. Often, the distinction between architecture and level design is difficult; for instance, settlement generation [39, 74] is attributed to the latter while texture changes on a building [103] is grouped under architecture. The main criteria for the classification as generated architecture was based on fidelity with real-world patterns and the focus on aesthetic appeal rather than gameplay.

Among other types of generated content, graphics are also prominent (in 8% of papers), and include particle effects [37], 3D meshes [1, 19] and 2D art [46, 72]. The generation of mechanics is equally popular (8% of papers), sometimes combined with level generation [25] or graphics generation [64]. Mechanics refer to actions players can take [25, 110], but in this case a broader view of mechanics is used to include gameplay parameters of weapons and character classes [45, 64, 69]. Generated story elements also feature in 5 papers, although the classification is broad since papers range from generating plots for mysteries [50], to NPC biographies [33], to satellite sentences [71]. Quests or missions are generated in 6 papers, either as a mission graph for a level [24, 26] or for challenges in serious games [59, 67], quests in RPGs [23], etc. The generation of full games is also tackled in 3 papers, two of which present actual game generators [9, 102].

Quite a few papers (12) propose generators that do not fall into the above categories: among those, some papers attempt to generate other aspects of games such as character locomotion [48], language [43], CAPTCHAs for human computation [108] or LEGO structures [62]. Other papers propose general algorithms without a specific type of content in mind, e.g. for graph matching [27], random number generation [61], or family tree representation [60]. Finally, 5 papers do not focus on any domain but instead propose new general generation methods [14, 28], or address philosophical issues such as the intentions of generators' coders [13, 82] and the research field of PCG as a whole [107].

Target Games

Looking at the types of games targeted in the research published in the PCG workshop, the above table attempts a classification of the work. It should be noted that five papers covered multiple game genres while 19 papers do not target a specific genre, and this will be discussed below.

Unsurprisingly, popular genres are platformers, roguelikes and RPGs. Super Mario Bros. (Nintendo, 1985) is the most common platformer being targeted [5, 32, 35, 66, 91, 96], but custom platformers have also been created by researchers either as commercial games, e.g. Sure Footing [21, 101], or as demonstrators of player-controlled PCG [16, 83]. In the context of this survey, roguelikes cover many tile-based dungeon-crawling games similar to Diablo (Blizzard, 1997) as well as action adventure games [24–26, 85, 104]. As expected, the generation of dungeons [2, 6, 31, 94] and caves [42] is a common trope of this genre. Role playing games (RPGs) also cover a broad range of games, and have been targeted in PCG workshop papers for different purposes including quest generation [23, 67, 109], class balancing [69], as well as the simulation of societies [39], their histories [33] and language [43, 73].

Among other genres mentioned in the above table, worth noting are serious games and real-world simulations. "Serious" games, or games with a purpose beyond entertainment, have been targeted by PCG workshop papers in the context of language learning [67], rehabilitation [3, 22], training scenarios [52, 59] and human computation [108]. Real-world simulations have also been common, simulating realistic urban planning [41, 75, 86, 93], architecture & interior design [7, 79, 103], environments [1], locomotion [48] or economies [52].

Five papers encompass more than one genre: two papers combine serious games with another genre [4, 52], two papers apply the same algorithm on multiple genres [12, 110], while [91] introduces a level corpus for different kinds of games (RPGs, roguelikes, arcade games and shooters). On the other hand, 19 papers also are genre-agnostic, primarily vision papers or papers on design tools, graphics, and general algorithms.

Algorithmic Processes

In the context of academic trends, perhaps the most relevant aspect of the PCG workshop papers is the algorithm used for generation. However, in many cases there is no clear-cut classification for such algorithms. We revisit this later in this section, and in the Discussion.

In the last decade, 14 papers published at the PCG workshop use artificial evolution to generate levels (8 papers) and/or mechanics (4 papers), or graphics [19, 37, 64]. Machine learning has also been applied in 7 PCG workshop papers, in the form of non-negative matrix factorization [109], random forests [5], self-organizing maps [72], deep learning [44, 45], and wave function collapse [46, 47]. Notably, 4 of these 7 papers are published from 2017 onwards. Declarative programming is also prominent, in 13 papers, including semantic constraints (5 papers) and answer-set programming (8 papers). Of these papers, 62% are published in the first 3 years of the workshop. 15 PCG workshop papers use some sort of grammar, either as templates for sentence/language generation [65, 71, 73, 88] or for more traditional PCG goals such as level generation (4 papers), architecture [79], design [62], or art [68]. However, the most common algorithmic approach in PCG papers of the last decade were constructive methods [76] which include industry-standard scripts such as cellular automata, agents, L-systems etc. Many papers in this corpus could qualify as using constructive methods, since this broad category likely encompasses anything that does not fall into the other better-defined families of algorithms. Even with stricter criteria, constructive methods are found in 14 papers, including cellular automata (7 papers), L-systems [1, 59], recursive subdivision [81], diamond-square [3], agents [86, 92]. Should we have included all scripted methods, the number of PCG papers that belong to this category wound increase dramatically, but would make classification difficult since ultimately any algorithm is a scripted method.

Other Aspects

Looking at other trends in the PCG workshop papers of the last decade, it is evident that design tools (for or with PCG) are popular (in 19 papers). Some papers focus on presenting the tool itself [2, 6, 8, 18, 58, 80] while most papers of this type focus on the algorithms and briefly expose a way for the designer to control these algorithms.

Other popular topics include the use of design patterns to help guide generation (7 papers), usually for level generation [2, 6, 21], the use of expressive range as an evaluation method (7 papers), and the real-time adaptation/modification of game content while the game is played (7 papers). It should be noted that expressive range as a concept was introduced in the first PCG workshop by Smith and Whitehead [84] as a method for evaluating level generators: indeed, all 7 PCG papers that use expressive range analysis apply it to level generation.

Of interest is also the fact that modelling the players' behavior or progress is attempted in 5 PCG workshop papers, all of which are published in the first 5 years of the workshop. Modelling is performed primarily to adapt the game's difficulty or challenge [5, 40, 56, 109] and to match players' preferences [57]. Close to the latter goal of modelling, 5 PCG workshop papers place content generation at the hands of the player, as part of the game mechanics [16] and/or a direct response to player actions [15, 83, 96]. Cases where players control the generator's settings [101] are more difficult to classify as their distinction from designer tools is not clear-cut, e.g. in Game-O-Matic [102].

Finally, it is important to note that 8 PCG papers introduce a vision and offer little technical details, proposing new types of content for PCG [106], new general methods for generation [14, 28, 51] or expose ethical or general issues that PCG is facing [13, 82, 97, 107]. Beyond these vision papers, 92% of papers published at the PCG workshop contain technical contributions, e.g. introducing new algorithms, corpora, games, methods, and a competition [32]. Despite the focus on technical contributions, only 32 papers include an empirical evaluation. We define "evaluation" as a quantitative analysis of multiple runs of the generator (with or without human testers) which reports aggregated statistics and/or performs significance tests. Instead, most surveyed papers include a couple of generated samples and a qualitative discussion on the workings of the system. Quantitative evaluation is primarily applied on level generation tasks (87% of evaluation papers), and in papers using constructive methods (26%), artificial evolution (22%) or declarative programming (19%). Evaluation is sometimes performed through expressive range analysis (22% of evaluation papers), sometimes based on experiments with human participants [2, 4, 22, 39], but usually focus on the statistical analysis of computation time (e.g. [27]), fitness (e.g. [98]), prediction error (e.g. [44]) and game-specific performance metrics such as map coverage [12] or game arcs [20].

Past and Future Trends

Following the quantitative analysis of different papers' topics, target games, algorithms, etc. the most popular aspects of this corpus will be tracked in terms of their prevalence in different years. Due to the small number of papers published every year, the analysis uses the ratio of relevant papers over all papers published at that year's PCG workshop. The visualization includes a trendline based on linear regression.

The above figure shows the most prominent types of content generated, and how they fluctuate across the decade examined. Level generation remains consistently high and fairly steady. Quest generation and architecture had been active topics early on but have been largely absent after the first four years of the workshop, while the generation of mechanics and graphics seems fairly stable in their – somewhat low – prominence. Interestingly, there is an upward trend in narrative generation with recent work at the PCG workshop focusing on aspects of stories [33, 50, 65, 71, 88]; possible reasons for this stronger focus on narrative are discussed in the Discussion.

The above figure shows the fluctuation in terms of the most prominent game genres targeted. The high fluctuations are evidence that researchers tend to focus on the algorithm or the type of content generated, while the game genre is mostly based on what's available or convenient. Perhaps the only major trend from this figure is the decrease in research interest on PCG for serious games. There is also a downwards trend for platformer games and an upwards trend for roguelikes, although the small percentage of papers for both seems to indicate that these trends largely depend on available testbeds and personal preferences of contributing researchers. While not in the above figure, worth noting is a recent increase in crafting games, spurred by the Generative Design in Minecraft competition introduced in 2018 [74] which provides a challenge and a codebase for interested participants. Crafting games are the focus of 11% of PCG papers in the last two years, with no presence in prior years.

The above figure shows the major families of algorithms applied for generation in the PCG workshop papers of the last decade. As with game genre, the low overall ratio for each algorithm leads us to believe that the choice of algorithm was often due to the personal preference (or existing codebase) of each researcher; however, some trends can be detected. Despite - or because of - their simplicity, constructive methods seem to be used steadily with no sign that they are falling out of favor. A previous surge in academic interest for declarative approaches seems to be dwindling (with exceptions), and grammars face a similar downward trend from its early popularity in 2010-2013. Artificial evolution seems to be a steady, popular algorithmic approach with a slight upwards trend. As noted in Algorithmic Processes, machine learning (ML) is the most upwards trending algorithmic method with 4 of 7 ML papers published from 2017 onwards. More importantly, all four of these latest ML papers combine machine learning with artificial evolution [45, 49] or declarative programming [46, 47]. The heightened interest of PCG workshop publications in machine learning are likely attributed to recent successes in deep learning more broadly, the availability of easy-to-use codebases for machine learning, and a game-specific academic interest in PCG via Machine Learning [90]. Therefore, perhaps the most significant trend of the above figure is the increased use of machine learning on its own or in conjunction with PCG mainstays such as artificial evolution or declarative programming.

Finally, the above figure tracks how some important aspects of PCG papers (highlighted in Other Aspects) have appeared at different times. It would seem that vision papers remain constant across the years, and workshop organizers should expect at least one vision paper published every year. Design tools and the use of design patterns are similarly retaining their popularity without showing any strong trends upwards or downwards. Interestingly, modeling players and generating personalized content was trending upwards in the first few years but has lost its appeal in the last four years of the workshop. Finally, it is promising that a steadily increasing amount of PCG papers contain a rigorous quantitative evaluation (i.e. not only a qualitative discussion on a few sample outcomes). While in the last two years of PCG papers evaluation is still absent in the majority of papers, the upward trend is irrefutable. Perhaps due to the increase in papers that perform evaluation, more and more papers use the notion of expressive range for the purposes of visualizing the generator's output.

Discussion

This paper attempted to categorize the 95 papers published at the PCG workshop in its first decade, as well as study the historical trends in topics covered within these papers. In many cases, classification was difficult and perhaps error-prone; as noted in Types of Content Generated, it is difficult to label e.g. a Minecraft (Mojang, 2011) settlement or building as level generation or parametric architecture. Boundaries between game genres are fuzzy for commercial games, and even fuzzier when prototype and simplified games are being generated. Similar issues arise when classifying algorithms, as semantic-based generation is sometimes following a declarative approach [56] and at other times a constructive one [93]. The Algorithmic Processes section highlighted the dangers of labeling everything as constructive if it does not test the generated artefact [100], and we opted for a stricter definition for constructive algorithms. Due to these issues of ambiguity, many papers ended up without a classification in terms of algorithms even when they included technical details of their generators. Although some readers (or papers' authors) may disagree with certain labels, the general trends found in the corpus of 95 papers are fairly indicative of the trends in PCG research.

Another caveat in this analysis is the nature of the workshop itself. Focusing on the 95 workshop papers rather than e.g. the complete PCG literature published in conferences such as FDG (of which the PCG workshop is part), journal papers or books, is bound to draw conclusions topical of the workshop microcosm. Workshop papers typically focus on algorithms that are work-in-progress or at a prototype stage, or invite a discussion on philosophical issues and new ideas. The large number of vision papers at the PCG workshop should thus not be surprising, and similarly the relative absence of experimental validation. In contrast, journal papers on topics related to PCG - e.g. in the IEEE Transactions on Games - include a robust quantitative evaluation. Finally, the fact that the workshop took place in different locations every year combined with the small community of researchers means that some of the trends can be attributed to lack of participation for a year or a shift in researchers' interests. PCG workshop authors follow their own trajectories (e.g. Joris Dormans organized 3 workshops and published 4 papers from 2010 to 2013 but has since moved to industry), and some iterations of the workshop had larger participation from some research groups than others. Keeping these caveats in mind, we argue that the 10 years of the PCG workshop are indicative of the broader trends in the PCG research field. Our reasoning is that the PCG workshop has been running for longer than other PCG-related academic venues, and its community has inspired most of the advancements in PCG of the last decade including the PCG book [77], special issues, PROCJAM, and workshops such as Experimental AI in Games and Computational Creativity and Games. While these larger movements were occurring, the PCG workshop continued to run and act as a node for PCG researchers to convene and co-ordinate.

On the other hand, we must admit that PCG research itself is a microcosm of the broader movements within AI research, and thus the above patterns may not reflect broader AI trends. A mundane observation is that there are other specialized workshops or conferences where topics targeted by a few PCG workshop papers are nigh-ubiquitous: for instance, the Intelligent Narrative Technologies hosted by the Artificial Intelligence and Interactive Digital Entertainment conference focuses heavily on generating narrative. The International Conference on Computational Intelligence in Music, Sound, Art and Design (EvoMusArt) and the International Conference on Computational Creativity focus heavily on generated graphics and audio, usually beyond games. These publication venues likely follow their own trends in terms of the algorithms that are currently rising in prominence. Beyond the PCG workshop (and games more broadly), the use of deep learning has been applied for a broad range of generative tasks, with an especially strong focus on graphics [11, 29] as well as music [36] and text [105]. Instead, papers using deep learning published at the PCG workshop exclusively focus on level design tasks. Therefore, PCG research understandably focuses on the game-specific concerns of generation and its trends may differ from broader movements even while it is still affected by them (e.g. the prevalence of deep learning).

Based on the analysis carried out in this paper and a more general view of research in procedural content generation (beyond the workshop itself), it is evident that level generation remains a popular and "easy" goal for PCG. This can be attributed to the fact that the game industry initially introduced PCG for level generation in games such as Rogue in the 1980s and level generation remains by far the most popular industry application of PCG today. From a practical perspective, moreover, level generation makes sense as games often rely on new levels to increase replayability and variety. Finally, level generation is easier due to the availability of more codebases (produced by researchers, game developers, and enthusiasts alike) and the prevalence of level generation competitions. The generation of complete games, while a popular vision in 2012-2015 [9, 28, 97, 102] has not quite taken off. While complete game generation is being attempted in small clusters [17, 54], the drop in interest likely comes from the lack of a big "success story" either in academia or the game industry. Another reason is the difficulty that a researcher must face when starting out on game generation in a non-toy domain given the lack of codebases, tutorials or assets. Narrative remains a strong direction for PCG research, likely encouraged by the increased interest on interactive narrative commercially (e.g. with LudoNarraCon) and academically (e.g. with the International Conference on Interactive Digital Storytelling conference and the Intelligent Narrative Technologies workshop).

In terms of algorithms, machine learning seems to be the up-and-coming trend in PCG research. This is hardly surprising given the widespread "success stories" and debate [63] on art created via deep learning [29, 30]. The broader concept of PCG via Machine Learning [90] has already driven research in the PCG workshop [47], while more established PCG algorithms such as artificial evolution or declarative programming are moving towards the integration of machine learning for content initialization [47], fitness prediction [45] and more. It is expected, therefore, that PCG research in deep learning will continue trending upwards, probably combining deep learning with other popular PCG algorithms.

As noted in Target Games, game genres usually favored by PCG researchers are largely due to personal preference, nostalgia, and convenience. In terms of the latter, the Super Mario Bros. level generation competition [78] has largely driven the popularity of platformer games for PCG, not only due to the extensive benchmarks established over the years but also due to the availability of a straightforward generation codebase and efficient gameplaying agents [95]. The entry level requirements for generation in Super Mario Bros. are lower than any other, and researchers can quickly put together something. It is surmised that similar levels of support by both the competition organizers and the community can also drive PCG research to other, overlooked genres. In that regard, Minecraft is likely be the next focus point of PCG research, in part due to the GDMC competition [74], the research by Microsoft on Minecraft gameplaying agents [38], and its associated MineRL competition [34]. There is extensive evidence that academic and practitioner interest hinges on an intuitive codebase as well as an activity that engages the community and produces reusable assets (agents, content, code) that can help future endeavors. Game generation, for which there has been dwindling interest, could thus be reinvigorated with any combination of benchmarks, competitions, easy-to-use codebases, tutorials etc. The same applies to any under-studied game genre, such as stealth games. At the minimum, researchers open-sourcing their generators would help drive future research in their preferred genre. However, building communities through competitions, tutorials, or collaborations with game companies would better act as catalysts to invigorate PCG research on topics that have not yet taken off.

With PCG research moving towards a more established if multi-disciplinary [107] field of study, it is heartening that more attention is given to methodically evaluating and reporting on generators' outcomes. Papers in the PCG workshop have been steadily improving their technical accuracy through quantitative tests with multiple runs and/or human users. Published papers on content generation in journals and conferences have similarly found ways of reporting the quality and diversity of their output. The concept of expressive range [84] is particularly suited for evaluation of PCG on both dimensions, and a number of enhancements have already been proposed [18, 89]. There is already an upwards trend in the application of expressive range for PCG evaluation, both in the PCG workshop and beyond. It would be beneficial - at least to the scientific facet of the PCG research field - to establish, contribute to and follow standards of evaluation such as the expressive range of the generator.

Conclusion

This paper celebrates the first decade of the workshop on Procedural Content Generation by providing an analysis of the 95 papers published during these ten years. The paper explored the types of content generated, the game genres, and the algorithms which were favored historically by the PCG workshop community. Drawing from the fluctuations of different topics over the years, the paper identified some trends in the past and the future of the workshop and discussed them in the broader context of PCG research. While the publications at the PCG workshop are a subset of publications on the topic of generation in journals, conferences, books etc., the consistency of the corpus allows us to draw conclusions that are generally in line with the broader academic trends in PCG. The paper is intended primarily as a historical overview, and secondarily as a view towards the future, the challenges and necessary steps for the next ten years of PCG research.

Acknowledgments

The author would like to thank the paper's reviewers for their insightful comments. While not all comments could be addressed in this paper, there is much that needs to be discussed on the future of the PCG workshop and the field at large.

References

[1] Ryan Abela, Antonios Liapis, and Georgios N. Yannakakis. 2015. A Constructive Approach for the Generation of Underwater Environments. In Proceedings of the FDG workshop on Procedural Content Generation.

[2] Alberto Alvarez, Steve Dahlskog, Jose Font, Johan Holmberg, Chelsi Nolasco, and Axel Österman. 2018. Fostering Creativity in the Mixed-Initiative Evolutionary Dungeon Designer. In Proceedings of the FDG workshop on Procedural Content Generation.

[3] Michael Andereck, Alan Price, and Roger Crawfis. 2013. Procedural Terrain Generation for Medical Rehabilitation. In Proceedings of the FDG workshop on Procedural Content Generation.

[4] Christopher Antoun, Matthew Antoun, James Owen Ryan, Ben Samuel, Reid Swanson, and Marilyn A. Walker. 2015. Generating Natural Language Retellings from Prom Week Play Traces. In Proceedings of the FDG workshop on Procedural Content Generation.

[5] Sander Bakkes and Shimon Whiteson. 2014. Towards Challenge Balancing for Personalised Game Spaces. In Proceedings of the FDG workshop on Procedural Content Generation.

[6] Alexander Baldwin, Steve Dahlskog, Jose M. Font, and Johan Holmberg. 2017. Towards Pattern-Based Mixed-Initiative Dungeon Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[7] J. Timothy Balint and Rafael Bidarra. 2019. A Generalized Semantic Representa- tion for Procedural Generation of Rooms. In Proceedings of the FDG workshop on Procedural Content Generation.

[8] Lionel Barret, Claudia Vance, and G. Michael Youngblood. 2011. Lessons in User Interface Design in the Procedural City Generation for Games Tool Ürban PAD. In Proceedings of the FDG workshop on Procedural Content Generation.

[9] Gabriella A. B. Barros, Antonios Liapis, and Julian Togelius. 2015. Data Adventures. In Proceedings of the FDG workshop on Procedural Content Generation.

[10] Cameron Browne, Georgios N. Yannakakis, and Simon Colton. 2012. Guest Editorial: Special Issue on Computational Aesthetics in Games. IEEE Transactions on Computational Intelligence and AI in Games 4, 3 (2012), 149–151.

[11] Yang Chen, Yu-Kun Lai, and Yong-Jin Liu. 2018. CartoonGAN: Generative Adversarial Networks for Photo Cartoonization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 9465–9474.

[12] Muntasir Chowdhury and Clark Verbrugge. 2016. Exhaustive Exploration Strategies for NPCs. In Proceedings of the FDG workshop on Procedural Content Generation.

[13] Kate Compton. 2017. Litle Procedural People: Playing politics with generators. In Proceedings of the FDG workshop on Procedural Content Generation.

[14] Kate Compton, Joseph C. Osborn, and Michael Mateas. 2013. Generative Methods. In Proceedings of the FDG workshop on Procedural Content Generation.

[15] Kate Compton, Adam Smith, and Michael Mateas. 2012. Anza Island: Novel Gameplay Using ASP. In Proceedings of the FDG workshop on Procedural Content Generation.

[16] Michael Cook and Simon Colton. 2016. Towards Procedural Generation As Gameplay: CLAY and Tombs of Tomeria. In Proceedings of the FDG workshop on Procedural Content Generation.

[17] Michael Cook and Simon Colton. 2018. Redesigning Computationally Creative Systems For Continuous Creation. In Proceedings of the International Conference on Computational Creativity.

[18] Michael Cook, Jeremy Gow, and Simon Colton. 2016. Danesh: Helping Bridge The Gap Between Procedural Generators And Their Output. In Proceedings of the FDG workshop on Procedural Content Generation.

[19] Isaac M. Dart, Gabriele De Rossi, and Julian Togelius. 2011. SpeedRock: procedural rocks through grammars and evolution. In Proceedings of the FDG workshop on Procedural Content Generation.

[20] Fernando de Mesentier Silva, Scott Lee, Julian Togelius, and Andy Nealen. 2018. Evolving Maps and Decks for Ticket to Ride. In Proceedings of the FDG workshop on Procedural Content Generation.

[21] Neall Dewsbury, Aimie Nunn, Matthew Syrett, James Tatum, and Tommy Thompson. 2016. Scalable Level Generation for 2D Platforming Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[22] Dajana Dimovska, Patrick Jarnfelt, Sebbe Selvig, and Georgios N. Yannakakis. 2010. Towards Procedural Level Generation for Rehabilitation. In Proceedings of the FDG workshop on Procedural Content Generation.

[23] Jonathon Doran and Ian Parberry. 2011. A Prototype Quest Generator Based on a Structural Analysis of Quests from Four MMORPGs. In Proceedings of the FDG workshop on Procedural Content Generation.

[24] Joris Dormans. 2010. Adventures in Level Design: Generating Missions and Spaces for Action Adventure Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[25] Joris Dormans. 2011. Level Design as Model Transformation: A Strategy for Automated Content Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[26] Joris Dormans. 2012. Generating Emergent Physics for Action-Adventure Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[27] Marlon Etheredge. 2012. Fast exact graph matching using adjacency matrices. In Proceedings of the FDG workshop on Procedural Content Generation.

[28] Marie Gustafsson Friberger, Julian Togelius, Andrew Borg Cardona, Michele Ermacora, Anders Mousten, Martin Møller Jensen, Virgil-Alexandu Tanase, and Ulrik Brøndsted. 2013. Data Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[29] Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 2414–2423.

[30] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Proceedings of the Neural Information Processing Systems Conference.

[31] Michael Cerny Green, Ahmed Khalifa, Athoug Alsoughayer, Divyesh Surana, Antonios Liapis, and Julian Togelius. 2019. Two-step Constructive Approaches for Dungeon Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[32] Michael Cerny Green, Ahmed Khalifa, Gabriella A. B. Barros, Andy Nealen, and Julian Togelius. 2018. Generating Levels That Teach Mechanics. In Proceedings of the FDG workshop on Procedural Content Generation.

[33] Jason Grinblat and C. Brian Bucklew. 2017. Subverting Historical Cause & Effect: Generation of Mythic Biographies in Caves of Qud. In Proceedings of the FDG workshop on Procedural Content Generation.

[34] William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Sean Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, Manuela Veloso, and Philip Wang. 2019. The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors. In Thirty-third Conference on Neural Information Processing Systems (NeurIPS) Competition track.

[35] Matthew Guzdial and Mark O. Riedl. 2015. Toward Game Level Generation from Gameplay Videos. In Proceedings of the FDG workshop on Procedural Content Generation.

[36] Gaetan Hadjeres, Franccois Pachet, and Frank Nielsen. 2017. DeepBach: a Steerable Model for Bach Chorales Generation. In Proceedings of the International Conference on Machine Learning.

[37] Erin J. Hastings and Kenneth O. Stanley. 2010. Interactive Genetic Engineering of Evolved Video Game Content. In Proceedings of the FDG workshop on Procedural Content Generation.

[38] Katja Hofmann. 2019. Minecraft as AI Playground and Laboratory. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (ChiPlay). ACM.

[39] Bas in het Veld, Ben Kybartas, Rafael Bidarra, and John-Jules Ch. Meyer. 2015. Procedural generation of populations for storytelling. In Proceedings of the FDG workshop on Procedural Content Generation.

[40] Martin Jennings-Teats, Gillian Smith, and Noah Wardrip-Fruin. 2010. Polymorph: Dynamic Difficulty Adjustment Through Level Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[41] Diego Jesus, Antonio Coelho, Carlos Rebelo, and Andre Cardoso. 2012. Modeling Urban Environments from Geospatial Data: A Pipeline for Procedural Modeling. In Proceedings of the FDG workshop on Procedural Content Generation.

[42] Lawrence Johnson, Julian Togelius, and Georgios N. Yannakakis. 2010. Cellular automata for real-time generation of infinite cave levels. In Proceedings of the FDG workshop on Procedural Content Generation.

[43] Mark R Johnson. 2016. Procedural Generation of Linguistics, Dialects, Naming Conventions and Spoken Sentences. In Proceedings of the FDG workshop on Procedural Content Generation.

[44] Daniel Karavolos, Antonios Liapis, and Georgios Yannakakis. 2017. Learning the Paterns of Balance in a Multi-Player Shooter Game. In Proceedings of the FDG workshop on Procedural Content Generation.

[45] Daniel Karavolos, Antonios Liapis, and Georgios N. Yannakakis. 2018. Pairing Character Classes in a Deathmatch Shooter Game via a Deep-Learning Surrogate Model. In Proceedings of the FDG workshop on Procedural Content Generation.

[46] Isaac Karth and Adam M. Smith. 2017. WaveFunctionCollapse is Constraint Solving in the Wild. In Proceedings of the FDG workshop on Procedural Content Generation.

[47] Isaac Karth and Adam M. Smith. 2019. Addressing the Fundamental Tension of PCGML with Discriminative Learning. In Proceedings of the FDG workshop on Procedural Content Generation.

[48] Ben Kenwright. 2012. Generating Responsive Life-Like Biped Characters. In Proceedings of the FDG workshop on Procedural Content Generation.

[49] Paul Hyunjin Kim, Jacob Grove, Skylar Wurster, and Roger Crawfis. 2019. Design-Centric Maze Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[50] Max Kreminski, Devi Acharya, Nick Junius, Elisabeth Oliver, Kate Compton, Melanie Dickinson, Cyril Focht, Stacey Mason, Stella Mazeika, and Noah Wardrip-Fruin. 2019. Cozy Mystery Construction Kit: Prototyping Toward an AI-Assisted Collaborative Storytelling Mystery Game. In Proceedings of the FDG workshop on Procedural Content Generation.

[51] Max Kreminski, Isaac Karth, and Noah Wardrip-Fruin. 2019. Generators that Read. In Proceedings of the FDG workshop on Procedural Content Generation. 10 Years of the PCG workshop: Past and Future Trends FDG ’20, September 15–18, 2020, Bugibba, Malta

[52] Stefan Leijnen, Paul Brinkkemper, and Anders Bouwer. 2015. Generating Game Mechanics in a Model Economy: a MoneyMaker Deluxe Case Study. In Proceedings of the FDG workshop on Procedural Content Generation.

[53] Antonios Liapis, Georgios N. Yannakakis, Michael Cook, and Simon Colton. 2019. Guest Editorial Special Issue on AI-Based and AI-Assisted Game Design. IEEE Transactions on Games 11, 1 (2019), 1–4.

[54] Antonios Liapis, Georgios N. Yannakakis, Mark J. Nelson, Mike Preuss, and Rafael Bidarra. 2019. Orchestrating Game Generation. IEEE Transactions on Games 11, 1 (2019), 48–68.

[55] Antonios Liapis, Georgios N. Yannakakis, and Julian Togelius. 2014. Com- putational Game Creativity. In Proceedings of the International Conference on Computational Creativity.

[56] Ricardo Lopes, Ken Hilf, Luke Jayapalan, and Rafael Bidarra. 2013. Mobile adaptive procedural content generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[57] Ricardo Lopes, Tim Tutenel, and Rafael Bidarra. 2012. Using gameplay semantics to procedurally generate player-matching game worlds. In Proceedings of the FDG workshop on Procedural Content Generation.

[58] Tiago Machado, Ivan Bravi, Zhu Wang, Andy Nealen, and Julian Togelius. 2016. Shopping for Game Mechanics. In Proceedings of the FDG workshop on Procedural Content Generation.

[59] Glenn A. Martin, Charles E. Hughes, Sae Schatz, and Denise Nicholson. 2010. The Use of Functional L-Systems for Scenario Generation in Serious Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[60] Peter Mawhorter. 2017. Efficiency, Realism, and Representation in Generated Content: A Case Study using Family Tree Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[61] Peter Mawhorter. 2019. Anarchy: A Library for Incremental Chaos. In Proceedings of the FDG workshop on Procedural Content Generation.

[62] Joseph Mazeika and Jim Whitehead. 2015. Design Motifs: A Grammar Based Approach. In Proceedings of the FDG workshop on Procedural Content Generation.

[63] Jon McCormack, Toby Gifford, and Patrick Hutchings. 2019. Autonomy, Authenticity, Authorship and Intention in Computer Generated Art. In EvoMUSART.

[64] Eric McDuffee and Alex Pantaleev. 2013. Team Blockhead Wars: Generating FPS Weapons in a Multiplayer Environment. In Proceedings of the FDG workshop on Procedural Content Generation.

[65] Chris Miller, Mayank Dighe, Chris Martens, and Arnav Jhala. 2019. Stories of the Town: Balancing Character Autonomy and Coherent Narrative in Procedurally Generated Worlds. In Proceedings of the FDG workshop on Procedural Content Generation.

[66] Santiago Londo no and Olana Missura. 2015. Graph Grammars for Super Mario Bros* Levels. In Proceedings of the FDG workshop on Procedural Content Generation.

[67] Cameron Olson, Daniel Kauffman, Allan Fowler, and Foaad Khosmood. 2015. Teaching Japanese through Game Mechanics: An exploratory study. In Proceedings of the FDG workshop on Procedural Content Generation.

[68] Johnathan Pagnutti, Kate Compton, and Jim Whitehead. 2016. Do You Like This Art I Made You: Introducing Techne, A Creative Artbot Commune. In Proceedings of the FDG workshop on Procedural Content Generation.

[69] Alex Pantaleev. 2012. In Search of Patterns: Disrupting RPG Classes through Procedural Content Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[70] Andrew Pech, Philip Hingston, Martin Masek, and Chiou-Peng Lam. 2016. Identifying Attributes for Characterizing Game Area Types in Virtual Terrain. In Proceedings of the FDG workshop on Procedural Content Generation.

[71] Aaron A. Reed. 2012. Sharing Authoring with Algorithms: Procedural Generation of Satellite Sentences in Text-based Interactive Stories. In Proceedings of the FDG workshop on Procedural Content Generation.

[72] Sebastian Risi, Joel Lehman, David B. D’Ambrosio, and Kenneth O. Stanley. 2014. Automatically Categorizing Procedurally Generated Content for Collecting Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[73] James Ryan. 2016. Diegetically Grounded Evolution of Gameworld Languages. In Proceedings of the FDG workshop on Procedural Content Generation.

[74] Christoph Salge, Michael Cerny Green, Rodgrigo Canaan, and Julian Togelius. 2018. Generative Design in Minecraft (GDMC): Settlement Generation Competition. In Proceedings of the FDG workshop on Procedural Content Generation.

[75] Chris Sexton and Bejamin Watson. 2010. Vectorization of Gridded Urban Land Use Data. In Proceedings of the FDG workshop on Procedural Content Generation.

[76] Noor Shaker, Antonios Liapis, Julian Togelius, Ricardo Lopes, and Rafael Bidarra. 2016. Constructive Generation Methods for Dungeons and Levels. In Procedural Content Generation in Games: A Textbook and an Overview of Current Research, Noor Shaker, Julian Togelius, and Mark J. Nelson (Eds.). Springer, 31–55.

[77] Noor Shaker, Julian Togelius, and Mark J. Nelson. 2016. Procedural Content Generation in Games: A Textbook and an Overview of Current Research. Springer.

[78] N. Shaker, J. Togelius, G. N. Yannakakis, B. Weber, T. Shimizu, T. Hashiyama, N. Sorenson, P. Pasquier, P. Mawhorter, G. Takahashi, G. Smith, and R. Baumgarten. 2011. The 2010 Mario AI Championship: Level Generation Track. IEEE Transactions on Computational Intelligence and AI in Games 3, 4 (2011), 332–347.

[79] Pedro Brandão Silva, Pascal Müller, Rafael Bidarra, and António Coelho. 2013. Node-Based Shape Grammar Representation and Editing. In Proceedings of the FDG workshop on Procedural Content Generation.

[80] Ruben Smelik, Tim Tutenel, Klaas Jan de Kraker, and Rafael Bidarra. 2010. Integrating procedural generation and manual editing of virtual worlds. In Proceedings of the FDG workshop on Procedural Content Generation.

[81] Adam M. Smith. 2011. Two methods for voxel detail enhancement. In Proceedings of the FDG workshop on Procedural Content Generation.

[82] Gillian Smith. 2017. What Do We Value in Procedural Content Generation?. In Proceedings of the FDG workshop on Procedural Content Generation.

[83] Gillian Smith, Elaine Gan, Alexei Othenin-Girard, and Jim Whitehead. 2011. PCG-based game design: enabling new play experiences through procedural content generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[84] Gillian Smith and Jim Whitehead. 2010. Analyzing the Expressive Range of a Level Generator. In Proceedings of the FDG workshop on Procedural Content Generation.

[85] Thomas Smith, Julian Padget, and Andrew Vidler. 2018. Graph-based Generation of Action-Adventure Dungeon Levels using Answer Set Programming. In Proceedings of the FDG workshop on Procedural Content Generation.

[86] Asiiah Song and Jim Whitehead. 2019. TownSim: Agent-based city evolution for naturalistic road network generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[87] Pieter Spronck, Elisabeth André, Michael Cook, and Mike Preuß. 2018. Artificial and Computational Intelligence in Games: AI-Driven Game Design (Dagstuhl Seminar 17471). Dagstuhl Reports 7, 11 (2018), 86–129. https://doi.org/10.4230/DagRep.7.11.86

[88] Anne Sullivan, Mirjam Palosaari Eladhari, and Michael Cook. 2018. Tarot-Based Narrative Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[89] Adam Summerville. 2018. Expanding Expressive Range: Evaluation Methodologies for Procedural Content Generation. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. 116–122.

[90] Adam Summerville, Sam Snodgrass, Matthew Guzdial, Christoffer Holmgård, Amy K. Hoover, Aaron Isaksen, Andy Nealen, and Julian Togelius. 2018. Procedural Content Generation via Machine Learning (PCGML). IEEE Transactions on Games (2018).

[91] Adam James Summerville, Michael Mateas, Sam Snodgrass, and Santiago Ontañón. 2016. The VGLC: The Video Game Level Corpus. In Proceedings of the FDG workshop on Procedural Content Generation.

[92] Tanel Teinemaa, Till Riemer, and Noor Shaker. 2015. A Procedural Approach for Infinite Deterministic 2D Grid-Based World Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[93] Edward Teng and Rafael Bidarra. 2017. A semantic approach to patch-based procedural generation of urban road networks. In Proceedings of the FDG workshop on Procedural Content Generation.

[94] Julian Togelius, Tróndur Justinussen, and Anders Hartzen. 2012. Compositional procedural content generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[95] Julian Togelius, S. Karakovskiy, and Robin Baumgarten. 2010. The 2009 Mario AI Competition. In Proceedings of the IEEE Congress on Evolutionary Computation.

[96] Julian Togelius, Emil Kastbjerg, David Schedl, and Georgios N. Yannakakis. 2011. What is Procedural Content Generation? Mario on the borderline. In Proceedings of the FDG workshop on Procedural Content Generation.

[97] Julian Togelius, Mark J. Nelson, and Antonios Liapis. 2014. Characteristics of Generatable Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[98] Julian Togelius, Mike Preuss, and Georgios N. Yannakakis. 2010. Towards multiobjective procedural map generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[99] Julian Togelius, Jim Whitehead, and Rafael Bidarra. 2011. Guest Editorial: Procedural Content Generation in Games. IEEE Transactions on Computational Intelligence and AI in Games 3, 03 (2011).

[100] Julian Togelius, Georgios N. Yannakakis, Kenneth Stanley, and Cameron Browne. 2011. Search-based Procedural Content Generation: A Taxonomy and Survey. IEEE Transactions on Computational Intelligence and AI in Games 3, 3 (2011).

[101] Tommy Tompson and Mathew Syret. 2018. ’Play Your Own Way’ Adapting a Procedural Framework for Accessibility. In Proceedings of the FDG workshop on Procedural Content Generation.

[102] Mike Treanor, Bryan Blackford, Michael Mateas, and Ian Bogost. 2012. Game-O-Matic: Generating Videogames that Represent Ideas. In Proceedings of the FDG workshop on Procedural Content Generation.

[103] Tim Tutenel, Roland van der Linden, Marnix Kraus, Bart Bollen, and Rafael Bidarra. 2011. Procedural filters for customization of virtual worlds. In Proceed- ings of the FDG workshop on Procedural Content Generation.

[104] Riemer van Rozen and Qinten Heijn. 2018. Measuring Quality of Grammars for Procedural Level Generation. In Proceedings of the FDG workshop on Procedural Content Generation.

[105] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. ArXiv abs/1706.03762 (2017).

[106] Jim Whitehead. 2010. Toward Proccedural Decorative Ornamentation in Games. In Proceedings of the FDG workshop on Procedural Content Generation.

[107] Jim Whitehead. 2017. Art and Science of Engineered Design: What Kind of Discipline is PCG?. In Proceedings of the FDG workshop on Procedural Content Generation.

[108] Hong Yu and Mark O. Riedl. 2015. Automatic Generation of Game-based CAPTCHAs. In Proceedings of the FDG workshop on Procedural Content Generation.

[109] Alexander Zook, Stephen Lee-Urban, Michael R. Drinkwater, and Mark O. Riedl. 2012. Skill-based Mission Generation: A Data-driven Temporal Player Modeling Approach. In Proceedings of the FDG workshop on Procedural Content Generation.

[110] Alexander Zook and Mark O. Riedl. 2014. Generating and Adapting Game Mechanics. In Proceedings of the FDG workshop on Procedural Content Generation.