Game Aesthetics
"Game aesthetics" is a broad and ambiguous term which encompasses believability, challenge, surprise, novelty, visual and aural appeal, interestingness, and more. Examining the "human" aspects of games from a computational perspective is of particular interest to my research.
Affect in Spatial Navigation: A Study of Rooms
Emmanouil Xylakis, Antonios Liapis, Georgios N. Yannakakis
Screenshot of PAGAN using the RankTrace tool during an arousal annotation task: navigation video (top) and continuous arousal annotation (bottom).
Abstract: How do spaces make us feel? What is the perceived emotional impact of built form? This study proposes a framework to identify and model the effects that our perceived environment can have by taking into consideration illumination and structural form while acknowledging its temporal dimension. To study this, we recruited 100 participants via a crowd-sourcing platform in order to annotate their perceived arousal or pleasure shifts while watching videos depicting spatial navigation in first person view. Participants' annotations were recorded as time-continuous unbounded traces, allowing us to extract ordinal labels about how their arousal or pleasure fluctuated as the camera moved between different rooms. Given the subjective nature of the task and the noisy signals from real-time annotation, a number of processing steps are applied in order to convert the data into ordinal relationships between affect metrics in different rooms. Experiments with random forests and other classifiers show that, with the right treatment and data cleanup, simple interior design features can be adequate predictors of human arousal and pleasure changes over time. The dataset is made available in order to prompt exploration of additional modalities as input and ground truth extraction.
in IEEE Transactions on Affective Computing, 2024 (accepted). BibTex
MAP-Elites with Transverse Assessment for Multimodal Problems in Creative Domains
Marvin Zammit, Antonios Liapis, Georgios N. Yannakakis
The MEliTA process: an elite is selected from the featuremap, and one of its modalities (image) is altered. Then, this new image is mapped to another row of cells in the featuremap and new candidate solutions are produced by combining the new image and the elites' old text modality. The best combination of new image and older text survives and replaces the previous individual or occupies a new cell.
Abstract: The recent advances in language-based generative models have paved the way for the orchestration of multiple generators of different artefact types (text, image, audio, etc.) into one system. Presently, many open-source pre-trained models combine text with other modalities, thus enabling shared vector embeddings to be compared across different generators. Within this context we propose a novel approach to handle multimodal creative tasks using Quality Diversity evolution. Our contribution is a variation of the MAP-Elites algorithm, MAP-Elites with Transverse Assessment (MEliTA), which is tailored for multimodal creative tasks and leverages deep learned models that assess coherence across modalities. MEliTA decouples the artefacts' modalities and promotes cross-pollination between elites. As a test bed for this algorithm, we generate text descriptions and cover images for a hypothetical video game and assign each artefact a unique modality-specific behavioural characteristic. Results indicate that MEliTA can improve text-to-image mappings within the solution space, compared to a baseline MAP-Elites algorithm that strictly treats each image-text pair as one solution. Our approach represents a significant step forward in multimodal bottom-up orchestration and lays the groundwork for more complex systems coordinating multimodal creative agents in the future.
in Proceedings of the International Conference on Computational Intelligence in Music, Sound, Art and Design (EvoMusArt), 2024. BibTex
Seeding Diversity into AI Art
Marvin Zammit, Antonios Liapis and Georgios N. Yannakakis
Images generated via Novelty Search with Local Competition on a latent vector previously optimized through a Generative Adversarial Algorithm.
Abstract: This paper argues that generative art driven by conformance to a visual and/or semantic corpus lacks the necessary criteria to be considered creative. Among several issues identified in the literature, we focus on the fact that generative adversarial networks (GANs) that create a single image, in a vacuum, lack a concept of novelty regarding how their product differs from previously created ones. We envision that an algorithm that combines the novelty preservation mechanisms in evolutionary algorithms with the power of GANs can deliberately guide its creative process towards output that is both good and novel. In this paper, we use recent advances in image generation based on semantic prompts using OpenAI's CLIP model, interrupting the GAN's iterative process with short cycles of evolutionary divergent search. The results of evolution are then used to continue the GAN's iterative process; we hypothesise that this intervention will lead to more novel outputs. Testing our hypothesis using novelty search with local competition, a quality-diversity evolutionary algorithm that can increase visual diversity while maintaining quality in the form of adherence to the semantic prompt, we explore how different notions of visual diversity can affect both the process and the product of the algorithm. Results show that even a simplistic measure of visual diversity can help counter a drift towards similar images caused by the GAN. This first experiment opens a new direction for introducing higher intentionality and a more nuanced drive for GANs.
in Proceedings of the International Conference on Computational Creativity, 2022. BibTex
AffectGAN: Affect-Based Generative Art Driven by Semantics
Theodoros Galanos, Antonios Liapis and Georgios N. Yannakakis
Two images generated via AffectGAN for a cityscape prompt: the left image is generated for "a happy cityscape" and the right image is generated for "a depressed cityscape".
Abstract: This paper introduces a novel method for generating artistic images that express particular affective states. Leveraging state-of-the-art deep learning methods for visual generation (through generative adversarial networks), semantic models from OpenAI, and the annotated dataset of the visual art encyclopedia WikiArt, our AffectGAN model is able to generate images based on specific or broad semantic prompts and intended affective outcomes. A small dataset of 32 images generated by AffectGAN is annotated by 50 participants in terms of the particular emotion they elicit, as well as their quality and novelty. Results show that for most instances the intended emotion used as a prompt for image generation matches the participants' responses. This small-scale study brings forth a new vision towards blending affective computing with computational creativity, enabling generative systems with intentionality in terms of the emotions they wish their output to elicit.
in Proceedings of the ACII Workshop on What's Next in Affect Modeling?, 2021. BibTex
Architectural Form and Affect: A Spatiotemporal Study of Arousal
Emmanouil Xylakis, Antonios Liapis and Georgios N. Yannakakis
First-person views of the 24 rooms examined in the AffRooms corpus, while a player navigates through them.
Abstract: How does the form of our surroundings impact the ways we feel? This paper extends the body of research on the effects that space and light have on emotion by focusing on critical features of architectural form and illumination colors and their spatiotemporal impact on arousal. For that purpose, we solicited a corpus of spatial transitions in video form, lasting over 60 minutes, annotated by three participants in terms of arousal in a time-continuous and unbounded fashion. We process the annotation traces of that corpus in a relative fashion, focusing on the direction of arousal changes (increasing or decreasing) as affected by changes between consecutive rooms. Results show that properties of the form such as curved or complex spaces align highly with increased arousal. The analysis presented in this paper sheds some initial light in the relationship between arousal and core spatiotemporal features of form that is of particular importance for the affect-driven design of architectural spaces.
in Proceedings of the IEEE International Conference on Affective Computing and Intelligent Interaction, 2021. BibTex
Modelling the Quality of Visual Creations in Iconoscope
Antonios Liapis, Daniele Gravina, Emil Kastbjerg and Georgios N. Yannakakis
Network architecture for late multimodal fusion of icons' visuals and summary statistics.
Abstract: This paper presents the current state of the online game Iconoscope and analyzes the data collected from almost 45 months of continuous operation. Iconoscope is a freeform creation game which aims to foster the creativity of its users through diagrammatic lateral thinking, as users are required to depict abstract concepts as icons which may be misinterpreted by other users as different abstract concepts. From users' responses collected from an online gallery of all icons drawn with Iconoscope, we collect a corpus of over 500 icons which contain annotations of visual appeal. Several machine learning algorithms are tested for their ability to predict the appeal of an icon from its visual appearance and other properties. findings show the impact of the representation on the model's accuracy and highlight how such a predictive model of quality can be applied to evaluate new icons (human-authored or generated).
in Proceedings of the 8th International Games and Learning Alliance Conference. Springer, 2019. BibTex
Orchestrating Game Generation
Antonios Liapis, Georgios N. Yannakakis, Mark J. Nelson, Mike Preuss and Rafael Bidarra
The key questions of facet orchestration: where does the inspiration come from, what is orchestrated and how, and who is the intended audience?
Abstract: The design process is often characterized by and realized through the iterative steps of evaluation and refinement. When the process is based on a single creative domain such as visual art or audio production, designers primarily take inspiration from work within their domain and refine it based on their own intuitions or feedback from an audience of experts from within the same domain. What happens, however, when the creative process involves more than one creative domain such as in a digital game? How should the different domains influence each other so that the final outcome achieves a harmonized and fruitful communication across domains? How can a computational process orchestrate the various computational creators of the corresponding domains so that the final game has the desired functional and aesthetic characteristics? To address these questions, this article identifies game facet orchestration as the central challenge for AI-based game generation, discusses its dimensions and reviews research in automated game generation that has aimed to tackle it. In particular, we identify the different creative facets of games, we propose how orchestration can be facilitated in a top-down or bottom-up fashion, we review indicative preliminary examples of orchestration, and we conclude by discussing the open questions and challenges ahead.
in Transactions on Games, vol. 11, no 1, pp. 48-68, 2019. BibTex
Data-driven Design: A Case for Maximalist Game Design
Gabriella A. B. Barros, Michael Cerny Green, Antonios Liapis and Julian Togelius
Examples of games along the dimensions of data versus game fidelity and functional versus decorative role.
Abstract: Maximalism in art refers to drawing on and combining multiple different sources for art creation, embracing the resulting collisions and heterogeneity. This paper discusses the use of maximalism in game design and particularly in data games, which are games that are generated partly based on open data. Using Data Adventures, a series of generators that create adventure games from data sources such as Wikipedia and OpenStreetMap, as a lens we explore several tradeoffs and issues in maximalist game design. This includes the tension between transformation and fidelity, between decorative and functional content, and legal and ethical issues resulting from this type of generativity. This paper sketches out the design space of maximalist data-driven games, a design space that is mostly unexplored.
in Proceedings of the International Conference of Computational Creativity, 2018. BibTex
Recomposing the Pokemon Color Palette
Antonios Liapis
Differences between the original color palette (top) and the evolved color palette (bottom), which removes the water type from all 141 water Pokemon in the dataset, assigning them instead to different types (and dual types).
Abstract: In digital games, the visual representation of game assets such as avatars or game levels can hint at their purpose, in-game use and strengths. In the Pokemon games, this is particularly prevalent with the namesake creatures' type and the colors in their sprites. To win these games, players choose Pokemon of the right type to counter their opponents' strengths; this makes the visual identification of type important. In this paper, computational intelligence methods are used to learn a mapping between a Pokemon's type and its in-game sprite, colors and shape. This mapping can be useful for a designer attempting to create new Pokemon of certain types. In this paper, instead, evolutionary algorithms are used to create new Pokemon sprites by using existing color information but recombining it into a new palette. Results show that evolution can be applied to Pokemon sprites on a local or global scale, to exert different degrees of designer control and to achieve different goals.
in Applications of Evolutionary Computation. Springer, 2018. BibTex
Mapping Chess Aesthetics onto Procedurally Generated Chess-like Games
Jakub Kowalski, Antonios Liapis and Lukasz Zarczynski
Indicative evolved pieces targeting the original chess visuals. The average piece (darker background) is a stepping stone targeted in the first evolutionary step, and it is the starting seed for individual evolution towards each individual piece.
Abstract: Variants of chess have been generated in many forms and for several reasons, such as testbeds for artificial intelligence research in general game playing. This paper uses the visual properties of chess pieces as inspiration to generate new shapes for other chess-like games, targeting specific visual properties which allude to the pieces' in-game function. The proposed method uses similarity measures in terms of pieces' strategic role and movement in a game to identify the new pieces' closest representatives in chess. Evolution then attempts to minimize the distance from chess pieces' visual properties, resulting in new shapes which combine one or more chess pieces' visual identities. While experiments in this paper focus on two chess-like games from previous publications, the method can be used for broader generation of game visuals based on functional similarities of components to known, popular games.
in Applications of Evolutionary Computation. Springer, 2018. BibTex
Modelling Affect for Horror Soundscapes
Phil Lopes, Antonios Liapis and Georgios N. Yannakakis
Web interface through which users ranked two sounds in terms of tension, arousal and valence.
Abstract: The feeling of horror within movies or games relies on the audience's perception of a tense atmosphere - often achieved through sound accompanied by the on-screen drama - guiding its emotional experience throughout the scene or game-play sequence. These progressions are often crafted through an a priori knowledge of how a scene or game-play sequence will playout, and the intended emotional patterns a game director wants to transmit. The appropriate design of sound becomes even more challenging once the scenery and the general context is autonomously generated by an algorithm. Towards realizing sound-based affective interaction in games this paper explores the creation of computational models capable of ranking short audio pieces based on crowdsourced annotations of tension, arousal and valence. Affect models are trained via preference learning on over a thousand annotations with the use of support vector machines, whose inputs are low-level features extracted from the audio assets of a comprehensive sound library. The models constructed in this work are able to predict the tension, arousal and valence elicited by sound, respectively, with an accuracy of approximately 65%, 66% and 72%.
IEEE Transactions on Affective Computing, vol. 10, no 2, pp. 209-222, 2019. BibTex
Sonancia: a Multi-Faceted Generator for Horror
Phil Lopes, Antonios Liapis and Georgios N. Yannakakis
Gameplay footage of a Sonancia level generated and sonified by a generative process.
Abstract: Fear and tension are the primary emotions elicited by the genre of horror, a peculiar characteristic for media whose sole purpose is to entertain. The audience is often lead into tense and fearful situations, meticulously crafted by the authors using a narrative progression and a combination of visual and auditory stimuli. This paper presents a playable demonstration of the Sonancia system, a multi-faceted content generator for 3D horror games, with the capability of generating levels and their corresponding soundscapes. Designers can also guide the level generation process, by defining an intended progression of tension, which the level generator and sonification will adhere to.
in Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG). 2016. BibTex
Framing Tension for Game Generation
Phil Lopes, Antonios Liapis and Georgios N. Yannakakis
The creative process of the Sonancia system: a randomly selected tension frame is used to evaluate an evolving tension progression, which guides the evolution of a level generator which attempts to place monsters and items to match the tension curve. finally, the evolved level and its derived tension curve are used to deterministically allocate sounds based on a crowdsourced model of aural tension.
Abstract: Emotional progression in narratives is carefully structured by human authors to create unexpected and exciting situations, often culminating in a climactic moment. This paper explores how an autonomous computational designer can create frames of tension which guide the procedural creation of levels and their soundscapes in a digital horror game. Using narrative concepts, the autonomous designer can describe an intended experience that the automated level generator must adhere to. The level generator interprets this intent, bound by the possibilities and constraints of the game. The tension of the generated level guides the allocation of sounds in the level, using a crowdsourced model of tension.
in Proceedings of the International Conference on Computational Creativity. 2016. BibTex
Discovering Social and Aesthetic Categories of Avatars: A Bottom-Up Artificial Intelligence Approach Using Image Clustering
Chong-U Lim, Antonios Liapis and D. Fox Harrell
Using non-negative matrix factorization we can find prototypes (left) which identify important features of the avatars. Using the alpha (top), brightness (middle) or hue (bottom) channel, the algorithm can identify features of the avatars' outline, or shapes and colors of their actual features respectively.
Abstract: Videogame avatars are more than visual artifacts - they express cultural norms and expectations from both the real world and the fictional world. In this paper, we describe how artificial intelligence clustering can automatically discover distinct characteristics of players' avatars without prior knowledge of a system's underlying data structures. Using only avatar images collected from a study with 191 players, we applied two clustering techniques - namely non-negative matrix factorization and archetypal analysis - that automatically revealed and detected (1) an avatar's gender, (2) regions that appeared to isolate shapes of items and accessories, and (3) aesthetic preferences for particular colors (e.g., bright or muted) and shapes for different body parts. These clusters correlated with players' preferences for character abilities, e.g., male avatars in dark clothes correlated with having high physical but low magic-casting attributes. These findings show that a bottom-up analysis of images can reveal explicit categories like gender, but also implicit categories like preferences of players. We believe that such computational approaches can enable developers to (1) better understand players' desires and needs, (2) quantitatively view how systems may be limited in supporting players, and (3) find actionable solutions for these limitations.
in Proceedings of the International Joint Conference of DiGRA and FDG. 2016. BibTex
Exploring the Visual Styles of Arcade Game Assets
Antonios Liapis
Spaceship styles in different colors, evolved by several fi-2pop genetic algorithms optimizing different style objectives.
Abstract: This paper describes a method for evolving assets for video games based on their visuals properties. Focusing on assets for a space shooter game, a genotype consisting of turtle commands is transformed into a spaceship image composed of human-authored sprite components. Due to constraints on the final spaceships' plausibility, the paper investigates two-population constrained optimization and constrained novelty search methods. A sample of visual styles is tested, each a combination of visual metrics which primarily evaluate balance and shape complexity. Experiments with constrained optimization of a visual style demonstrate that a visually consistent set of spaceships can be generated, while experiments with constrained novelty search demonstrate that several distinct visual styles can be discovered by exploring along select, or all, visual dimensions.
in Proceedings of Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt). Springer, 2016. BibTex
Targeting Horror via Level and Soundscape Generation
Phil Lopes, Antonios Liapis and Georgios N Yannakakis
Horror levels of different sizes, evolved according to a linearly increasing tension model. Levels contain monsters (green triangles), the main quest item (blue square) and subquest items (pink circles); the player starts the level from the darker room and must reach the main quest item to finish the level.
Abstract: Horror games form a peculiar niche within game design paradigms, as they entertain by eliciting negative emotions such as fear and unease to their audience during play. This genre often follows a specific progression of tension culminating at a metaphorical peak, which is defined by the designer. A player's tension is elicited by several facets of the game, including its mechanics, its sounds, and the placement of enemies in its levels. This paper investigates how designers can control and guide the automated generation of levels and their soundscapes by authoring the intended tension of a player traversing them.
in Proceedings of the AAAI Artificial Intelligence for Interactive Digital Entertainment Conference, 2015. BibTex
Sonancia: Sonification of Procedurally Generated Game Levels
Phil Lopes, Antonios Liapis and Georgios N Yannakakis
The critical path of a player traversing a haunted mansion from a starting room to a final room (left image) is used as the objective of a level generation module but also as input to a sonification module which adds background music to each room (right image).
Abstract: How can creative elements brought from level design effectively be coupled with audio in order to create tense and engaging player experiences? In this paper the above question is posed through the sonification of procedurally generated digital game levels. The paper details some initial approaches and methodologies for achieving this core aim.
in Proceedings of the ICCC workshop on Computational Creativity & Games, 2015. BibTex
AudioInSpace: Exploring the Creative Fusion of Generative Audio Visuals and Gameplay
Amy K. Hoover, William Cachia, Antonios Liapis and Georgios N. Yannakakis
Weapon particle trajectories in AudioInSpace which are generated according to a procedurally generated game soundtrack.
Abstract: Computer games are unique creativity domains in that they elegantly fuse several facets of creative work including visuals, narrative, music, architecture and design. While the exploration of possibilities across facets of creativity offers a more realistic approach to the game design process, most existing autonomous (or semi-autonomous) game content generators focus on the mere generation of single domains (creativity facets) in games. Motivated by the sparse literature on multifaceted game content generation, this paper introduces a multifaceted procedural content generation (PCG) approach that is based on the interactive evolution of multiple artificial neural networks that orchestrate the generation of visuals, audio and gameplay. The approach is evaluated on a spaceship shooter game. The generated artifacts - a fusion of audiovisual and gameplay elements - showcase the capacity of multifaceted PCG and its evident potential for computational game creativity.
in Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt), vol. 9027, LNCS. Springer, 2015. BibTex
Computational Game Creativity
Antonios Liapis, Georgios N. Yannakakis and Julian Togelius
Abstract: Computational creativity has traditionally relied on well-controlled, single-faceted and established domains such as visual art, narrative and audio. On the other hand, research on autonomous generation methods for game artifacts has not yet considered the creative capacity of those methods. In this paper we position computer games as the ideal application domain for computational creativity for the unique features they offer: being highly interactive, dynamic and content-intensive software applications. Their multifaceted nature is key in our argumentation as the successful orchestration of different art domains (such as visual art, audio and level architecture) with game mechanics design is a grand challenge for the study of computational creativity in this multidisciplinary domain. Computer games not only challenge computational creativity and provide a creative sandbox for advancing the field but they also offer an opportunity for computational creativity methods to be extensively assessed (via a huge population of gamers) through commercial-standard products of high impact and financial value.
in Proceedings of the fifth International Conference on Computational Creativity, 2014. BibTex
Towards a Generic Method of Evaluating Game Levels
Antonios Liapis, Georgios N. Yannakakis and Julian Togelius
Map sketches for different game genres (top) and the complete maps they generate (bottom). The map sketches were evolved using heuristics of balance, area control and exploration.
Abstract: This paper addresses the problem of evaluating the quality of game levels across different games and even genres, which is of key importance for making procedural content generation and assisted game design tools more generally applicable. Three game design patterns are identified for having high generality while being easily quantifiable: area control, exploration and balance. Formulas for measuring the extent to which a level includes these concepts are proposed, and evaluation functions are derived for levels in two different game genres: multiplayer strategy game maps and single-player roguelike dungeons. To illustrate the impact of these evaluation functions, and the similarity of impact across domains, sets of levels for each function are generated using a constrained genetic algorithm. The proposed measures can easily be extended to other game genres.
in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2013. BibTex
Transforming Exploratory Creativity with DeLeNoX
Antonios Liapis, Hector P. Martinez, Julian Togelius and Georgios N. Yannakakis
Progressive complexification of generated spaceships trained with new features in every iteration.
Abstract: We introduce DeLeNoX (Deep Learning Novelty Explorer), a system that autonomously creates artifacts in constrained spaces according to its own evolving interestingness criterion. DeLeNoX proceeds in alternating phases of exploration and transformation. In the exploration phases, a version of novelty search augmented with constraint handling searches for maximally diverse artifacts using a given distance function. In the transformation phases, a deep learning autoencoder learns to compress the variation between the found artifacts into a lower-dimensional space. The newly trained encoder is then used as the basis for a new distance function, transforming the criteria for the next exploration phase. In the current paper, we apply DeLeNoX to the creation of spaceships suitable for use in two-dimensional arcade-style computer games, a representative problem in procedural content generation in games. We also situate DeLeNoX in relation to the distinction between exploratory and transformational creativity, and in relation to Schmidhuber's theory of creativity through the drive for compression progress.
in Proceedings of the Fourth International Conference on Computational Creativity, 2013, pp. 56-63. BibTex
Adapting Models of Visual Aesthetics for Personalized Content Creation
Antonios Liapis, Georgios N. Yannakakis and Julian Togelius
User interface for interactively evolving spaceship hulls.
Abstract: This paper introduces a search-based approach to personalized content generation with respect to visual aesthetics. The approach is based on a two-step adaptation procedure where (1) the evaluation function that characterizes the content is adjusted to match the visual aesthetics of users and (2) the content itself is optimized based on the personalized evaluation function. To test the efficacy of the approach we design fitness functions based on universal properties of visual perception, inspired by psychological and neurobiological research. Using these visual properties we generate aesthetically pleasing 2D game spaceships via neuroevolutionary constrained optimization and evaluate the impact of the designed visual properties on the generated spaceships. The offline generated spaceships are used as the initial population of an interactive evolution experiment in which players are asked to choose spaceships according to their visual taste: the impact of the various visual properties is adjusted based on player preferences and new content is generated online based on the updated computational model of visual aesthetics of the player. Results are presented which show the potential of the approach in generating content which is based on subjective criteria of visual aesthetics.
IEEE Transactions on Computational Intelligence and AI in Games 4(3), 2012, pp. 213-228. BibTex
Optimizing Visual Properties of Game Content through Neuroevolution
Antonios Liapis, Georgios N. Yannakakis and Julian Togelius
Different methods for calculating aesthetic metrics on the balance (first and second images), alignment (third image) and containment (fourth image) of spaceship hulls.
Abstract: This paper presents a search-based approach to generating game content that satisfies both gameplay requirements and user-expressed aesthetic criteria. Using evolutionary constraint satisfaction, we search for spaceships (for a space combat game) represented as compositional pattern-producing networks. While the gameplay requirements are satisfied by ad-hoc defined constraints, the aesthetic evaluation function can also be informed by human aesthetic judgement. This is achieved using indirect interactive evolution, where an evaluation function re-weights an array of aesthetic criteria based on the choices of a human player. Early results show that we can create aesthetically diverse and interesting spaceships while retaining in-game functionality.
in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2011. BibTex