Game Aesthetics

"Game aesthetics" is a broad and ambiguous term which encompasses believability, challenge, surprise, novelty, visual and aural appeal, interestingness, and more. Examining the "human" aspects of games from a computational perspective is of particular interest to my research.

MAP-Elites with Transverse Assessment for Multimodal Problems in Creative Domains

Marvin Zammit, Antonios Liapis, Georgios N. Yannakakis

Abstract: The recent advances in language-based generative models have paved the way for the orchestration of multiple generators of different artefact types (text, image, audio, etc.) into one system. Presently, many open-source pre-trained models combine text with other modalities, thus enabling shared vector embeddings to be compared across different generators. Within this context we propose a novel approach to handle multimodal creative tasks using Quality Diversity evolution. Our contribution is a variation of the MAP-Elites algorithm, MAP-Elites with Transverse Assessment (MEliTA), which is tailored for multimodal creative tasks and leverages deep learned models that assess coherence across modalities. MEliTA decouples the artefacts' modalities and promotes cross-pollination between elites. As a test bed for this algorithm, we generate text descriptions and cover images for a hypothetical video game and assign each artefact a unique modality-specific behavioural characteristic. Results indicate that MEliTA can improve text-to-image mappings within the solution space, compared to a baseline MAP-Elites algorithm that strictly treats each image-text pair as one solution. Our approach represents a significant step forward in multimodal bottom-up orchestration and lays the groundwork for more complex systems coordinating multimodal creative agents in the future.

in Proceedings of the International Conference on Computational Intelligence in Music, Sound, Art and Design (EvoMusArt), 2024. BibTex

Seeding Diversity into AI Art

Marvin Zammit, Antonios Liapis and Georgios N. Yannakakis

Abstract: This paper argues that generative art driven by conformance to a visual and/or semantic corpus lacks the necessary criteria to be considered creative. Among several issues identified in the literature, we focus on the fact that generative adversarial networks (GANs) that create a single image, in a vacuum, lack a concept of novelty regarding how their product differs from previously created ones. We envision that an algorithm that combines the novelty preservation mechanisms in evolutionary algorithms with the power of GANs can deliberately guide its creative process towards output that is both good and novel. In this paper, we use recent advances in image generation based on semantic prompts using OpenAI's CLIP model, interrupting the GAN's iterative process with short cycles of evolutionary divergent search. The results of evolution are then used to continue the GAN's iterative process; we hypothesise that this intervention will lead to more novel outputs. Testing our hypothesis using novelty search with local competition, a quality-diversity evolutionary algorithm that can increase visual diversity while maintaining quality in the form of adherence to the semantic prompt, we explore how different notions of visual diversity can affect both the process and the product of the algorithm. Results show that even a simplistic measure of visual diversity can help counter a drift towards similar images caused by the GAN. This first experiment opens a new direction for introducing higher intentionality and a more nuanced drive for GANs.

in Proceedings of the International Conference on Computational Creativity, 2022. BibTex

AffectGAN: Affect-Based Generative Art Driven by Semantics

Theodoros Galanos, Antonios Liapis and Georgios N. Yannakakis

Abstract: This paper introduces a novel method for generating artistic images that express particular affective states. Leveraging state-of-the-art deep learning methods for visual generation (through generative adversarial networks), semantic models from OpenAI, and the annotated dataset of the visual art encyclopedia WikiArt, our AffectGAN model is able to generate images based on specific or broad semantic prompts and intended affective outcomes. A small dataset of 32 images generated by AffectGAN is annotated by 50 participants in terms of the particular emotion they elicit, as well as their quality and novelty. Results show that for most instances the intended emotion used as a prompt for image generation matches the participants' responses. This small-scale study brings forth a new vision towards blending affective computing with computational creativity, enabling generative systems with intentionality in terms of the emotions they wish their output to elicit.

in Proceedings of the ACII Workshop on What's Next in Affect Modeling?, 2021. BibTex

Modelling the Quality of Visual Creations in Iconoscope

Antonios Liapis, Daniele Gravina, Emil Kastbjerg and Georgios N. Yannakakis

Abstract: This paper presents the current state of the online game Iconoscope and analyzes the data collected from almost 45 months of continuous operation. Iconoscope is a freeform creation game which aims to foster the creativity of its users through diagrammatic lateral thinking, as users are required to depict abstract concepts as icons which may be misinterpreted by other users as different abstract concepts. From users' responses collected from an online gallery of all icons drawn with Iconoscope, we collect a corpus of over 500 icons which contain annotations of visual appeal. Several machine learning algorithms are tested for their ability to predict the appeal of an icon from its visual appearance and other properties. findings show the impact of the representation on the model's accuracy and highlight how such a predictive model of quality can be applied to evaluate new icons (human-authored or generated).

in Proceedings of the 8th International Games and Learning Alliance Conference. Springer, 2019. BibTex

Orchestrating Game Generation

Antonios Liapis, Georgios N. Yannakakis, Mark J. Nelson, Mike Preuss and Rafael Bidarra

Abstract: The design process is often characterized by and realized through the iterative steps of evaluation and refinement. When the process is based on a single creative domain such as visual art or audio production, designers primarily take inspiration from work within their domain and refine it based on their own intuitions or feedback from an audience of experts from within the same domain. What happens, however, when the creative process involves more than one creative domain such as in a digital game? How should the different domains influence each other so that the final outcome achieves a harmonized and fruitful communication across domains? How can a computational process orchestrate the various computational creators of the corresponding domains so that the final game has the desired functional and aesthetic characteristics? To address these questions, this article identifies game facet orchestration as the central challenge for AI-based game generation, discusses its dimensions and reviews research in automated game generation that has aimed to tackle it. In particular, we identify the different creative facets of games, we propose how orchestration can be facilitated in a top-down or bottom-up fashion, we review indicative preliminary examples of orchestration, and we conclude by discussing the open questions and challenges ahead.

in Transactions on Games, vol. 11, no 1, pp. 48-68, 2019. BibTex

Data-driven Design: A Case for Maximalist Game Design

Gabriella A. B. Barros, Michael Cerny Green, Antonios Liapis and Julian Togelius

Abstract: Maximalism in art refers to drawing on and combining multiple different sources for art creation, embracing the resulting collisions and heterogeneity. This paper discusses the use of maximalism in game design and particularly in data games, which are games that are generated partly based on open data. Using Data Adventures, a series of generators that create adventure games from data sources such as Wikipedia and OpenStreetMap, as a lens we explore several tradeoffs and issues in maximalist game design. This includes the tension between transformation and fidelity, between decorative and functional content, and legal and ethical issues resulting from this type of generativity. This paper sketches out the design space of maximalist data-driven games, a design space that is mostly unexplored.

in Proceedings of the International Conference of Computational Creativity, 2018. BibTex

Recomposing the Pokemon Color Palette

Antonios Liapis

Abstract: In digital games, the visual representation of game assets such as avatars or game levels can hint at their purpose, in-game use and strengths. In the Pokemon games, this is particularly prevalent with the namesake creatures' type and the colors in their sprites. To win these games, players choose Pokemon of the right type to counter their opponents' strengths; this makes the visual identification of type important. In this paper, computational intelligence methods are used to learn a mapping between a Pokemon's type and its in-game sprite, colors and shape. This mapping can be useful for a designer attempting to create new Pokemon of certain types. In this paper, instead, evolutionary algorithms are used to create new Pokemon sprites by using existing color information but recombining it into a new palette. Results show that evolution can be applied to Pokemon sprites on a local or global scale, to exert different degrees of designer control and to achieve different goals.

in Applications of Evolutionary Computation. Springer, 2018. BibTex

Mapping Chess Aesthetics onto Procedurally Generated Chess-like Games

Jakub Kowalski, Antonios Liapis and Lukasz Zarczynski

Abstract: Variants of chess have been generated in many forms and for several reasons, such as testbeds for artificial intelligence research in general game playing. This paper uses the visual properties of chess pieces as inspiration to generate new shapes for other chess-like games, targeting specific visual properties which allude to the pieces' in-game function. The proposed method uses similarity measures in terms of pieces' strategic role and movement in a game to identify the new pieces' closest representatives in chess. Evolution then attempts to minimize the distance from chess pieces' visual properties, resulting in new shapes which combine one or more chess pieces' visual identities. While experiments in this paper focus on two chess-like games from previous publications, the method can be used for broader generation of game visuals based on functional similarities of components to known, popular games.

in Applications of Evolutionary Computation. Springer, 2018. BibTex

Modelling Affect for Horror Soundscapes

Phil Lopes, Antonios Liapis and Georgios N. Yannakakis

Abstract: The feeling of horror within movies or games relies on the audience's perception of a tense atmosphere - often achieved through sound accompanied by the on-screen drama - guiding its emotional experience throughout the scene or game-play sequence. These progressions are often crafted through an a priori knowledge of how a scene or game-play sequence will playout, and the intended emotional patterns a game director wants to transmit. The appropriate design of sound becomes even more challenging once the scenery and the general context is autonomously generated by an algorithm. Towards realizing sound-based affective interaction in games this paper explores the creation of computational models capable of ranking short audio pieces based on crowdsourced annotations of tension, arousal and valence. Affect models are trained via preference learning on over a thousand annotations with the use of support vector machines, whose inputs are low-level features extracted from the audio assets of a comprehensive sound library. The models constructed in this work are able to predict the tension, arousal and valence elicited by sound, respectively, with an accuracy of approximately 65%, 66% and 72%.

IEEE Transactions on Affective Computing, vol. 10, no 2, pp. 209-222, 2019. BibTex

Sonancia: a Multi-Faceted Generator for Horror

Phil Lopes, Antonios Liapis and Georgios N. Yannakakis

Abstract: Fear and tension are the primary emotions elicited by the genre of horror, a peculiar characteristic for media whose sole purpose is to entertain. The audience is often lead into tense and fearful situations, meticulously crafted by the authors using a narrative progression and a combination of visual and auditory stimuli. This paper presents a playable demonstration of the Sonancia system, a multi-faceted content generator for 3D horror games, with the capability of generating levels and their corresponding soundscapes. Designers can also guide the level generation process, by defining an intended progression of tension, which the level generator and sonification will adhere to.

in Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG). 2016. BibTex

Framing Tension for Game Generation

Phil Lopes, Antonios Liapis and Georgios N. Yannakakis

Abstract: Emotional progression in narratives is carefully structured by human authors to create unexpected and exciting situations, often culminating in a climactic moment. This paper explores how an autonomous computational designer can create frames of tension which guide the procedural creation of levels and their soundscapes in a digital horror game. Using narrative concepts, the autonomous designer can describe an intended experience that the automated level generator must adhere to. The level generator interprets this intent, bound by the possibilities and constraints of the game. The tension of the generated level guides the allocation of sounds in the level, using a crowdsourced model of tension.

in Proceedings of the International Conference on Computational Creativity. 2016. BibTex

Discovering Social and Aesthetic Categories of Avatars: A Bottom-Up Artificial Intelligence Approach Using Image Clustering

Chong-U Lim, Antonios Liapis and D. Fox Harrell

Abstract: Videogame avatars are more than visual artifacts - they express cultural norms and expectations from both the real world and the fictional world. In this paper, we describe how artificial intelligence clustering can automatically discover distinct characteristics of players' avatars without prior knowledge of a system's underlying data structures. Using only avatar images collected from a study with 191 players, we applied two clustering techniques - namely non-negative matrix factorization and archetypal analysis - that automatically revealed and detected (1) an avatar's gender, (2) regions that appeared to isolate shapes of items and accessories, and (3) aesthetic preferences for particular colors (e.g., bright or muted) and shapes for different body parts. These clusters correlated with players' preferences for character abilities, e.g., male avatars in dark clothes correlated with having high physical but low magic-casting attributes. These findings show that a bottom-up analysis of images can reveal explicit categories like gender, but also implicit categories like preferences of players. We believe that such computational approaches can enable developers to (1) better understand players' desires and needs, (2) quantitatively view how systems may be limited in supporting players, and (3) find actionable solutions for these limitations.

in Proceedings of the International Joint Conference of DiGRA and FDG. 2016. BibTex

Exploring the Visual Styles of Arcade Game Assets

Antonios Liapis

Abstract: This paper describes a method for evolving assets for video games based on their visuals properties. Focusing on assets for a space shooter game, a genotype consisting of turtle commands is transformed into a spaceship image composed of human-authored sprite components. Due to constraints on the final spaceships' plausibility, the paper investigates two-population constrained optimization and constrained novelty search methods. A sample of visual styles is tested, each a combination of visual metrics which primarily evaluate balance and shape complexity. Experiments with constrained optimization of a visual style demonstrate that a visually consistent set of spaceships can be generated, while experiments with constrained novelty search demonstrate that several distinct visual styles can be discovered by exploring along select, or all, visual dimensions.

in Proceedings of Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt). Springer, 2016. BibTex

Targeting Horror via Level and Soundscape Generation

Phil Lopes, Antonios Liapis and Georgios N Yannakakis

Abstract: Horror games form a peculiar niche within game design paradigms, as they entertain by eliciting negative emotions such as fear and unease to their audience during play. This genre often follows a specific progression of tension culminating at a metaphorical peak, which is defined by the designer. A player's tension is elicited by several facets of the game, including its mechanics, its sounds, and the placement of enemies in its levels. This paper investigates how designers can control and guide the automated generation of levels and their soundscapes by authoring the intended tension of a player traversing them.

in Proceedings of the AAAI Artificial Intelligence for Interactive Digital Entertainment Conference, 2015. BibTex

Sonancia: Sonification of Procedurally Generated Game Levels

Phil Lopes, Antonios Liapis and Georgios N Yannakakis

Abstract: How can creative elements brought from level design effectively be coupled with audio in order to create tense and engaging player experiences? In this paper the above question is posed through the sonification of procedurally generated digital game levels. The paper details some initial approaches and methodologies for achieving this core aim.

in Proceedings of the ICCC workshop on Computational Creativity & Games, 2015. BibTex

AudioInSpace: Exploring the Creative Fusion of Generative Audio Visuals and Gameplay

Amy K. Hoover, William Cachia, Antonios Liapis and Georgios N. Yannakakis

Abstract: Computer games are unique creativity domains in that they elegantly fuse several facets of creative work including visuals, narrative, music, architecture and design. While the exploration of possibilities across facets of creativity offers a more realistic approach to the game design process, most existing autonomous (or semi-autonomous) game content generators focus on the mere generation of single domains (creativity facets) in games. Motivated by the sparse literature on multifaceted game content generation, this paper introduces a multifaceted procedural content generation (PCG) approach that is based on the interactive evolution of multiple artificial neural networks that orchestrate the generation of visuals, audio and gameplay. The approach is evaluated on a spaceship shooter game. The generated artifacts - a fusion of audiovisual and gameplay elements - showcase the capacity of multifaceted PCG and its evident potential for computational game creativity.

in Evolutionary and Biologically Inspired Music, Sound, Art and Design (EvoMusArt), vol. 9027, LNCS. Springer, 2015. BibTex

Computational Game Creativity

Antonios Liapis, Georgios N. Yannakakis and Julian Togelius

Abstract: Computational creativity has traditionally relied on well-controlled, single-faceted and established domains such as visual art, narrative and audio. On the other hand, research on autonomous generation methods for game artifacts has not yet considered the creative capacity of those methods. In this paper we position computer games as the ideal application domain for computational creativity for the unique features they offer: being highly interactive, dynamic and content-intensive software applications. Their multifaceted nature is key in our argumentation as the successful orchestration of different art domains (such as visual art, audio and level architecture) with game mechanics design is a grand challenge for the study of computational creativity in this multidisciplinary domain. Computer games not only challenge computational creativity and provide a creative sandbox for advancing the field but they also offer an opportunity for computational creativity methods to be extensively assessed (via a huge population of gamers) through commercial-standard products of high impact and financial value.

in Proceedings of the fifth International Conference on Computational Creativity, 2014. BibTex

Towards a Generic Method of Evaluating Game Levels

Antonios Liapis, Georgios N. Yannakakis and Julian Togelius

Abstract: This paper addresses the problem of evaluating the quality of game levels across different games and even genres, which is of key importance for making procedural content generation and assisted game design tools more generally applicable. Three game design patterns are identified for having high generality while being easily quantifiable: area control, exploration and balance. Formulas for measuring the extent to which a level includes these concepts are proposed, and evaluation functions are derived for levels in two different game genres: multiplayer strategy game maps and single-player roguelike dungeons. To illustrate the impact of these evaluation functions, and the similarity of impact across domains, sets of levels for each function are generated using a constrained genetic algorithm. The proposed measures can easily be extended to other game genres.

in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2013. BibTex

Transforming Exploratory Creativity with DeLeNoX

Antonios Liapis, Hector P. Martinez, Julian Togelius and Georgios N. Yannakakis

Abstract: We introduce DeLeNoX (Deep Learning Novelty Explorer), a system that autonomously creates artifacts in constrained spaces according to its own evolving interestingness criterion. DeLeNoX proceeds in alternating phases of exploration and transformation. In the exploration phases, a version of novelty search augmented with constraint handling searches for maximally diverse artifacts using a given distance function. In the transformation phases, a deep learning autoencoder learns to compress the variation between the found artifacts into a lower-dimensional space. The newly trained encoder is then used as the basis for a new distance function, transforming the criteria for the next exploration phase. In the current paper, we apply DeLeNoX to the creation of spaceships suitable for use in two-dimensional arcade-style computer games, a representative problem in procedural content generation in games. We also situate DeLeNoX in relation to the distinction between exploratory and transformational creativity, and in relation to Schmidhuber's theory of creativity through the drive for compression progress.

in Proceedings of the Fourth International Conference on Computational Creativity, 2013, pp. 56-63. BibTex

Adapting Models of Visual Aesthetics for Personalized Content Creation

Antonios Liapis, Georgios N. Yannakakis and Julian Togelius

Abstract: This paper introduces a search-based approach to personalized content generation with respect to visual aesthetics. The approach is based on a two-step adaptation procedure where (1) the evaluation function that characterizes the content is adjusted to match the visual aesthetics of users and (2) the content itself is optimized based on the personalized evaluation function. To test the efficacy of the approach we design fitness functions based on universal properties of visual perception, inspired by psychological and neurobiological research. Using these visual properties we generate aesthetically pleasing 2D game spaceships via neuroevolutionary constrained optimization and evaluate the impact of the designed visual properties on the generated spaceships. The offline generated spaceships are used as the initial population of an interactive evolution experiment in which players are asked to choose spaceships according to their visual taste: the impact of the various visual properties is adjusted based on player preferences and new content is generated online based on the updated computational model of visual aesthetics of the player. Results are presented which show the potential of the approach in generating content which is based on subjective criteria of visual aesthetics.

IEEE Transactions on Computational Intelligence and AI in Games 4(3), 2012, pp. 213-228. BibTex

Optimizing Visual Properties of Game Content through Neuroevolution

Antonios Liapis, Georgios N. Yannakakis and Julian Togelius

Abstract: This paper presents a search-based approach to generating game content that satisfies both gameplay requirements and user-expressed aesthetic criteria. Using evolutionary constraint satisfaction, we search for spaceships (for a space combat game) represented as compositional pattern-producing networks. While the gameplay requirements are satisfied by ad-hoc defined constraints, the aesthetic evaluation function can also be informed by human aesthetic judgement. This is achieved using indirect interactive evolution, where an evaluation function re-weights an array of aesthetic criteria based on the choices of a human player. Early results show that we can create aesthetically diverse and interesting spaceships while retaining in-game functionality.

in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2011. BibTex