System for automated interactive lighting

Video games have grown to a billion dollar entertainment industry. Beyond entertainment, games are increasingly the target of serious research, from learning to “serious play”, training, therapy, and communication. Alongside the expansion of game applications has come the expansion of game media. Games are now incredibly complex and expressive multimedia platforms that rival the highest fidelity tools of film or television. Independent of their applications, most games are visual works and any researcher or entertainer interested in games must first purposefully control and manipulate the visual substance of a game to achieve a goal. Therefore, this dissertation presents a focus on one factor that is inseparable from the visual experience of a game: lighting. Our real world visual experience is shaped by light. The very structure of modern life is possible because we have transcended the constraints of natural day and night cycles and can effectively operate 24 hours a day. Games too are now shaped by light. Modern computer visualization, the process of generating an image of a virtual thing, now offers a physically realistic illumination model. Virtual illumination can produce nearly photorealistic images (literally, images that look like a photograph). However, successful lighting in video games requires more than a physically realistic illumination model. Aesthetics and function are of equal or greater importance. Lighting can control visual focus, directing a player’s gaze to important objects. It can also control mood and perceived emotion, communicating a narrative and engaging a player in a game world. The role of a lighting designer in games is therefore crucial. Lighting designers control and purposefully manipulate light. A lighting designer in film will carefully control lighting, scene by scene and even frame by frame, to precisely convey emotion, control visual perception, and communicate intent. Game lighting designers face a challenge however, in that omniscient knowledge of a game’s state at any time is not available to them in advance. A player can make many decisions that change the game world. For example, a film lighting designer might carefully arrange lighting to strike a specific visual cue when the lead actor crosses the threshold of a front door. However, a game lighting designer might also need to predict and accommodate a player coming in through a bedroom window, a player destroying the front door, a player never coming to the house at all, or a player coming to the house at a point in the story where the visual cue is no longer appropriate. Under an assumption that game lighting designers require tools to design light that assist with the challenge of adapting to player interaction at runtime, this dissertation presents the design, development, and final product of research into a lighting design tool, the System for Automated Interactive Lighting (SAIL). The goal of this research was to develop an adaptive system that maintains lighting design goals (aesthetic and functional) in the context of unpredictable, interactive experiences. This dissertation makes several contributions. As an artifact, SAIL offers functionality not available in existing automated lighting systems. As a design tool, SAIL may be useful as a pre-visualization or storyboarding tool. As an experimental instrument, SAIL offers the possibility of simplifying existing experiments into automated lighting and facilitating new types of experiments. SAIL’s contributions are discussed in depth in Chapter 8. These contributions are supported by a qualitative study discussed in Chapters 6 and 7. These chapters also present several other interesting observations that may be useful in general to the field of automated lighting.