- Dr. Francis Quek,
- Diane Cahill Bedford (MFA)
- Sharon Lynn Chu
- Dr. Ricardo Gutierrez-Osuna
- Jose Quintana
- Andre Thomas
- Dr. Ico Bukvic, Asst. Professor Department of Music
- Dr. Yong Cao, Asst. Professor Computer Science
- Dr. Dane Webster, Assoc. Professor School of Visual Arts
- Xiao Lin, Ph.D. Student Computer Science
- David Mills, Undergrad Student School of Visual Arts
- SeungIn Park, Ph.D. Student Computer Science
- Mi Peng, Ph.D. Student, Computer Science
- Geraldine Aui, Summer intern from Singapore Polytechnic"
- Peng Chao, Ph.D. Student Computer Science
- Bing Fang, Ph.D. Student Computer Science (Completed)
- Huiying Goh. "Summer intern from National University of Singapore
- Yao Wei Tan, Summer intern from Singapore Polytechnic"
This project is a collaborative effort for integrating engineering with music (and other performing arts) and visual arts to realize a new game genre that combines skilled performance with casual game engagement in multi-level participatory gameplay. the gameplay itself is designed to engage two classes of players in creative interaction: skilled performers and casual participants. The skilled performance takes place in the real world with real instruments and creative action. Computation is employed to translate these into game control signals. We expect that to the degree that the casual participants engage in the game, they will also be drawn into the performance in an ‘engagement of commitment’ the same way partisans in a college football game invest their energies in the skilled performance of their athletes. The difference in our game is that the casual participants are able to affect the outcome of the game directly. The performers, in turn, engage the audience both through the performance per se, and through the joint gameplay.
Imagine a massive-interaction game in which players control large forces of ancient ‘Terra-cotta Armies’ in battle on large double-sided theater-scale screens (top-left of Fig. 1). The warriors arranged in several large battalions per side are controlled in real-time by teams of players (a battalion per player). As in ancient times, there are no radio communications, so the players signal the troops using ‘battle drums’ (top-right of Fig. 1) to execute mass battle actions like advancing, retreating, flanking right or left, or forming up in a variety of pre-defined formations (top-right of Figure 1). The battalions march in response to the drums using real-time crowd simulation. The game begins as the drummers beat out the cadence to awake their respective armies from their suspended animation. When the two armies come into contact, artificial intelligence kicks in so that the warriors engage in combat, executing group activities such as double-teaming an opponent and support-ing a compatriot that is being double teamed, all the while moving to obey the direction of the human drummers. Depending on the type of troops engaged (e.g. elite troops or infantry), warriors in personal combat have different likelihoods of winning or falling. Warriors that fall are returned terra-cotta clay.
A larger audience participates in the action by cheering for their side (bottom-right of Fig. 1). The volume of the cheering modifies the morale of their troops, shifting their likelihood of winning in individual combat. Cheerleaders encourage each audience block to move and cheer in unison. As the coordination of the audience movement of a side increases, the morale of their terra-cotta soldiers becomes stronger.
Imagine, further, that this grand spectacle is played in large venues such as during half-time shows of evening football games. Theater-scale projectors display scenes showing the perspective from behind each army on both sides of large projection screens at midfield. Side screens on each side show fly-over views. These views are controlled by synthetic cameras that are manipulated over a horizontal display showing a plan view of the battle, allowing zoom-ins of sections of the battle being fought. As the drums sound, synthetic music that match their cadence and beat play over the large speakers (middle-right of Fig. 1). The music is set to match the drum fugue being performed as the players try to out-maneuver their opponents.
The system is constructed on several components. These are:
- Communications Support: The system is composed of different pieces on different platforms. These components are linked through a network of UDP network links that support the inter module communication.
- Game Input Module: The game input module processes the drum signals and interprets these as a drumming pattern id and a repetition speed. These are fed to the simulation, game play, music engines to control the different soldier units.
- Simulation Engine: This module takes the desired movements of the soldier units and converts this into the motion simulations of all individual soldiers. A Game Play Module within the simulation engine applies game play rules to determine the effect of combat actions in the game.
- Animation Engine: This module receives the soldier locations and types of action (e.g. run, or walk) from the Simulation Engine and animates it from a camera viewpoint.
- Music Engine: The music engine takes the drumming input (type of action and pace) and activity that affects mood (e.g., whether the action is to attack or retreat), and generates accompanying synthetic music for the game.
- Audience Participation Input Module: Since we expect the Drummer Game to be performed in the evenings, our audience participation module detects movements from light patterns of audience participant mobile phones (e.g., doing 'the wave' with the phone held up).
The simulation engine is responsible to provide interesting soldier behaviors, including formation-based movement, collision avoidance, and fighting for all of the soldiers in the game.
In order to imitate realistic soldier formation, a lattice approach creating invisible spring forces to provide attraction and separation between neighboring soldiers are devised. This approach can be thought of as a follow-the-leader movement – a forward movement from the front line will propagate necessary forward movement throughout all other rows in order to stabilize the distances between each soldier. To simulate a combat mode, a potential force-based approach is used. This method involves balancing attractive and repulsive forces, analogous to physical electric ones, to enable soldiers moving around and battling with other soldiers while not colliding with them.
To guarantee the real-time performance of simulating thousands of soldiers, all the movement of soldiers are calculated on the GPU using Nvidia’s CUDA technology. This requires various strategies for parallelism depending on the algorithm for updating the soldiers.
The role of the animation engine is to visualize the whole battle field. The game engine receives the commands from the simulation engine by a local area network, and shows the soldiers, arrows and their action in a 3D virtual battle field. It also shows the legends on top of each army corresponding to the commands.
Technique Details: The animation engine is based on the OpenGL and uses several techniques to speed up the performance due to this massive-interaction game.
- Pipeline: The animation engine receives each soldier's position, action, orientation, and based on the commands to load the corresponding animation data, which generated by the Maya software.
- LOD: In the animation engine, we use level of detail(LOD) technique to speed up the performance. Based on the distance between the camera and the soldiers, the animation engine shows different resolution of the soldiers. Now the animation engine can support 4 resolutions solders.
- Motion Transition: To make sure the different animations to smoothly change, we define a motion transition file to integrate different animaiton.
This research has been partially funded by NSF grants, Drummer Game: A Massive-Interactive Socially-Enabled Strategy Game, NSF IIS-0940723, 7/01/09 - 8/1/12., and CRI: Interfaces for the embodied mind, IIS-0551610.