(c) 2017 Petri Lankoski, v0.1 (a very drafty draft.)
The purpose of observational design in DGR is to understand how players interact with the game, how the system shapes the play and the player’s reactions to the game events. An observational design also means that there is someone else than the researcher playing. The researcher playing and recording the play is another type of study design (see analyzing previous designs). Also, if the focus is in the player-player interactions or relations in, for example in MMORPG ethnography or participatory observations might suit better for that than the observations described here. For ethnography to study MMORPG play, see Brown (2015).
Observations can be used in qualitative, correlational, quasi-experimental, and experimental to mixed methods research designs.
The goal of an observational design is to study game systems via observing gameplay. The basic set-up is let players play a game and observing them while they play. Recording the game session to in some way is advisable. For video recording, researchers should record the game screen(s) and the player(s) at the same time (see fig below). In addition, it is possible to use event information from game logs, server requests the game makes, or instrument the game to send or log the gameplay events of interest. Using logs or instrumenting the game requires is that the game is developed for research or the game can be modded to log events.
Figure. An example of a laboratory setup for data recording data
A variation of observations is observations with the think-aloud protocol. Note that the think-aloud protocol is hard for players when the game is demanding. In the think-aloud protocol, players are asked to explain why they are doing what they are currently doing. (An alternative to the think-aloud protocol is using the stimulated recall interviews. About stimulated recall, see Pitkänen, 2015, where after the game session, the players are showed specific game events and asked about what they were thinking in those situations and why they performed in that way, etc.).
Observation data can be analyzed qualitative, quantitative, or mixed methods approach. However, in all cases, the data needs to be coded. (Note that there are ways to automate the coding of game events in some extend [cf., Marczak and Schott, 2015]) .
Below, coding analysis approach is described in more detail. Discriptions about other possible qualitative analysis methods are on section Qualitative data analysis.
There are two approaches for data coding: pre-set codes and emergent codes. In both coding approaches, researchers connect an event in the data with a code and time stamp so that the event is easy to find later. It is also important that one code represents a type of event and all that kinds of event are content with that code.
The codes names itself should be brief and descriptive. It is a good practice to have a more verbose description of what code means as well as some examples of that code.
The coding schemata are theory-based. As an example of a theory based coding schemata, Ravaja, et al. (2005) set-up codes for observing events in Super Monkey Ball () based on psychological theory: any events that are likely to provoke emotional responses. They ended up with following pre-set codes:
“Start” Playing starts (the end of loading) “End” Playing ends (starts loading) “Go!” Level begins, “go!” is displayed (+sound) “Small fall” A small fall onto a platform below “Seeing stars” Hitting the platform so that the monkey sees stars “Sparkles” Going so fast that sparks fly “Home stretch” Finish line comes into view without obstacles, home stretch begins “Goal post bump” Bumping into a goal post (instead of finishing the race) “Finish” Finishing the race “Time bonus” Getting a time bonus (after finishing the race?) “No time bonus” Not getting a time bonus (after finishing the race?) “Replay” Replay starts “Big fall” Falling off the platform “Switch” Pressing a switch on the ground “Banana” Picking up a banana “Banana bunch” Picking up at least five bananas or one bunch of bananas “Map zoom” Zooming map “Bump” Bumping into a wall, bump, and so on (Ravaja, et al., 2005)
This is a data-driven approach where codes emerge from inspecting data. These emergent codes describe the events in the data. The main rule is that researchers should not make the data fit the codes, but create codes that describe the data. Using emerging codes is a recursive process: after new codes are created, the data needs to be analyzed again using the new set of codes.
Table. An example of coded data from Watchdogs 2 (Ubisoft Montreal, 2016).
Player 1 (man, 29 years old) Coded event Timestamp Notes ... Lift Hack 10:10 Raising scissorlift Selfie 10:29 Lift hack 10:30 Moving lift Selfie 10:42 Selfie now with a graffiti on backfround Jump 10:51 Jumping down from the lift Damage 10:52 Landing from too high; receiving damage Run 10:52 Hit 10:56 Hit by a bus, died Game cont. 11:21 Run 11:21 Running on street Jump 11:23 Jump over a car Run 11:26 Attack 11:28 Striking a bystander Run 11:28 Jump 11:34 Jumping over a car Run 11:35 Climb 11:44 Climping a wall Run 11:45 Pet 12:00 Petting a dog Walk 12:01 Pet 12:06 Hack/Stealing 13:11 Run 13:11 Jump 13:24 Jumping over a fence ...
Of course, coding everything is not usually feasible as there are just too many events happening. The research question gives focus for coding and only what is relevant for the research question should be coded.
This coding approach uses pre-set codes and emergent codes. For example, the player’s reactions to game events could be coded using Paul Ekman’s Facial Action Coding System (FACS) and game events relating to those emotional reactions can be coded using emergent codes.
Second phase coding
In the second phase, initial codes are grouped together by creating categories from the interconnected codes.
In coding, researchers would be asking, for example, if there are context, conditions, consequences or actions that connect the coded events.
From the categories and connections created in the axial coding process, researchers build a generalized description what happens based on the data. That description can even be a model and hypothesis.
Back to Watchdog 2. As Watchdog 2 is a sandbox game that allows multiple different play styles and behaviors, research question about how players are using the game are actually playing the game is an interesting one. By coding multiple players actions we could end up something as follows. [NEED TO WORK THIS…]
- Travel (players prefer different ways of travel)
- Random hostility
- Combat in Quests
- Quest play (players prefer different ways of travel)
- Stealth / Marcus
- Stealth / Remote control
- Non-quest behaviour
- Vehicle stunts
Qualitative analysis example (with a real data)
Example coming here…
Basics of quantitative approaches are described in section Quantitative approaches to study games. In this section, we give an example about quantitative analysis of observational data.
Mixed methods design
Aims of the study
- description of what parts will be used and how; difficulty level, …
What data is gathered and how
- Video record players and gameplay. If players will be recorded, how many cameras is used…
- game logs, what is logged
- should something blinded; if so, how
- how the data is stored and how long it is stored, data protection
- destroying the data vs anonymising the data (video or voice recordings are hard to anonymise)
- Quantitative: correlation or experimental
Video and voice recordings that show participants and where you hear participants voices pose potential ethical and legal issues. The recordings can reveal who participants are and the recordings are hard to anonymise. The data can reveal personal and sensitive details about participants so it is important that the data is handled with care.
- Participants are aware and accept how the recordings are used and how long the
- Recordings should be anonymised or destroyed after the data has been analyzed and reported
- Braun and Clarke, 2006
- Loland, J., Snow, A., Anderson, L. and Lofland, L.H., 2006. Analyzing social settings: A guide to qualitative observations and analysis, 4th ed. Bemont: Wadsworth Cengage Learning.
- Ravaja, Saari, Laarni, Järvinen, 2005. The Psychophysiology of Video Gaming: Phasic Emotional Responses to Game Events. In: DiGRA 2005 conference proceedings.
- Saldaña, J., 2016. The coding manual for qualitative researchers. Los Angeles: SAGE.