Game-based learning (GBL) has transformed how learners engage with content, offering interactive and immersive experiences that enhance traditional education. However, to ensure GBL is effective in achieving learning goals, game-based learning evaluation is essential. This article explores how to evaluate the effectiveness of game-based learning using two key frameworks: Connolly, Stansfield, and Hainey’s seven aspects of evaluation (2008) and J.W. Rice’s rubric for assessing higher-order thinking in video games. Together, these approaches provide a comprehensive method for evaluating both educational outcomes and cognitive engagement.
Connolly, Stansfield, and Hainey’s Seven Aspects of Game-Based Learning Evaluation
In their 2008 framework, Connolly, Stansfield, and Hainey identified seven key aspects of game-based learning evaluation that should be examined:
1. Learner Performance
One of the primary goals of game-based learning is to improve learner performance. This can be measured by assessing whether learners demonstrate improved understanding, knowledge, or skills after engaging with the game. Pre- and post-game assessments, quizzes, and task performance metrics are common ways to track progress and evaluate learning outcomes.
2. Motivation
Motivation is crucial in game-based learning, as it drives engagement and persistence. Evaluating how motivated learners are during gameplay is essential for understanding how well the game captures and sustains interest. Motivated learners are more likely to spend time in the game, leading to better retention and mastery of concepts.
3. Perceptions
Understanding learner perceptions of the game provides insight into how they view its relevance, usability, and enjoyment. Perceptions can be gathered through surveys, interviews, or focus groups, and they help determine whether learners find the game engaging and beneficial to their learning process.
4. Attitudes
In addition to perceptions, evaluating changes in attitudes toward the subject matter is critical. Game-based learning evaluation should assess whether learners’ attitudes toward a topic have improved after playing the game. For instance, does a history game make students more interested in learning about history? Tracking attitude shifts can reveal the game’s long-term impact on learners.
5. Collaboration
Games often encourage collaboration, where players work together to solve problems or complete tasks. Evaluating how well learners communicate and collaborate during gameplay is essential for understanding the social aspects of the learning experience. Collaborative gameplay can enhance critical thinking, teamwork, and social learning.
6. Preferences
Understanding learner preferences for game-based learning is key to refining and improving future games. Do learners prefer competitive or cooperative elements? Are they more engaged with narrative-driven games or puzzle-based challenges? Evaluating these preferences allows educators to design more effective game-based learning experiences.
7. Environment
The learning environment—both physical and digital—plays a crucial role in the success of game-based learning. Evaluating the environment involves assessing whether it supports focused gameplay. Are there distractions? Is the technology accessible and functional? Ensuring a conducive learning environment is critical for the effectiveness of game-based learning.
J.W. Rice’s Rubric for Evaluating Higher-Order Thinking in Games
In addition to Connolly, Stansfield, and Hainey’s framework, J.W. Rice’s rubric (2007) offers a method for evaluating game-based learning by focusing on how games promote higher-order thinking. Rice’s criteria examine the cognitive complexity and engagement that games provide, helping to ensure that they challenge learners beyond rote memorization.
1. Role-Playing and Perspective Shifting
Games that require players to assume a role and process information from different perspectives encourage higher-order thinking. This promotes critical thinking as learners analyze scenarios from new viewpoints, an essential element of effective game-based learning evaluation.
2. Interaction and Dialogue with NPCs
Rich, meaningful interactions with NPCs (non-playable characters) deepen a learner’s engagement with the game. Dialogue that is dynamic and responsive enhances critical thinking and encourages learners to explore the game world, making this a vital aspect of evaluating a game’s effectiveness.
3. Complex Storylines and Emotional Investment
Games with complex storylines and characters that players care about drive emotional engagement. When learners are emotionally connected to the narrative, they are more likely to retain information and apply critical thinking to in-game decisions.
4. Problem Solving with Complex Puzzles
Educational games should feature complex puzzles that challenge learners to apply knowledge, think critically, and test various strategies. These puzzles encourage higher-order thinking and are an important part of the game-based learning evaluation process.
5. 3D Graphics and Multiple Views
Games that offer 3D graphics and multiple viewpoints provide a more immersive experience. These visual elements support spatial reasoning and allow players to engage more deeply with the content, making them a key feature in evaluating the effectiveness of game-based learning.
6. Simulating Complex Processes
Simulating complex processes that learners can manipulate fosters hands-on learning. These simulations help players test hypotheses, learn from mistakes, and understand real-world applications, all of which are crucial for higher-order learning.
7. Interactive Objects and Avatars
Games that allow for interactive objects and lifelike avatars create a more engaging and immersive environment. Players’ ability to manipulate objects and interact with their avatars enhances their connection to the learning material, making this an important factor in game-based learning evaluation.
8. Real-World Replication and Replayability
Games that replicate real-world scenarios and allow for replayability with varying outcomes encourage learners to explore different strategies. This variability helps learners develop critical thinking skills and enhances the replay value of educational games, which is an important aspect of their long-term impact.
Combining Connolly, Stansfield, and Hainey with Rice’s Rubric for Comprehensive Evaluation
To effectively measure the success of game-based learning, educators should combine Connolly, Stansfield, and Hainey’s seven aspects with Rice’s rubric for higher-order thinking. Together, these frameworks provide a holistic approach to game-based learning evaluation, ensuring that both learner outcomes and cognitive complexity are addressed. By evaluating factors such as learner performance, motivation, and collaboration (Connolly et al.) alongside role-playing, problem-solving, and real-world replication (Rice), educators can gain a clear understanding of how well a game fosters meaningful learning.
How Odeum Studio Aligns with Game-Based Learning Evaluation Criteria
At Odeum Studio, our educational games are designed with these evaluation frameworks in mind. Our platform incorporates immersive storytelling, interactive NPC dialogue, and complex problem-solving, ensuring that learners are deeply engaged and challenged cognitively. By aligning with both Connolly, Stansfield, and Hainey’s and Rice’s criteria, Odeum Studio delivers games that are not only enjoyable but also highly effective in promoting deep learning.
Final Thoughts
Evaluating the effectiveness of game-based learning requires a structured approach that considers both educational outcomes and cognitive engagement. Using frameworks like Connolly, Stansfield, and Hainey’s seven aspects and J.W. Rice’s rubric for higher-order thinking, educators can conduct a comprehensive game-based learning evaluation to ensure that their games are achieving learning objectives. With platforms like Odeum Studio, educators have the tools to create, implement, and evaluate high-quality educational games that align with these proven principles.
