Ancient Three Kingdoms main menu

Home About Services Testimonial Contact

Welcome to the Ancient Three Kingdoms (ATK) research study!

This is the official website of the Carnegie Mellon University research study "Exploring Collaborative Narration in Shared Digital Mutimedia Spaces." The research study space (i.e. "the game") is titled Ancient Three Kingdoms, or ATK.

This research study has been approved by Carnegie Mellon University's Institutional Review Board (IRB), has been issued a Federalwide Assurance Number by the United States Dept. of Health and Human Services to conduct research, and is also deployed in several classrooms as a teaching tool.

ABOUT THE GAME

ATK is a 2D adventure role-playing game that takes place in the ancient "Three Kingdoms" period in Korea's history. Players will inhabit various roles as they adventure through the Three Kingdoms and beyond! What makes this game so special is the community that creates and drives storylines with each other, the events players host for each other, and the multitude of collaborations that define and evolve the game. Its nostalgic graphics and gameplay require very low system resources.

Click on "PARTICIPATE!" in the menu to DOWNLOAD THE GAME

About the study

STUDY NAME
Exploring Collaborative Narration in Shared Digital Multimedia Spaces

STUDY BACKGROUND
In a 2009 grant-funded study, the PI for this study illustrated how the experiential learning of "wreading" described by Emily Carr aligned with many online collaborative writing activities found in MMORPG games, and that much of what George Landow described in his book "Hypertext 2.0" is relevant to studying this particular space as not a game, but rather a network of multimodal hypertext where users build their own creative resource network through highly-specialized discourse communities. This led to the term "Digital Narrative Network" (DNN), which holistically describes those shared digital spaces that rely on user engagement and "wreading" to experience a narrative, contribute to it, and advance it for other users to then participate in. Despite the breadth of online games to flood the modern market, there are very few contemporary examples of DNNs. This study aims to build one that can be used as both a research tool and as an in-class teaching instrument.

The objects of this study are the participants' written communications in a highly-customized and substantially modified version of an online game loosely based on Kru Interactive Inc (a licensee of Nexon Inc)'s "NexusTK: The Kingdom of the Winds." The various creative and strategic writing that participants produce as part of advancing their character and evolving the game world will be analyzed using contemporary Machine Learning and Natural Language Processing techniques in Python, as well as ran through several communication assessment rubrics to unpack rhetorical and linguistic choices. The game itself will also be used in formal course instruction and activities in various classrooms at Carnegie Mellon University.

There is a strong history in modifying portions of online games for education and research, the most notable of which is a 2009 book, "Digital Simulations for Improving Education." It includes a chapter from C.S. Loh and Jae Hwan Byun that used a modified version of Bioware's 2002 game "Neverwinter Nights" to study player movement and decision-making. Loh's previous 2006 study on player movement tracking in another modified version of Neverwinter Nights was presented at the Association for Educational Communications and Technology and sebsequently received federal funding. Loh established research on "information trails" in another study to assess player learning and presented that research at the 10th International Conference for Computer Games. Dr. Loh served on the dissertation advising committee for this study's PI.

ABOUT THE PI
The PI for this research study has enjoyed over a decade of research in digital learning environments. He has previously taught at The Massachusetts Institute of Technology (MIT) and Case Western Reserve University, and is presently serving on the faculty at Carnegie Mellon University where he recently completed a research appointment for the Office of the Vice Provost. His work in computational text analysis was used in a university-wide teaching initiative as well as in rubric revision efforts across the curriculum. He presented research on machine learning and natural language processing techniques at the 2019 Eberly Teaching and Learning Summit. His forthcoming book chapter in "Writing STEAM" (Spring 2021 by Utah St. University Press) describes this study's research methodologies and how they can be used in data-driven writing classrooms. His previous work in developing and deploying mobile learning solutions saw seventeen writing centers across the country use his mobile app. He is collaborating with faculty from CMU's Language Technologies Institute, School of Computer Science, and Entertainment Technology Center to further explore this multimedia userspace as a research and teaching tool.

RESEARCH PURPOSE AND QUESTIONS
The central purpose of this research is to encounter and better understand how small groups of digital users build collaborative written projects in a shared, multimedia virtual space and then act upon those projects using various linguistic and rhetorical strategies. This line of inquiry primarily resides in the fields of New Media Studies, Digital Literacies, and Computational Linguistics.

The following research questions will be addressed through this study:

(1) In what ways do writers collaborate in shared digital spaces when they are segmented by narratively distinct discourse communities?

(2) What situations do writers select and deflect in contributing to narrative collaboration and what linguistic and rhetorical strategies do they employ in addressing those situations?

(3) In what ways can interactive multimedia environments and DNNs be leveraged in writing classrooms to encourage participation and foster critical communication skills?

STUDY DESIGN
A highly modified game client (i.e. the "research study space") will be made accessible for study participants to connect to after providing consent for participation. The narrative structures for user engagement will be developed by the PI. Study participants will then inhabit these narrative structures and produce written content in centralized locations. Aside from creative writing, participants may choose to collaborate in "classroom" spaces that are designated by their CMU course numbers and titles. For example, a student from the "Writing about Data" class may go to the "76107 - Writing about Data" classroom space and post information on its bulletin board, or chat with other students from that course. All of this content will be recorded and then analyzed in service towards addressing the three research questions above.

Participants will inhabit the space in any way they choose, for any amount of time they choose, and would be creating "posts" of creative content on various, public bulletin boards inside the space. These posts would be collected and coded according to a rubric of communication elements. The text from the posts will also be sent through a text analysis program the PI built in Python, which was used in Carnegie Mellon's Core Competency Initiative. This program assessed the communication features of nearly 600 syllabi across ten departments using contemporary natural language processing and machine learning techniques. The text will also be analyzed using "LightSide," a machine learning and feature-extraction tool developed by CMU's Language Technologies Institute with grants from the National Science Foundation and the Office of Naval Research. This mixed-methods approach of collecting rubric-centered and computational linguistics data allows the research to focus on the "what is" of participant generated interactive writing and understand it through different assessment channels. Substantial modifications of the game's codebase have been made to accomodate these analytical methods.

Any "community events" that are generated as a result of these shared narrative posts would be similarly analyzed through a similar rubric and subject to computational text analysis.

RESEARCH PARTICIPANTS
This study aims to reach a large user base (300+ participants) by recruiting participation primarily inside the CMU community, but will also be made available to members outside the CMU community as per this study's IRB approval.

Participation will be entirely anonymous, as users will only be identified by their study usernames. Usernames will not contain real names or any identifying information that could be tied to individual identities. The PI will not ask for any real-life identifying information during any part of the study, including but not limited to name, gender, race, or educational level. E-mail addresses are never collected. Should any user disclose any real life information, their written contributions will be removed from the data record. No personal information is ever gathered from any participants at any point of the study.

Participants who sign up for study participation will choose their own username, which will be kept in a secured database for future reference. Any posted communication created by a participant may also be voluntarily deleted at any time by that participant.

Participants will have the right to delete their posted communications at any time, as well as remove their access from the study space at any time. Participants must agree to these terms and those outline on the Study Consent page of this website before being granted access to the study space. There is, and will never be, any renumeration or compensation of any type for participation in this study.

EDUCATIONAL/CLASSROOM USE BEYOND THE STUDY
Portions of this research project as well as the study space itself (i.e. "the game") will be used in various courses at Carnegie Mellon University for formal instruction and classroom activities, including 76-101, 76-102, 76-107, 76-108, and 76-270. The PI is actively collaborating with faculty from CMU's Language Technologies Institute (LTI) and Entertainment Technology Center (ETC) to develop a co-taught course using this multimedia userspace as both research object and teaching tool.

Logistics & Frequently Asked Questions

Is this a real research study?

Yes! This research study has been approved by Carnegie Mellon University's Institutional Review Board (IRB) as per 45 CFR 46.110 and 21 CFR 56.110. It has also been issued a Federalwide Assurance Number (FWA) from the United States Department of Health and Human Services. Please see the official documentation below. The "game" itself and its attendant objects of study will also be used for teaching purposes in classrooms across CMU.



What exactly are you researching?
Broadly, this study explores various linguistic and rhetorical choices as they are made inside a narrative-driven community, and how those choices impact the narratives that drive community development and facilitate the building and performance of its participants' digital identities. The study uses a combination of contemporary Machine Learning and Natural Language Processing (NLP) techniques to extract language data, as well as communication rubrics to explicate and give shape to these choices. Secondary to this, it tests this multimedia eduspace as a pedagogical intervention tool and has game features built specifically for in-class writing activities and student collaboration.

How will you use this to teach?
This study space (i.e. "the game") will be displayed in the classroom as part of a module on shared digital spaces and collaboration in digital narrative networks, following Szaforn et al's 2005 study on how interactive story-building might work inside a classroom. It may also be used to demonstrate pedagogical intervention practices that necessitate the use of digital communities of peer learners and will leverage custom classroom spaces and functions built into the game by the PI. Player interactions will be critically discussed as part of a "Digital and Multimodal Writing" course instance of 76-101: Interpretation and Argument or its advanced counterpart 76-102, subject to course availability. It will further be deployed in some capacity in the following courses at Carnegie Mellon University: 76-101, 76-102, 76-107, 76-108, 76-270. Those five courses already have classroom spaces inside the game and more will continue to develop over time as need arises. Additionally, I am presenting this project at CMU's Language Technologies Institute (LTI) in the Spring of 2020 and am currently collaborating with faculty from both CMU's Language Technologies Institute and the Entertainment Technology Center to develop a multi-disciplinary course using this space as an object of instruction.

What have you changed from the original game?
This research project has substantially transformed both the front end elements (what the participants experience in the multimedia space) and more substantially the back-end (game source and database code) of the "game." There are numerous major changes to the front and back end source codes made specifically for research and teaching purposes, including but not limited to database features that allow for methods of computational linguistics analytics, features that allow participants to more easily produce creative content in the game, features that promote virtual classroom pedagogies, in-game spaces created to facilitate distance learning/e-learning and peer collaboration, etc. In addition to changing various in-game locations and names to match the historical Korean spellings, this project has written thousands lines of dialogue and text options within the game's literary corpus and this number grows each week as new literary and creative content is being added. The project leverages open-source audiovisual assets from OpenGameArt.org where possible. There are several player quality-of-life adjustments, as well as dozens of new areas, mechanics, items, quests, spells, NPCs, minigames, and events not present in the original.

Is this a legal use of copyrighted material?
Yes! Codified in the Copyright Act of 1976 at 17 U.S.C. § 107, the preamble to the Fair Use Doctrine lists favored purposes: criticism, comment, teaching, scholarship, and research, and this educational project operates in each of these. For this reason, the United States Supreme Court has recognized that the fair use doctrine "affords considerable latitude for scholarship and comment." Learn more about fair use for research and education here. This project is also in full compliance with The Technology, Education, and Copyright Harmonization (TEACH) Act of 2002.

This research and educational project necessitates the use of a limited amount of copyrighted material under the following Fair Use for Education provisions, including but not limited to portions of audiovisual appearance, source code, literary works, and other graphical components. This non-commercial research study fully addresses each of the favored purposes in compliance with the Copyright Act of 1976, 17 U.S.C § 1, et seq.:

Criticism
Pursuant to 17 U.S.C. § 110(1) - performance and display in classrooms, this study and the limited amount of copyrighted material and its attendant modifications and player interactions will be critically discussed as part of a "Digital and Multimodal Writing" course instance of 76-101: Interpretation and Argument or its advanced counterpart 76-102, subject to course availability. It will further be deployed in some capacity in the following courses at Carnegie Mellon University: 76-101, 76-102, 76-107, 76-108, and 76-270.

Teaching
Similar to above, this game and study space will be displayed in the classroom as part of a module on shared digital spaces and collaboration in digital narrative networks, following Szaforn et al's 2005 study on how interactive story-building might work inside a classroom. It may also be used to demonstrate pedagogical intervention practices that necessitate the use of digital communities of peer learners in the courses listed above, such as the custom classroom spaces and functions built into the game by the PI. There is an active collaboration between the PI and faculty from Carnegie Mellon's Language Technologies Institute (LTI) and Entertainment Technology Center (ETC) in developing a co-taught course using this study space as both a research object and instructional tool.

Scholarship
The PI intends to collect user writing data on a two month interval, since this is an iterative experience and it relies on community and community building. This longitudinal study accounts for the number of participants changing over time as a core experience of building digital narrative networks (DNNs) over time. The goal is to eventually gather enough data to have an adequate sample size to release meaningful results in publications and at conferences. The PI is publishing a forthcoming book chapter in "Writing STEAM" (Utah St. U Press, Spring '21) on the research methodologies of this project and their attendant uses in teaching data-driven writing courses.

Research
The purpose of releasing a version of this interactive multimedia space is for research and educational purposes at outlined on this page and the "About Study" section of this website.

Comment
Along with teaching and criticism, mindful and critical comment will be a necessary component of an academic conversation surrounding digital narrative networks, online identities and new media literacies, and these tools' capabilities as both interactive classroom tools and pedagogical intervention strategies.

Persuant to the Section 107 policies, Carnegie Mellon University has identified several "Fair Use Factors" in determining fair use. This project has been confirmed to be in full compliance with each factor.

Transformative Use
Addressed under sections 1.A.i and 1.A.ii., "Transformative Use" is defined as where the new use does not supersede the originally intended use. This study addresses this provision by not only substantially transforming the purpose and nature of this commercial work, but also by: placing strict limitations on both original graphical, audiovisual, literary, and other creative assets by only using a portion of the whole library, using open-source audiovisual assets where available, not using the latest content developed by Nexon Inc and/or Kru Interactive Inc (i.e. the new maps, events, and areas released recently), reconfiguring portions of existing assets, adding new assets, and not using any commercialization components of Nexon Inc and/or Kru Interactive Inc's work such as paid accounts and a microtranscation marketplace. Any new content for Ancient Three Kingdoms will be entirely generated by the PI, the study participants, and not Nexon Inc. or Kru Interactive. This non-commercial research study and teaching tool is not affiliated with Kru Interactive Inc or Nexon Inc in any sense.

In accordance with 1.B.i and 1.B.ii, the nature of the particular copywritten work used in this research study and teaching tool is indeed published, but will be transformative for the purpose of this project. Although it is worth noting that legally "the repurposing or re-contextualization of a work can be transformative, even if the work itself is not modified," this study does indeed transform both the front-end (what the participants experience in the multimedia space) and more substantially the back-end (game source and database code) of Nexon Inc and/or Kru Interactive Inc's copyrighted work of NexusTK. There are substantial changes to the front and back end source codes made specifically for research and teaching purposes, including but not limited to database features that allow for methods of computational linguistics analytics, features that allow participants to more easily produce creative content in the game, features that promote virtual classroom pedagogies, in-game spaces created to facilitate distance learning/e-learning and peer collaboration, etc.

This project has written thousands of lines of dialogue and text options within the game's creative and literary corpora and has developed new literary works within the corpora. It has also leveraged open-source audiovisual assets from OpenGameArt.org to replace much of the sound and music. There will still be some small amounts of copyrighted work that will not be modified for the purpose of this research study and its attendant educational uses in classrooms, including but not limited to portions of audiovisual appearance, portions of source code, literary works, and other graphical components; however, the use of these assets has been substantially transformed for the purposes of research and teaching. The transformative use of portions of reused copyrighted assets has been found to be in full compliance with this provision.

Amount and substantiality of portion used
This project only reuses a portion of the whole copyrighted graphical, literary, artistic, and audiovisual assets of NexusTK. For example, there are 67,546 total graphical "tiles" and objects available in the client to create the in-game spaces that users inhabit. Likewise, there are 65,440 map locations in the database and over 50,000 combined items, character appearance features, and audio assets available in the client. This project only uses a small portion of each of those numbers, respectively. Considering Nexon Inc and/or Kru Interactive Inc has continued over the years to substantially add to the depth and number of copyrighted works contained within NexusTK, and this project leverages many open-source sounds and music from OpenGameArt.org, it further demonstrates the proportionally limited use of copyrighted assets by this non-commercial research study and teaching tool. The substantiality of the portion used, vis a vis the number of total copyrighted assets not being used in this study, has been found to be in full compliance with this provision.

Effect of use upon the potential market or value of copyrighted work
Since this IRB-approved study is a non-commercial research and teaching project, the secondary use of limited amounts of copyrighted material as a research and teaching tool is extremely unlikely to usurp the market for the original work. Any market loss on the original copyrighted work cannot be rightfully attributed to this transformative, non-commercial research and educational project and is purely coincidental given the contemporary online gaming market.

Compliance with The Technology, Education, and Copyright Harmonization (TEACH) Act of 2002:
The Technology, Education, and Copyright Harmonization (TEACH) Act of 2002 facilitates and enables the performance and display of copyrighted materials for education by accredited, non-profit educational institutions.

There are seven provisions that must be adhered to for compliance. The following section will review how this project meets all seven provisions:

1. The institution must be an accredited, non-profit educational institution.
Carnegie Mellon University is a non-profit, major research university with campuses across the world.

2. The use must be part of mediated instructional activities.
There are many classroom spaces built into this project that will be used for both asynchronous and synchronous instructional activities and educational experiences such as peer review and classroom collaboration. Additionally, the project itself will be central in course discussions of digital identity, online collaboration, and digital media literacies.

3. The use must be limited to a specific number of students enrolled in a specific class.
The course caps for each respective, individual classroom represented in the game space will serve as provision 3’s specific number. The “specific class” will refer to specific courses, such as 76-101, 76-107, 76-108. As there are multiple sections of each course, provision 3’s limited number will be set by multiplying the course cap for the respective course by the number of course sections taught by the PI each semester. This specific number is far less than the total number of enrolled students in any class.

4. The use must either be for ‘live’ or asynchronous class sessions.
The distance education technologies afforded by “message boards” in the digital classrooms within the study space will serve as the asynchronous educational touchpoints. The “chat” feature present in the study space will be used in live class sessions.

5. The use must not include the transmission of textbook materials, materials “typically purchased or acquired by students,” or works developed specifically for online uses.
There is no use of textbook materials anywhere in this study space. There are no materials purchased or acquired by students in order to explicitly participate in the study space.

6. The institution must have developed and publicized its copyright policies, specifically informing students that course content may be covered by copyright, and include a notice of copyright on the online materials.
Carnegie Mellon University has made its copyright policies explicit on the following website: https://www.cmu.edu/policies/administrative-and-governance/intellectual-property.html There is a message posted on each digital classroom’s message board that specifically informs students that course content may be covered by copyright.

7. The institution must implement some technologies and measures to ensure compliance with these policies, beyond merely assigning a password.
This project ensures compliance through various technological means such as but not limited to: print-disabling (a system’s default print commands such as CRTL-P or CMD-P does not work in this project); decoupling CMU’s Course Management System (Canvas) from this project’s multimedia user space, ensuring that such content cannot be directly shared; limiting the types of user communication modes and not allowing users to directly post images, audio, and video anywhere in project’s multimedia user space; building and maintaining dedicated “virtual classroom” spaces for synchronous and asynchronous instructional activities and student collaboration

Study Consent Form

As per 45 CFR.46.110 and 21 CFR 56.110, Carnegie Mellon University's Institutional Review Board (IRB) has granted a waiver of written documentation of informed consent for the study. The information and terms below is provided for participants to consider before they join the study. A participant's digital consent is therefore granted and signified by accessing the study space (i.e. "the game.") by clicking PARTICIPATE in the main menu of this site and following the directions to download and login to the study space (i.e. "the game").

Consent to participate in the research study: "Exploring Collaborative Narration in Interactive Multimedia Spaces"

Summary

The main principle of this research is to explore and unpack writing and collaboration habits in an online game that is built upon creating shared stories. Participants of this study will have the opportunity to inhabit the role of a character of their choosing in the game and produce small, creative writing "posts" (public, written discourse) towards advancing these community-shared stories and narrative features. They will also have the opportunity to collaborate with other participants in developing stories, events, and narrative writing for the entire participant community to enjoy and engage with.

Purpose

The central purpose of this research is to encounter and better understand how small groups of digital users build collaborative written projects in a game environment and then act upon those projects to develop shared stories as narrative experiences with other users.

Procedures

An online game will be made accessible for study participants to connect to, create a character, and inhabit the game world with other participants. Central storylines will be developed by the PI, which includes community storylines, group-focused written projects, and individual character stories. Study participants will then interact with these stories by iterating their own creative written work onto them, producing written content for all participants to access, and thereby advancing the various storylines and character stories.

Participants would inhabit the space in any way they choose, for any amount of time they choose, and would be creating short, written, creative content (such as individual character stories or stories about groups of characters) on various, public bulletin boards inside the game. This written content will be collected by the PI every two months. We expect to have the game open to collect data for a period of three years (at which time the PI will "check in" to the IRB for continuance).

The minimum amount of time participants could spend on this study is approximately five minutes, with the maximum determined by personal choice for how long a participant chooses to continue playing the game.


Participant Requirements

Participation in this study is limited to individuals age 18 and older. Participation will be entirely anonymous, as users will only be identified by their study usernames. Usernames will not contain real names or any identifying information that could be tied to individual identities. The PI will not ask for any real-life identifying information during any part of the study, including but not limited to name, gender, race, or educational level. Should any user disclose any real life information, they will be immediately disqualified from participation in the study and their written contributions will be removed from the data record. No personal information is ever gathered from any participants.

Participants who sign up for study participation will choose their own username, which will be kept in a secured database for future reference.

Participants will have the right to delete their posted communications at any time, as well as remove their access from the study space at any time. Participants will have to agree to these terms before being granted access to the study space. There is, and will never be, any renumeration or compensation of any type for participation in this study.


Risks

The risks and discomfort associated with participation in this study are no greater than those ordinarily encountered in daily life or during other online activities. A participant might present negative or disparaging content when interacting with other participants in the game space. The amount of computer screen time that participants are exposed to is not greater than any other online gaming experience.


Benefits

There may be no personal benefit from participation in the study but the knowledge received may be of value to humanity. Participants might enjoy participating in a shared narrative experience in the study space and develop writing and peer communication skills.


Compensation & Costs

There is no compensation for participation in this study.


There will be no cost to participate in this study.


Future Use of Information

In the future, once we have removed all identifiable information from your data (i.e. usernames from posted, creative writing samples), we may use the data for our future research studies, or we may distribute the data to other researchers for their research studies. We would do this without getting additional informed consent from you (or your legally authorized representative). Sharing of data with other researchers will only be done in such a manner that you will not be identified.


Confidentiality

The data captured for the research does not include any personally identifiable information about you. Your IP address will not be recorded into a permanent database and is only received by connecting to the server on which the interactive multimedia space is hosted, in order to prevent fraud. By participating in this research, you understand and agree that Carnegie Mellon may be required to disclose any consent form(s) if applicable (not applicable), data and other information as required by law, regulation, subpoena or court order. Otherwise, your confidentiality will be maintained in the following manner: Your data and consent form will be kept separate if applicable (not applicable). Your consent form will be stored in a secure location on Carnegie Mellon property (the department-issued computer of the PI) and will not be disclosed to any third parties (this is not applicable due to the granted waiver of written document of informed consent). By participating, you understand and agree that the data and information gathered during this study may be used by Carnegie Mellon and published and/or disclosed by Carnegie Mellon to others outside of Carnegie Mellon. Note that per regulation all research data must be kept for a minimum of 3 years.


Voluntary Participation

Your participation in this research is voluntary. You may discontinue participation at any time during the research activity. You may print a copy of this consent form for your records.


I am age 18 or older. Yes No

I have read and understand the information above. Yes No

I want to participate in this research, wish to have access granted to the game. Yes No

Your agreement to the above questions in the affirmative is indicated by your following of the instructions on the "PARTICIPATE" page to access the research study space (i.e. "the game")

If you have any questions, you may contact the PI at this email address.

Participate in the Study/Access the game

If you agree to the terms and participant information on the Study Consent Form page on this website, you may DOWNLOAD THE GAME FILES HERE.

You will also need to install Microsoft's .NET v3.5 framework and Visual C++ runtimes to run the game.

You may also choose to join the project's official Discord server but it is not required for your participation.

There is no account manager because there are no accounts. Personal information, including email addresses, is never asked for by the PI nor should it ever be given in the study space.