This literature review will support the goal of creating a more robust Narrative Tree Complexity Metric for the field of Interactive Digital Narrative Design.
To build a case, we will examine:
Interactive Digital Narrative(IDN) involves the creation and use of digital tools toward multi-authored narratives (Koenitz, 2015). An example would be a story based video game. IDN Theory differs from Narrative Theory because of the branching nature of an IDN tool. The IDN designer creates potential branching narratives and procedural mechanics so the player participates in the story creation. In a book, you read from the first page to the last, the author of the book has laid out one path through the narrative, but in a narrative video game, the player makes choices that co-create one or more versions of the narrative.
Although the field is now discussing a wide variety of interactive mechanics(Kreminski & Wardrip-Fruin, 2018), the most basic IDN pattern is a decision tree structure. Because the data for this project is with beginning IDN designers, the quantification of the tree structure and its relation to other features of the IDN design and process is of foundational importance(Daiute et al., 2021).
That said, measuring the basic IDN tree structure has remained elusive.
Complexity evades a formal definition, as each person has their own intuitive definition of what is complex and what is not. Even among scientists this holds true,
“complex systems … involve many (relatively) simple individual elements interacting locally with one another”(Polančič & Cegnar, 2017)
Narrative tree structures are made up of the most basic component parts of branches and nodes(Nelson, n.d.). The nodes are where the story elements are contained, such as characters actions and objects. The branches are links to subsequent or recursive story nodes. In order to be engaging IDNs must provide rich enough opportunities to be engaging for the player.
Complexity Depends on the Observer
“Only when observations are made, as produced by an acquisition model, is when the question of complexity becomes relevant: after the observer’s model is incorporated.”
Emergent levels of complexity
If the complexity of the most basic IDN can be quantified, then this could lead to the quantification of higher level IDNs in the future. Funes suggests “interactions at a lower level of organization (e.g. subatomic particles) result in higher levels with aggregate rules of their own”(Funes, 2001)
There is an interesting history of measuring Complexity by comparison. One particular relevant study took place in 1982 by Dr. MacEachren entitled “Map Complexity: Comparison and Measurement.” The initial procedure was to determine the subjective complexity of the test maps. This was accomplished by generating a psychological complexity scale to which physical complexity measures for each map could subsequently be compared.(MacEachren, 1982)
In the study “Map Complexity: Comparison and Measurement” MacEachren sought to quantify the consensus that chloropleth maps were more complex than isopleth maps. Isopleth maps define features by densities, while chloropleth maps define them by boundaries(Słomska-Przech K., 2021) The procedure was to obtain the subjective complexity of the maps by participants by having them visually compare and rate the complexity of the different types of maps. This data was used to create a “psychological complexity scale,” which was then compared to physical features of each type of map. In essence they had participants select the more complex map, which they then generated a scalar metric from, and then modeled this metric from the features of the maps.
At the heart of this project is the need for a ranked list of narrative tree structures to produce the scalar element of complexity as referred to in Machlean’s study. Thurstone’s Law of Comparative Judgement implies that there is evidence that comparing alternatives pairwise versus ranking all alternatives or selecting one from the set of alternatives is more effective (for humans)(Fürnkranz & Hüllermeier, 2011). Pairwise comparison has been leveraged for machine learning applications with the Learning by pairwise comparison (LPC) paradigm(Fürnkranz & Hüllermeier, 2011), which will be addressed in the Data and Modeling sections.
The method of Pairwise Comparison I am proposing is as follows:
This data can be aggregated in many different ways, such as a win/loss metric or not aggregated at all, with the raw data used to train specific kinds of models.
The term complexity is widely used in the IDN space. In fact there is a group called the The INDCOR project (Interactive Narrative Design for Complexity Representations). They are referring to the complexity of the “space” in which IDNs exist socially. They work at the abstract level of the ideal IDN, but do not look empirically(Perkis, n.d.)
‘A Complexity Analysis Matrix for Narrative Userly Texts’ by Noam Knoller, Christian Roth and Dennis Haak develops a method for measuring the complexity of the IDN(Knoller et al., 2021).
This research adapts the ‘Learning Progression Model’ (Yoon et al., 2019)
breaks complexity of a system down into 6 complex system ideas.
This project applies Yoon’s work to create a complexity vector space. They then manually code these vectors for the IDN, and a users opinion on the completed IDN. They then measure the euclidean distance between these vectors to determine how much the student understands the system.
This is a metric for the overall complexity of all the aspects of a completed IDN, that can transcend the dimensionality of the story.
What I am focusing on is not the overall complexity of the completed narrative, but strictly a 2 dimensional Tree Structure Complexity of the proto-story elements. This basic structural analysis (nodes and branches) and the patterns of node and branch sections, not the story text. If the narrative tree structure complexity is robust, it provides an independent measure of basic structure to correlate with other elements, such as analysis of the narrative text, and designer/player interaction process.
In addition, this basic metric could provide quantifiable measures of structural change during the IDN design process, which is especially relevant to beginning designers.
In one study with beginning IDN designers, researchers used a connection-density measure adapted from neural connectivity research(Daiute et al., 2018 ; Rubinov & Sporns, 2010).
They computed branches divided by nodes for a measure of connection density. The quantitative metric did not correlate well with their qualitiative Narrative analysis, which is why they are attempting to update the metric. In addition this measure was further updated(Daiute et al., 2021) to address logical flaws. The connection density metric could be manipulated by special cases of tree structures so the metric was incoherent. An example would be a single node with 4 branches would have a higher connection density than an infinitely long tree structure with 2 branches per node. A version of the connection density metric created to correct these issues was created(Daiute et al., 2021) based loosley on fractal dimensionality, with length and maximum pathway variables included, but additional kinds of logical incoherence could be created with special cases of tree structures.
The identification of these prior measurement issues motivated the rationale, measure and design for this project. When two tree structures are drawn, the tree struture complexity metric should mirror the expert’s intuition on which of the trees is more complex. Pairwise ranking of a sufficiently large dataset of narrative tree structures leads to the computation of a psychological complexity continuum. This continuous variable can then be modeled and the model can be used to compute the intuition of an expert on trees outside of the training dataset.
The sample of the data will be fairly small due to the restrictions of the number of research participants in the workshop. There are an expected 50-100 trees to be collected during the data collection phase, and their features will likely need to be hand coded.
METHOD:
The data will be collected from a set of IDN experts opinions on the tree-structure complexity through a tree-structure complexity game using segmented training data set from a research workshop. The results will be stored on an SQL server, recording each rating event. The event variables retained will be:
The reason this data will not be aggregated at the time is to potentially take advantage of the EloChoice algorithm that will be discussed in the modeling section.
The results of the expert survey will be aggregated in two ways:
eloChoice algorithmELO is a dominance rank ordination method. In brief, ELO looks at competitive matchup (think two chess players playing a match) and will assign points (positive to the winner, negative to the loser.) The amount of points assigned depends on the models pre-match assumptions of who is going to win. competitors with ELO in the same range, will have a relatively similar amount of positive and negative points assigned, while a match with a large difference in ELO will assign small values if the pre-match assumption is met, or a large amount of points assigned if it is upset(Albers & Vries, 2001).
The winrate metric does not take into account any pre-match assumptions, but will be included in the exploratory stage as a comparison with ELO.
Linear regression and ensemble methods will be performed. Most likely \(R^2\) and \(AIC_c\) will be used to compare models, because the sample sizes will be relatively small. \(AIC_c\) is a version of Akaike’s Information Criteria that has a correction for small sample sizes(Hurvich & Tsai, 1989).
Despite the relative performance of the models, because the tree-structure complexity metric is being created for use and interpretation by researchers in the field, a version of the linear model will be produced that will be maximized for intelligibility.