Modelling Interpretation in 3DH: New dimensions of visualization

Introduction
Visualization techniques developed in the sciences normally focus on the (re)presentation of empirical data. But how can we graphically express interpretations? This paper presents the intellectual framework

underpinning the 3DH project (Three-dimensional Visualizations for the Digital Humanities), a collaborative project conducted at the University of Hamburg from 2016 to 2019. The project foregrounds data interpretation and develops a visualization paradigm from the epistemological perspective of the humanities. The “third dimension” required in DH visualization techniques is therefore not merely that of an additional quantitative z-axis. Rather, it is an axis that can ‘unflatten' (Sousanis 2015) the objectivist notion of visualized data. In our presentation, we will do three things:

• Digital and visual turn: Review existing visualization paradigms that emphasize the representational approach. We start with the epistemological issues raised by the digital and visual turn.

• Visual modelling: Outline and discuss an interpretative modelling alternative through two case studies of existing tools, CATMA and Voyant, and Temporal Modelling, a platform for creating data through graphical means.

• “Hermeneuticizing” visualization: Discuss the design of a full visual framework. We will present possible conventions and prototypes that use them. These inform our case studies and the envisaged infrastructure.

Case studies in our presentation will be drawn from CATMA (a collaborative mark-up & text analysis environment), Voyant (a text analysis platform), and humanities research projects using base images (historical maps) and original models (for non-standard chronologies).

The digital and the visual turn: a hermeneutic ceterum censeo

For centuries, academic discourse in humanities disciplines has relied predominantly on text. In DH, however, visualizations increasingly claim the status of arguments and proofs that play a decisive role in the development and presentation of ideas, findings, and conclusions.

The visual and the digital turn have thus gone hand in hand - but the way in which this synergy manifests itself remains constrained in a symptomatic way. We can print a chart or render it on screen just as we can print or display a text in various media, but we normally cannot subject the chart to in-depth critique in the way we can question and respond to the text. Inadvertently, once generated and communicated as ‘output', visualizations seem to take on a quasi-dogmatic quality - they are hard to deconstruct, let alone reconfigure; they state their case but seem removed from critical reflection.

Most current DH visualizations are thus epistemological one-way avenues toward knowledge, from data via rendering algorithm to visual display. Charts, graphs, interactive maps, timelines, and similar representations are by and large imports from the natural and social sciences (Friendly 2008). Many of them emanate from domains of empirical research that conceptualize knowledge production as a function of empirical observation and objective measurement followed by analysis, inference, and conclusion. These approaches to visualization, however, hide two critical aspects, namely

(a) the underlying human modeling of the represented phenomena as data, which is already an interpretive and meaning-creating act that often oscillates repeatedly between observation and interpretation (Kitchin 2014), and

(b) the meaning-lessness of certain visual effects that are owed to contingent technological constraints (screen size, rendering, scaling, choice of color, etc.).

DH is in a unique position to investigate the domains of human experience and of its expression in symbolic practices and artefacts from two complementary methodological vantage points: the numeric, which models them as statistical phenomena, and the hermeneutic, which explores them as phenomena of meaning and thus by definition as a function of interpretation (Rockwell & Sinclair 2016). Where meaning comes into focus, our theories, object models, and practices

must therefore be conceptually aligned and ‘herme-

neuticized' - just as numeric approaches come with the pre-requisite of quantification. Against this backdrop, we propose to reintroduce the dimension of interpretation into visualization: Methodological principles of hermeneutic approaches, such as multi-per-spectivity, subjectivity, and context-boundedness present a challenge which representational visualization cannot and which interpretational visualization must meet.

Two questions arise: What are the defining principles of a genuinely humanistic and hermeneutically oriented approach to visualization? And how can we graphically express and support interpretation in DH visualizations - both as an activity and as a product of humanistic enquiry?

Visual modeling of interpretation vs. visualization of data

In the 3DH project, we address the former question by conceptual analysis and critique of existing approaches to visualization in DH, and then by systematically specifying and developing a visualization environment that can support higher level data interpretation rather than base-level data representation. In the presentation, we will share our survey of existing tools and their affordances but focus on two tools that we have developed, CATMA and Voyant.

Figure 1: Visualization of interpretive text annotation in CATMA

Our premise is that interpretation happens through the deliberate activity of an individual engaging with an image, text, display, or other artifact to create an argument about its meaning and a way it should be read.

For example, in CATMA (Figure 1) such an activity - in this instance the interpretive act of text annotation - is executed and represented by (a) highlighting a string on screen, (b) assigning it a tag, and (c) storing the annotation in a stand-off markup file. However, the annotation is at the same time (d) visually expressed as colored underlining. Moreover, via its visual representation on screen - the colored underlining - the markup data can also be (e) inspected, analyzed, manipulated directly, and even (f) enriched with meta-annotation. This is but one example of interpretative modeling.

Current representational ‘one-way' techniques like topic modeling (see Figure 2) are seen as a way to deal with scale, they process large amounts of data into summary abstractions called topics that can be displayed as lists or in other ways (Montague et. al 2015). In our second case study, we will therefore show how we are adapting scale tools to create a prototypical bidirectional 3DH visual modeling environment for big

Figure 2: Galaxy Viewer

data. We believe visual modeling can support not only interpretative close reading of primary data but also the reading of large collections like the collections of the Hathi Trust.

‘Hermeneuticizing' base-level visualization through activators: the 3DH framework of interpretive parameters and dimensions

A key goal of the 3DH project is to develop a set of generic graphic features that can be used to create interpretative attributes and/or inflections of visual representations of data, alter underlying data structures, and activate three-dimensional space in the service of interpretative activity. These features which aim to ‘hermeneuticize' visualizations are termed activators. In the presentation we will show the framework of the

activator set that was developed during a series of cha-

rettes (design workshops) in 2016.

Figure 3: Framework of Concept Modeling workspace: Shows features, activators, and dimensions from various pictorial conventions.

The visual activators in our feature set are not simply graphical marks or animations on a screen display: They perform data structuring functions and as such provide a conceptual framework for ‘hermeneu-ticizing' existing base-level data visualization techniques (see Fig.3). The individual features of this framework indicate and facilitate interpretative moves made by the user, such as a qualification of visualized data structures in terms of salience, irrelevance, uncertainty, degree of completeness, and other attributes or inflections. For example, uncertainty can be expressed by overlaying a standard graph with visual effects such as blur or shading, whereas the introduction of additional interpretative dimensions, such as point of view systems, parallax, relative scales, and other conventions from the visual arts, will support higher levels of interpretative critique and reflection, such as explicitly marking the historicity and context-dependency of underlying data.

Conclusion

As Pinker (1990) argues, the ease with which a particular graph can be understood is a function of the processing effort that goes into the exercise: The more we can rely on ‘hard-wired' encoding connections between the visual and the conceptual and the more we are guided by established graph and comprehension schemata (such as Gestalt phenomena), the less ‘intelligent' effort we have to put into reading a graph. Yet in a humanities perspective such conventionalized ‘ease of comprehension' is a double-edged sword: It may optimize the process of (re)cognition - but it also progressively obscures the constructedness of a visualization, turning it into an apparently self-evident object of perception. The 3DH project counters this anti-hermeneutic tendency toward reification by moving from a conceptualization of the principles of visualization as interpretative modeling to the development of a visual language framework, and finally the instantiation of the principles and language in two case studies. In terms of implementation, this approach is supported by drawing on Bertin's Semiology of Graphics and the high-level object-oriented Grammar of Graphics approach outlined by Wilkinson (2005), and features from game engines, three-dimensional modelling, and other pictorial conventions (Panofsky (1991) and Burgin (1991)).

To conclude, we will discuss next steps toward developing a 3DH environment that can act as a generic, project independent infrastructure for introducing user parameterized enunciative functionality into graphical displays. This infrastructure will make it possible to inscribe into visualizations the critical features of authorship, speaking/spoken subject, and an epistemological perspective grounded in situated and constructed approaches to knowledge. These interpretative principles are well mapped in, e.g., critical theory, narratology, visual studies, and cultural studies, but they have not been integrated into a graphical environment for hermeneutic practice yet: the methodological lacuna which the 3DH project tries to address.

Bibliography

Bertin, J. (1983). Semiology of Graphics: Diagrams, Networks, Maps. Madison, WI, University of Wisconsin Press.

Burgin, V. (1991). “Geometry and Abjection”. In: J. Donald

(ed.), Psychoanalysis and Cultural Theory: Thresholds.

London, Macmillan Education, pp. 11-26 .

Chandrasekaran, B. & Lele, O. (2010). “Mapping Descriptive Models of Graph Comprehension into Requirements for a Computational Architecture: Need for Supporting Imagery Operations”. In: A. K. Goel, M. Jamnik & N. H. Narayanan (eds.), Diagrammatic Representation and Inference. 6th International Conference, Diagrams 2010, Portland, OR, USA, August 9-11, 2010. Proceedings. Berlin & Heidelberg, Springer Verlag, pp. 235-242.

Drucker, J. (2011). “Humanities approaches to Graphical

Display”. In: DHQ, Digital Humanities Quarterly, 5 (1). http://digitalhumani-

ties.org/dhq/vol/5/1/000091/000091.html [March 17 2017].

(2014). Graphesis, Cambridge, Harvard University Press.

Friendly, M. (2008). “A Brief History of Data Visualization”. In: C.-H. Chen, W. K. Hardle & A. Unwin (eds.), Handbook of Data Visualization. Heidelberg, Springer-Verlag, pp. 134.

Kath, R., Schaal, G. S. & Dumm, S. (2016). „New Visual Hermeneutics“. In: Cybernetics and Human Knowing, 23 (2), pp. 51-75.

Kitchin, R. (2014). The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences. Los Angeles, SAGE.

Montague, J., Simpson, J., Brown, S., Rockwell, G. & Ruecker, S. (2015). “Exploring Large Datasets with Topic Model Visualization”. Conference paper at DH 2015, University of Western Sydney, Australia.

Panofsky, E. (1991). Perspective as Symbolic Form; C. Wood, trans.; New York, Zone Books.

Pinker, S. (1990). “A Theory of Graph Comprehension”. In: R. Feedle (ed.), Artificial Intelligence and the future of testing. Marwah, NJ, Erlbaum Hillsdale, pp. 73-126.

Rockwell, G. & Sinclair, S. (2016). Hermeneutica. Cambridge, MS & London, MIT Press.

Sousanis, N. (2015). Unflattening. Cambridge, MS & London, Harvard University Press.

Wilkinson, L. (2005). The Grammar of Graphics. 2nd ed.; New York, Springer.