3D Imaging and Processing of Damaged Texts

INTRODUCTION

This paper addresses the technical framework for using 3D imaging as a means of creating, manipulating, restoring and carefully measuring features on digital facsimiles of texts. This effort, which is part of the "Digital Atheneum"[1] project at the University of Kentucky, is concerned with developing new techniques that will ultimately make severely damaged texts more accessible and useful to scholars. In particular, we address the problems and limitations of 2D imaging when a document is warped and damaged, and the difficulties this can present to subsequent image processing operations.
The presentation is divided into two topics. We address why and when it is preferable to acquire 3D shape information, and how our system works. Second, we discuss the uses of the 3D representations beyond that of facsimile.

3D IMAGING
An image of a text is a very useful representation that is familiar and easy to create with current technology. As a primary representation for preservation and manipulation, however, the 2D image has limitations. Modeled mathematically, a photographic image is a mapping of light rays onto a planar surface. If the image is not of a planar object, information is lost during the projection which creates the image [2]. This is generally not a problem because the majority of text collections are sufficiently flat. But the surface shape of many older and damaged items may be warped and bent. Although scholars may be able to discern from the image that the underlying surface is not flat, it can still be difficult to disambiguate the contribution of the shape distortion to the shape of the markings: is the shape of a letter-form accurately discerned by a scholar, or does it appear distorted as a by-product of the non-planar shape of the object? In addition, there are items that inherently have 3D relief, such as book covers, coins, wax seals, and other similar items. For these items, a quality facsimile must capture shape. Figure (1) compares a 3D model with its 2D counterpart.
We advocate the use of 3D imaging when digitizing non-flat materials. A 3D representation more accurately preserves the "look and feel" of the original. To acquire a 3D facsimile, we have designed and implemented an acquisition system that integrates into 2D digitization setups based on a stationary digital camera. By augmenting the camera with a light projector[3], we are able to acquire 3D data points on the surface of the text through a triangulation-based computer vision technique known as "structured-light". Structured-light is similar to laser scanning, but uses a light projector in place of a laser. The projector emits a vertical or horizontal stripe of light onto the surface of the manuscript or artifact. The camera observes this stripe and can determine the 3D shape of the underlying material based on how the stripe is warped (see figure 2). The projector repeats this process, sweeping a line across the entire object until the shape has been recovered.

By design, the acquisition system integrates well with existing digitization setups. This minimizes operational costs and allows the digiteur or directing scholar the freedom, without a change in setup, either to acquire a complete 3D model or just the 2D image. This 3D image acquisition system uses the same 2D image that would otherwise have been captured; in essence, nothing changes in the overall digitization procedure except that shape information is also collected.

USES OF 3D DATA
METRIC MEASUREMENTS
Our 3D imaging technique creates a digital 3D model which has, to a very high level of accuracy, the same metric dimension as the corresponding physical material. This makes it possible to make accurate metric measurements on the surface of the digitized text. Although we as yet have no experience with how expert editors and scholars will use these capabilities in a production system for creating a digital edition, we envision that scholars will find it useful to make very accurate measurements of letter-forms, markings, and other regions of interest in damaged manuscripts. The acquired 3D shape representation provides the opportunity to extract accurate measurements, in millimeters for example, with these measurements correctly and exactly corresponding to the physical features of the object. Such measurements are impossible to make on the 2D image alone, and are likely impossible, or tedious at best, to make directly on the physical object. We also envision that experts may have interest in using metric measurements as another tool for monitoring damage, deterioration and deformation of a text or other object over time. These measurements would be possible by making periodic 3D scans to measure and track features such as the sizes of holes, tears, and crinkles.
DIGITAL FLATTENING OF A MANUSCRIPT
We demonstrate preliminary work that uses 3D data to digitally flatten a warped and crinkled text. This is done by using a physics-based Mass-Spring model, similar to that used in computer graphics to model cloth. The Mass-Spring model uses Newtonian physics to approximate the motion of the acquired 3D points as they are pushed back to a flat surface, thereby flattening the 3D model. This restored 3D model represents what the original material might have looked like in its original, flat state, before the distortion due to aging and damage. The virtual, or digital, flattening of the 3D manuscript model can be especially useful when the original text cannot physically be flattened without the risk of further damage.

Virtual flattening serves two purposes. First, in many cases it will make the document more readable. Text that would appear warped by the underlying shape in a photograph will now look undistorted. This is a desirable outcome from a perceptual standpoint, and we hope in the future to obtain the opinion of manuscript experts as to the perceptual usefulness of the technique. The approach also serves a less subjective but very useful purpose. Images used to create electronic editions often are heavily processed. Some image operations, such as scaling and rotation, are not dependent on the content of the image. However, more sophisticated processing, such as those used to help the user automate or semi-automate extraction of information in the image, often rely on the assumption that original object is flat. For example, Optical Character Recognition (OCR) suffers if the input image is from a warped and crinkled text. In addition to OCR, there are many other useful and desirable automated image processing algorithms, such as line detection, word grouping, and image-based searches for user-specified regions (such as letter forms). The performance of these automated image processing algorithms can benefit greatly by using the "flattened" image as input instead of the image of the warped and crinkled document.

CONCLUSION
The research presented in this paper reflects new technical approaches that we envision will be of value to those researchers who are interested in editing and manipulating damaged texts. The goal is to provide new technical approaches that will serve as a framework that can enable scholars to create, manipulate and process digital representations of physical materials in ways that were previously impossible. In many cases, the "virtual restoration" performs operations which physically are risky or impossible, or for which there is no physical analogue. Our work focuses on the need for the 3D representation of materials that are assumed flat but prove otherwise. When addressing very old and damaged texts containing profound shape distortions, the acquisition of a 3D model can serve many useful purposes. It provides a useful and effective representation of the original, and moreover, allows the possibility for making complex metric measurements, applying new restoration techniques, such as flattening, and monitoring deterioration over time.

NOTES

[1] "The Digital Atheneum" http://www.digitalatheneum.org/

[2] A photographic image is a projection of a 3D objects on a 2D image plane. One dimension, the "depth" of the object, is collapsed in the projection process and thus lost. Although cues in the image, such as shadows, give hints about the shape of the objects, actual depth values cannot be recovered without making assumptions about lighting and object properties.

[3] Standard XGA light projectors, such as those used by laptop and computers for presentations.

REFERENCES

Brown M. S. and W. B. Seales, "Beyond 2D Image: Effective 3D imaging for library materials", 5th ACM Conference on Digital Libraries , San Antonio 2000

W. Brent Seales, James Griffioen, Kevin Kiernan, C. J. Yuan, and Linda Cantara. "The Digital Atheneum: New Technologies for Restoring and Preserving Old Documents." Computers in Libraries 20:2 (February 2000), 26-30.

Kiernan, K. S. "Digital Image Processing and the Beowulf Manuscript.", Literary and Linguistic Computing 6 (1991): Special Issue on Computers and Medieval Studies (Edited by Marilyn Deegan with Andrew Armour and Mark Infusino), 20-27

Prescott, Andrew. "The Electronic Beowulf and Digital Restoration." Literary and Linguistic Computing 12 (1997), 185-95

Lecolinet, E. et al "An Integrated Reading and Editing Environment for Scholarly Research on Literary Works and Their Handwritten Sources", 3rd ACM Conference on digital Libraries , Pittsburgh 1998

M. Sonka, V. Hlavac, and R. Boyle, "Image Processing, Analysis, and Machine Vision", PWS Publishers , Second Edition, 1998

Xavier Provot. "Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior", In Proc. Graphics Interface '95, 1995, pp 147-154.

Valkenburg R. J. and McIvor A. M., "Accurate 3D measurements using a Structured Light System", Image and Vision Computing 16:2, February 1998, pp 99-110