10.6092/issn.1973-9494/1228
In the past years, many different new technologies for Cultural Heritage Diagnostic have been developed. In particular laser scanner surveys with digital photogrammetry and also multi-spectral surveys are becoming very useful and inalienable tools for non invasive diagnosis. In the SIDART Project (Integrated System for Cultural Heritage diagnosis), we develop a software able to visualize and elaborate triangulated surfaces coming from high resolution laser scanner survey. In this paper, we want to present the most innovative aspect of our study, that is the possibility to visualize and work in default mode or in immersive Stereoscopy (3D mode). This lets the operator perceive the third dimension and the “virtual investigation” of the object becomes more realistic. This lets us take into consideration in a more simple, natural and correct way and also reduce the possibility to make wrong evaluation due to the false prospective of the classic visualization.
application/pdf
1973-9494
1228
en
Francesco Fassi
Conservation Science in Cultural Heritage
3D visualization and virtual reality for cultural heritage diagnostic
Text.Serial.Journal
https://conservation-science.unibo.it/
7
Articles
2008-07-03
2007-07-08
2015-11-02
CSCH
Francesco Fassi
2007
10.6092/issn.1973-9494/1228
1973-9494
en
3D visualization and virtual reality for cultural heritage diagnostic
https://conservation-science.unibo.it/article/download/1228/654
7
3D       VISUALIZATION AND VIRTUAL REALITY
         FOR CULTURAL HERITAGE DIAGNOSTIC

 Lucio Colizzi, Fabio De Pascalis
 Centro di Progettazione, Design e Tecnologie dei Materiali (CETMA), Brindisi
 Francesco Fassi
 Politecnico di Milano




 1. Introduction

    This work takes place inside SIDART (Integrated System for Cultural Heritage
 Diagnostic) project, whose objective is to develop hardware and software package for
 cultural heritage diagnostic. The project involves some partner such as ENEA Group of
 Frascati, CNR-INOA and CETMA that is a private research center located in the south-
 east of Italy and also leader of the project. The aim is to develop an integrated system to
 acquire, integrate, process and distribute information produced by different survey
 instruments (laser scanner, multi-spectral camera, calibrated metric photogrammetric
 camera, thermography) used in cultural heritage diagnostic. This would be a new com-
 pleted tool useful to restoration in architectural and cultural study application.
    The Laboratory of Survey, Digital Mapping, GIS of Politecnico di Milano has cooper-
 ated with CETMA to create the software able to display and to model laser scanner points


                                                                                               /
 clouds. The analysed clouds come from a new prototype of laser scanner product from
 CETMA with Enea (Frascati Group) competence [1]. This instrument has a very good              Conservation Science in Cultural Heritage
 accuracy (at test phase) and guarantee 10-1 space resolution. All test and other survey
 measures, such as multi-spectral and thermographic survey are conducted by CETMA
 with Politecnico permanent collaboration.


 2. The software: laser clouds
 2.1. The language: VTK Library

    The software has been developed using VTK library. The Visualization ToolKit (VTK)
 [2] is an open source, freely available software system for 3D computer graphics, image
 processing, and visualization used by thousands of researchers and developers around
 the world. VTK consists of a C++ class library, and several interpreted interface layers
                                                                       including Tcl/Tk, Java, and Python. VTK supports a wide variety of visualization algo-
                                                                       rithms including scalar, vector, tensor, texture, and volumetric methods; and advanced
                                                                       modeling techniques such as implicit modeling, polygon reduction, mesh smoothing, cut-
                                                                       ting, contouring, and Delaunay triangulation.


                                                                       2.2. The PLY format

                                                                          The software is able to load files with PLY format. The PLY file format is a simple
D visualization and virtual reality for cultural heritage diagnostic




                                                                       object description that was designed as a convenient format for researchers who work
                                                                       with polygonal models. Early versions of this file format were used at Stanford University
                                                                       and at UNC Chapel Hill. A PLY file consists of a header followed by a list of vertices and
                                                                       then a list of polygons. The header specifies how many vertices and polygons are in the
                                                                       file, and also states what properties are associated with each vertex, such as (x, y, z)
                                                                       coordinates, normal and colour. The polygon faces are simply lists of indices into the ver-
                                                                       tex list, and each face begins with a count of the number of elements in each list. These
                                                                       files come directly from the laser instrument that provides the triangulated surface of the
                                                                       acquired object and infrared colour information.
                                                                          Here is a script to load and view 3d scanned object in ply format:

                                                                                vtkPLYReader *reader=vtkPLYReader::New();
                                                                                          reader→SetFileName(file);
                                                                                vtkPolyDataMapper *mapPD = vtkPolyDataMapper::New();
                                                                                          mapPD→SetInput(reader→GetOutput());
                                                                                vtkActor *actorPD = vtkActor::New();
                                                                                          actorPD→SetMapper(mapPD);
                                                                                  vtkRenderWindow *renWin = vtkRenderWindow::New();
                                                                                vtkRenderer *renderer=vtkRenderer::New();
                                                                                renWin→AddRenderer(renderer);
                                                                                renderer→AddActor(actorPD);
                                                                                renWin→Render();
L Colizzi F De Pascalis F Fassi




                                                                       2.3. Data matching

                                                                          The most important application implemented in the software is the possibility to cre-
                                                                       ate a data matching from the triangulated surface and digital images.
                                                                          The aim is to place the colour information obtained in other diagnosis instrument
                                                                       images on the 3D laser surface model.
                                                                          The implemented algorithm is the DLT equation (Direct Linear Transformation) solved
                                                                       with least squares method [3-4]. We need a set of points that are known in the two
                                                                       spaces; each point enables to write two equations. As a result, by matching at least six
points between the 3D object and 2D
image, we are able to know for each 3d
coordinates the corresponding point on
the image. From the coordinates it is then
possible to extract the RGB values and to
associate them at the laser cloud.                                                           Conservation Science in Cultural Heritage

   The DLT methods guarantee in this
type of analysis adequate precision
because the laser surface and digital
image have very deep resolution.
   The presented software allows to          Figure 1. Our first test. Clockwise from the
                                             top: The raw image of a niche of the S.
georeference different kinds of digital
                                             Stefani Crypt in Poggiardo, Vasto (Lecce)1.
image. We test with multispectral image
(infrared, ultraviolet) and also with image from thermocamera. Integration of different
diagnosis data and technique (Laser scanner, thermographic, multispectral) is a good val-
uation method of the state of the cultural object (fig. 1). Different colour layers can be
                                                                       added to 3d object, furthermore every layer is stored in a list, so the user can switch
                                                                       between different views of the object.
                                                                          A perfect point matching is possible because the laser SIDART 100 enables to
                                                                       acquire both depth and intensity images (in infrared band): so it is easy to find features
                                                                       in common between the 3D object and the corresponding image.
                                                                          The software allows to use a particular interceptor manipulator to place the marker
                                                                       (fig. 2). Furthermore a better accuracy is possible using the stereoscopic visualization.
                                                                       This manipulator has been developed inheriting functions by the vtkInteractorObserver
D visualization and virtual reality for cultural heritage diagnostic




                                                                       class, whose instances are generally used to control the camera and provide simple key-
                                                                       press and mouse oriented techniques [2]. This manipulator allows to place in correct
                                                                       position the marker, viewing around the object a gridbox on which the user can move a
                                                                       mobile intersector line. It is not difficult that way to choose the point with good accuracy.

                                                                       2.4. Other function

                                                                          With the software it is possible to carry out some elaboration of the 3D surface:
                                                                       ¾ export in ascii file and in 3ds format, cut with plane, filtering of large triangles, deci-
                                                                          mate, modify light illumination, shave light simulation, exporting section in ascii or dxf
                                                                          format.

                                                                       3. 3D Visualization and Virtual Reality

                                                                          The “Virtual Cultural Heritage” is a recent area where some spectacular growth look
                                                                       place over the past few years.
                                                                          At the beginning the Virtual Reality was used in design or industry contest to have a
                                                                       whole and immersive representation of the projected objects or in the entertainment field
                                                                       for immersive games and virtual adventure.
                                                                          Nowadays VR can be considered the most powerful “knowledge tool” developed at
                                                                       the beginning of this millennium. To understand this importance we must realize what the
L Colizzi F De Pascalis F Fassi




                                                                       Virtual Reality is. The VR allows to reproduce an environment or an object by a software
                                                                       technology, but this is done in a way that it is not distinguishable from reality. So it is pos-
                                                                       sible evolve in this environment, observe and examine it and also work with it inside it, as
                                                                       though it were a real and natural world. So the VR allows an intuitive, unconscious,
                                                                       almost childish approach to a phenomenon: this is the first and the most natural knowl-
                                                                       edge form [5].
                                                                          Today Virtual Reality (VR) technologies are becoming very useful techniques also in
                                                                       the field of cultural heritage to represent, show, explain historical and architectonical
                                                                    Conservation Science in Cultural Heritage




Figure 2. Example of 3D Surface scanned with Laser SIDART100. The
left image shows the raw surface and the right image the coloured
mesh. The example shows the fresco in Saint Mary Antiqua2.
                                                                       object to a large public. It also provides a useful and powerful instrument for the opera-
                                                                       tors in restoration fields.
                                                                          We believe that the combination of computer graphics and VR technique may provide
                                                                       an important set of information to better understand the decay phenomenon and choose
                                                                       the most appropriate restoration intervention.

                                                                       3.1. Virtual Reality Device

                                                                          Our VR device is a Table Projector Baron (fig. 3) [6], with workstation equipped with
D visualization and virtual reality for cultural heritage diagnostic




                                                                       nVidia Quadro 4500 graphic card.
                                                                       Such system allows us to make
                                                                       both passive and active stereo-
                                                                       scopic visualization.
                                                                          During the active 3D projection,
                                                                       the viewer wears special eyewear
                                                                       (two IR-controlled LCD light shut-
                                                                       ters working in synchronization
                                                                       with the projector). When the pro-
                                                                       jector displays the left eye image,
                                                                       the right eye shutter of the active
                                                                       stereo eyewear is closed, and vice
                                                                       versa. The used projector must be
                                                                       able to display at a very high
                                                                       refresh frequency so that the view-
                                                                       er cannot perceive the flicker effect.   Figure 3. Table Projector Baron (Barco).
                                                                          We also develop the passive
                                                                       (Anaglife) technology, so the two images (the right and the left ones) are processed with
                                                                       two filters of complementary colours. In accordance with the International Stereoscopic
L Colizzi F De Pascalis F Fassi




                                                                       Union the red colours is used for the left image and Cyan for the right one. The 3D image
                                                                       can be seen with the famous “red and green glasses”.
                                                                          The first technology gives better results and is certainly more useful than the second
                                                                       one. We develop the two techniques only to give the 3D possibility also when is not pos-
                                                                       sible to use or to have the active visualization that is more expensive.
                                                                          We have also tested the software inside a CETMA’s Virtual Reality Center. This sys-
                                                                       tem is based on a ORAD cluster architecture (fig. 4) [7] with twelve DVG (digital video-
                                                                       graphic).
Figure 4. ORAD
cluster architecture
with twelve DVG.


   Each DVG is composed of two render units. Each unit has: a Motherboard ASUS PC-
DL Deluxe 2, two INTEL XEON CPU 3.06 GHz (on each motherboard), a BFG 6800
ULTRA by NVIDIA graphic card. These renderers are linked by means 1Gb ethernet net-
work and their output is made up with a genlock.
   The virtual theatre (fig. 5) is composed of three mobile screens that can assume two
configurations like traditional cad wall or immersive cave.                                         Conservation Science in Cultural Heritage
   The system uses six Infitec Technology Projectors (two for each screen).
   Infitec is the latest stereo technology. It delivers superior stereo separation without
ghosting, with full freedom of motion, independent of head tilt. Left and right eye image
are simultaneously displayed, while the Left/Right image separation is made through
Infitec Filter Technology.
   This technology requires two projectors with two optical Infitec filters that split the
colour spectrum in two parts: one for the left and one for the right eye information. If each
of the two Infitec filters is mounted into the optical light path of its own projector, a stereo-
scopic image can then be made by putting the left eye information in the image of one
projector, the right eye information in the image of the other and by using the matching
D visualization and virtual reality for cultural heritage diagnostic




                                                                            Figure 5. The virtual theatre is composed of three mobile screens that can
                                                                            assume two configurations like traditional cad wall or immersive cave.

                                                                       Infitec filter in the pair of Infitec glasses in front of the corresponding eye. Both projectors
                                                                       display their information at standard refresh rate.
                                                                           Compared to active stereo it is completely free of flickering, more efficient and comes
                                                                       with more affordable glasses. Compared to passive stereo it achieves superior color
                                                                       fidelity.

                                                                       3.2. Stereoscopic Visualization (fig. 6-7)

                                                                           This software can work using classic prospective visualization or with 3D visualiza-
                                                                       tion. The high resolution of the laser product in the project plus the 3D vision allows to
                                                                       detect anomalies easily. The 3D picking of the points for data matching will be more accu-
                                                                       rate and enables to produce more precise colour transfer. The possibility to add to the
L Colizzi F De Pascalis F Fassi




                                                                       geometry colour information obtained by diagnostic instruments such as thermo-camera,
                                                                       multi-spectral machine, allows to investigate the damage cause in a deeper level.
                                                                           Using VTK is simple enough to manage the stereo capability; indeed the library pro-
                                                                       vides a complete set of classes to develop rendering window adapt to view the two
                                                                       scenes (right and left) for stereoscopic visualization.
                                                                           Here is below a part of the code in order to manage passive and active stereo.

                                                                                   renWin→StereoCapableWindowOn();
                                                                                   renWin→SetStereoTypeToCrystalEyes(); or renWin->StereoTypeToRedBlue
                                                                                   renWin→StereoRenderOn().
Figure 6. Passive 3D visu-
alization of frescoed wall
in Saint Mary Antiqua in
Rome3.




                             Conservation Science in Cultural Heritage




Figure 7. 3D pick-
ing in passive
stereo.
                                                                       Notes
                                                                       1   The S. Stefani Crypt in Vasto province of Lecce (LE). It is excavated in the stone and is formed
                                                                           by three naves separated with pillars. The crypt is decorated with frescos, even if some of them
                                                                           are in deterioration state. The frescos show images of Saints, Archangels, Our Lady and Christ.
                                                                           The original frescos date back to the Xth century, but they were substituted with other paintings
                                                                           afterwards (around the XIVth and XVth century). The laser SIDART 100 and the software with high
                                                                           resolution photogrammetric digital camera Rolleiflex 6008 were tested.
D visualization and virtual reality for cultural heritage diagnostic




                                                                       2   Saint Mary Antiqua (Rome) was erected in the middle of the 6th century on the Palatino Hill; it is
                                                                           the most ancient and important Christian monument of the Roman Forum. All the walls are paint-
                                                                           ed with frescos. We test our technology before the beginning of restoration works.
                                                                       3   You can see this image with Anaglife glasses.


                                                                       References
                                                                       [1] RICCI R., FANTONI R., FERRI DE COLLIBUS M., FORNETTI G., GUARNERI M.,
                                                                           POGGI C. (2003), High-resolution laser radar for 3D imaging in artwork cataloging,
                                                                           reproduction, and restoration, Optical Metrology for Arts and Multimedia. Edited by
                                                                           Salimbeni, Renzo. Proceedings of the SPIE, vol. 5146, pp. 62-73.
                                                                       [2] SCHROEDER W., MARTIN K., LORENSEN W. (1998), The visualization Toolkit: An
                                                                           Object-Oriented Approach to 3D Graphics, Prentice-Hall.
                                                                       [3] FANGI G. (1995), “Photogrammetry Notes”.
                                                                       [4] TSAI, ROGER Y. (1986), An efficient and Accurate Camera Calibration technique for
                                                                           3D Machine Vision, Proceedings of IEEE Conference on Computer Vision, pp. 364-
                                                                           374.
                                                                       [5] http://www.mediamente.rai.it/biblioteca/biblio.asp?id=13&tab=int#link009;
                                                                           www.barco.com.
                                                                       [6] www.orad.tv.
                                                                       [8] BRUMANA R., FREGONESE L., FASSI F., DE PASCALIS F. (2005), 3D Laser
                                                                           Scanner Points Clouds and 2D Multi-spectral images: a data matching software for
L Colizzi F De Pascalis F Fassi




                                                                           cultural heritage conservation, CIPA 2005, Torino, XX International Symposium.
                                                                       [9] BRUMANA R., FASSI F., FREGONESE L., MONTI C., POTENZA A., COLIZZI L., DE
                                                                           PASCALIS F. (2005), SIDART-A New Integrated System for diagnostic of the cultu-
                                                                           ral heritages, CIPA 2005, Torino, XX International Symposium.


                                                                       Summary
                                                                       In the past years, many different new technologies for Cultural Heritage Diagnostic have been devel-
                                                                       oped. In particular laser scanner surveys with digital photogrammetry and also multi-spectral surveys
                                                                       are becoming very useful and inalienable tools for non invasive diagnosis. In the SIDART Project
                                                                       (Integrated System for Cultural Heritage diagnosis), we develop a software able to visualize and
                                                                       elaborate triangulated surfaces coming from high resolution laser scanner survey. In this paper, we
want to present the most innovative aspect of our study, that is the possibility to visualize and work
in default mode or in immersive Stereoscopy (3D mode). This lets the operator perceive the third
dimension and the “virtual investigation” of the object becomes more realistic. This lets us take into
consideration in a more simple, natural and correct way and also reduce the possibility to make
wrong evaluation due to the false prospective of the classic visualization.


Riassunto
Negli ultimi anni sono state sviluppate diverse tecnologie informatiche per la diagnostica dei beni cul-
turali. In particolare analisi morfologiche condotte con laser scanner o tecniche di fotogrammetria
digitale, o ancora imaging multispettrale, cominciano ad affermarsi e nello stesso tempo ad imporsi
per la diagnosi non invasiva. Nell’ambito del progetto SIDART (Sistema integrato per la diagnostica
dei beni artistici), è stato sviluppato un software per visualizzare ed elaborare superfici triangolariz-
zate ottenute da un radar ottico ad altissima risoluzione. In questo articolo vogliamo evidenziare gli
aspetti più innovativi del software con particolare attenzione alle potenzialità avanzate di visualizza-
zione anche stereoscopica, con le molteplici possibilità di ispezionare i modelli 3D in modo più
immersivo e realistico. La visualizzazione immersiva consente di effettuare considerazioni in modo
naturale e semplice, riducendo la probabilità di errore.


Résumé
Ces dernières années, différentes nouvelles technologies ont été développées dans le domaine du
patrimoine culturel. Tout particulièrement, les enquêtes laser scanner à photogrammétrie digitale et
multi-spectrales sont devenues des instruments de plus en plus importants pour effectuer un dia-
gnostic non invasif. Dans le projet SIDART, nous développons un logiciel qui permet de visualiser et
élaborer des surfaces triangulaires tirées de l’enquête à haute résolution laser scanner. Dans cet
article, nous voulons présenter l’aspect le plus innovateur de notre étude, la possibilité de visualiser
et travailler in default ou en stéréoscopie (3D). Ceci permet à l’opérateur de percevoir la troisième
dimension et l’enquête virtuelle de l’objet devient plus réaliste. La visualisation immersive nous per-
met de faire des analyses d’une manière plus simple, plus naturelle et plus correcte, et surtout de
réduire la possibilité d’effectuer une mauvaise évaluation due à une fausse perspective de la visua-
lisation classique.


Zusammenfassung
In den letzen Jahren wurden viele verschiedene neue Technologien zur Diagnostik des Kulturerbes
entwickelt. Im Besonderen Laserscanner mit digitaler Photogrammetrie und multi-spektraler                   Conservation Science in Cultural Heritage
Bildverarbeitung, nützlich und unverzichtbar für die nicht invasive Diagnostik. Mit dem SIDART
Projekt haben wir eine Software entwiceklt, die imstande ist, trianguläre Flächen aus hochaufgelö-
sten Bildern zu visualisieren und elaborieren.
In diesem Beitrag möchten wir den innovativsten Aspekt dieser Studie präsentieren, nämlich die
Möglichkeit in Standardverfahren oder immensiven Stereocopien zu visualisieren.
Das erlaubt dem Bediener die 3. Dimension wahrzunehmen, um die virtuelle Recherche des Objekts
realistischer zu machen.
Das ermöglicht, es in einem simpleren, natürlicheren und korrekteren Weg zu erkennen und redu-
ziert die Möglichkeit, einer falschen Evaluation als Ergebnis einer falschen Prospektive der klassi-
schen Evaluierung.


Resumen
En los últimos años se han desarrollado diferentes tecnologías para el diagnóstico de los bienes cul-
turales. En particular, se están volviendo cada vez más usados para diagnósticos no invasivos los
análisis morfológicos con láser escáner, técnicas de fotogrametría digital o imagen multiespectro
                                                                       imaging. En el ámbito del proyecto SIDART (Sistema integrado para el diagnóstico de los bienes
                                                                       artísticos), se ha desarrollado un software para visualizar y elaborar superficies triangularizadas
                                                                       obtenidas con un radar óptico de altísima resolución. En este artículo queremos poner de manifies-
                                                                       to los aspectos más innovadores del software, que permite también una visulización estereoscópi-
                                                                       ca: esto hace que se puedan inspeccionar los modelos 3D de manera más inmersiva y realista. La
                                                                       visualización inmersiva permite realizar consideraciones de forma más natural y simple, reducien-
                                                                       do las probabilidades de error.
D visualization and virtual reality for cultural heritage diagnostic
L Colizzi F De Pascalis F Fassi
