C
ORSO DI
L
AUREA
M
AGISTRALE IN
I
NFORMATICA
U
MANISTICA
Anno accademico 2015-2016
MUSE-TOOLS: EXTENDING THE UNITY EDITOR TO SUPPORT THE
DESIGN OF VIRTUAL AND REAL MUSEUM EXHIBITIONS
Candidato: Riccardo Galdieri Relatore: Prof. Marcello Carrozzino Controrelatore: Prof. Matteo Dellepiane
“It turns out that the killer application for virtual reality is other human beings. Build a
world that people want to inhabit, and the inhabitants will come.”
Contents
1. Introduction . . . 1.1 Human Heritage and the urge to preserve it . . . 1.2 New technologies to preserve the past . . . 1.3 Muse-Tools – a suite to simulate realistic Human Heritage exhibitions . 2. Theoretical Background . . .
2.1 Defining what museums and virtual museums are . . . 2.2 Virtual museums as modern ways to resolve old problems . . . 2.3 Serious games and gamification . . . 2.4 Spreading VR: From 3D scanning to user-oriented application . . . . 2.5 Unity3D and the Human Heritage digitization . . . 3. Questionnaire . . . 3.1 Questionnaire structure . . . 3.2 Reaching an invisible audience . . . 3.3 Results . . . 3.4 Comments and Discussion . . . 4. Muse-Tools . . . 4.1 Unity, Unreal, and the duality Videogames - Museums . . . 4.2 Wrapping the tools around the Engine: basic Unity introduction . . . . 4.3 Plugin structure . . . 4.4 Features . . . 4.4.1 HumProjector . . . 4.4.2 InfoTable . . . 4.4.3 SunPos . . . 4.4.4 Ruler . . . 4.4.5 RoomBuilder . . . 4.4.5.1 Mesh creation . . . 4.4.5.2 Consistency check . . . 4.4.5.3 Node uniqueness check . . . 4.4.5.4 Walls creation . . . 4.4.5.5 Rooms detection . . . 4.4.5.6 Floors and Roof creation . . . 4.4.5.7 Tessellation . . . 4.4.6 WallsHandler . . . 4.4.6.1 Adding a Fixture . . . 4.4.6.2 Deleting a Fixture . . . 4.4.6.3 Moving a Fixture on its side . . . 4.4.6.4 Changing a Fixture’s width . . . 4.4.6.5 Changing a Fixture’s height . . . 4.4.6.6 Changing a Window’s height from ground . . . 4.5 Creating the assets . . . 5. Project in Action: The Demo . . .
5.1 Optimization . . . 5.2 The rooms . . . 5.3 Discussion and Further Developments . . . 6. Conclusions . . . Bibliography . . . Appendix I: Questionnaire . . . Appendix II: Standard Email Sample . . .
11 6 7 9 11 14 16 18 21 23 25 29 30 34 37 39 43 46 48 48 49 51 52 53 59 60 60 61 63 66 67 69 71 77 78 80 81 82 82 89 89 91 97 98 100 104 112 6
ABSTRACT
In a world where new technologies are modifying the way people perceive the spaces around them, museums need to embrace the change by using a combination of general and dedicated tools to attract the broadest possible audience. Amongst the new softwares that museums can use, 3D engines are some of the hardest to handle for people not coming from the Information Technology field, creating a technological gap between the museological space and the third-millennium public’s expectations that result in a lower interest for the displayed artefacts. With the support of Unity3D, one of the most famous and reliable VR-ready 3D engines available on the market, a suite of tools called Muse-Tools was developed to reduce this gap, extending the engine’s editor functionalities to provide museums curators with enough tools to plan both real and virtual exhibitions without relying on expert programmers or 3D artists.
After the first section, where the general problem of using modern technologies in combination with the human heritage is described, section two will explore the current state of the art of virtual museums, discussing not only the problem of defining what a virtual museum is, but also referencing to other similar projects involving VR in public events and their outcome. Section three will present a questionnaire made to understand the current museums’ curators level of familiarity with technologies and Virtual Reality and what they would expect from Muse-Tools. In section Four the entire suite will be described, explaining why Unity3D was chosen and how the suite was designed to maximise its fit with Unity interface and engine, what components were created and how they were internally structured. Section number five introduces a demo to show the potentiality of the suite in combination with some basic engine optimizations, describing in detail how the environment was created to reproduce a fictitious museum. In section six the results of the project will be commented and future implementations will be described.
1 - INTRODUCTION
1.1 – Human Heritage and the urge to preserve it
Recent facts in Syria and Iraq have reminded the world the importance of human heritage preservation through time and space. Only in the last three years millenary buildings such as the palace of Ashurnasirpal II in Nimrud, the temple of Bel in Palmyra and the Dair Mar Elia monastery of Mosul were completely destroyed by the tremendous consequences of the new policies applied in that region by the self-proclaimed Islamic state of Iraq and the Levant (ISIL). Unfortunately the irreparable loss of precious artefacts and constructions is not a recent issue, and going back in time many other examples could be listed: in 2000 under similar circumstances the Buddhas of Bamiyan were destroyed by war, many temples were lost during the Japanese earthquake in 2011, a lot of precious paintings and statues were damaged by the 1966 flood of Florence, and many more were lost in the great fire of London in 1666. All those events have contributed to build a general awareness of how vulnerable the cultural heritage can be, and the responses to preserve from destruction some very unique artefacts through which local identities are shaped have been increased, with joint efforts by the scientific community to find newer solutions to protect it from every possible threat.
While most of the artefacts we can access are very well preserved and periodically restored, unpredictable events such as those we have just described can compromise the appearance of the object as we know it, or make them unavailable to public display for years if not forever. Even when carefully conserved artefacts will never be safe from unforeseeable events, it has happened in more than one occasion that due to the lack of experience of restorers some paintings were devastated, and too often unique pieces of art are stolen to be sold on the black market. Moreover, sometimes it is physically impossible
to have continuous and direct access to an object, making any attempt of restoration impossible.
Since the last years of the 20th century, several groups of researchers have approached the issue of preserving cultural heritage with the help of modern technologies, creating three-dimensional replicas to preserve valuable artistic objects, and experimenting using different methods of acquisition. The results have so significantly improved since then that it is now possible to reconstruct many different artefacts in a three-dimensional space, even without any direct contact with them, with a very limited budget and with a great overall quality.
These processes have been used so far for two main purposes: on one side, they have been used for complex studies about the artefact’s structure that could not be done on the real object without damaging it, such as analysing the most stressed points within the structure; on the other side the same models were used to create interactive models to be placed in dynamic virtual applications and distributed through different means. Those models have already proved to be attractive for the general public, who very much enjoyed the chance to manipulate and play with the 3D object as if they were dealing with the real one. This second case of use is the one this project will discuss and study in deep.
1.2 - New technologies to preserve the past
The unpredictable spread of technology we have seen in the last decade, together with the simplification of software interfaces, have revolutionized the way digital worlds are perceived, used and developed by the mass. If fifty years ago computers were still considered an exclusive tool that could be handled by experts only, technology has done several steps towards a broader audience, and nowadays smart device can be afforded and
used by almost everyone. This social change was made possible not only by the reduced costs of hardware production, but also by changing the way software is thought. Concepts such as usability and accessibility have become fundamental subjects in modern industries as they can grant the best possible user experience to a wider public increasing the profits, making user interfaces (UI) as important as the software they describe. Many products nowadays have extremely simple UI, specifically designed to be understood by the widest possible audience, from the curious one-time user to the professional that make heavy use of it. This simplification has brought many people to invest in technology, creating new products and new professional figures that can be used by museums to expand their appeal in a newly interconnected world.
Some museums have already started to show a practical interest in those new technologies by creating advanced positions to enhance real exhibitions, providing the visitors with a more immersive experience. Unfortunately so far the broad spectrum of contexts in which the Virtual Heritage (VH) was used has only reached a small percentage of its potential: Virtual Reality (VR) or Augmented Reality (AR) stations are quite expensive and require a huge amount of work to be programmed and installed, they often have a fixed position within the exhibition that could create security problems, distort the attention from the real opera, and can only be used in the best case scenario by very few people at the same time.
Amongst all the new technologies that appeared on the market in the recent years, one in particular has shown the potential to revolutionize the way people can use VR in everyday life: the Head Mounted Display (HMD). Despite the idea of wearing Augmented Reality helmets is everything but new, it is only since 2012 that it was transformed from a
cool futuristic concept to an actual prototype to be released on the market. The first company to reach this goal was the Oculus VR company with its first Oculus rift prototype. From that moment, several companies have joined the market with models that could be applied to different platforms and that could match different hardware settings. In the last few months, the number of VR-HMD products have increased considerably, and big companies such as Sony, Lenovo and Valve – this last one in cooperation with HTC, the mobile phone manufacturer - have released their own headsets. Even Facebook, the social network colossus, joined the competition by acquiring the Oculus VR company for over US $2 billions.
Real museums could immensely benefit from this important innovation. In fact, museums users satisfaction could be significantly improved by creating new types of experience based on this immersive devices. Technologies have demonstrated to be mature enough to support complex experiences such as exploring a museum in its digital form simply by wearing a HMD from home. Thanks to the support of JavaScript libraries like WebGL, Nexus and SpiderGL, transmission and rendering of 3D scenes with millions of polygons have already been successfully achieved through online-based platforms, making those product even more shareable. Of course building a complex 3D scene such as a museum will bring many complex challenges, but technologies are already mature enough to be successfully applied to complete these tasks.
1.3 – Muse-Tools – a suite to simulate realistic Human Heritage
exhibitions
In order to move from their old static idea of museums as mere collection of objects to a new concept of museums as open digital organizations, institutions will need dedicated
tools to facilitate the digitalization of their own environments and Digital Humanities experts that will help them using those assets in the most profitable way. Being the creation of dedicated software to humanities research a very recent approach, very few technologies have been created with the specific purpose of recreating cultural heritage exhibitions so far and almost none of them have reached a broader audience yet.
This dissertation describes Muse-Tools, a new suite of tools that will help curators and museums in general to recreate any location, real or fictional as it could be, making users able to manipulate their own world without limitations by simplifying complex operations such as UV-unwrapping, mesh processing and measurements calculation. Such a tool could drastically drop the costs of planning an exhibition, things such as lights, objects, supports and materials could be tested in a rich environment with almost no costs, and different people could work on the same model without even touching the real object, increasing the overall preservation level and producing a better result with a reduced effort. Another advantage coming from the use of Muse-Tools could be more user oriented: in fact, once a three dimensional environment is available, with a few simple modifications the same project can be moved from being an internal demo to something that can be explored by the broader audience in numerous different ways, making it the perfect tool to recreate pleasant experiences outside the testing environment.
An error in which many project incur is to assume that the software they need must be build from scratch. It is not a coincidence that of all the developed projects almost none of them has ever been used by anyone else other than the team that developed it. There is a misconception that building a software to answer a specific needs is the same thing of building a software to answer a general question that many people could have. This assumption is usually false, while choosing a software to use, especially while working
with Virtual Environments (VE) related fields, it is important to have the maximum solutions at disposal coming from the same software rather than having many of them to answer a simple question. For this reason most of the software that was specifically designed to be used in the humanities field has failed to reach a broader audience, it was not structured to solve different problems or to perform multiple tasks. After these considerations, it seemed logical not to develop a new software from scratch but to find and extend the functionalities of a specific pre existing one.
2 – THEORETICAL BACKGROUND
2.1 - Defining what museums and virtual museums are
An important shift of paradigm took place at the end of the 80s, when the old idea of museums as mere collection and display of objects shifted to a modern concept of museums as public institutions responsible for producing and sharing information (Pearce 1986, MacDonald/Alsford 1989, MacDonald/Alsford 1991, Alsford 1991, MacDonald 1992). Following this new paradigm, in 1991 the first International Conferences on Hypermedia and Interactivity in Museums was created to "explore policy, legal, social,
economic, technological, organizational and design concerns of digital culture and heritage, from the perspective of cultural policy makers, institutions and cultural participants.”1 (Bearman, 1991). This change was motivated not only by the curiosity that emerging technologies such as the internet were arousing in the media and the academic environment, it was also demonstrated that for a general public information was even more important than the mere object. According to several researches, if no significant connection could be built between the object and the viewer, people would not have any interests towards it and they will easily not pay enough attention to it. (Treinen 1996, p. 65)
With this newer conception of museums coming forward, a new definition of museum was proposed in 2007 by the International Council of Museums (ICOM), describing a museum as “A non-profit, permanent institution in the service of society and its development, open
to the public, which acquires, conserves, researches, communicates and exhibits the tangible and intangible heritage of humanity and its environment for the purposes of education, study and enjoyment." (ICOM, 2007).
Such a definition can be considered complete and exhaustive, but defining what a Virtual Museum is has proved to be a harder task because of the many different meanings and areas this new concept is associated to. In 1998 Schweibenz defined a virtual museum as follows:
“a logically related collection of digital objects composed in a variety of media,
and, because of its capacity to provide connectedness and various points of access, it lends itself to transcending traditional methods of communicating and interacting with the visitors being flexible toward their needs and interests; it has no real place or space, its objects and the related information can be disseminated all over the world” (Schweibenz, 1998)
While being a good attempt to define what purposes Virtual Museums can have it was a too broad definition that did not specific any boundary to the subject nor did help to define what a virtual museum is or is not.
In 2004 Schweibenz again tries a different approach to the problem by shifting the focus from finding a general definition to finding a way to subdivide them. He defined a categorization of virtual museums by their presence on the web, reserving the role of
Virtual Museums only to those web-based application running on a web-based platform, removing any possible space for mobile applications or downloadable contents. (Schweibenz 2004). A sense of general confusion was also evidence by Bruno in 2010, which specified how the “Virtual Museum” definition is used to describe two distinct types of VR technologies:
“the first one is a reconstruction of an already existing museum, in
which the user can simulate an immersive visit, and enjoy the works of art and the archaeological finds; the second one is the ex novo creation of a virtual environment, not corresponding to existing structures, in which the user can navigate and watch the reconstruction of the objects" (Bruno, 2010).
As it clearly appears from this distinction, a long series of applications can be categorized under the name of Virtual Museum, such as a virtual environment running on browser, an Advanced Reality application running on a personal smartphone or even a kiosk position in a real museum can be considered as Virtual Museums application. Without a clear distinction of roles it is hard to find a homogeneous comprehensive definition, and surely we cannot consider as exhaustive the many ones proposed so far. In 2010 Carrozzino and Bergamasco proposed a new categorization method of such technologies, based on VR’s level of immersivity (Fig. 1) rather than their application, from a starting level of a totally detached scenario displayed by a 2D monitor to a fully immersive environment such as a CAVE (Carrozzino and Bergamasco, 2010). While being a good solution to distinguish different environments belonging to the virtual world, a fully shareable definition of a Virtual Museum is still missing.
2.2 - Virtual museums as modern ways to resolve old problems
Despite a common shared definition of virtual museums is still lacking, many studies that autonomously categorized themselves under this label were performed, showing how Virtual Museums could significantly increase the visitor’s experience by providing new solutions to older problems while not creating any disaffection or counterproductive effects on the museum itself, as feared by many museums’ curators (Pierdicca 2015). One of the biggest challenges in modern museums is to choose what to display and how to display it, as evidenced by Walczak "Museums around the world hold
countless artefacts that they cannot exhibit to the public due to limited space, the items’ fragility, or the prohibitive cost of creating and managing appropriate displays.". Virtual
Reality can offer a digital solution to prevent this objects from being forgotten while giving the museum institution a feedback on what users are really interested in or not. (Walczak, 2006). Moreover, the same instance of a virtual museum can possibly reach those categories of people that would not be able to visit the real instance, spreading the audience of a museum beyond its physical borders.
Virtual museums have also the capacity to tackle the problem of detached experiences that many users have lamented about. Several different experiments were conducted to understand what level of interactivity and customization users would expect from future museums, from which it emerged that the majority of people would demand more interactive applications, with the possibility to be challenged during the visit (Fig.2), and even showed the necessity to be guided by a room attendant while visiting an exhibition. (Pagano 2015, Choi 2016). These results brought the attention to the problem of unqualified or even missing staff that very often is not able to follow the visitors in their experience. As explained by Swartout, “a well-informed guide or interpreter can have a
tremendous influence on the quality of a museum visitor’s experience. The best guides not only provide information but also engage the visitor in an interactive exchange that can
lead to deeper understanding and promote excitement about museum content. ". This
problem can easily handled by Virtual Museums thanks to pre defined avatars that could guide the users, interacting with them, and functioning just like a professional guide with no drawbacks of sort. In fact, as shown by several studies (Reeves 1996, Krämer 2003, Gratch 2007), people react to virtual humans in much the same way that they react to real people, increasing their level of awareness and making custom experiences more enjoyable for different types of audience (Robles-Ortega: 2011, Katz 2015, He 2016)
2.3 - Serious games and gamification
Problem solving is not the only motivation that brought humanities scholars to develop a certain interest in game technologies, in fact at the beginning of this decade new researches have demonstrated how interactive scenarios could significantly improve the sense of satisfaction of the audience, especially in the youngest people, leading to a better understanding of the environment (Champion, 2008). Unfortunately, this younger public is more accustomed to VE and to attract them towards museums and their applications, many aspects of modern videogames must be implemented to increase the sense of familiarity with the scene and improve the consequential learning process. (Neto 2011, Pietroni 2012) From these researches new fields of study were derived, and concept such as “Gamification”, “Meaningful play” and “Serious games” have quickly risen (Fig. 3) amongst the scientific community (Hamari, 2014), not only when applied to museums and humanities research, but also when applied to several other subjects such as “finance,
health, education, sustainability, as well as news” (Deterding, 2011b). Being Gamification not strictly related to the Humanities research, the general definition proposed by Deterding of Gamification as “the use of video game elements (rather than full-fledged games) to improve user experience and user engagement in non-game services and applications.”, for as true as it is, cannot be enough to define Gamification in relation to the humanities field. (Deterding, 2011a)
Similar problems were faced while defining serious games, in 2005 Zyda proposed a definition of serious games as “applying games and simulations technology to non-entertainment domains”, and while this definition is broadly accepted, it still cannot be applied to a specific field (Zyda, 2005).In 2013, a better definition that also included the human heritage aspect, was proposed by Lercari:
“A serious game for heritage (SGH) is a 3D real-time application that uses
games technology for the simulation, visualization, and communication of cultural heritage. Some of the key aspects of a SGH are the accuracy of visualized cultural information, transparency of the sources or data generation processes, reliability of the narrative, and a thorough validation of the results”
(Lercari, 2013).
While respecting the same basic design principles (Malone, 2008), SGH often face some minor challenges deriving from having different needs from the classic commercial videogames. (Woolford 2013, Hamari 2013). However, by using proper dedicated tools those problems can easily be tackled in order to create engaging applications that increase the level of appreciation of the general public for a museum (Aparicio, 2012).
2.4 Spreading VR: From 3D scanning to user-oriented
applications
A first overview of the “Virtual Heritage scene”, as it was recently defined (Koller, Frischer and Humphreys, 2009), was given in 1997 (Forte and Silotti, 1997), when the first 3D representations of object with artistic interest were modelled by hand. In 1998 the first significant results in automatically converting real artistic objects to 3D models were achieved by “The Digital Michelangelo Project”, when a joint force of Stanford University and University of Washington scholars was able to digitalize 10 different statues from the Italian artist Michelangelo in over two years of work (Fig. 4). The project was the first one to be conducted on a large scale, and it pointed out several critical points of such a task, including the huge costs of moving laser scanners and handling raw data outputs. (Levoy et al., 2000)
Since that work several other projects were able to produce better results with smaller impacts of said critical points: in 2003 Stumpfel (Stumpfel et al, 2003) created a 3D reconstruction of the original Parthenon by merging laser scansions from reproductions; in 2004 another Italian team (Callieri et al, 2004) successfully evidenced invisible damages on Michelangelo’s David computing effect of physics, time and dust exposure by using a three-dimensional model.
The time and costs required by these projects have significantly decreased since the
first attempts, cheaper and faster laser scanners have been developed to maximise the output on both medium and big objects, reducing the effective costs of professional equipment. Even home-made scanners were demonstrated to be as good as professional ones when working with small size objects like coins. (Dellepiane and Scopigno, 2012).
Despite using a laser scanner is still the most common way of acquiring Virtual Heritage, photometry technique has also been widely used to acquire 3D models. In 2000, a Japanese team guided by Professor Miyazaki from University of Tokyo (Miyazaki, 2000) scanned a 15 meters tall bronze statue of Buddha, using photometric scanning and an algorithm to compensate calibration errors, producing a complete model with lower human supervision.
As direct consequence of refining these techniques, an increasing number of scholars has pointed out the importance of using them to improve the way museums make their contents available to public. In 2008 University of California in cooperation with other institutes started a project to create a Virtual Museum of the Chinese Han dynasty (Forte et al., 2010). In 2010 Hirose and Tanikawa from University of Tokyo started a
MEXT founded project about augmented reality in museum explorations, mixing VR with real objects during the museum exploration (Hirose and Tanikawa, 2010). In 2014 a team from Ritsumeikan University in Japan, went another step forward, recreating a whole moving scene: in their project they collected data about the traditional Yamahoko Parade (Fig. 5) in Kyoto to build a VR system able not only to show the parade in its own real context, but also to make the users feel like they were participating in it (Li et al, 2014).
With an increased overall quality, 3D scans and consequent application have been lacking a mean of interaction so far. The solution proposed in recent years by an increasing number of scholars is to use the web as the mean for accessing the contents. The standardization of 3D contents on the web in 2007 (Don Brutzmann, 2007) was a significant step in that direction, but the real turning point was the introduction of WebGL standard, promoted by Khronos Group in 2009, which allowed browsers to access graphic hardware handling by using JavaScript language only (Khronos Group 2009). For instance, after digitalizing the Ruthwell Standing Cross in Ruthwell Kirk, Scotland, "The Dream and the Cross" project created a web interactive version of the whole model by adding an information panel alongside the
model itself (Scopigno et al, 2015), giving the users not only the chance to remotely access the cross, but also to read additional information alongside it (Fig. 6). Again, in order to give scholars a new tool that did not require any programming background, CNR-ISTI released
Fig. 6 During the navigation, the user has detected and activated a Spot and consequently the link between the carved panel on the Ruthwell Cross and the corresponding portion of text describing it is visually enhanced (Scopigno et al, 2015)
3DHOP2, a JavaScript library that provides a web interface where to load meshes with an order of magnitude of tens of millions of polygons across the web, with no significant wait (Callieri, 2014).
2.5 - Unity3D and the Human Heritage digitization
Of all the project related to the VH (Granström, 2013) developed in recent years, a significant percentage of them used Unity3D to build a user-oriented application. While being a recent software that is still trying to implement state-of-the-art rendering and lighting, it already appears to be the best choice available on the market because of its simple and intuitive interface and its huge extensibility. Unity is the perfect tool for a broad spectrum of circumstances, and it is not an overstatement to say that no other engine could offer the same features with such a simple approach. In regards of this matter, the technology news and media network The Verge wrote that “Where Epic has worked on creating state-of-the-art graphics, Unity started with the goal of making game development universally accessible. Since then, they've been moving towards each other. And Unity 5 is a long-awaited step towards that future”3. In 2014 Wessels enforced this impression writing that:
“all the other engines were found to either be complex to use or limited in their
ability, which lead to the choice of Unity 3D as the enabling platform. Also the Unity 3D community has 2.5 million registered developers and 500 000 active developers as of March 25th 2014, which would indicate that it is well accepted in the Virtual Reality community”(Wessels 2014).
26-Unity has also shown to be the best option when building web-based application, as it allows users to build a virtual application with its engine while exporting the contents to a non invasive HTML page that requires no plugin to be executed (Arends, 2010). While many powerful libraries such as BabylonJS and three.js can reproduce 3d contents on browser, none of those has a powerful editor such as Unity, making it the best options for non-expert programmers. A deeper analysis of Unity and its main competitors will be performed in section 4.1.
Developing a VH-related application significantly inherits from the standard videogame development pipeline, and face the same challenges that game makers have faced in years of work. One of the advantages of adopting this technology many years after it was first introduced is that many problems that could affect the development of a Virtual Museum have already been faced and solved by other disciplines. One of them, as reported
Fig. 7. Raster height-map of Fort Ross area (left), blank terrain (center), and painted terrain in Unity (right)
. (Lercari, 2013)
Fig. 8. Nine cameras system on the left, as seen in Unity3D, and the corresponding AR view on the
by Magrini, is the translation of acquired data from dense point clouds to usable textured meshes (Magrini, 2016). As reported by the same article, a solution to save performance is to translate the point cloud to create a height-map that will be used by Unity terrain generator to recreate the area (Fig. 7). In 2012 Another Unity feature used by many projects is the ability to load and visualize different levels of details (LOD) according with the camera position, so that models can be loaded in real time, granting the best overall mesh to those that are closer to the viewer (Merlo, 2012). Unity was also chosen by many other projects for other features, such as its Animation interface (Neto, 2011), its dynamic textures and light effects (Magrini, 2016), its standard assets (Kontogianni, 2015), and even its active support given by the community (Wessels, 2014). In 2015 the Culturama project pushed Unity even forward, building a plugin containing an advanced AR system that rendered nine cameras together on a big-screen (Fig. 8), with fully implemented mouse movements and no pixel overlapping. (Farouk, 2015). A perfect summary of Unity potentials can be found in (Wessels 2014):
"Unity 3D was found to be a very suitable and flexible software for the creation of a Virtual Tours. This project is on-going and it is proposed to add functionality to the tour to facilitate scientific analysis, interactive learning-oriented games, virtual tour guides, multiple users, and other applications.”
3 – QUESTIONNAIRE
Curators are usually responsible for organizing exhibitions, but this operation is very complex and requires to handle numerous subtasks. While preparing a public space,
spots with the risk of being damaged by accidents, and external figures such as architects must be involved in the process to ensure the feasibility of the project. Making curators able to “play” with their own exhibitions before to built them in a real space, to reconstruct their spaces inside a computer and to explore them, could be crucial to improve the final product overall quality while dropping the costs, minimizing the staging creation time, and making the curators more independent from the other professional figures.
During the first project drafting phase a wide range of possible features was listed, but being the authors main background not from the museology discipline, it was only a broad guessing based on what it was imagined to be useful for curators, it was therefore important to prioritize that list in terms of usefulness to the aimed audience. After some informal chats with several students coming from related fields such as history of art, architecture and cultural heritage preservation, the number of features was reduced to a shorter list that could be reasonable both in terms of development time and usefulness, and a survey draft was created. Before asking people to take part in the survey, we took advantage of the extreme courtesy of Dr. Caterina Bay, curator at San Matteo museum in Pisa, Italy. On what was a very interesting interview, we discussed in depth the role of curators in modern museums, their daily tasks, the tools they tend to use and the digital supports they would like to have.
The impression coming from this chat was that, even though the San Matteo museum had several digital kiosks and a fair number of interactive points of interest created with the supervision or Dr Bay, the interviewed subject was not familiar at all with the technologies used on those projects and had no idea of what other tools could have been used to further expand those implementations. It was therefore important to expand the questionnaire to explore any possible correlation between the role of curator and the
new emerging technologies.
3.1 – Questionnaire structure
Being the purpose of the project to create a suite of tools to help museums curators to develop and fine-tune their digital spaces, it was important to have an estimate of the average curator’s ability with computers, what was their experience with Virtual Museums and Virtual Reality in general, what they would expect from such a tool, if they thought it could be useful to improve their daily tasks and if they would be willing to use it in their daily activity.
The questionnaire was subdivided in six different sections, each one containing a list of multiple choices questions and, in some cases, a non-mandatory open question regarding specific fields. This solution was adopted for two main reasons, the first one being the ability to perform better statistical analysis on schematic results, and the second one being those types of questions faster to answer. Subjects were informed that the average time to complete the survey was estimated between 8 and 10 minutes, according to three tests performed by volunteers just before making the questionnaire public.
Section number one, named “Personal data”, was composed of four multiple choices questions regarding personal data of the interviewed subject such as gender, age, qualification, and how many years he had spent working with museums and exhibitions. By keeping the personal questions to a very small number and as open as possible, it was easier to earn people’s trust and get them to continue the survey, yet these questions were important not only to create a general profile of modern museums’ curators, but also helped understanding museums willingness to invest in new staff and what degree of instruction
they expect their employees to have.
Section number two, named “Institution info” was composed of 8 mixed-type questions and it was aimed to categorize the institutions in which the interviewed subjects worked and what relationship with modern technologies did this institution have. The first three question were aimed to understand what type of contents the institution was interested in, dividing them by historical period, type of displayed contents and type of museums. To avoid data sparseness, users were allowed to choose multiple answers from predefined lists, with a general “Other” entry at the end of each one of them: the historical period was divided in eight periods, going from Ancient Rome to Modern times; the contents’ types were divided in thirteen categories, including everyday life tools, statues, paintings, religious relics, books, minerals and some others; the category question was divided in six different categories, named “Archaeology”, “Science”, “History”, “Food and Drink”, “War”, and “Art Gallery”. The fourth question asked the interviewed subjects how big they believed their institution was, making them choose between five answers going from small to big. The fifth question was an open question and asked what position they did currently hold in their institution. The sixth and seventh questions were simple yes-no questions, and asked subjects if their institutions offered a virtual tour on the web and if it did offer any type of virtual experience, through mobile applications, digital screens or video projections. The eight and last question was aimed to understand how common it is for modern museums to have an IT team and asked how many technologies experts the institution currently involves, going from none to 5+.
Section number three, named “Previous Experiences with technology VR \ AR” was introduced after meeting Dr. Bay and it was aimed to understand curators’ personal
level of familiarity with some of the most common modern technologies that could be used to create new types of Virtual Museums, and if their institutions happen to use any of those. The first two questions asked the subjects to rate their experience from 1 to 10 with several broad categories of tools, named “Professional game Engines”, “Amateur game Engines”, “3D modelling tools”, “Graphics tools”, “Web-specific scripting languages” and “Laser scanners”, once according to their own personal experience, and once for what they believed the use of those softwares was in their institutions. An open question followed, asking if there was any other software their institution was making heavy use of that was not listed above. The section concluded with another rating question, subjects were asked to rate from 1 to 10 their personal experience with modern VR devices, including the most common VR headsets, fixed-positions touch-screens and CAVE.
Section number four, named “Museum Experiences”, was intended to investigate the personal relationship of curators with museums in general and not only in relation to the institution they are working with. This section had no particular reference to the curatorial job and could have been compiled by a broader audience. The first question asked when was the last time interviewed subjects visited a museum website, allowing them to choose between five different answers going from “last week” to “before last year”. The second question asked for what purpose the interviewed subjects usually check museum websites and allowed them to select multiple answers from a list that included “Checking the opening hours”, “checking tickets prices”, “Exploring the collection” and others. The third and fourth question were simple yes-no questions and asked respectively if the subjects would be willing to pay to visit a virtual museum, and if they would be willing to use their personal smartphone to improve the visit. A non mandatory question followed, asking them what could make a visitor experience bad or unpleasant according to
their own personal experience. The last section asked to rate how much they agreed with a series of definitions of museums on a scale going from 1 to 10. The full list of sentences can be found on appendix I.
Section number five, named “Organizing Exhibitions”, was the first section to introduce questions that were strictly related to the activity of curators and the practical organization of an exhibition, and it was aimed to understand what duties do curators have in their daily occupation and how they approach the creation of a new space. The first question asked users to rate how much time their institution usually spend in a list of activities on a scale from 1 to 10, including “Deciding what to exhibit”, “Writing labels and descriptions”, “where to put Artificial Lights” and several others, followed by an open question that asked to list any other time consuming activity they could think of. A similar set of questions followed, asking users to use the rating scale to evaluate how much money their institution spend in the same activities listed above. An open question followed, asking users to list any other possible expensive activity for their institution. Questions three and four asked the interviewed subject to rate from 1 to 5 how would they feel about creating a dedicated mobile application to visit their exhibitions and how would they feel about creating a dedicated website with digital asset for their exhibitions. The last question asked users to rate from 1 to 10 the importance of several features while planning an exhibition. Those features will be described in section 4.4.
Section number six of the questionnaire was named “Project-Related Questions”, and it was used to relate curators’ needs to the suite development by asking what kind of features they would expect to find in such a tool. The first question asked users to rate from 1 to 10 the importance of the same features mentioned in the previous paragraph, followed
by an open question that asked what other features they would like to see implemented. An identical structure was used on the following question while asking users to rate a list of features that are not only related to the planning of an exhibition, but that could be considered useful to enhance the creation of a Virtual Museum not only for testing purposes, but also to commercialize it. A last open question asked users to suggest a non-listed feature they would like to have.
3.2 - Reaching an invisible audience
Reaching a reasonable number of answers for the survey has been a harder task than expected. Museum staff nowadays is composed of a rather small number of curators, and they tend not to be mentioned directly by any museum website. It was therefore hard to get in touch with enough people to obtain a significant number of answers. After a few rounds of emails sent to Italian curators, the number of answers was limited to twenty-one.
To increment the number of answers a stronger approach was needed, it was therefore decided to systematically parse museums websites from all over the world looking for curators contacts, and sending each one of them an email asking to fill the questionnaire. This procedure brought to more significant results: searching for a country's name on Google, followed by the keyword "museums", returned a list of around forty museums ordered by importance. Manually checking every single museum website was a tedious and time-consuming operation, most of the websites did not have any staff list and only provided simple contact forms, but at the end of the day it brought to significant results. After sending more than 350 to museum curators all over the world, the number of answers raised to 63. Considered the very peculiar type of audience I was aiming at, the obtained sample size was considered significant enough to proceed with statistical analysis.
It is important to notice that many museums curators replied directly to us before actually completing the survey, wondering if we really wanted them to answer those "technical questions" or if we wanted them to forward it to the IT office. In hindsight the idea of asking how interviewed subjects were familiar with some specific technologies has been misleading to them, and probably caused some of the receiver to drop the survey before actually completing it. The results obtained by those who answered, as discussed in section 3.4, still shown relevant data, but it is logical to assume that the number of answers could have been higher if section five and six were positioned right after section two, or by adding a disclaimer before the actual questions. However, only by receiving these emails the impression had while speaking with museum curators in person was confirmed.
3.3 - Results
The average audience who took part to the survey represented a well-distributed heterogeneous sample. Out of the 63 people who answered, the 55 percent of them was female, only the 11 percent was younger than 30 years old, and with only one exception, all of them had at least an Undergraduate degree. More than the 60 percent of the interviewed had a higher qualification than a masted degree, with the 22 percent of them having completed a PhD.
Section two has shown that the relative majority of the answers came from people who works in institution having a high interest in Arts, with the 46 percent of them listing their institution as “Art gallery”. This tendency was confirmed by “Paintings” being the type of content museums are more interested in, and “Modern Times” being the historical period of interest that had most preferences. Surprisingly, ancient times appeared to be the less interesting period, receiving on average less than 25 percent of preferences amongst
the reached institutions. This data is particularly significant as more than 50 percent of the participants listed their institution as above average or big, in fact it would be expected from big institutions to be interested in different kinds of contents coming from
different periods and not to be focused on only one specific category. Interesting results came out from the other two questions: 68 percent of the interviewed belonged to institutions that do not offer any virtual tour on the web, even though the 70 percent of them positively answered when asked if their institution offered any virtual experience in general (Fig. 9). This result indicates that currently museums are still investing in virtual technologies they can only by used within the tour and do not prefer to invest in web technologies.
Section three certified the total detachment of museum curators in regards to Virtual Environments and any possible software linked to three-dimensional objects. Over 80 percent of them have indicated that from 1 to 10, their personal experience with game engines or modelling tools was 1, with more homogeneous and equally distributed answers for the Graphic Tools. Above 80 percent of answers indicated a level of knowledge of web-specific technologies below 5, with a peak of 44.5% of 1. Similar results were obtained in relation to the institution interest in the same fields, with the only exception of the graphic tools, rated 8 or above more than 50 percent of the times, and around 50 percent of preferences going from 6 to 10 in regards to web specific languages. Personal experience with all the listed VR headsets was rated 1 for each one of them more than 85 of times,
Fig. 9, Number of institutions that offer a tour on the web
on the left, and number of institutions that offer virtual experiences on the right.
with the exception of Google Cardboards that was unexpectedly indicated with 1 only 66 percent of the times.
From section four it is important to evidence how 73 percent of the curators expressed a negative opinion in having to pay any money to visit a virtual museum, while 84 percent of them would be willing to use their own personal hand-held devices while visiting an exhibition (Fig.10). When asked to list what could make a visit unpleasant, the most cited problems were the feeling of unsuitability visitors can have in front of non-intuitive interfaces, technical labels they cannot fully understand and linguistic barriers. Many aspects related to the museum’s organization were also listed, with rude staff, unclear paths to follow, excessive noise and overcrowded spaces amongst the most prominent problems. When asked to rate their level of agreement with certain definitions of museums, contrasting and ambiguous answers were given. If above 95 percent of them agreed in considering a museum “an institution that collects, preservers and displays objects”, above 50 percent of them did not consider museums bounded to physical objects. The vast majority of subjects has shown an overall appreciation of Virtual Museums as communicative experiences, with 70 percent of them being mildly positive in
considering them a way to enhance the perception of the world around us.
Fig. 10, Number curators willing to pay to visit a Virtual
museum on the left, and number of curators willing to use a personal smartphone on the right.
Fig. 11. Number of answers to the question “How much time does
The results of section five have been very homogeneous and not very indicative. 10 has been the most selected answer for every listed activity, even though positioning artificial lights has not been considered a time-consuming activity by over 30 percent of the subjects. On the other side, answers of question two followed a Gaussian distribution, with 5 being often the most common answer (Fig. 11). Almost all the interviewed subjects have shown a very high interest in creating digital contents for their exhibitions, and when asked to rate the importance of certain elements while planning their exhibitions, all answers but one have received a majority of votes of 10, with the exception of the sun position.
Results in section six followed those of section five, all listed features have received a relative best score
of 10 every time, with the exception of the sun position. However, the possibility to recreate the position of a given windows has been considered not a priority by many users (Fig. 12), who specified in the dedicated section how windows tend to be plugged when possible. The second question has given much more interesting results: 34 percent of curators would not use the possibility of making people pay to visit a virtual museums (76 percent from 1 to 5) and they would not use any automated system to parse information from untrusted sources to speed up the creation of
Fig. 12 How curators rated the importance of recreating windows
positions.
Fig. 13 Curators interested in Human-looking avatar guides for the
contents. They were also against the possibility of a human-looking avatar used as museum guide through the exhibition (25 percent voted 1, 69 percent from 1 to 5, Fig. 13), while many researches have promoted them as much attractive as real human figures for a general public. A navigation system to guide the users and implementing touch screen were instead considered useful by most of the interviewed subjects.
3.4 - Comments and Discussion
On average, the answers followed the author’s expectations. Most of the curators, regardless of their age, gender, role or previous experiences have expressed a total detachment to new technologies, demonstrating how museums and temporary exhibitions tend not to involve any technology expert while planning exhibitions, even when they could. The profile of current curators coming out from section one is representative of an individual who has a high degree of instruction and at least ten years of experience working with museums and exhibitions, but that has no idea whatsoever of what kind of possibilities new technologies, especially VR-related ones, could offer to their work. Some of them have shown a previous experience with graphics tools such as Gimp or Photoshop, but unfortunately there is no way to know where these skills were learned or used.
From section two it is curious to notice that many people believed that web-specific languages were not in use at all in their institution. In fact, almost 49 percent of them stated that their institutions do not make use of those languages very often (from 1 to 5), while all contacts were found on new up-to-date websites. It is logical to assume that many of these people are not fully involved in the process of organizing a museum, and the answers they provided while considering their institution were somehow restricted to their office or department, and not the general institution organization.
If the results given by section three, in regards to VR headsets could be predictable, a better score of Google Cardboards in favour of the more famous Oculus rift were not. While still being absolutely low, almost 15 percent of people have answered with a value of 6 or more in regards of the Cardboards, with the top 8 percent of them scoring over 8, less than 10 percent of them has actually positively rated their experience with the Oculus Rift and none of them has expressed a higher score than 7. Also the very low score of fixed-position touch screens was not foreseen, considering that most of the users did specify that their institutions provide a virtual experience which is not web-related it is logical to wonder what kind of experiences those curators were referring to. Moreover, those types of displays are pretty common everywhere, from airports to fast-foods, the most reasonable explanation for this result, besides the incongruence with section two, is that users calibrated their answers only on their museum experiences, and not their everyday life. If this was the case, the reliability of this data should be matter of investigation.
Results of section four have offered some interesting insights on how curators consider new technologies, especially if compared with those from previous sections. 95 percent of the interviewed subjects have visited a museum website up to seven days before compiling the questionnaire, with the purpose of exploring exhibitions almost 100 percent of times. Curiously, the majority of them would not pay to access virtual museums, even though at least 84 percent of them were positive about using their own device to enhance the visiting experience. It is important not to lose the focus on the audience here, the answers were not provided by general public but by those who curate museums spaces, it is therefore significant to notice how the reluctance towards new technologies was reflected
in considering virtual museums not worth of any expense. This tendency is somehow contrasted by how the users evaluated the sentence “Museums are bounded to physical
objects, there can’t be any museum without artefacts", as 50 percent of them did not agree
with it. According to the data, museums are not bounded to physical objects, and curators are more than willing to use their own devices to enhance their experience, but still they would not pay any money for that kind of experience. A hint to understand this incongruence comes from the other definition of museums on which the interviewed subjects agreed more: "museum is a permanent institution in the service of society and its
development, open to the public, which acquires, conserves, researches, communicates and exhibits the tangible and intangible heritage of humanity and its environment for the purposes of education, study and enjoyment". It is possible that users did not believe that
culture should be subject to costs and therefore it should be freely shared between museums and audience, but these are mere considerations, and unfortunately the amount of data collected does not offer any further analysis on the matter.
Results from section five and six, which were supposed to be the most important in regards to the project’s design failed to gather any significant data. All listed features and situations received a score of 10 in almost all the questions, and no significant answers were given to prioritize the working schedule. Yet another apparent contradiction appeared from the data: many of the interviewed subjects listed marketing and advertising as two of the most expensive activities for their institutions, but despite the urge of collecting money, as they have said in some open answers, the overall majority of them believed that giving potential Muse-Tools users the possibility to make their audience pay a ticket to visit a virtual museum was not worth it. Once again the ambiguity to consider a virtual museum a positive enhanced experience that is not worth any money from the general public appears
with no intuitive explanation. Also, the other poorly rated feature, the possibility to have a human-looking avatar to guide guests through the exhibition contrasts with the problem of rude or unprepared staff that was so often considered as major problem in making a visitor’s experience unpleasant and with the data provided in section 2.2.
While the purpose of collecting useful data to plan the project’s development cannot be considered as completely fulfilled, more interesting considerations about how curators would be willing to interact with such an application can be made. All the features that somehow expanded the physical bounds of museums were poorly rated, even when they could be the answer to specific problems not only for curators but for the general audience as well. Curators seemed to be interested only in the possibility to recreate real spaces, with no enhanced effects or surrealistic feature that can not have a corresponding counterpart in the real world, it would be good for them to be able to simulate spaces, interact with the scenery and the exhibition to improve their models and ideas, but every other further use was perceived as unnecessary, not worth a cost for the users, and most importantly not the solution for real problems museums have. Also, there seems to be an overall tendency not to be updated by the state of the art of newer technologies applied to virtual museums, many answers contrasted with what was instead predicted by most recent publications.
4 – MUSE-TOOLS
Creating a tool for a specific audience presented some specific challenges related to the peculiar job those people do: as it was shown by the survey, curators are still reluctant to adopt new technologies in their daily workflow, and to ensure the project’s success it
comes to simplification, the risk of over-doing it is always present, and being the objective of this project to be functional for both old-generation curators, such as those who are representative of the traced profile above, as much as technology experts working in museums and people that had familiarity with computers and modern technologies, it was extremely important while projecting the suite to give enough importance to have the simplest possible User Interface (UI) without taking any possible functionality away.
Following the data coming from the questionnaire, it was evident how curators needed the highest possible degree of simulation for their spaces in terms of measurements, materials and lighting, and it was important to put users in the condition to create realistic lighting models submerged inside a believable context, with precise measurements to be used when building an exposition, without asking them to do too much work. To answer all these needs, the suite was designed to be used in two very distinguished contexts: on a side editor tools needed to be designed to provide curators to experiment their own exhibitions in their virtual spaces with the highest level of realism and with no interest in releasing it as a proper user-oriented experience, and on the other side the exact opposite situation was also taken into account, giving users the possibility not only to build their spaces, but also to release their project for the general public. It was also important to make each single feature as optional, giving the user total freedom to decide what they wanted to use from the suite.
It was during this preliminary phase that an external platform needed to be found, something that allowed the suite to implement all the required features with the smallest efforts keeping some minimum quality standards at the same time, such as running on as many platform as possible, both in development that in release phase, a reasonable weight in terms of disk space and memory consumption, and to be expanded by custom scripting.
project, but with the declared purpose to build not only a small dissertation project with no practical uses and to create a real tool with possible applications in real contexts, it was very clear that given the project specifications listed above and the analysis done in section 2.3, scripting the tool from scratch would have taken an unrealistic amount of time. The error of starting a project from scratch, with the presumption of making something good for the community, is too often made in the scholar world. Many projects start and die after being the subject of some publications when their creators decide to migrate on new projects and stop to update the old ones. Experimenting and sharing new solutions is important for humanities research now more than ever, and the costs of starting a project from the bottom inside institutional walls are too often underrated. The results are often good piece of software that perform good in a specific situation but fail to integrate with the surrounding world, condemning the project to a certain end. The best solution to reduce the amount of time needed to produce satisfying results was therefore to integrate the suite with some other existing software that could provide enough quality while respecting all the requirements listed above. Even when the authors will move to a new project, in case of interest the community will always be able to keep the project updated by modifying its source code.
4.1 - Unity, Unreal, and the duality Videogames -Museums
Considering the list of features that emerged during the planning phase, it was extremely important to find a software that could help the developers not to “reinvent the wheel” by implementing basic rendering functions and a simple assets importers, features that will be necessary for the project but that would also take months to implement. The ideal solution was offered by two of the biggest game-engine companies in the world, Unity Technologies and Epic Games, that within a period of just two days of distance from
each other, in March 2015 decided to change their business plan in favour of a revenue-based model, making their tools free to use for everyone, including institutions, as long as the projects they are used on do not reach a certain amount of revenue. With the purpose to extend the default behaviour of any chosen tool and being the project absolutely revenue-free, the possibility to build Muse-Tools around one of those was considered the one that better suited the project’s needs. The decision to look at modern game engines allowed the project to respect minimum quality standards in terms of visual appearance and distribution, with a minimal complexity increment for the final users. With no additional costs it was possible to start working with tools that already allowed complex productions with few functions, and the time used to learn those engines was significantly lower of what would have taken to develop similar simplified features from scratch.
The duality between videogames and instruction is a relatively recent topic, but many studies are already showing how interactive and recreational environments could be exploited to improve people’s learning processes. With the urge to be rebuilt modern museums structures’ to fit a modern audience, the use of new technologies is a needing that will become a priority in the next years, and refusing to take inspiration from what has proved to be not only an incredibly catchy activity but also a possible informative source for people could influence the gap between the new generations and culture. There is therefore no shame to take advantage of a well-established market to explore the possibilities that modern technologies offer to further improve the way human heritage is perceived. Moreover, modern videogames, regardless of the platform they are built for, provide a visual standard that will be unavoidably used as comparison for whatever virtual space museums could be able to build. Besides, the videogames industry can provide solutions to problems that will inevitably faced when developing virtual applications for
museums. By adopting the same standards and software of modern videogames it will be easier and cheaper for museums to find and hire competent staff, as they will not need to put any resource teaching them how to use any internal software. In other words, regardless of the concept of gamification and serious gaming, the videogames industry can offer museums those tools that they so desperately need.
In a market where a limited number of choices is possible, it was easy to restrict the list of possible candidates for the project. From an initial list containing all the game engines that were available on market at time, all of those that required a subscription fee were immediately discarded, followed by those that did not provide any 3D feature. Amongst these, the only two engines that demonstrated a continuous support, an active community, an extensive documentation, state-of-the-art lighting, shading and rendering, possibility to use non custom programming languages, and most important the possibility to extend the default editor to integrate custom functionalities have been Unity, developed by Unity Technologies, and Unreal Engine (UE), developed by Epic Games. Using this second engine was the first choice to be explored, as it was already in use in the laboratory where Muse-Tools was developed, and it seemed to have a better reputation amongst the experts. However, it very soon revealed a series of criticalities that could affect the final product. First and most important, its interface is extremely complex and counter-intuitive, and despite the fact that it is advertised as “fully extensible”, there is no actual way to shrink the default UI the engine comes with if not by compiling its source from scratch and redistributing the entire engine. Moreover, the C++ version in use by the engine is heavily customized and custom classes are used to handle even the simplest things such as pointers, making its learning curve exponentially higher for those who would try to use it alongside the suite.
