Assignment #4: Augmented and Virtual Reality

 Odds are you’ve heard of augmented and virtual reality. Maybe you know what virtual reality is because of gaming consoles, but you’re not sure what augmented reality is. In this post, we will be explaining both terms so that you can understand what each term means and the difference between them.

Augmented reality is a version of the real physical, real-world reality, whose elements are enhanced (or augmented) by computer-generated or real-world extracted sensory input. This can be audio, video, graphics or haptics.^[1]

Virtual reality, on the other hand, is a computer-generated scenario that simulates a realistic experience.^[2]

In summary, the main difference between them is that virtual reality creates a new environment that resembles reality, whereas augmented reality adds digital elements to a real environment.

A brief history of augmented reality:

• 1962: Morton Heilig created the Sensorama, a multi-sensory machine shaped like an arcade video game. There were five short films, a stereoscopic color display, fans to provide wind, odor emitters, stereo-sound system, and a moving chair to simulate motion. However, the Sensorama was not interactive.^[3] Here is an interview with Morton Heilig.
• 1968: Ivan Sutherland and Bob Sproull created The Sword of Damocles, the first virtual reality head-mounted display system that connected to a computer.
• 1974: Myron created the Videoplace, which projected onscreen silhouettes and allowed users to interact with its content.
• 1990: Tom Caudell coined the term “Augmented Reality.”
• 1992: Louis Rosenberg created the first augmented reality system, which was a Virtual fixtures system for the Air Force.^[4]
• 1999: The NASA X-38 spacecraft used a Hybrid Synthetic Vision system that overlaid map data using augmented reality.
• 2013: Car manufacturers began to use augmented reality as vehicle service manuals.
• 2014: Google announced Google Glass devices.

A brief history of virtual reality:

• 1962 and 1968: See above. The history of augmented and virtual reality intercepted at the beginning.
• 1987: Jaron Lanier coined the term “Virtual Reality.”
• 1991: Virtuality Group launched VR arcade games.
• 1993: Sega announced the Sega VR headset for the Sega Genesis, but it never left the prototype phase.
• 1995: Nintendo released Virtual Boy, the first portable gaming console with 3D graphics. It was a commercial failure.
• 2007: Google launched Street View.
• 2014: Sony announced Project Morpheus, which debuted the PlayStation 4 VR system.

Description of each technology:

Augmented reality: Augmented reality technology can work in any of three ways: SLAM (Simultaneous Localization and Mapping), recognition based augmented reality or location based augmented reality. New Gen Apps says the following about these technologies:

About SLAM:

“SLAM simultaneously localizes sensors with respect to their surroundings, while at the same time mapping the structure of the environment.

SLAM is an approach to solve complex AR simulation problems and is not any specific algorithm or software. The SLAM system is, in fact, a set of algorithms aimed at solving simultaneous localization and mapping problem. This can be done in multiple ways and now every augmented reality development kithas its upon to providing SLAM functionality.”

About recognition based augmented reality:

“Recognition (or marker) based augmented reality uses a camera to identify visual markers or objects, such as a QR/2D code or natural feature tracking (NFT) markers, to showcase an overlay only when the marker is sensed by the device. Marker-based AR technology depends upon device camera to distinguish a marker from other real-world objects.”

About location based augmented reality:

“Location-based AR relies on a GPS, digital compass, velocity meter, or accelerometer to provide data about the location and the augmented reality visualizations are activated based on these inputs. It is also known as markerless augmented reality. The location detection features in smartphones make it easy to leverage this type of augmented reality technology. Some common uses of location-based AR include mapping directions, finding nearby services, and other location-centric mobile apps.”

Virtual reality: An article by Tomasz Mazuryk and Michael Gervautz says:

“The user is equipped with a head mounted display, tracker and optionally a manipulation device. As the human performs actions like walking, head rotating, data describing his/her behavior is fed to the computer from the input devices. The computer processes the information in real-time and generates appropriate feedback that is passed back to the user by means of output displays.
In general: input devices are responsible for interaction, output devices for the feeling of immersion and software for a proper control and synchronization of the whole environment.”

An illustration of the technology is available on page 14 of the article.

 Augmented reality apps:

• GIPHY World: AR GIF Stickers: This app lets you add GIF stickers to your videos as you record them.
• IKEA Place: This app lets you visualize what IKEA furniture would look like in your home before you purchase it.
• Housecraft: Similar to IKEA place, but it has more than just IKEA furniture.
• Tap Measure – AR utility: This app measures objects, lets you create 3D floor plans, makes sure picture frames are level and lets you export your files to SketchUp.
• Plane Finder – Flight Tracker: This app lets you track planes in real time.

Virtual reality apps:

• Google Arts & Culture: This app has many functions. The VR aspect of it is that it lets you take virtual trips to famous museums around the world.
• Discovery VR: This app has VR videos of different places around the world.
• Sisters: A VR Ghost Story: This is a horror VR game.
• InCell VR (Cardboard): This is a biology VR game that takes place inside of a human cell.
• NYT VR – New York Times: The New York Times tells stories through virtual reality.
• VR Roller Coaster: Experience being on a roller coaster without actually being on one.

Academic references:

• Characterizing first and third person viewpoints and their alternation for embodied interaction in virtual reality

According to the paper, sense of embodiment comes from the sense of agency, the sense of body ownership and self-location. The authors claim that sense of embodiment is more malleable than people assume. The paper’s purpose is to use virtual reality to experiment with people’s sense of embodiment.

Galvan Debarba H, Bovet S, Salomon R, Blanke O, Herbelin B, Boulic R (2017) Characterizing first and third person viewpoints and their alternation for embodied interaction in virtual reality. PLoS ONE 12(12): e0190109. https://doi.org/10.1371/journal.pone.0190109

• Missing depth cues in virtual reality limit performance and quality of three dimensional reaching movements

This paper is not so different to the one above, in that it deals with the ways virtual reality may limit the perception of a person’s target, thus affecting movement planning and execution. The authors conclude that head-mounted displays provide more perception and depth than 2D screen setups.

Gerig N, Mayo J, Baur K, Wittmann F, Riener R, Wolf P (2018) Missing depth cues in virtual reality limit performance and quality of three dimensional reaching movements. PLoS ONE 13(1): e0189275. https://doi.org/10.1371/journal.pone.0189275

• Comparing virtual and location-based augmented reality mobile learning: emotions and learning outcomes

This study tested learners’ ability to learn about historical differences between past and present historical locations through two mobile AR apps in a laboratory and outdoors.

Harley, J., Poitras, E., Jarrell, A., Duffy, M., & Lajoie, S. (2016). Comparing virtual and location-based augmented reality mobile learning: emotions and learning outcomes. Educational Technology Research & Development, 64(3), 359-388.

• When the Virtual and Real Worlds Collide: Beginning to Address the Clash Between Real Property Rights and Augmented Reality Location-Based Technologies Through a Federal Do-Not-Locate Registry

This paper discusses the legal questions about augmented reality games such as Pokémon GO, and how they fit with modern property laws.

• The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

This paper talks about sensory cues required to perform different tasks in virtual reality environments. They explain that tactile and auditory cues are particularly helpful, especially when someone has to reach out and grab a virtual object.

Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., & Meyer, G. (2018). The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. Plos ONE, 13(2), 1-25. doi:10.1371/journal.pone.0191846

Websites about virtual or augmented reality:

• WITHIN: WITHIN is full of story-based virtual and augmented reality content from VR creators around the world. They have content such as entertaining fictional tales and informational documentaries. A headset is required to view the content. They’re partners of Apple, Google, Facebook, Sony, HTC, nvidia, Samsung, The New York Times, Universal, and others.
• AumentedReality.Org: AumentedReality.Org is a website that educates about the potential and power of AR; connects the best talent to AR professionals for free advice and a network of mentors; and helps launch AR initiatives. It’s a useful resource for people who are trying to start a career in AR.
• 8i: 8i is a company that creates realistic virtual, augmented and mixed reality experiences. They create human holograms that look and feel real. The main page on their website shows incredible examples of the work they do, including someone boxing, a man modeling clothes, and a woman dancing inside of a donut box.

In the video above, Meron Gribetz, founder and CEO of Meta, gives a TED Talk explains the potential for augmented reality in the future while emphasizing on humans being the operating system or augmented reality.

Here are a few Twitter accounts of VR and/or AR experts:

• Edward Miller is the co-founder and CEO of Scape Technologies, which is a “real-time localization engine allowing AR content and digital services to be tied to specific locations in the real world.^[5]” He has been working in the AR and VR field for 8+ years.

• Meron Gribetz is the founder and CEO of Meta, a company that strives to make augmented reality “an extension of who we are.”

• Emily Savage is a VR/AR specialist and immersive technology program coordinator at Digital Catapult.

• Taylor Freeman is the CEO and co-founder of Upload, which “exists to accelerate the success of the VR/AR industry.”

• Aldis Sipolins is the Head of Virtual Reality & Game Design at IBM Research.