What is Augmented Reality?
Augmented reality is the technology that expands our physical world, adding layers of digital information onto it. Unlike Virtual Reality (VR), AR does not create the whole artificial environments to replace real with a virtual one. AR appears in direct view of an existing environment and adds sounds, videos, graphics to it. (ThinkMobiles Team, 2018)
What Type of Augmented Reality?
Before I can start looking at augmented reality applications I need to determine what type of app I want to build. I was able to find out that there are four types of AR apps.
Marker-Based
Marker-based augmented reality mobile applications are based on image recognition. They use a camera of a portable device to detect certain patterns or markers, such as QR codes or images. Once a pattern is recognized, the app overlays digital information on this marker. The orientation of the AR object depends on the position of the marker. The most advanced augmented reality SDKs support recognition of 3D markers, i.e. real-life objects. (Mykola, 2018)
Markerless AR.
A.k.a. location-based or position-based augmented reality, that utilizes a GPS, a compass, a gyroscope, and an accelerometer to provide data based on user’s location. This data then determines what AR content you find or get in a certain area. With the availability of smartphones this type of AR typically produces maps and directions, nearby businesses info. Applications include events and information, business ads pop-ups, navigation support.
Projection-based AR.
Projecting synthetic light to physical surfaces, and in some cases allows to interact with it. These are the holograms we have all seen in sci-fi movies like Star Wars. It detects user interaction with a projection by its alterations.
Superimposition-based AR.
Replaces the original view with an augmented, fully or partially. Object recognition plays a key role, without it the whole concept is simply impossible. We’ve all seen the example of superimposed augmented reality in IKEA Catalog app, that allows users to place virtual items of their furniture catalogue in their rooms.
Source : https://thinkmobiles.com/blog/what-is-augmented-reality/
Augmented Reality App Research
For this project, I will be using augmented reality to incorporate my 3D Japanese creatures in the real world. I want to achieve this by creating various masks and programme the app to recognise it which causes the model to come to life. As I discussed in the previous post I will not be programming my own app because it will too time-consuming and the app is not actually what I want to focus on for this project. Therefore I will need to find an existing app or programme that will allow me to achieve the ability to import my model.
Zappar
“With Zappar you can turn almost anything (print, POS, product, packaging, place) into an interactive delivery channel serving video, animation, games, competitions, additional information, data capture mechanics, social shares and more. Zappar adds a new visual dimension to the world seen through your device as a digital discovery channel. Making every touchpoint an engaging, measurable, accountable and delightful short-form experience.”
Zappar works by:
“30 times a second the app reads an image from the camera and searches it for little distinctive corners and shapes. Once it’s found a good number of them it compares them with a database of corners and shapes that it knows to be looking for – the distinctive features of the “target” image that it’s going to bring to life.
Matching features let the app know not only that it’s found the target, but also where that target is in the camera image. Armed with this knowledge the app places the 3D experience in that place. ”
Based on the showcase videos the company produces zapcodes (their own QR codes) which the app then recognises and the 3D model appears. As I want the app to recognise the masks itself this app won’t work for my project because I want the mask to be exclusively my design rather than trying to incorporate the QR code into the design. However, if I cannot find an app that can recognise the masks I will have to use Zappar as my back up plan.
Vuforia
“Vuforia is one of the best platforms for augmented reality development, as it offers an impressive set of functions. Let’s go over some of them:
- The Vuforia augmented reality SDK can recognize 2D and 3D objects (such as cylinders, boxes, and consumer products).
- Text recognition. Vuforia has a standard English vocabulary with over 100,000 words, or you can use your own custom vocabulary.
- Video playback. Vuforia can play videos when target surfaces are detected.
- VuMarks are Vuforia’s own barcodes that not only encode data but act as markers as well.
- Local (on-device) or cloud recognition.
- Virtual buttons allow you to use various surfaces as a touch screen.”
Exhibition Test
At the start of the term, I discussed my idea with a previous student. She suggested I download this app made by her friend. The app has the ability to draw objects and anchoring them to a surface. I found this quite informative as it games me an idea of what I need from my chosen AR App. However, with this app, I can import my own models meaning I can’t use it for my project.
PaintSpaceAR
Similar to the test above, this app only allowed me to draw and anchor the paintings onto a surface. The only difference is the ability to bring in an image and creating an AR plane.
AR Studio
With Ar Studio you’re able to import your own models into Sketchfab and use them to anchor it to a surface. You’re also able to access other people work and play around with it. Below is an example of using someone’s model and anchoring it to my laptop keyboard. I want to use this ability within my own app and having my model anchoring to the masks. The problem with this app is that you will need to purchase the pro version if you want to use the 3D recognition. In addition to this, within the app you’re able to resize and turn the model which I don’t want as I want the viewer to walk around the model to study it rather than turning the model via the app.
Vuforia Test
I then went on to researching various ways I can create the AR app without coding it from scratch. I managed to find a tutorial which uses Vuforia’s code within unity to create the AR within the program. By signing up for a developer account I was able to obtain a tutorial code and import an image to use as the scannable image for the model to scan. within the preview, I was able to import a 3D model and once unity recognised the image the model appeared.
I also used different models and anchor it to the same image. This simple test does exactly what I want the program to do. I want it to simply recognise the specific image which causes another specific model to appear. I also discovered that Vuforia not only has the ability to use single image target, they also can use 3D Object targets meaning I can import my 3D Masks into the program and have it recognise the masks itself
The only problem I discovered when experimenting with Unity was the ability to write the app out. I found errors when I tried to render the app into Xcode, I spoke to Ryan about this and he will have a look at the error within the project and help assist me further after assignment 1’s submission.
Animation Project 