Software and hardware used:

• Unity 3D - software
• Vuforia – plug-in
• Macbook

Original Idea:

The challenge I wanted to face was that, since I am close to applying for jobs after school, there is a drastic need to both sell and differentiate myself from the crowd. The issue with this is that since everyone will have a resume and cover letter, the method of how you convey that you’re the best candidate is almost identical to other people hoping to win that position. By working in a field that is all about communication, advertising, it’s come to my attention that if you can show not only your work but that you are able to build new methods of communications that circumvent issues like this than you immediately can stand out from the crowd.

This was why I wanted to create an augmented reality resume. The experience would be as follows, during an interview or while connecting with a potential employer, I would show them my resume. Upon telling them that I had a film reel or have done some photo restoration, instead of bringing up the content on my phone, I would point my camera at the resume. Using an app I’d built, multiple interactive prompts would pop up in correspondence to specific sections of my resume such as my past work, the web address to my reel or my contact information. If they pressed any of these prompts on the resume, a 3-d object would render from the sheet of paper (seen via the phone) showing my work with another prompt available to go back to the main menu.

I accepted that, with little programing experience, that I might not be able to make it interactive but I was intending to tell a story though other means.

Attempt 1: Follow one target and automatically play a video that is a texture tracking that image.

• Progress
  • Made target and was able to have it tracked by the program
  • Video was easily made into a texture that would appear when target was tracked
• Issues
  • Texture the video appeared on had to be scaled as to not warp/distort the video
  • The playback quality was not ideal

The intent of this objective was to have the program recognize and track one embedded target, and then render a texture on it that would play my reel. This was successful, with the video easily attaching to a plane that was sized appropriately as to be the same size of the video.

There were two issue I wasn’t able to remedy. The first was the quality of the playback that would usually have some issues and make the video appear to be low quality or the video would glitch entirely. This is most likely due to the size of the video file and the processing that Unity is going through as to track the target, render the objects and play the video all at the same time. I tinkered around but had found no immediate solution.

Attempt 2: Make it interactive

• Progress
  • Found that you can use and connect playmaker to the augmented reality assets
  • Buttons come with Vuforia
• Issues
  • Lack of programing experience
  • Tutorials/code were made for past versions of Unity
  • Could not connect with buttons with objects or commands

This was my foray into the programing side of things when it came to the project. The goal was to simply add one button that would play and pause the video and from there I was going to expand the number of buttons and screens. Though it is technically possible and the plug in I used, Vuforia, has pre programed buttons in its tools, they do require coding to execute prompts and all the tutorials and sample code I could find were made years ago. Due to Unity updating itself every year this code was usually outdated or would bring up issues. This got so bad that at one point I had to scrap the first project and start a whole new one due to how many issues I was receiving from this old code.

On the bright side, it is possible not only to make an augmented reality program interactive via Vuforia and have it work with Playmaker, the issue being that it does require an understanding of how to code in Unity.

Attempt 3: Track multiple targets

• Progress
  • Successfully can have multiple targets on paper, each rendering something different
• Issues
  • There are some issues with tracking

With the interactive angle not possible with the time and experience I had I wanted to try and make this augmented reality resume have a story element. At this point I experimented with having multiple targets, with the whole sheet acting as one big target while each corner would be its own target that would render something different if the camera trained on it alone.

In effect, if you followed the sheet like you would follow a resume you would be following the story of my professional career. This is assuming that the user is following it in the order you intend them too but this is a persistent issue with all interactive forms of storytelling where you have to trust that the user does what you intend them to.

I avoided putting numbers or indicating where you should be looking as this both gives a feeling of the story being inorganic and takes away the point of interactive story telling which is to allow the user to explore. There were some issues with tracking or switching from one target to the next but overall the system worked.

Final thoughts and product:

The biggest take away I’ve had from this project is being able to be flexible when faced with a limited amount of time and experience. Augmented reality as a whole is new to both me and most people and I was a little ambitious with where I wanted to take it for this project. All in and all though I was able to learn the basics of tracking and developing an augmented reality program with multiple targets and present the information from my resume in a new way.