The Reality of Augmented Reality
You walk into your local pub, planning to meet friends. You scan the draft beer chalkboard with your handheld – bookmarking the pop-up descriptions for two or three which seem interesting, and order your favorite brew at the bar. Pulling a chair up to an empty table, you use your tablet computer to log in to your poker game, say something rude to the 3D avatar of your friend from the West Coast who has also logged in, have a laugh, and deal a virtual hand. You’ve just had an augmented reality experience which allowed you to make an informed decision about your beverage, and share time with remote friends that went way beyond FaceTime.
In my admittedly short time in this field, I have noticed that the words Augmented Reality and AR are thrown around constantly. They are used to describe a whole spectrum of technologies, which do some very cool things, but are somewhat less than the above ideal. Despite the relative simplicity of some of these examples, we are going to consider them augmented reality because they all augment reality with computer generated content.
The simplest of these technologies is the QR code. I’m sure you’ve seen these codes around – I recently just saw a large one in front of a house for sale, which was intended to be used from a moving car; I’ve even seen them on my tomato based condiments. These codes can be considered augmented reality because they add computer-generated information to the environment, albeit in a very static way. QR codes aren’t fancy, they do their job of pulling up web content or allowing you to post about your location or whatever they’ve been programmed to do, and then they are done. Note that any web content that is pulled up is not overlaid on the current environment – it is simply shown in a browser. This sort of technology works very well for our interactive beer list idea – allowing you to bookmark things for later reference. And, as you will see later on, we may be able to walk away from abstract codes very soon.
A next step towards our ideal poker game can be seen in applications like ARIS and Tagwhat. Both of these are location based – you do not need to scan codes to access the extra information.
ARIS is a platform for augmented reality games where you can interact with simple characters and pick up virtual objects to complete tasks for these characters. In their Dow Day game, reaching certain locations will trigger photos and videos which further the game’s storyline. What makes this less than ideal is that the characters, objects and media are not overlaid on the environment – they are not visible within the environment (through the medium of the smart phone), but instead replace the current environment, which is more like virtual reality. While this may seem to be a picky distinction, one of the great strengths of augmented reality is that it encourages you to engage with your environment more deeply. By replacing the environment with other content, you are pulled more deeply into using your device, rather than engaging with your environment. If we get really persnickety with the ideal, it would be great to move away from hand-held devices and into more hands-free technologies (yes, I’m talking about those silly headsets from the movies!), so that you can still manipulate items in the environment (ie, drink your beer) while you are playing augmented reality poker!
TagWhat, unlike ARIS, overlays their stories on the environment, so that you can see where there are nearby stories – but the content is much less interactive, displaying statically over the environment, directing your attention away from reality. Again, this is useful for things like our interactive beer list, but less so for our poker game.
Two of the coolest new additions to the augmented reality world are the Nintendo 3DS and Layar.
The 3DS has a dual camera system which allows it to display content in 3D over a card (sans QR code) in the environment. I recommend watching the short video in the link to get a feel for what this looks like. I even tweeted about the coolness of their AR about 3 minutes after I finished charging my 3DS. While not the most informative content ever, it does allow you to interact with virtual items which are overlaid on the environment. However, it also requires the use of the AR cards to help position the graphics – which limits how the user can move and interact with the various characters. If the 3DS had GPS capabilities, this would make for a much stronger contender for ideal AR.
Layar does something similar with its display, but in a less interactive way – it has been touted as the harbinger of doom for QR codes. Layar will recognize specific two-dimensional items in the environment and display information when it does so. So far Layar seems to be used similarly to QR codes, by companies trying to leverage social media, get more feedback from customers, or push visitors to their websites, none of which is extremely complicated. However, until GPS can pinpoint our locations to within a few feet (rather than within 30 yards) technology like this or Google Goggles may be the best method for providing more high-level interactions as well.
What is encouraging is that each of these technologies has some of the pieces we need in order to get to where we want to go, or to the extent we can take this. If we mix them together, we are actually not very far from going to the local pub to play cards with friends in other cities – as people becoming more demanding of their augmented experiences, it will be interesting to see how the technologies merge to meet user expectations.
Game Cognito 
This is great: I haven’t seen the conversation put so clearly in this way before. Thank you! A couple points:
*I think QR Codes are object hyperlinking and not AR.
This is a squares-and-rectangles sort of syllogism, but I content that QR codes are object hyperlinking: the connection of physical items to online content. QR codes can, and are, used to create games that augment reality, but QR codes themselves are no more augmented reality than they are transmedia. They’re a tool that can be used for either of those purposes, or other purposes. How about a device that will release a different smell depending on which QR code you scan? Robert Rodriguez should pioneer that.
*I’ve created some ARIS games – have you created any? Or played any?
This is more a brag than anything else. But the ARIS game Greg and I made in Griffith Park is awesome, and the one I did solo in Jack London Square, though flawed, is also good, and has the best inside joke about being mayor of Oakland that anyone ever wrote. If you want to play our LA game, go to tinyurl.com/galacticgrove. If I have a larger point other than trying to get more people to play our game, it’s just to say that there are ways to tie the physical world in with the device that aren’t related to graphics or the camera feature. I think if the content is skillful enough at intertwining the real and virtual worlds, the game takes on properties of augmented reality and, in your words, “encourages you to engage with your environment more deeply.” This is a question of taste, so altogether subjective, but our tastes seem to be in synch. The Galactic Grove game has content outside of ARIS. Nobody has found it yet, and I didn’t take much care in hiding it, but it’s out there. It’s on Yelp actually.
I’ve never used Tagwhat, though, so I’m going to check that out. Looks neat. Do you know about Six To Start’s project Wanderlust? Or have you seen this video: http://sixtostart.com/onetoread/2011/putting-the-game-back-in-args/
* I’ve heard good things about Layar. Have you seen Aurasma?
Anyway, this was great. Thanks. I’m going to share with my father-in-law Jed, whom I hope will add his thoughts.
August 30, 2011 at 5:12 pm
I agree that including QR codes was a bit of a stretch, but I have heard people refer to them as augmented reality, so I wanted to talk about their limitations.
We are looking at ARIS as a platform for an upcoming project, which is what inspired this post, actually. I was impressed with what you could do, but felt like some pieces of it actually detracted from exploration of the environment. However, I think that that was likely an artifact of the example I saw, rather than a limitation of the technology – in other words, it was the game design, and not the platform design. I’m interested in seeing what you’ve done with it – it seems like a really powerful tool.
I will definitely check out Wanderlust and Aurasma – I had not come across them when I was doing my research, but am always looking for cool tools for ARGs and learning experiences.
Thanks for the feedback – good to know I’m not speaking into the void!
August 31, 2011 at 11:36 am