More than a billion people have interacted with Facebook’s Spark AR augmented reality platform. But that might mean that we’ve all had cat faces put on our heads in silly messages.
Some of us are hoping that the technology will be useful for something more than that silliness. But Facebook is working hard on helping the technology grow up. It has launched its Portal smart camera that lets people hold long-distance video calls in their living rooms.
It also extended the Spark AR Studio AR app creation platform to the Windows and Mac platforms. That’s all about making it easier to create AR apps — and hopefully some higher quality apps than the kind we’ve seen so far.
I spoke with Matt Roberts, product manager for Spark AR, at Facebook’s recent F8 conference about the potential of augmented reality.
Here’s an edited transcript of our interview.
Above: Matt Roberts is product manager for Spark AR.
Image Credit: Dean Takahashi
GamesBeat: Spark AR got a lot of airtime during the keynote. What are your views on the coolest things people are doing, as opposed to Snapchat and other things like that? It’s all a bit silly, but I’m curious how you might perceive the differences between really well-done AR and more basic AR.
Matt Roberts: As Mark mentioned yesterday, we’ve had more than a billion people try it in the past 12 months. The vast majority of those people tried it perhaps for the first time, or just the first few times, and it was for a more playful experience like you describe. Having come from the game industry, the power of play and the power of people experimenting with technology through games and playful applications is a natural and necessary part of the evolution of any technology. You see that on the front line of new technologies very often.
I’m encouraged by the fact that it has that much appeal. As far as where we think it can go and what’s next, I think the playfulness and fun are going to continue. Social media and sharing are about connecting with people, expressing yourself, having a good time, sharing jokes, sharing things you like, an affinity for brands. That’s going to continue for quite a long time. It’s going to get better as well, the types of things you can build.
Beyond that we’re very interested in things like shopping and ads, things that help you make better decisions as a consumer using AR. We think that’s a very big strike for the technology. When we talk to advertisers for brands–we’re running an advertising beta right now with AR on Facebook. You can see an ad in your news feed, open the camera, and preview a product there. We’re working with 20 brands on that right now. Things like glasses and makeup and other consumer goods, or brands in media and film.
We think that’s pretty powerful. When we talk to advertisers and brands, they pretty consistently tell us the same thing. One, they’re looking for ways to increase the interaction and engagement and richness of their advertising experience with customers. They want them to spend more time thinking about their products, getting more nuance, more detail. Personalizing it to their needs.
Makeup is a good example of a product that’s classically difficult to buy online, because you’re not actually buying makeup. You’re buying the look the makeup delivers. It’s a powerful difference to be able to try that out virtually versus just seeing the colors on a model. That’s an example of personalization and engagement that’s really different, and AR can enable that.
Advertisers and brands also want to increase customer satisfaction, obviously. People want to have the product they think they’re buying and have it deliver on the expectations they had when they were buying it. We think that for a lot of verticals, especially personalized things, things you can wear, but also physical goods – will this fit in my space? What’s the color, the look, the texture? You can increase satisfaction by having that be more realistic and more interesting.
Above: Facebook’s Spark AR has been used by more than a billion people.
Image Credit: Dean Takahashi
GamesBeat: On makeup, do you think that’s better in AR or VR?
Roberts: I don’t know if it’s better. I would say we’ve seen a lot of—we’re happy with the results when people are trying on makeup. We have a lot of brands interested in doing more with makeup and looks. We’re also expanding our tech to cover new types of styles. You can change your hair color. You can try on jewelry and watches and things like that, have the jewelry with your makeup at the same time. These are the types of combinations that people try in real life, and we want to be able to simulate that.
The last thing we hear from brands, and retailers too, is that they want customers to mix themselves up with their brands. They want people to become brand advocates. A couple of weeks ago at the Game of Thrones season launch, for example, we had a lot of success with a Spark effect on Instagram where people could express their affinity for light or dark or ice and share that on social media. We had a lot of people using that.
It’s a bit like the first case, where you mentioned playfulness, but from a brand perspective, that’s a radically different engagement with a brand than me telling you about it. “I’m gonna watch Game of Thrones.” I’m mixing myself in with the characters, picking a side, showing my point of view creatively. AR is the type of technology that lets you do that. Agencies are chomping at the bit to come up with new experiences like that.
GamesBeat: A lot of that seems to involve tech that you guys were showing down there, like picking up which way your face is looking, so you can put things on the right part.
Roberts: The technology, in the end—we think about it as something we call machine perception and learning. This underlies a lot of different technologies. You may have seen something like the VR demo of Guardian that prevents you from bumping into the walls. At its core, it’s a similar concept. You’re understanding what’s going on in the world and giving the user real-time feedback based on some sort of context that’s relevant.
We have a couple of areas that we want to focus on. First of all, we want to make the performance of that machine perception really fast, really diverse. Our Spark AR runs on more than 1.5 billion devices across Android and iOS. If you’re a new person trying to build AR and thinking about where to invest your time to have the most impact and reach the most people, instead of having to choose the Android path or the iOS path, we hope you can go down the Spark path and distribute on top of our family of apps.
Another thing is the creation itself needs to be streamlined and easy. We want a lot of people using this technology. We think the technology is only as good as the content that people make for it. The types of people who are attracted to the technology will shape the character of what happens.
Above: Facebook Portal head of product management John McCarthy onstage at the F8 developer conference held April 30, 2019 at the McEnery Convention Center in San Jose, California
Image Credit: Khari Johnson / VentureBeat
GamesBeat: Detecting where surfaces are, that’s something that has to happen in real time?
Roberts: We talk about it like world tracking or face tracking. We’re interested in, also, is this a soda can? Is that a couch? Having some understanding of what’s going on in the world is the next level of what we’re calling machine perception.
GamesBeat: I talked to a company that’s doing product placement for movies and TV shows – things like replacing a can with a Coke can if that’s what advertisers want. Where would that kind of tech fit in?
Roberts: Is that virtual product placement, something like that?
GamesBeat: It’s able to do it quickly, but it’s getting baked into things like a TV show. They can do it post-production, changing objects on a set without taking them out physically. A digital object can be there that wasn’t there before.
Roberts: That makes me think about the flexibility and power you have authoring media, free from the constraints of time and physics. You can have something that’s in your imagination be in the scene, or some kind of representation. That’s core to our creative philosophy about what we build. Our channel for this is not TV, obviously. It’s Instagram or Facebook. This past week we announced that we’ll be opening anybody to be able to build on Instagram. You can build AR in Spark AR Studio, distribute it on Instagram, and have the reach of the Super Bowl if you’re famous enough on Instagram. But it’s that same notion of, how am I going to craft this media and land with the audience?
It’s only limited by imagination and time, mixing graphic elements and changing—there’s an interesting trend on Instagram for virtual avatar influencers. They’re not real people. They’re artist’s representations or virtual characters. This came out of somebody’s imagination. It’s a creative exercise. AR is the type of thing that enables that exploration and creativity. Our focus is on making it as fast, as low cost, and as available to people as possible. We think about it like a new medium that has its own rules.
GamesBeat: Are the creators and developers making much money selling these things they create?
Roberts: The idea behind Spark is that it’s free for everyone. Anyone can get going on it. It should complement whatever your existing strategy is. Different people are using social media for different things. Some people want to use it as a creative outlet for—say I’m a musician. We announced a ton of new features around responsive music and audio. If I’m a musician, my product is to some extent myself, my music, my show, whatever it is. AR allows you to create a real time interactive audio and visual experience that can somehow land with your audience and help them experience in a new way.
Above: Spark AR reached a billion people via smartphones.
Image Credit: Facebook
GamesBeat: It can help your brand, but does AR monetize well in itself? Are there obvious ways people are making money?
Roberts: We’re looking at products like advertising and commerce. Obviously that’s related to selling things. We think that can improve, like I mentioned, with buying confidence, customer satisfaction, affinity for a brand. This is all valuable in some way. Right now selling AR content—it’s the same way with photography. We really do think of it as a medium. It’s a way to communicate with people or with customers or with friends. That’s the core of what we’re thinking about.
We’re not trying to be too prescriptive about what you do with it in a sense of—we want it to be useful. We talk to customers. We say, “What’s useful to you?” “I’m a musician. I want a better way to connect with my audience.” That leads to more powerful audio features in AR. If I’m a beauty influencer and I have a large number of people looking to me for tips on how to apply styles, can we make it easier to have virtual beauty trials? Can we make it easier to try a piece of furniture or other physical object with the lighting and rendering and physically-based characteristics it actually has? It looks like glass or plastic or a textured surface. These are things that improve the experience with that product.
But it’s just a medium. You can point the camera at whatever you want. What are you trying to communicate? I think that’s the key question that every AR creator should ask themselves.
GamesBeat: How far along are you on some of the things in the keynote, like being able to recognize faces and genders?
Roberts: Yeah, the inclusive AI? We’ve made a lot of progress on that. It’s an industry challenge. This is the type of thing that will never be finished, because as the technology improves, the ability to have it match the diversity of the planet will only improve as well. We’re committed to that. That’s something that’s not only really important to us as a product category, but incredibly hard technology that’s very interesting to tackle. It’s a tough problem.
GamesBeat: Do certain technologies make this a lot better at different points on the road map? I assume glasses might be a big improvement.
Roberts: Since I started working on AR, I’ve observed two camps around this. There’s the camp that will be interested when they can wear it, when they have this new type of computer. By the way, as Mark mentioned yesterday, we’re trying to build a new computer platform. That’s the whole goal of Oculus and AR. We think that computers that can see and sense the world and immerse you in their simulations and mixed realities—that’s going to be a massive change. That’s a decade or more of engineering, although we’ve come quite far.
People are really excited about that and say, “That’s great. Let me know when it happens.” They check out until then. But then there are people like me who’ve started to see the beginnings of this and feel we need to be a part of it. For me, it reminds me a bit of when I got started on smartphone games. I knew, when Steve Jobs announced the iPhone in 2008, that I needed to make games for it. I was at a console studio at the time, and I went to my general manager and said, “We should make games for the iPhone!” He says, “Are you crazy? We make PlayStation games.” But I was so fascinated by it. I went and jumped into that industry for 10 years.
It’s not the same, obviously, but I have a similar excitement around the unbounded potential of AR. I want to start working on it now. With Spark, what’s unique is we have a strong commitment inside Facebook, and also to our creators, to do the most we can with the devices you own today. There’s a lot of smartphones out there that can do very interesting things with AR. Those phones are only getting better. More cameras, better sensors, more understanding of what’s going on. The phones of five years from now are going to be extremely interesting platforms for AR experiences. We’re building into that.
One day people will wake up and realize that AR is here, has been here, and now all of a sudden there’s some perfect device with a bow on it. That’ll be great. That’ll be a different phase of the technology. But as a team we’re looking at what we can do at this stage, knowing what we know.
OhNoCrypto
via https://AIupNow.com
Dean Takahashi, Khareem Sudlow
