Understanding What the Smartphone Camera Sees
Photos and videos are a huge part of our cultural conversation. 2.8 trillion photos were shared on social networks in 2017 (set to increase 15 percent each year). What happens when computers can suddenly see and understand that visual conversation at scale? And what can brands do with this newfound power? No need to wait. 2018 will be the year of social vision, with artificial intelligence (AI) transforming the smartphone camera into a tool that brands can use to learn about, communicate with, assist, and ultimately sell to consumers in new ways.
Vision, Part of the Bigger Picture
Vision is just one of four pillars in which we see AI transforming marketing. But it is perhaps the most exciting for brands and advertisers, because it promises to change how we learn about and talk to our consumers in such a fundamental way.
Say It With the Camera, We’ll Listen
Shared social photos are the cornerstone of conversation today. Snapchat sprung into adoption because people wanted to snap their messages rather than simply type them. And now Facebook, one of the 10 largest companies (by market capitalization) in the world, is telling its users to “let your camera do the talking.”
The question is, can we listen to this conversation? Over 3 billion social photos are shared daily. Of the photos that include a brand, 85 percent do not have a caption that references the brand. But with advances in computer vision, many social listening tools are gaining the capability of visual listening. By applying AI to the firehose of public social photography, companies like Ditto Labs use visual listening to help brands understand their place (or the place of their competitors) in consumers’ lives in a way that text conversation cannot.
This conversation provides context for consumption. Where are people most likely to be enjoying my product, with whom, when? It also provides information about competitive products so you can also target based on those interactions.
Source: Ditto Labs
Visual listening is only the beginning. New ad products are also making it possible to use social vision AI to target consumers. For instance, Cluep Pics allows advertisers to serve ads to audiences who have posted photos that feature specific competitors or environments.
Image courtesy of Cluep
The Rise of the Lens
Google, Facebook, Snapchat, and Pinterest are all working to turn the smartphone camera from a simple tool for capturing photos into a lens through which people can discover their world. Enabled by image recognition, these companies want to change how you see and search, what you share, and what you can do anytime you are using a smart device.
Change How You See and Search
Today, you might search for knowledge about companies, products, and people by typing or talking to Google. In 2018, searching will be done with lens and images. And it’s no surprise that Google is at the forefront of this shift. With the release of Google Lens in September 2017, people can identify anything from famous artwork to buildings (including storefronts), look up product information, and save dates from posters to your calendar.
Google CEO Sundar Pichai nodded at the ambition behind Google Lens when he said, “all of Google was built on first understanding text and web pages.” What he doesn’t mention is that much of the wealth of Google came from using its search engine to help brands advertise themselves with messages and relevant content. If tomorrow’s search engine is a lens, it follows that a lens is where tomorrow’s search advertising will live. In 2018, brands should be ready to take advantage of search ads beyond the search bar of a computer.
Using these ads, brands have the opportunity to begin to interact with consumers as they search and experience the real world.
For a preview of where Google may be headed, it is helpful to look at Alibaba, the second-largest Internet company in the world after Google. Alibaba is integrating computer vision into each step of the journey from search to purchase. For example, a customer who lenses a picture of a music album in the real world will be presented with the option of visiting the social profiles of the musicians, seeing more details about the event, or buying tickets. If they want to purchase tickets, they can then confirm payment just by smiling at the app.
Alibaba’s Alipay App – Source: Alibaba
Change How You Share
Because the smartphone camera is inherently a social camera, lenses built on social networks are worth paying attention to. Snapchat, perhaps the first social lens, recently rebranded itself as a “camera company” and filed a patent on object recognition ads (e.g., detecting a cup of coffee in a shared photo would display a miniature Dunkin’ Donuts shop on the breakfast table).
Not to be outdone, Facebook recently released a suite of social lensing products. As Mark Zuckerberg introduced these new products at the 2017 F8 Conference, he stated that Facebook wants to “turn smartphone cameras into the first AR platform.”
In his demo of these products (Camera Effects Platform and AR Studio), Zuckerberg shows the Facebook Camera app detecting a bottle of wine and adding an information card, making object recognition ads (like those in Snapchat’s patent) seem not only inevitable but imminent. Through lenses, augmented reality (AR) can change what people see. But because the smartphone camera is inherently social, these experiences also change what people can “say” by sharing what they see.
Though a runner may not be wearing any Nike clothing, the brand uses Facebook Camera to become part of how they share their run with friends. Source: Facebook
Brands should explore opportunities with AR products and social lenses that change what people see about their brand. Advertisers must hold these experiences to a high standard, perhaps even more so than other ads, as the social lens – which uses our most personal device to change how we see ourselves and our world – is an intimate one. Thus, ads that use the social lens must be useful, interesting, or worth sharing.
For more on how AR is becoming a powerful marketing force, read Utility Over Novelty: AR’s New Marketing Focus.
Change What You Do
Since the dawn of the Internet, people have been finding products on shelves and showroom floors, then searching for them online in hopes of finding a better price. With social vision, the barrier between physical products and the buy button is eroding.
In February 2017, Pinterest released Pinterest Lens, a camera function that finds pins related to anything a user can point their camera at. Already, Pinterest recognizes pictures of branded goods, logos, and locations. But discovery is only half the story for Pinterest Lens. A growing number of pins on the Pinterest platform feature the “Buy Now” button, letting inspiration turn into transaction in the span of a moment. In September of this year, Target announced that it will be integrating Pinterest Lens into its app and website.
This is not limited to packaged products. Social vision will give us new ways to instantly act when we see nearly anything. Soon, the information card that pops up when you point your camera at a restaurant may include a “Reserve Now” button.
This expression of intent may even take the form of a screenshot. For example, rewardStyle’s new LIKEtoKNOW.it app harnesses image recognition technology to convert screenshots into an itemized list to be shopped. If the image in the screenshot is created by an “influencer” in rewardStyle’s network, the users will automatically receive an email with shoppable links to each of the featured products.
Image courtesy of rewardStyle
Your Ads, Through a New Set of Eyes
Ultimately, the rise of social vision and lensing makes all ads and products Internet-connected, interactive, and inevitably social. Tomorrow, people will be looking at anything from a magazine to a billboard with a new set of digital eyes. Companies need to ensure their products and ads are recognizable and are designed with the next steps that computer vision apps make possible in mind. Here are some tips to ensure that your brand is seen in this new world:
Use visual listening tools to understand how your brand and your competitors fit into the culture and lives of your consumers.
Consider using visual targeting ads based on what consumers have shared in their images.
Be Easy to Find
Make your storefronts recognizable and ensure your location data is current and accessible. Make sure product and logo images are widely available and easily identified by image recognition software.
Create content that is tied to a greater experience. Encourage the use of social lenses to link experiences and ads to products for purchase (e.g., “snap this ad using Pinterest Lens to see the whole collection”). Look for ways to use augmented reality ads to showcase your product in a new light when different objects, people, or places are recognized.
Overall, social vision is a big milestone on our pathway to “reconciling our dual citizenship in the physical and digital worlds” (as Hiroshi Ishii of MIT’s Media Lab put it). It will radically change life and marketing in 2018 and beyond, rewarding brands who take advantage of visual listening, new ad types, and the social lens.