On the hunt for new shoes? Racking your brain for a recipe idea? When you’re trying to find information, typing out words in a search bar is probably the first thought that comes to mind.
Words can conjure up images and evoke emotions, but as the saying goes, a picture is worth a thousand words.
Now, companies including Pinterest, Amazon and others are giving consumers more ways to search just by taking a photo with your smartphone camera.
While some visual search tools focus on the discovery of similar objects or new ideas, others are more precise about finding the exact item you’re seeking.
Home Depot’s app allows customers to snap a photo to find tools and other objects that the business is selling online or in stores.
In the coming months, Google is also releasing a visual search tool for its assistant app called Lens that lets users, for example, point their camera at a restaurant and get reviews.
“Currently, the challenge is really figuring out what the use case is. What is the problem that’s hard to put in words? How can you make your customer’s life simpler?” said Yoram Wurmser, an analyst with eMarketer.
I put five visual search tools to the test, snapping photos on the streets of downtown San Jose, Calif., in a shopping mall, at the office and around my home.
Sometimes I got exactly what I was searching for — new recipe ideas, cheaper products, tips or more information — but I also got some surprises. And I had to be careful about exactly how I composed a photo.
I found the features a useful alternative to searching via text, but also discovered there’s still room for improvement.
———
Pinterest Lens
Pinterest started beta testing Lens this year, rolling out the tool worldwide for users on iOS and Android who have their mobile device set to English.
To find the tool, you tap on the search bar, and then tap on a red and white camera icon. A circle surrounded by white space pops up, and you click on it to snap a photo.
Pinterest also rolled out a new way to access Lens from the iPhone’s home screen by pressing longer on the Pinterest app.
Colorful dots quickly start swirling around. Words describing the object start hovering around the circle followed by a series of visually similar images and ideas.
I snap a photo of San Jose City Hall’s transparent dome building, hoping Lens will identify the architect who designed the structure.
None of the pins identified the architect — Richard Meier, who also designed the Getty Center in Los Angeles — but I did learn about other transparent dome buildings, such as the Biosphere in Montreal.
I took photos of everything from cars and home decor to food, shoes and even my own face. I not only received a list of visually similar objects, but pins with ideas such as recipes, beauty tips and tutorials about how to create a terrarium.
When I took a photo of strawberries, Lens showed me recipes for strawberry chia seed jam and a summer berry spinach salad, which was exactly what I was looking for.
“It’s more about understanding your intent,” said Jeffrey Harris, a product manager for Pinterest’s visual discovery team. “If you take a photo of a living room, we don’t need to find the exact chairs that you want, but we may notice that you’re into modern taste.”
Spending my day visually searching, though, also made me more aware of the subtle differences between objects. And sometimes, I got results I didn’t want.
I got beef recipes when I snapped a photo of ground pork. I got pasta recipes higher in the search results when I snapped a photo of spiralized zucchini noodles. And taking a photo of my friend’s earring gave me search results for pendant necklaces.
Harris said Pinterest is focusing on getting more precise about how Lens understands food, such as the differences between a salmon burger, hamburger or chicken burger. The tech firm also rolled out a new feature this month so people who take a photo that shows multiple items can focus in on one object.
And if Lens doesn’t get it right? There’s a plus sign you can click on near the photo that lets you type out a description of the item.
———
CamFind
This visual search app surfaced different types of results: related images, shopping results and web results.
Julia Gallagher, a consumer outreach associate at CamFind, said people use the app for a variety of reasons, from shopping for clothes to finding movie showtimes and trailers.
The app, which is available on iOS, Android and Google Glass, recently surpassed 3 million downloads.
“We believe that visual search is going to take over text searching since that’s the way technology is going anyways. Most things are more visual,” she said. “If you saw something on a street — a car, shoes or even a bag — it’s much easier to take out your phone and open an app and take a quick picture.”
At Crate and Barrel, I took a photo of a gray tufted sofa and got search results from eBay, Amazon, Overstock.com and other retailers.
The camera in this app, though, didn’t focus closer on an object like Pinterest Lens did, so sometimes I got search results for other items in the photo.
When I snapped an image of a building on San Jose’s East Santa Clara Street, the app thought I was searching for a one-way traffic sign. I took a photo of my glass container for a terrarium and got results for a white concrete wall. And that same image of ground pork was identified as a pink and white textile.
Unlike Pinterest, though, it did show me a history of all the items I searched for in the physical world. The app also allows you to see popular searches and share your searches in a public feed.
———
Amazon Shopping
Amazon’s visual search tool doesn’t even require that you snap a photo of an object. You just point at a product and tiny blue dots start twinkling around the item as it searches the online retailer’s inventory.
To access the tool, just click on the camera icon on the search bar.
I pointed my camera at the “The Circle,” a book by Dave Eggers, and found out it was retailing for $9.52 on Amazon. I held up my phone to an orange-red ceramic lamp and got 8,822 results for red desk lamps.
But this tool also took longer to surface search results than any of the other apps I tested. My patience wore thin as I waited for the blue dots to stop twinkling.
And for some reason, when I captured an image of a yellow patio table, I got results for trampolines.
———
Slyce
Clothing and accessories can be pricey, especially if they’re by a well-known designer.
But what if you could get something similar at a cheaper price?
At Santana Row, I stopped by the Gucci store and took a photo with Slyce of a printed leather bag. The app showed me 30 results for a women’s floral printed satchel, surfacing results for other bags and floral clothing from Forever 21, Aeropostale and Amazon.
Clicking on the item brings you to the retailer’s website, allowing you to purchase the product.
But sometimes I got items I just wasn’t interested in buying.
Snapping a photo of a pair of pointed Christian Louboutin heels, which cost $1,195 at Nordstrom, yielded results for a multi-stripe tank and stud earrings from Aeropostale.
———
Realtor.com
Some visual search tools are clearly meant for a specific purpose.
Realtor.com’s app has a tool that allows users to take a photo of a real estate sign for home details. You access it by clicking on the camera icon in the search bar.
I took a photo of a sign outside a home at San Jose’s Communications Hill. The app showed me that the home was listed for $818,000, had three beds and two bathrooms and was 1,776 square feet.
It also had listings for open houses, property value history, tax history and a way to contact the agent.
Photos were displayed at the top, allowing me to swipe through images of the house without even walking in the door.
About the Author
