For a while it seemed like smartspeakers were the new black of digital audio, driving sales ever higher.
But a new study by Score Publishing suggests all is not well in the smartspeaker market, with book-related search queries at best delivering accurate results only 72.5% of the time, and the one we’d expect to do best among the worst offenders with an accurate result turning up only 42% of the time.

The study, the results of which were presented at the London Book Fair’s Quantum conference, took the market-leading devices Google Assistant, Apple Siri, Microsoft Cortana, Samsung Bixby and Amazon Alexa and asked each identical book-related questions.
One might expect Alexa to have come out on top, given the device was designed to sell Amazon products and Amazon is the USA’s biggest book, ebook and audiobook seller, but in fact Microsoft managed 72.5% relevance, Cortana 60.8% relevance and Alexa a worrying low 44.2% relevance.
Not good news for publishers pinning their hopes on the twin values of smartspeakers delivering audiobooks to consumers who found that content in the first place by asking the smartspeaker for recommendations.
Score Publishing calculate that New York Times bestselling authors and their publishers stand to lose as much as $17 million this year in books sales because of poor voice assistant search recognition.
In a classic example of tunnel vision it seems the smartspeaker teams spent too much time making sure the question was understood by the person asking, and not enough time making sure the questions could be answered.
The questions were not exactly taxing. Examples included:

  • Simply stating the book’s title and author. For example, “Becoming, by Michelle Obama”
  • Saying, “I want to read [BOOK TITLE] by [AUTHOR].”
  • Saying, “I want to listen to [BOOK TITLE] by [AUTHOR].”
  • Saying, “I want to order [BOOK TITLE] by [AUTHOR].”

Brett Kinsella, summarising the Score Publishing report for VoiceBot AI, explains,

Score Publishing assumed that only about 20% of these failed queries led to a lost sale and that the average revenue per book is $12. The analysis then took publicly available data on questions asked of voice assistants to arrive at the $17 million in loss figure.

adding,

Amazon is often thought of as the king of audiobooks because of its Audible service, but it was only successful in the “listen to” questions for five books out of 30, only one point ahead of Bixby.

At which point let’s turn to What’s New In Publishing, where we delve deeper into the numbers:

  • More than 3.5M book-related queries are made by voice, daily, across the world
  • More than 1.9 million of these queries are failed to be understood or properly answered by voice assistants
  • Two percent of these queries are attempted purchases: more 70,000 per day
  • More than 38,000 attempted book purchases via voice fail daily

Finally, we arrive at dollars and cents: More than US$46,000 is lost, per day, right now due to nonfunctional voice queries, equaling over US$17 million in 2019.

Just how badly did the so-called smartspeakers perform? Back to VoiceBot AI:

Graphic via VoiceBot AI.

The book with the lowest overall composite score came from the non-fiction category. “Bad Blood” by John Carreyrou only scored a 3 and was recognized as a book by both Google Assistant and Cortana. Microsoft Cortana also was able to direct the user to a place to purchase the book. No other voice assistant recognized the book for any of the questions. This is not a new book. It was first published in May 2018 so there has been plenty of time for the search engines behind the voice assistants to learn about it.
“Where The Crawdads Sing” by Delia Owens was the fiction book with the lowest overall composite score of 6. This also isn’t a new book. It was originally published in August 2018. Only Google Assistant and Cortana recognized it at all with both responding correctly to three of the four questions.

While it’s fair to say smartspeakers are bought mainly for the audio-content they deliver, not as convenient search engines for shopping, the results are deeply embarrassing for Apple and Samsung, but stand to hurt Amazon the most.
But speaking as a layman in all matters tech, I must wonder if the solution, for publishers at least, lies in their metadata.
Might it be the case that content aimed at the audio market where smartspeakers play a significant role in connecting products with consumers, need to create new metadata specifically geared to what smartspeakers are trained to find?