Smart Speakers or Smart Listeners? Privacy Concern Explained
What is a smart speaker?
Smart speakers are internet-connected devices that can recognize speech and act in response to voice requests. You can ask a speaker whether there’s life on Mars, request it to put on some upbeat Balkan music or set an alarm for 3 a.m. (check out the chart below for more ideas). The goal of most smart speakers is to help you with your daily tasks and entertain you—all in a convenient hands-free mode.
Let’s face it: smart speakers with digital assistants are trendy gadgets that are here to stay and probably evolve. According to a 2019 Tech Trends Report by Future Today Institute, digital assistants are now becoming widespread in nearly all industries—and the trend will continue.
26.2% of all adults in the US already use smart speakers at home. If you are among the owners, you can skip the next paragraph. But if you don’t usually talk to plastic objects (and expect them to respond), check out this quick overview.
The most popular vendors of smart speakers are Amazon, Google, and Apple. When you communicate with a device, you talk to their virtual personas: Amazon’s Alexa, Google Assistant and Apple’s Siri. Other vendors have created their own assistants as well.
How do smart speakers listen and talk to you?
So, how exactly do these devices work? Well, that’s easy. Inside, there are tiny creatures with Encyclopædia Britannica and a few trumpets. *Crickets chirp* Just kidding.
In fact, any given smart speaker is stuffed with sophisticated electronics that allows speech detection and voice recording, then sends it to a server over the internet and provides appropriate outcomes. The latter may include talking back to you through a speaker or exchanging information with other devices, apps, and websites to turn on the lights, order pizza or add a meeting to your calendar.
To start this whole process, the speaker has to be triggered by an activation phrase or a “hotword.” For Google Home, the words are “OK, Google,” while for Amazon’s device it’s “Alexa.”
Wait, so are smart speakers always listening? Honestly, they probably have to keep their mics on all the time to “catch” the hotword. In fact, they do, but normally this information is not sent anywhere.
Here’s how Google explains it:
“Google Home listens in short (a few seconds) snippets for the hotword. Those snippets are deleted if the hotword is not detected, and none of that information leaves your device until the hotword is heard.”
Now, when the device hears "OK, Google", it records the following request and sends that recording to Google in order to fulfill the request.
So, are smart speakers safe? It’s not that straightforward. Let’s see why.
Do smart speakers affect your privacy?
Well, it’s one thing is to share something private to any group large or small. It’s another thing to share your needs and desires with a large corporation over a potentially not-so-safe Wi-Fi connection.
Even if nothing goes wrong and the device handles your requests as designed, there are some far-reaching consequences to your privacy. While you get new pie recipes and check out gas prices, the vendor of your smart gadget learns what you are interested in. A lot of personal information gets linked to your account—more than you may even realize: when you are home and what you do; when and where you ride by taxi; who you call and when; which foods you prefer, how often, and associated with which activities. The list goes on.
Google’s business is based on the targeted advertising of third-party services, so receiving information on user behavior is crucial for the company. Likewise, Amazon sells goods and wants to advertise them in accordance with the consumers’ interests and needs. Eventually, you are bombarded with personally-tailored ads through various channels. While sometimes it can be handy, in a certain way it can also feel too invasive.
As Rupert Pople, founder of YourSmartHomeGuide, says, “The gathering of data in itself is a stepping stone in the escalation of monitoring, surveillance, and control. This can lead to an abuse of power that fundamentally changes who has control in our society, which in the long term will lead to negative outcomes for citizens.”
However, not all devices work the same way. For instance, Apple’s HomePod anonymizes all interactions and has no third-party extensions, unlike Google’s and Amazon’s smart speakers. This way, your experience with HomePod is more secure but, at the same time, less diverse and convenient.
Does this aspect of privacy—or lack of it—bother the users? As it turns out, it does. Nearly 67% of US consumers are at least mildly concerned about smart speaker privacy risks.
What can go wrong with a smart speaker?
Issue 1: You don’t really know what it can do
In February 2019, there was a scandal around Nest Secure, Google's smart home security system. Nest Secure was supposed to protect homes via an alarm, a keypad, a motion sensor, a special sensor for windows and doors, and a security access tool (keychain fob).
Yet, it turned out that since its launch in 2017, Nest Secure also had a hidden microphone that wasn’t mentioned in the product description. For years, the users just didn’t know that the device could technically listen to them.
This case is an unpleasant precedent. If even a trusted company allows such omissions, can we trust other smart devices? “When hidden microphones started being built into all our home devices, from security devices to toaster ovens, technology is no longer something to make our lives easier, it is something to control us rather than us controlling it,” says Andrew Selepak, PhD and media professor at the University of Florida.
Issue 2: It mishears requests and acts weird
Another story that made it to the news recently was centered around Amazon Echo privacy. A married couple in Oregon discovered that their Amazon Echo Dot was sending their private conversations by email to the husband's employee.
The wife called Amazon, and the company investigated the case. They concluded that what happened was “an extremely rare occurrence.'' Alexa misinterpreted speech a few times in a row, which led to sending audio files to a person from their contact list.
Apart from Alexa listening inattentively, multiple users reported activity that was plain weird. The voice assistant activated randomly and laughed in a creepy way. This is probably something you don’t want to hear in the dead of night at home, right?
Issue 3: It has been tricked to order stuff online
More Amazon Echo security concerns arise when Alexa tries to buy something the user didn’t want. How come? Well, some smart advertisers figured out that they can talk to your Alexa… out of the TV! Shoot a commercial where you say “Alexa, order this”, air it and voilà—Amazon speakers all over the country start buying the advertised stuff.
It all started with an accident. A little girl ordered a doll house through Amazon Echo, and a morning show host told this story ending it with a phrase “I love the little girl saying ‘Alexa, order me a dollhouse.’” Soon, the channel received a number of reports from viewers, whose devices attempted to order a dollhouse after hearing the broadcast.
There were a few more cases, this time with people intentionally provoking smart speakers to take action. In April 2017, Burger King targeted Google devices. In a commercial, a Burger King server starts to describe a Whopper sandwich and ends by asking a Google assistant to complete this description. While Google reacted quickly disabling their devices’ response to this ad spot, the advertisers released four more versions of the commercial prolonging the trick.
Next, in October 2017, another similar ad was released. In this video, a man asked Amazon's Alexa to order Purina cat food. Commenting on this situation, Amazon explained that no order would be completed unless the user specifically confirms it.
As smart assistants evolve, protection mechanisms for such problems will appear. For example, you can set up a confirmation code for online shopping on a home device. Smart speakers by Amazon and Google can now recognize different voices and associate requests with separate accounts of family members. Still, they generally obey anyone who addresses them with a hotword.
Issue 4: It gets hacked
For now, connected household devices are less protected than laptops and smartphones. The reason is that this market is new and security standards are not yet in place. Yet, there are significant reasons for criminals to hack Google or Alexa privacy. An intruder that hacks a smart device gets access to a lot of private information. Further, the attacker can follow the user, blackmail them, and so on.
Frighteningly, it’s not that hard to do. In 2017, a British security expert Mark Barnes demonstrated how an Amazon Echo could be hacked. The installed malware allowed the researcher to sneakily stream audio from the device to his private server.
Similarly, Google Home and Chromecast were caught with a vulnerability that allowed discovery of their exact location. To make it happen, the user merely had to click on a dangerous link while being connected to the internet through the same network as the smart home device.
Issue 5: It learns to do more than expected
Google and Alexa privacy concerns are not likely to decrease in the future. A disturbing fact is that the vendors keep filing patent applications that indicate that they have truly invasive technologies. Consumer Watchdog, a US nonprofit organization, analyzed a number of such patents, and its findings are both futuristic and creepy. For instance, the described technologies allow the following:
- Analyzing emotions and behaviors by speech, even when a user has not addressed the device
- Targeting ads based on observations of users’ sleeping, cooking, entertainment, and showering schedules
- Recommending products based on furnishings observed by a smart home security camera
- Inserting paid content into the responses provided by digital assistants
Both companies stated that these patents do not mean that such technologies will be implemented in consumer devices. "Like many companies, we file a number of forward-looking patent applications that explore the full possibilities of new technology. Patents take multiple years to receive and do not necessarily reflect current developments to products and services," Amazon stated. Likewise, Google said, “Prospective product announcements should not necessarily be inferred from our patent applications.”
How can you protect your privacy?
Let’s not discuss the obvious and radical approach of totally avoiding smart speakers and digital assistants. Although, if by now you made such a decision, it would be no surprise.
Yet, if you prefer to enhance your life with these technologies, at least consider the safest ways to use them.
1. Select your device wisely. The more diverse and adaptive services it provides, the more personal details it will collect from you and possibly share with third parties. Generally speaking, out of the big vendors, Amazon and Google are team “Cool experience,” while Apple is team “Security.” By the way, the Mozilla Foundation runs a “Privacy not included” project to analyze the safety of connected devices. You can check out their conclusions on various smart home products.
2. Buy from trusted vendors. Be especially careful with second-hand devices. Hacking a smart device is not that hard, and it is way easier for a person who has physical access to it. For the same reason, request technical support only from trusted providers.
3. Connect carefully. It is a good idea to have one router for connected devices and another one for frequently used smartphones and computers. In any case, protect your Wi-Fi network with a strong password, and make sure it is WPA2 encrypted.
4. Disable all the features you don’t need. First, check out what your device can actually do and decide whether you want it all. If you don’t plan to use a camera, a microphone, or a shopping option for a while (or ever), just turn them off.
5. Keep in mind what you are sharing. Do not share sensitive details to your voice assistant unless it is absolutely necessary. These may include you payment information, Social Security number, passwords, answers to security questions and so on. If you accidentally let it slip, be sure to delete your commands—all popular speakers have this option.
Remember that the use of modern devices is often a trade-off between privacy and comfort. Be sure to make your choice consciously.