5 Amazing, 5 Shocking, and 4 (Creepy?) Things about Amazon Go
I’ve been to New York City twice in the last month: once over the holidays with my family, and the second time alone for a work conference. Both times I visited the Amazon Go store at the corner of 53rd and Lexington, with my wife on the first trip, and a prominent CIO on the second. Both times I was amazed by how easy and in most ways flawless the experience was. And both times I was shocked by obvious misses in the in-store and Go App experience.
To take one step back, Amazon Go lets you enter by scanning the QR code the Go App shows you on your home screen. Once you scan in, everything you take off the shelves and don’t put back will be charged to a saved payment method in your Amazon account when you leave. For a general overview, go here.
Back to the story. After reflecting on my two visits, I realized just how much more Amazon knew about me now (or could/should know about me), and, even though data-driven decision making, retail strategy, and customer experience are at the heart of much of what I do for a living, the potential accuracy and completeness of my Amazon customer profile was starting to creep me out. More on that soon. For now, let’s enter the store and get to the amazing and the shocking…
Amazon Go — Initial Impressions.
Just to get us all on the same page, let me first give you an overview of my initial impressions. I was curious as to how the experience would feel to me as a customer with exceedingly high expectations. I also wanted to see how solid the experience was, as in, would it feel reliable, secure, clean, trustworthy, and efficient? Or would it seem gimmicky, half-baked, and hollow. From a professional point of view, I wanted to understand how it worked operationally, technically, and functionally. I advise retailers regularly and I wanted to have informed opinions to offer, and my own ideas on how to apply what I saw to my clients. Did I have to download an app? (I had to before I could enter). Was the shopping a seamless experience? (It mostly was.) Was the Go App tied somehow into my Amazon account? (It was.) Would it know me instantly just by my being there, or would I have to scan in? (I had to scan in.) Would it pick up my existing payment methods from my Amazon account? (It did.) What would the return policy be like? (More on this in the shocking section…) These questions and more were on my mind as my wife and I approached the store and entered the double glass doors.
The Amazing
AMAZING THING 1: PRICES
My wife and I were pleased at how affordable the items in the store were, especially considering that we were in Manhattan. The prices easily beat those of the street-corner vendors (as you would expect), and the prices seemed reasonable enough to me that I didn’t feel compelled to shop at any local convenience stores for a better deal. My suspicion is that the prices on average were probably lower than even the local grocery / drug store chains which have a much larger footprint. Amazing though the prices were, I was surprised the prices weren’t digital, as that would facilitate real time pricing and offers just for you (more on this in the creepy section).
AMAZING THING 2: QUALITY
We were both uber-impressed (sorry Uber) with the quality of the products in the store. My wife told me she’d be comfortable prepping for a small dinner party for friends or a romantic meal for the two of us with the high-quality meal kits, cheeses, sparkling waters and other beverages, and assorted yummy foods we found. I was just psyched that there were twelve-packs of La Croix and bars of dark chocolate with sea salt, even though these were made/branded ‘for Amazon Go’. And despite the high-quality, the prices were (again), totally reasonable.
AMAZING THING 3: ASSORTMENT
Amazon clearly did its research in determining what the hotel-going tourists like us and hipster locals would want when they did the assortment plan for this location. Among six kinds of yogurt (including 365, the Whole Foods brand), 10+ brands of water, 10+ options for salads (including green stuff, fruit, and hearty varieties), emergency staples like tampons, small-box dog food, and Pringles, (the dog food and Pringles were hilariously under the same ‘Wellness’ banner as the tampons), an assortment of medicines, hot coffee, 100+ soft-drink choices courtesy of the “Freestyle” soda fountain, cleaning products, and more, we actually felt that if we lived here, we could shop here regularly and not have to deal with checkout lines, crowds, and dingy old stores, at least not until we needed something we couldn’t find at Amazon Go.
AMAZING THING 4: ASSOCIATIVE SHOPPING
On my first visit, my wife and I weren’t sure how Amazon Go was going to deal with the two of us shopping together since I was the only one with the Go App. On entering though, it was made plain to us that what we both pulled from the shelves and put back and carried out the door would be tracked, even if we weren’t next to each other. Honestly, what Amazon is doing here is phenomenal. I call it Associative Shopping, and sorry Mr. Bezos, I call dibs on that term. My wife and I, and on my second trip, my CIO-friend and I did all we could to confuse and fool the system, and it got everything right, both times. Pick up a bottle and put it back? No charge. Put a candy-bar in your shopping bag? Charge. Put it in your pocket? Used to be shoplifting. Now it’s $.79 cents. Pour yourself a soda and drink it in the store and throw away the cup? Charge. Take the same soda, pour out the contents, and put the wet cup back into the sleeve? Gross. No idea, I didn’t try that, but my guess is that you’d be charged. Someone try it and let me know… We tried many permutations of pull, put, move, toss to each other and put back, and assorted sneaky tricks I can’t tell you about, and the tech got it all correct. Quantities, prices, item identification. Really, really, really amazing.
While forever changing in a good way the shopping experience for us law-abiding citizens, shoplifters better watch out: you don’t get to come in without scanning, and apparently Amazon is going to know if you so much as eyeball a Snickers, let alone put it in your pants, so your old tricks aren’t going to cut it any longer. In fact, the average retail store loses something like 1% of sales to shoplifting and employee theft each year. What a boon this tech will be to loss prevention professionals (though we might need fewer of you). The accumulated annual loss prevention savings would probably pay for the tech many times over.
AMAZING THING 5: IN-STORE EXPERIENCE
I’m not sure why we were so surprised by this, but everything was just so (love you Syrio Forel— did you wind up dying? Hope not. You were whipping those knights with that wooden practice sword when they cut the scene…). Products were almost too perfectly arranged, the floors too clean, the (free) eco-friendly bags too cool, and the soda fountain and coffee areas too spotless for us to comprehend how it was possible. My wife was especially impressed. There was no one in the store to take care of things, so how was Amazon pulling this off? Well, we unraveled that mystery when we came across the Associates Only area. Apparently there are people back there with mops, brooms, sponges, and the will to use them. We found one of these Amazon recluses snooping around mid-shop, and in a hushed voice, he told us that they were “still working out the kinks”. Not sure he was supposed to say that, but okay. To wit, there were several empty slots on shelves. Kind of think Amazon would have a better handle on inventory management and forecasting (see Shocking Thing 1 for more). Anyway, this particular associate was scanning shelf label QR codes to take inventory and do some of that replenishment I just mentioned (presumably). After he knew we weren’t up to no good, he was really nice, and certainly seemed proud of himself and his store. I don’t blame him.
I can’t tell you just how clean and organized a grocery store, of all places, has to be to impress my wife. She’s the first to notice little things that are off. Take movies. She’ll tell me two or three times a movie (usually during very important but quiet dialog) that the actor’s hair was different between different shots in the same scene, or how there was a magnifying glass on the desk when looking down from the actor’s point of view, but not when from another actor’s point of view in the same scene. Point is, if my ultra-detail-oriented wife is impressed with a small-format grocery store, believe me, it must be impressive.
Now for what you’ve all been waiting for: The shocking.
The Shocking
SHOCKING THING 1: STOCK OUTS
There were LOTS of empty slots and some empty shelves. That was shocking to me. I mean, more than any company, Amazon knows fulfillment and inventory management. And EVERY purchase of EVERY item is available in real time to the database that underlies the app, and I cannot imagine that Amazon would not have had this dialed in before opening. But, there it was. Lots of missed sales right there in front of us. BOO. Some of the empties were due to true stockouts, and featured a ‘SO GOOD IT’s GONE!’ sign (nice touch with the messaging and smiley Amazon arrow). Others were either due to bad real time data and hence no alerts in the employee area, or perhaps this reason.
SHOCKING THING 2: CLOSED AT 7:00 PM
Okay, so maybe this isn’t shocking to you, and it’s pretty easy to fix, but really? A convenience store in Manhattan that closes over the weekend at 7:00 PM? I was at dinner with a client down the street at Hudson and Malone as part of a “dinner then store tour fun/work appointment” I had carefully planned. Only I was foiled by the ridiculous fact that not only was the store not open at 10:00 PM when we finished dinner, but it closed at 7:00 PM! I was shocked. That’s why it’s in this shocking section, I suppose.
To be fair to Amazon Go, and to appropriately humble myself, I did have a picture of the store window (it’s in one of the opening set of pics in this article) and it did clearly show the hours the store was open, and during the week (as I just saw literally as I checked the photo just now), the store is open til 9:00 PM, but I am still shocked. I mean, the store is pretty much an auto-pilot type of deal. One person could run the whole thing. As long, again, as they didn’t have this problem.
SHOCKING THING 3: NO REAL TIME SHOPPING CART ON THE APP
Okay, this was pretty bad. I was sure that when I looked at the App after doing all those fun ‘break the store tech’ tests I described earlier I was going to get to gleefully proclaim: “GOTCHA!” But not only could I not feel superior to Amazon, I didn’t have any idea what was going on, because the app (as of January 13 anyway), didn’t have a real-time shopping cart helping me keep track of my purchases, and helping me verify if Amazon was getting all the items I was messing around with properly sorted. I had stuff in my free Amazon Go shopping bag, but nothing was registering in the App. Disappointing. Didn’t seem like the cameras and sensors and other tech should struggle to register items as belonging to my account, and I’m pretty sure we’re past the days of 300 baud modems, so what was the reason for the delay? I spent 10 and 20 minutes in the store on my 1st and 2nd visits, and both times it wasn’t until after I left the store, perhaps 3 to 5 minutes after, that I received an SMS that my receipt was ready.
After receiving this message, I opened the app to ‘Receipts’ and saw that my shopping cart had finally populated (correctly as it turned out… see Amazing thing 4) and my Amazon account’s saved AMEX card been charged.
Even though I was rightly amazed by the accuracy of the system, was it by design that the cart didn’t track activity in real time? I’m trying to think of good reasons to design the system the way Amazon did, and I’m coming up snake-eyes. Perhaps, as Guy Kawasaki recommends, Amazon cares more about launching and getting customer feedback than they do about building a perfect product (which would certainly include a shopping cart that lets you know what you’re buying and how much it costs before you leave the store). IMHO, this one is a shocking miss, but OKAY — kudos to Amazon for being bold and being willing to experiment. And if I offended Amazon with this one, perhaps someone will click the link for Guy Kawasaki and buy his book from them, which will be kind of an apology from me.
SHOCKING THING 4: NO MISTAKES
It’s worth repeating that in two separate trips, both of which involved me and a guest shopping together, and trying to game the system, Amazon Go got it all correct. Actually, the second trip was me with my account and my CIO guest and her husband doing the Associative Shopping thingy on her account. See Amazing Thing 4 again. I imagine the tech could be glitchy at times, or perhaps affected by poor lighting, a failure in electricity, an internet outage, or something else I haven’t thought of, but it’s darned impressive how accurate it is. Bodes very well for cashierless stores, and very poorly for cashiers.
SHOCKING THING 5: HOW REFUNDS WORK
I’m sure you’ve all been wondering about this question since you started to think about how YOU would game the system. At least as of February 1st, 2020 the answer will shock you. Ready? Wait for it. WAIT FOR IT…
If you want a refund for something, you just swipe left on that item from your shopping cart (in the app) and choose the return option. No questions asked. No need to return the item. Are you listening oh ye huddled and hungry masses? Get a smartphone, get the app, and get to Amazon Go! Well, if only it were really that simple. And I’M KIDDING. Don’t do that. Don’t be dishonest. Be a good person. I suppose you could get some free stuff from Amazon easily enough with no repercussions, but I doubt they will let you do it too often. Finding patterns of fraud with AI and ML these days is getting easier. My guess is that if you do this often, or maybe even more than once in a while, you might find yourself banned from the App, or the store(s), or maybe even lose your Amazon Prime benefits. Or, as we move toward the creepy section of this article, maybe Alexa will start to admonish you for being a thief while you’re nuking one of those yummy food kits (see Amazing Thing 2). Or perhaps she’ll refuse your next request to play your favorite 80’s hits like Wild Wild West (ok, that’s one of my favorites), and instead play you a dirge from Cool Hand Luke. In any event, in case you don’t believe me about Shocking Thing 5, please see below the actual Return rules from the app (as of February 1, 2020).
The Creepy
As I reflected on my visits, and started to connect the dots on what it all meant to me, and to Amazon, I began to have thoughts (and worries) on how thoroughly and accurately Amazon will be able to understand their omnichannel users, like me. Imagine, your entire buying history on Amazon Prime married to your interactions with Alexa, combined with the many ways Alexa can know that you are not home, your life patterns Amazon learns about through your in-home Amazon subscriptions, your media consumption patterns from Prime Video, and data gathered about you while you’re in Amazon Go stores. These incredibly rich sources of data could result in a uniquely powerful Customer Segmentation and Personalized Marketing model that Amazon could leverage in its every interaction with you or perhaps sell to others who would like to better target you (if you don’t believe Amazon would do this, have a look at this little known company called Facebook to see what it did with your data). So here goes:
CREEPY THING 1: AMAZON KNOWS WHAT YOU LOOK LIKE
When you pass the gates of an Amazon Go store, the sophisticated cameras and technology begin to record your every move. It’s not a stretch that if it wanted to, Amazon could use facial and image recognition techniques as well as analysis of recorded video to know what you look like, accurately estimate your height, weight (and if you’ve gained or lost a lot of it between visits), hair color, eye color, whether you walk with a limp (like I do), the kinds of clothes you wear, if you have on expensive jewelry, or maybe the type of watch you wear. While it’s a bit Orwellian to think that putting all this data together and treating you differently as a result is a likely outcome, it’s not beyond reason. At least not from a profit-motive perspective. If I can determine that you’re a well-heeled person a bit thick in the paunch and combine that data with the fact that you just bought a bunch of candy, sugary soda, and ice-cream, perhaps I might start to send or show you advertisements for dieting books, healthy eating choices, or exercise equipment the next time you’re browsing Amazon Prime.
How about if your appearance has changed between visits or over a series of visits? Maybe you’re balding (guilty), are visibly heavier, are pregnant, or are using crutches. All of these attributes can be added to your Amazon profile, better informing the next decision Amazon makes about how to interact with you. Now, there is heavy regulation covering the subject of Personally Identifiable Information which arguably would include any biometric data (of which most of what I described could be considered). In Europe, companies can be fined up to 20 million Euros or 4% of annual turnover (whichever is greater) for a single violation of certain provisions of the General Data Protection Regulation, otherwise known as GDPR. Here’s a Wiki article about the fines levied under GDPR thus far. As to whether Amazon will leverage its burgeoning profile capabilities, or there are any risks of abuse, just see how much money Facebook prints from its targeted advertising business, and don’t forget the Cambridge Analytica data scandal of 2018.
CREEPY THING 2: AMAZON KNOWS WHO YOU ARE WITH
Were you alone in the Amazon Go store, or with friends? A woman? A man? Your wife or husband? Might go towards informing that profile of yours Amazon keeps. Might not. But there it is. It’s definitely creepy to think about.
CREEPY THING 3: AMAZON KNOWS WHERE YOU ARE
OK, so does Facebook or other social networks if you are active users. But social networks don’t have the ability to change prices on an electronic label as you walk up to a shelf of products, knowing you’re likely on vacation or a work trip and thus a bit less focused on saving money and a bit more focused on getting what you need or want. The leverage I see Amazon gaining here is in the real time modification of your personalized marketing profile and immediate use of same while you’re shopping in the store. I am pretty sure no one else can really do this the way Amazon now can. Of course, there are only 26 Amazon Go stores today, but who’s to say that Whole Foods doesn’t start to deploy this technology across its 500 stores, and use the Amazon Go application?
Today Amazon controls between 37% and 45% of the online commerce market, and perhaps 4%–5% of overall retail sales in the United States. Their physical store footprint pales in comparison to the likes of large grocers and convenience store chains like 7–11 (north of 68,000 stores worldwide and north of 8,500 in the USA) or OXXO (north of 20,000 stores overall and north of 14,000 in Mexico). Considering Amazon’s incredible size and strength though, it’s hard to imagine that we won’t soon be seeing thousands of Amazon Go stores in just about every kind of location you can think of: airports, malls, street corners, office buildings, within other stores, on cruise ships, etc. The technology, financial strength, and the value in what Amazon knows about us shoppers makes this seem a very likely outcome to me.
CREEPY THING 4: HOT MIC
Yeah. Hot mic. As in, what you say while you’re in an Amazon Go store is likely being recorded. Under the guise of product research and user experience improvement, Amazon might like to listen into everything we’re saying while in the store. It makes sense. I made a lot of comments, as did my wife, and the prominent retail CIO who was my guest on my second trip. A lot to learn from us perhaps, but at a minimum, a lot of feedback about the experience, both positive and negative. To be safe, and to be a good corporate citizen, Amazon could simply aggregate / anonymize all voice recordings and use them like those companies who use our telephone calls with their agents for “Training Purposes”. But they don’t have to. Even if they say they won’t. Companies can change commitments to you simply by changing terms and conditions of the apps and services you use and getting you to agree the next time you access the app or service. And come on, who really reads that stuff? Well, in the case of Amazon Go’s Terms of Use, which are linked to Amazon Services Terms of use, I do. I’ve taken some screen shots (just the juicy ones about data use and privacy) from the Terms and Conditions accessible via the Amazon Go app. Have a look at just how much data about us Amazon has, and look carefully at the rights to use that data Amazon claims.
Well, that’s about all for this article. I am sure Amazon Go will continue to deliver on the Amazing, probably fix or minimize the Shocking, and who knows what will happen as far as Amazon’s use of our data goes. But, if Amazon is not careful, I won’t be the only one thinking about the potential creepiness of connecting all the dots of a Prime user’s life. President Trump has a history of fighting with Bezos, and given the imminent failure of Democrats to remove him from office, any kind of data or security issues related to user data might give Trump the ammunition he needs to sick the DOJ on Amazon. It’s happened before.
If you visit an Amazon Go store, please comment with your own observations and experiences. If you haven’t, visit one and let me know what you think!
Jerry Lewis is a Partner with IBM Digital Strategy and IX with 20+ years of consulting experience in digital commerce, omnichannel, agile ways of working, and customer experience led digital transformation. He’s held senior roles with multiple retailers, often speaks at conferences, and 15 years ago married his best friend in Northern Ireland, in a castle. He loves to take his two girls fly-fishing and skiing. Jerry’s thoughts, analyses, and opinions are his own, and do not represent those of the IBM Corporation.
