amazon创建sns_我们如何在36小时内重新创建Amazon Go

amazon创建sns

by Subhan Nadeem

由Subhan Nadeem

我们如何在36小时内重新创建Amazon Go (How we recreated Amazon Go in 36 hours)

My colleagues and I wanted to create something that would make people go “wow” at our latest hackathon.

我和我的同事们想创造一些能让人们在我们最新的黑客马拉松比赛中大吃一惊的东西。

Because imitation is the sincerest form of flattery and IoT is incredibly fun to work with, we decided to create our own version of Amazon Go.

因为模仿是最真诚的奉承形式，并且物联网的使用非常有趣，所以我们决定创建自己的版本的Amazon Go。

Before I explain what it took to make this, here’s the 3 minute demo of what we built!

在我解释如何做到这一点之前，这是我们构建的3分钟演示！

There were four of us. Ruslan, a great full-stack developer who had experience working with Python. John, an amazing iOS developer. Soheil, another great full-stack developer who had experience with Raspberry Pi. And finally, there was me, on the tail end of an Android developer internship.

我们四个人。 Ruslan ，一个伟大的全栈开发人员，曾在使用Python的经验。 John，一个了不起的iOS开发人员。 Soheil，另一位伟大的全栈开发人员，曾在Raspberry Pi上工作。 最后，是我，在Android开发人员实习的尾声。

I quickly realized that there were a lot of moving parts to this project. Amazon Go works on the basis of real-time proximity sensors in conjunction with a real-time database of customers and their carts.

我很快意识到，这个项目有很多活动内容。 Amazon Go在实时接近传感器以及客户及其购物车的实时数据库的基础上工作。

We also wanted to take things a step further and make the entry/exit experience seamless. We wanted to let people enter and exit the store without needing to tap their phones.

我们还想更进一步，使出入体验变得无缝。 我们希望让人们无需点击手机即可进入和退出商店。

In order to engage users as a consumer-facing product, our app would need a well-crafted user interface, like the real Amazon Go.

为了使用户成为面向消费者的产品，我们的应用将需要精心设计的用户界面，例如真实的Amazon Go。

On the day before the hackathon, I put together a pseudo-design doc outlining what we needed to do within the 36 hour deadline. I incorporated the strengths of our team and the equipment at hand. The full hastily assembled design doc can be seen below.

在黑客马拉松比赛的前一天，我整理了一份伪设计文档，概述了我们在36小时的期限内需要做的事情。 我吸收了我们团队的力量和手头的设备。 匆忙组装的完整设计文档如下所示。

There were six main components to EZShop, our version of Amazon Go.

我们的Amazon Go版本EZShop有六个主要组件。

Kairos面部识别API (The Kairos Facial Recognition API)

The Kairos facial recognition API was a fundamental component for us. It abstracted the ability to identify and store unique faces. It had two APIs that we used: /enroll and /verify.

Kairos面部识别API是我们的基本组件。 它抽象了识别和存储独特面Kong的能力。 它有两个我们使用的API： /enroll和/verify 。

/enroll is described as:

/enroll描述为：

Takes a photo, finds the faces within it, and stores the faces into a gallery you create.

拍摄照片，在其中找到面Kong，然后将面Kong存储到您创建的画廊中。

We enrolled all new customers into a single “EZShop” gallery. A unique face_id attribute would be returned and stored with the customer’s registered name in our real-time database.

我们将所有新客户纳入一个“ EZShop”画廊。 将返回唯一的face_id属性，并将其与客户的注册名称一起存储在我们的实时数据库中。

When we wanted to verify a potential customer’s image, we would POST it to the /verify endpoint. This would return the face_id with the highest probability of a match.

当我们想要验证潜在客户的图像时，我们会将其发布到/verify端点。 这将返回具有最高匹配概率的face_id 。

In a real-world implementation, it probably would have been a better idea to use a natively implemented facial recognition pipeline with TensorFlow instead of a network API. But given our time constraints, the API served us very well.

在实际的实现中，将本机实现的面部识别管道与TensorFlow而不是网络API一起使用可能是一个更好的主意。 但是考虑到我们的时间限制，该API为我们提供了很好的服务。

实时Firebase数据库 (The Realtime Firebase Database)

The Firebase database was another fundamental piece to our puzzle. Every other component interacted with it in real time. Firebase allows customized change listeners to be created upon any data within the database. That feature, coupled with the easy set-up process, made it a no brainer to use.

Firebase数据库是我们困惑的另一个基本方面。 每个其他组件都与它实时交互。 Firebase允许根据数据库中的任何数据创建自定义的更改侦听器。 该功能，再加上简单的设置过程，使它使用起来毫不费力。

The schema was incredibly simple. The database stored an array of items and an array of users. The following is an example JSON skeleton of our database:

该模式非常简单。 该数据库存储了一组项目和一组用户。 以下是我们数据库的示例JSON框架：

{  "items": [    {      "item_id": 1,      "item_name": "Soylent",      "item_stock": 1,      "price": 10    }  ],  "users": [    {      "face_id": 1,      "name": "Subhan Nadeem",      "in_store": false,      "cart": [        1      ]    }  ]}

New users would be added to the array of users in our database after registering with the Kairos API. Upon entry or exit, the customer’s boolean in_store attribute would be updated, which would be reflected in the manager and personal app UIs.

向Kairos API注册后，新用户将被添加到数据库中的用户数组中。 在进入或退出时，客户的boolean in_store属性将被更新，这将反映在管理器和个人应用程序UI中。

Customers picking up an item would result in an updated item stock. Upon recognizing which customer picked up what item, the item’s ID would be added to the customer’s cart array.

客户提货会产生更新的库存。 识别出哪个客户拿起了什么商品后，该商品的ID将添加到客户的cart阵列中。

I had planned for a cloud-hosted Node/Flask server that would route all activity from one device to another, but the team decided that it was much more efficient (although more hacky) for everybody to work directly upon the Firebase database.

我曾计划过使用一台云托管的Node / Flask服务器，该服务器会将所有活动从一台设备路由到另一台设备，但是该团队认为，每个人都可以直接在Firebase数据库上工作，效率更高(尽管更加hacky)。

经理和个人客户应用 (The Manager and Personal Customer Apps)

John, being the iOS wizard that he is, finished these applications in the first 12 hours of the hackathon! He really excelled at designing user-friendly and accessible apps.

John是他的iOS向导，在黑客马拉松的前12小时内完成了这些应用程序！ 他在设计用户友好和可访问的应用程序方面确实非常擅长。

经理应用 (The Manager App)

This iPad application registered new customers into our Kairos API and Firebase database. It also displayed all customers in the store and the inventory of store items. The ability to interact directly with the Firebase database and observe changes made to it (e.g. when a customer’s in_store attribute changes from true to false) made this a relatively painless process. The app was a great customer-facing addition to our demo.

该iPad应用程序将新客户注册到我们的Kairos API和Firebase数据库中。 它还显示了商店中的所有顾客和商店物品的清单。 直接与Firebase数据库进行交互并观察对其所做的更改(例如，当客户的in_store属性从true更改为false )的能力使此过程变得相对in_store 。 该应用程序是我们演示中面向客户的重要补充。

个人购物应用 (The Personal Shopping App)

Once the customer was registered, we would hand a phone with this app installed to the customer. They would log in with their face (Kairos would recognize and authenticate). Any updates to their cart would be shown on the phone instantly. Upon exiting the store, the customer would also receive a push notification on this phone stating the total amount they spent.

注册客户后，我们会将安装了该应用程序的手机交给客户。 他们将用自己的脸登录(Kairos将识别并认证)。 他们购物车的任何更新都会立即显示在手机上。 离开商店后，客户还将在此电话上收到推送通知，说明他们的消费总额。

物品架，传感器和相机 (The Item Rack, Sensors, and Camera)

Soheil and Ruslan worked tirelessly for hours to perfect the design of the item shelf apparatus and the underlying Pi Python scripts.

Soheil和Ruslan不懈地工作了数小时，以完善商品货架设备和底层Pi Python脚本的设计。

There were three items positioned in rows. At the end of two rows, an ultrasonic proximity sensor was attached. We only had two ultrasonic sensors, so the third row had a light sensor under the items, which did not work as seamlessly. The ultrasonic sensors were connected to the Raspberry Pi that processed the readings of the distance from the next closest object via simple Python scripts (either the closest item or the end of the rack). The light sensor detected a “dark” or “light” state (dark if the item was on top of it, light otherwise).

行中放置了三个项目。 在两行的末端，安装了一个超声波接近传感器。 我们只有两个超声波传感器，因此第三行在这些物品下面有一个光传感器，它不能无缝地工作。 超声波传感器连接到Raspberry Pi，后者通过简单的Python脚本(距离最近的物品或架子的末端)处理距下一个最近物体的距离的读数。 光线传感器检测到“暗”或“亮”状态(如果物品在其上方，则为暗，否则为亮)。

When an item was lifted, the sensor’s reading would change and trigger an update to the item’s stock in the database. The camera (Android phone) positioned at the top of the tower would detect this change and attempt to recognize the customer picking up the item. The item would then instantly be added to that customer’s cart.

提起物料后，传感器的读数将发生变化，并触发数据库中物料库存的更新。 位于塔顶的摄像头(Android手机)将检测到此更改，并尝试识别出正在提货的顾客。 然后，该商品将立即添加到该客户的购物车中。

入口和出口摄像机 (Entrance and Exit Cameras)

I opted to use Android phones as our facial recognition cameras, due to my relative expertise with Android and the easy coupling phones provide when taking images and processing them.

我之所以选择使用Android手机作为我们的面部识别相机，是因为我在Android方面具有相对的专业知识，并且在拍摄和处理它们时所提供的易于耦合的手机也提供了这种功能。

The phones were rigged on both sides of a camera tripod, one side at the store’s entrance, and the other at the store exit.

这些电话被固定在相机三角架的两侧，一侧在商店的入口，另一侧在商店的出口。

Google has an incredibly useful Face API that implements a native pipeline for detecting human faces and other related useful attributes. I used this API to handle the heavy lifting for facial recognition.

Google有一个非常有用的Face API ，该API实现了用于检测人脸和其他相关有用属性的本机管道。 我使用此API处理面部识别的繁重工作。

In particular, the API provided an approximate distance of a detected face from the camera. Once a customer’s face was within a close distance, I would take a snapshot of the customer, verify it against the Kairos API to ensure the customer existed in our database, and then update the Firebase database with the customer’s in-store status.

尤其是，API提供了检测到的面部与相机之间的近似距离。 一旦客户的脸近在咫尺，我将拍摄该客户的快照，并根据Kairos API对其进行验证，以确保该客户存在于我们的数据库中，然后使用该客户的店内状态更新Firebase数据库。

I also added a personalized text-to-speech greeting upon recognizing the customer. That really ended up wowing everybody who used it.

在识别出客户之后，我还添加了个性化的文字转语音问候。 最终，这让所有使用它的人都惊叹不已。

The result of this implementation can be seen here:

该实现的结果可以在这里看到：

Once the customer left the store, the exit-detection state of the Android application was responsible for retrieving the items the customer picked up from the database, calculating the total amount the customer spent, and then sending a push notification to the customer’s personal app via Firebase Cloud Messaging.

一旦客户离开商店，Android应用程序的退出检测状态将负责检索客户从数据库中提取的商品，计算客户花费的总额，然后通过以下方式向客户的个人应用发送推送通知Firebase云消息传递。

Of the 36 hours, we slept for about 6. We spent our entire time confined to a classroom in the middle of downtown Toronto. There were countless frustrating bugs and implementation roadblocks we had to overcome. There were some bugs in our demo that you probably noticed, such as the cameras failing to recognize several people in the same shot.

在这36个小时中，我们睡了大约6个小时。我们整个时间都被限制在多伦多市中心的一间教室里。 我们必须克服无数令人沮丧的错误和实施障碍。 您可能已经注意到了我们的演示中的一些错误，例如相机无法识别同一张照片中的多个人。

We would have also liked to implement additional features, such as detecting customers putting items back on the rack and adding a wider variety of items.

我们还希望实现其他功能，例如检测客户将物品放回货架上并添加更多种类的物品。

Our project ended up winning first place at the hackathon. We set up an interactive booth for an hour (the Chipotle box castle that can be seen in the title picture) and had over a hundred people walk through our shop. People would sign up with a picture, log into the shopping app, walk into the store, pick up an item, walk out, and get notified of their bill instantly. No cashiers, no lines, no receipts, and a very enjoyable user experience.

我们的项目最终在黑客马拉松比赛中获得第一名。 我们设置了一个小时的互动式展位(在标题图片中可以看到Chipotle盒子城堡)，并且有一百多人走过我们的商店。 人们会签上一张图片，登录购物应用程序，走进商店，拿起一件商品，走出去，并立即收到账单通知。 没有收银员，没有排队，没有收据，以及非常愉快的用户体验。

I was proud of the way our team played to each individual’s strengths and created a well put-together full-stack IoT project in the span of a few hours. It was an incredibly rewarding feeling for everybody, and it’s something I hope to replicate in my career in the future.

我为我们的团队发挥每个人的优势而感到自豪，并在几个小时的时间内创建了一个精心组织的全栈物联网项目。 这对每个人来说都是一种令人难以置信的收获，这是我希望将来在我的职业中复制的东西。

I hope this gave you some insight into what goes on behind the scenes of a large, rapidly prototyped, and hacky hackathon project such as EZShop.

我希望这能使您对大型，快速原型化的，骇人的黑客马拉松项目(例如EZShop)的幕后工作有所了解。

Follow me on Twitter and Medium if you’re interested in more in-depth and informative write-ups like this one! I’m always opening to connecting with and learning from other software developers.

如果您对像这样的更深入，更翔实的文章感兴趣，请在Twitter和Medium上关注我！ 我一直乐于与其他软件开发人员联系并向他们学习。

The project is open-source and can be found on Github here. Be warned, hackathon code isn’t pretty!

该项目是开源的，可以在Github上找到 。 警告，hackathon代码并不漂亮！

翻译自: https://www.freecodecamp.org/news/how-we-recreated-amazon-go-in-36-hours-e32a4101d5f0/

amazon创建sns