The U.S. Patent and Trademark Office filed for a patent for a non-contact Biometric ID System that could apply to future libraries, hospitals, offices and stores and more than likely their Amazon Go stores because one of the inventors is listed as Sora Kim, Hardware Engineering Manager at Amazon Go. Another inventor listed is Korwin Smith – Director, Software Development, Amazon Go and Amazon Books. An image of the Amazon Go store is presented as our cover graphic above and when you enlarge it you’ll see a sign in the store window stating: “No lines. No Checkout. Just Walk out Shopping.” ‘Amazon Go’ is a chain of convenience stores in the U.S. operated by Amazon. As of 2019, there are 18 stores open in Seattle, Chicago, San Francisco and New York City.
Amazon’s patent application states that accurate and fast identification of a user provides useful information that may be used in a variety of ways. For example, entry to a material handling facility (facility), office, transportation facility, or other location may be controlled based on user identity.
In another example, information about the identity of the user may also be used to associate particular actions made by that particular user with an associated account.
The facility may include, or have access to, a facility management system. The facility management system may be configured to maintain information about items, users, condition of the facility, and so forth based at least in part on sensor data obtained by one or more sensors. The facility management system may use the sensor data to determine interaction data.
The interaction data may include information about a type of item involved in an interaction, quantity involved in the interaction, whether the interaction was a pick or place, who performed the interaction, and so forth. Interactions may include the user picking an item from an inventory location, placing an item at the inventory location, touching an item at the inventory location, rummaging through items at the inventory location, and so forth. For example, the facility management system may generate interaction data that indicates what item the user picked from a particular lane on a shelf, and then use this interaction data to adjust the count of inventory stowed at that lane.
As the user enters the facility, they may be identified using the devices and techniques described in their patent filing. Once identified, they may be located while in the facility. Information obtained by other sensors, such as weight sensors, cameras, and so forth, in the facility may be used to determine the interaction data.
This interaction data may then be associated with the particular user who has been previously identified, and subsequently used for billing or other purposes. For example, the interaction data and identification data may be used to bill an account associated with the identified user for the item that was picked.
Traditional systems for identifying users suffer from several significant drawbacks including susceptibility to fraud, speed, accuracy, and operational limitations. For example, a traditional system to identify a user by presenting a token, such as an identification card, may be used by someone other than an authorized user. As a result, systems that involve only the use of “something you have” are vulnerable to misuse.
Biometric identification systems deal with this by using a characteristic of the particular individual that is difficult or impossible to copy or be transferred. However, operation of traditional biometric identification systems introduces operational problems and may also exhibit serious latencies in heavy-use environments. For example, traditional palm-based biometric identification systems require physical contact between the user’s hand and a scanning device. This physical contact may be deemed unsanitary and may be difficult to accomplish for some users.
Existing systems are also relatively slow to gather and process information. These and other factors result in existing systems being unsuitable for use in situations where rapid identification of users is called for without significantly impeding the flow of user traffic. For example, the delays introduced by existing systems would produce serious negative impacts such as delays at an entry to the facility which services tens of thousands of users in a given day.
Amazon’s invention covers a system that provides for non-contact biometric identification of users. A scanner device is used to obtain raw images of a user’s palm that is within a field of view of the scanner.
The scanner obtains a first set of one or more raw images that use infrared light with a first polarization and a second set of one or more raw images that use infrared light with a second polarization. The first set of images depict external characteristics, such as lines and creases in the user’s palm while the second set of images depict internal anatomical structures, such as veins, bones, soft tissue, or other structures beneath the epidermis of the skin.
The raw images undergo initial processing to provide a set of images obtained using the first and second polarizations that contain a hand, that the images are well illuminated, in focus, show the hand in a particular orientation, show the hand in a particular canonical pose, rectified, which hand is presented (left or right), and so forth.
Images in this set of images are then divided into sub-images or “patches”. For example, an image that depicts external characteristics may be divided into a set of 15.times.15 sub-images. In some implementations each sub-image may overlap with an adjacent sub-image, while in other implementations the sub-images may exhibit no overlap.
A neural network may be used to determine the feature vectors. For example, a neural network may be trained to recognize features in sub-images. Once trained, the neural network may accept as input a sub-image and produce as output a feature vector that characterizes one or more features present in the sub-image.
Amazon’s Patent FIG. 1 below illustrates a system to identify a user at a facility; FIG. 8 illustrates a flow diagram of a process to rank and select a particular candidate user identifier based on rankings of sub-images.
Amazon’s patent FIG. 2 below illustrates implementations of a scanner used to acquire raw image data of a user’s hand.
Amazon’s patent FIG. 10 below is a block diagram illustrating a materials handling facility (facility) using the system.
The U.S. Patent and Trademark Office published Amazon’s patent application 20190392189 on December 26, 2019. Amazon originally filed the patent on June 21, 2018. To dive deeper into the details of this invention, click here.
Whether Apple will consider such a biometrics system for Apple Store in the future is unknown at this time. However, something similar would definitely be a deterrent to many of the hit and run robberies that Apple Stores experienced in the last two years (01 & 02).
