Humans like to feel known and understood. It is both practical and comforting to be in situations where your needs can be anticipated and personalized, to who you are and what you like. The hit 80’s TV show Cheers struck a chord in popular culture, highlighting the emotional satisfaction humans feel when they are some place where everyone knows them. The concept of showing up, sitting down and your favorite drink being prepared for you, without you ever saying a word, has a near universal emotional appeal. Of course in Cheers, it was a human bartender who recognized the unspoken need.
More recently, the global block-buster movie character Tony Stark highlights the practical advantages of using advanced human machine interface technologies, J.A.R.V.I.S., to combine acquired knowledge of his personal preferences with his verbal commands, to easily and privately execute precise action sequences in real-time.
Friction-free interfacing is the concept of leveraging rapidly emerging machine learning capabilities, such as voice and face recognition, to enable your everyday devices to anticipate and personalize their responses to meet your specific needs. In human terms, it combines the emotional appeal of having your preferences addressed, without ever needing to utter a single word, with the practical advantage of using voice to effect more complex actions.
Going beyond whimsical TV shows and science fiction characters, the first generation of low-cost, pervasive, friction-free interfaces is now upon us. The best way to think of the value these machine interfaces can provide consumers is to think of them as a mix of the Cheers bartender, who immediately recognizes someone and provides them with what they want and J.A.R.V.I.S. like voice recognition capabilities, which combine preferences with verbal commands to drive real-time actions.
At the IFA 2019 show in Berlin Germany, September 6, NXP announced two additional low-cost MCU-based solutions for offline local voice commands and face recognition. These solutions, based on NXP’s i.MX RT1060 crossover MCU family, join the previously announced i.MX RT106A MCU-based solution for Amazon’s Alexa Voice Service.
These solutions address the need for consumers to have their interactions with their devices, big and small, actively account for their personal preferences and anticipate their needs. If you always do your laundry with warm water, why have to enter that every time? Or maybe you always want crushed ice, or warm tea or hot coffee. If you have a larger family like I do, you don’t want to keep reconfiguring your machines. In short, consumers want to feel known in their own homes, as well as where they work and where they socialize. Why can’t you just walk up to your machine and have it recognize who you are and respond automatically by anticipating what you want, based on its private records of your past behavior? If by chance you want to do something different or more complex, you can simply tell the machine via short verbal command.
Forward thinking OEMs/ODMs understand these trend lines and recognize the rapidly emerging need to differentiate their products by going beyond the standard higher friction user interfaces available today, including legacy technologies such as smartphone apps, touch panels, push buttons, rotary knobs and LEDs and replacing them with interfaces with radically less friction.
Alexa started the trend a few years ago but represents just the first piece of the puzzle and isn’t well suited for all situations and all people. Alexa does most of the heavy lifting in the cloud but has connectivity requirements that result in cost, latency and privacy concerns. Also, until NXP announced the first (and still only) low-cost MCU based solution qualified by Amazon for Alexa, traditional voice and face recognition systems required an MPU based system operating on Linux.
By adding MCU-based local voice commands and facial recognition to its IoT Solutions portfolio, NXP significantly increases the capabilities available to innovative OEMs and ODMs, to reduce the friction and cost of their interfaces for their high volume price sensitive devices, turning science fiction into reality for everyday people, not just movie super heroes like Tony Stark.
NXP’s MCU-based solution for offline far field automatic speech recognition (ASR) leverages the i.MX RT106L crossover MCU, enabling developers to quickly and easily add local voice control capabilities to their products. This turnkey design, with ultra-small form factor, comes completely integrated with all software and speech recognition models for an example wake word and four sets of commands, enabling easy out-of-the-box local voice control proof of concept (PoC) development.
The local commands solution provides OEMs with a fully integrated, self-contained, software and hardware solution that includes both the MCU and an NXP smart audio amplifier with speaker protection. It comes with an ASR engine for the local recognition of commands and wake word and all required far field audio processing algorithms, including noise suppression, beamforming and echo cancellation (barge-in) capabilities, to enable use in acoustically difficult environments. This cost-effective, easy to use implementation facilitates the demand for low latency and private voice control embedded in a diverse variety of products across home, commercial and industrial applications. It gives users hands-free voice control, without the privacy concerns associated with cloud-based services.
NXP’s MCU-based machine vision solution leverages the i.MX RT117F crossover MCU enabling developers to quickly and easily add face recognition capabilities to their products. This ultra-small, form-factor, production ready hardware design comes with fully integrated software running on FreeRTOS, for quick out of the box implementation. This turnkey solution minimizes time to market, risk and development effort enabling OEMs to easily add machine vision to their smart home and smart appliance products without the need for an expensive Linux-based MPU implementation and without Wi-Fi and cloud connectivity, addressing the privacy concerns of many consumers.
This face recognition solution provides OEMs with a fully integrated, self-contained, software and hardware platform. It includes the NXP OASIS s face and emotion recognition runtime library, a set of pre-integrated machine learning algorithms, as well as required drivers for peripherals such as camera, memories and optional connectivity features. From an optimized, small form factor reference hardware to fully tested, documented and supported software, the solution is designed to accelerate time to market and reduce complexity for OEMs and ODMs.
These MCU-based solutions for local voice control (i.MX RT106L) and face recognition (i.MX RT106F) enable designers to integrate voice and face recognition capabilities into a wide variety of smart home, appliance, retail and industrial products including:
Bill Krakar is a Software and Systems Product Manager for Internet of Things (IoT) solutions, with key responsibilities defining IoT solutions for Kinetis and i.MX processors. Bill has a Bachelor of Science in Electrical Engineering from University of Illinois. If Bill's not putting together an IoT solution, you'll find him exploring on a bike.
