Simplified and Optimized Generative AI at the Edge with eIQ GenAI Flow
eIQ Gen-AI Flow is a modular, end-to-end software pipeline optimized to bring large and small language models (LLMs and SLMs) to NXP applications processors like the i.MX 95, i.MX 93 and i.MX 8M Plus. The pipeline includes wake word, speech-to-text, text-to-speech and Gen-AI models to run on device CPU and/or NPU. Gen-AI Flow also includes retrieval augmented generation (RAG) to securely and easily fine-tune LLMs with custom, private knowledge sources like PDFs.
Software Details
Features
Wake Word Engine (WWE)
- Recognize key phrases and triggers to wake up the conversational AI system. This feature is enabled by NXP Voice Intelligent Technology (VIT).
Automatic Speech Recognition (ASR)
- Convert human generated audio into text, intent (Speech to Intent) and tokens for further processing. ASR uses Whisper_small and Moonshine AI models for mutlilingual speech-to-text.
Large Language Models (LLMs)
- Support for leading open foundation models like Llama, Qwen and Danube with pre-optimized ONNX models quantized and able to run on CPU or NPU.
Text-to-Speech (TTS)
- Convert text and LLM outputs into AI-generated speech for a fully conversational interface. TTS uses the VITS model available in ONNX.
Retrieval Augmented Generation (RAG)
- Fine tune LLMs and SLMs with private and domain-specific knowledge sources. Simply input a PDF and get a compact database out. The database is stored on-device and accessed by the AI model on-demand for efficiency and data privacy.
System Requirements
eIQ Gen-AI Flow Demonstrator
- This demo requires a Linux BSP
eIQ RAG Database Generator
- Linux PC environment. Python 3
Downloads
Quick reference to our software types.
2 downloads
-
Application Example Software
eIQ RAG Database Generator
-
Libraries
eIQ GenAI Flow Demonstrator
Note: For better experience, software downloads are recommended on desktop.
