Embedded AI: Artificial intelligence for decentralized decisions at the network edge

Embedded AI is increasingly becoming a central driver of modern edge systems. With this technology, artificial intelligence is moving from the cloud directly to where it is needed - to the devices at the edge of the network. Embedded hardware with low power consumption ensures cost and energy savings. Local processing protects data and allows offline operation, while edge AI supports real-time decision making with minimal latency. This is required for autonomous systems and industrial automation.

Market growing rapidly thanks to specialized hardware

The market for embedded AI is growing rapidly, driven by specialized hardware components such as neural processing units (NPUs) and heterogeneous architectures that are increasingly being integrated directly into microcontrollers and systems on chips (SoCs). This presents engineers with the challenge of implementing complex models on devices with limited memory and computing resources. Techniques such as quantization, pruning and other methods of model compression are therefore essential. In addition to hardware, powerful libraries and tools are becoming increasingly important in order to make AI reliably usable across different platforms.

A concrete example of this trend is the combination of wake word recognition with object recognition and tracking on Qualcomm Snapdragon platforms. YOLOX-based networks are used for object recognition, with inference outsourced to the Hexagon NPU. The NPU is only activated when necessary if a corresponding wakeword (audio signal) was previously given and this was recognized by a second, more energy-efficient NPU. This approach shows how heterogeneous architectures that combine a low-power NPU with Hexagon DSP enable real-time image processing tasks while maintaining energy efficiency.

Shift Left for smart AI

As the complexity of embedded AI increases, so does the need for clearly structured workflow processes. Model-based design provides a framework for this: Instead of writing low-level code, engineers model their algorithms visually in Simulink. Requirements, models and test artifacts are brought together in a uniform digital thread, which supports collaboration and traceability throughout the entire product life cycle.

A key advantage of this approach is early validation. With the help of hardware-in-the-loop (HIL) and processor-in-the-loop (PIL) tests, potential problems can be identified at an early stage and development can be accelerated, in line with the «shift left» principle. How this approach works can be explained using an example from the automotive industry. In AI-based trajectory planning and control on Infineon AURIX TC4x, neural network controllers are designed and validated using model-based design before deployment. By using a parallel processing unit (PPU) on the hardware, this solution achieves 50 percent higher accuracy and 5 percent energy savings compared to traditional approaches - an example of how structured workflows and hardware-aware optimization deliver tangible benefits.

From Matlab to the microcontroller

MathWorks provides an integrated environment for developing and deploying AI on embedded systems. Engineers can design and train their models in Matlab or import pre-trained models from frameworks such as PyTorch, TensorFlow and ONNX. Using automatic code generation tools such as Matlab Coder and GPU Coder, these models can be translated into optimized C, C++, CUDA or HDL code for CPUs, GPUs, FPGAs and MCUs. This closes the gap between high-level design and hardware implementation.

Optimization workflows for quantization, pruning and compression are also integrated into the toolchain. They enable use on devices with limited resources without compromising performance. In addition, verification tools such as Polyspace ensure additional reliability by analyzing the generated code statically and dynamically, detecting errors early in the development cycle - or proving their absence. These functions are particularly important in safety-critical areas to ensure compliance and robustness.

A practical example of this workflow is temperature prediction for electric motors on TI C2000 hardware. Virtual sensor models are developed and trained in Matlab or Python, compressed for use and validated using processor-in-the-loop tests. This approach replaces physical sensors with software-based estimators, reducing cost and complexity while maintaining or even increasing accuracy.

Use in numerous application areas

Embedded AI is already being used in numerous areas of application: in healthcare, for example, it is used for real-time signal processing in ECG analysis on STM32 boards. Deep learning and signal processing algorithms are combined here. Sensor data can be processed in real time with the help of automatic C/C++ code generation. Embedded AI solutions are also used in the field of occupational safety: the detection of personal protective equipment on Raspberry Pi and pose detection on Nvidia Jetson platforms show how embedded image processing and GPU acceleration enable compact, powerful AI solutions for monitoring and compliance.

These examples are indicative of a broader trend: embedded AI is no longer a niche technology, but is becoming a standard for intelligent systems in various industries. Through structured workflows and the use of integrated toolchains, engineers can accelerate their development processes, ensure their reliability and optimize performance for edge deployments. As generative AI and advanced workflows move ever closer to the edge, efficient methods and optimization with hardware in mind will be key to realizing their full potential.

MathWorks will be presenting various key topics and application scenarios to engineers and developers at embedded world from March 10 to 12, 2026 in Nuremberg (Hall 4, Booth 110) in a series of technical presentations. Visitors to the stand will gain insights into various demos, including AI-based trajectory planning on Infineon AURIX TC4x, object tracking on Qualcomm Snapdragon, AI-based temperature calculation for electric motors and real-time signal processing on IoT devices.

This article originally appeared on m-q.ch - https://www.m-q.ch/de/embedded-ai-kuenstliche-intelligenz-fuer-dezentrale-entscheidungen-am-netzwerkrand/