Industry 4.0, Industrial IoT or Smart Factories. No matter which term you use, all of them refer to the same goal:
increasing efficiency, reliability, security and safety on the factory shop floor. At NXP, we do not only design and
produce innovative chips that are at the core of the next generation of Industry 4.0 applications, we also integrate
Industry 4.0 concepts to increase the efficiency of our factories even further. In this blog, we will show you how
NXP is leveraging its own technology in conjunction with Basler’s advanced embedded vision technology to create a
new visual inspection concept for our factories.
NXP has silicon wafer factories around the world. Within each facility, quality and efficiency are at a very high
level. However, we always seek ways to exceed the requirements of the already demanding semiconductor market. A
typical silicon wafer manufacturing process ends in a delicate packaging process of the wafers in a transport box
before they leave the factory, as Figure 1 illustrates.
Figure 1 – Last stage in silicon wafer manufacturing
Once the transport box is closed, before wrapping, an operator performs a visual inspection. Amongst others, the
inspection intends to detect any incorrect positioning of the wafers within the box. This check is crucial in order
to avoid issues downstream in the supply if wafers are shipped in the wrong position.
The error rate in the boxing process is very low. As a consequence, an operator rarely encounters an error. With such
low hit rates, natural human behavior leads to a higher chance that the rare error is not spotted by the manual
inspection. As part of our continuous improvement activities, the wafer factory is testing a new concept for
automated visual inspection. This automation adds value in terms of efficiency. Also, it will provide evidence of
the status of the transport boxes before they leave the factory.
Behind the Technology
To make this initiative a reality, we joined forces with our official partner Basler AG, a leading global manufacturer of computer vision applications.
Building on the close collaboration in the development of vision-based edge devices, we were able to take advantage
of the strong synergy between our companies. Widely recognized for their extensive expertise on factory automation
and embedded vision solutions, Basler was capable of building the complete vision application at hand. In addition
to delivering a cost-efficient and powerful vision system, Basler also provides the complete application software to
perfectly complement our use case. The powerful industrial-qualified i.MX 8 Family of applications processors from
NXP ensures the high performance needed to efficiently run the algorithms in real time.
For this specific use case, we integrated a Basler Add-on Camera Kit daA2500-60mci-IMX8-EVK, which comes with a 5 Megapixel dart
camera module, cable, lens and a BCON for MIPI to Mini SAS adapter, supporting up to 750 MB/s of data transfer.
Additionally, Basler provided optimized drivers specifically designed for easy integration with various i.MX 8M evaluation
As processing platform, we chose the NXP i.MX 8M Mini evaluation board 8MMINILPD4-EVK, exhibiting an industrial qualified i.MX 8M
Mini Quad applications processor with 4x Arm® Cortex®-A53 @ 1.6 GHz as main CPU platform and 1x Arm
Cortex-M4 @ 400 MHz as low power/security CPU, making this board an excellent candidate for efficiently computing
image processing routines. Besides, the i.MX 8M Mini evaluation board provides the right peripherals to meet the
requirements of this proof of concept. Not only does it support a MIPI-CSI Camera mini-SAS connector; it also comes
with a 1Gbit Ethernet port allowing direct communication with the factory’s manufacturing execution system (MES). It
is a power efficient board and exposes enough digital IO signals capable of interacting with other devices on the
Figure 2 Basler Add-on Camera (daA2500-60mci-IMX8-EVK) with i.MX 8M Mini Evaluation board
The Proof of Concept
The i.MX 8M Mini evaluation board, together with the Basler Add-On Camera Kit, was installed at the station where the
visual inspection takes place. The applications processor was already running an image processing algorithm tuned by
Basler for the specific illumination conditions expected in the clean room. The algorithm uses “classic” image
processing methods, such as segmentation and blob analysis. To communicate with the MES backend via Ethernet, the
application sends/receives JSON messages to/from an MQTT broker.
To start the automatic visual inspection, the transport box with the wafers is positioned on top of an
industrial-grade backlight platform, as Figure 3 shows. The Basler camera is located approximately one meter above
the transport box and is connected to the i.MX 8M Mini board. The applications processor starts the checking process
once it receives a message from the MES backend’s MQTT broker, or when the operator manually activates a digital
input connected to the evaluation board.
Figure 3 – Wafer box finished – Proof of Concept
Once the image is captured and pre-processed by the Basler camera, the actual image processing algorithm searches the
box in the image and aligns it for the subsequent steps (just in case the box is tilted).
Figure 4 Wafer box centered and aligned with regards to subsequent processing steps, even
if box was placed tilted on the backlight
In the following processing steps, the algorithm looks for empty slots as well as for slanted wafer disks or disks
not fully inserted into their slots. Additionally, the disks in the box are counted. As a result of this image
processing routine, the i.MX 8M Mini sends an MQTT message to MES with the conclusion of the inspection as follows:
- OK message if there was no wafer in wrong position or NOK message if at least one wafer was in wrong position.
In case of NOK, a new picture is taken to double check the first result.
- Message with the number of wafers within the transport box.
- Message containing an index that identifies what wafers had a wrong position.
Figure 5 – Automatic identification of position errors
Finally, the 16-bit images of the analyzed wafer are sent to the MES and stored for the traceability of the factory’s
The use of embedded vision technology in the field of factory automation applications is a rising trend as the
benefits are obvious. Due to the form factor that continuously grows smaller, power consumption getting lower and
growing processing performance, it becomes much easier to substitute big PC-based vision systems with embedded
technology—ensuring a lean and cost-efficient solution.
We are closely collaborating with our partners to bring innovative solutions of machine vision for factory
automation. This automated visual inspection system is only one example of what the strong synergy of NXP and Basler
technology can solve in the area of factory automation. We encourage you to visit our webpage and learn more about
our portfolio at: