If you’re a game developer, you’ve probably heard of Vulkan. If you’re an automotive instrument
cluster or infotainment developer, maybe not so much. But you soon will. Vulkan was released in February 2016 and as
a low-overhead graphics and compute API, and it’s for more than just for mobile and desktop computer games.
Vulkan “display” window system integration extension in automotive applications
Automotive suppliers like Continental and NXP participated in the Vulkan specification. We
believe that Vulkan will become one of “the” graphics APIs in automotive.
In just less than a year after it was released, you can find several articles, tutorials and videos about Vulkan on
the web. However, the content goes partly into deep depth and it’s usually
focused on computer games. Here’s where I want to fill the gap with this blog, by providing information about
the potential impact and use of the Vulkan graphics API in automotive. (I am using it in automotive for Human
Machine Interface, Head-Up Display and Infotainment System software applications.)
If you have read about Vulkan, you’ve probably heard about its optimization potential, like memory
consumption, rendering performance and explicit execution control. I’ll explain how automotive applications
can take advantage of the Vulkan API. From architecture to source code snippets related to Vulkan in automotive are
provided in this and following blogs.
About the Vulkan extension in automotive
Vulkan in its core is specified without any windowing system. Any windowing system is added through several
platform dependent extensions. This approach allows more flexibility for defining future or enhancing current
windowing systems for Vulkan. Furthermore, it allows windowless Vulkan applications for compute only without
violating the Vulkan specification.
If something needs to be shown on a display, at the point of writing, the window system integration (WSI) extensions
VK_KHR_swapchain and VK_KHR_surface are required for any operating system. Basically, the swap chain provides the
functionality to bring rendering results done with the Vulkan core API on a surface. The above surface extension
does not define a function to create one. This is defined in several windowing system dependent extensions. For the
Microsoft windows platform, there is the WSI extension VK_KHR_win32_surface, for Linux there are several ones for example,
In automotive applications, there are setups like instrument clusters, where either a windowing system is not
provided by the operating system or the overhead of a windowing system is not wanted at all. In case a Wayland
backend has to be used, the WSI extension VK_KHR_wayland_surface does exist.
I want to have a closer look at the VK_KHR_display extension, as it provides features, which are useful and/or
requested in automotive.
The other WSI extensions normally do take a windowing system dependent argument to create a surface for example, HWND on
Windows. With the discussed extension, no operating system specific types do need to be passed.
This is an advantage for target development:
For example, if the R&D for the target device is done on embedded Linux and later the production code has to run
on another automotive operating system like Integrity or QNX, no adaption has to be done from application side. The
HMI just calls the VK_KHR_display extension functions. The rest is done by the Vulkan driver.
Vulkan WSI extension avoids platform specific code
Usually the development of an HMI is not directly done on the target. Instead, the GUI and other elements are
designed on a desktop system. Typically, Microsoft Windows is used. Unfortunately, even technically possible, the
Vulkan drivers for desktop systems do not provide the VK_KHR_display extension at the moment. In this case, the
native surface extension of the desktop windowing system has to be used. Also, NXP will provide a custom layer,
which is simulating the VK_KHR_display extension on desktop GPUs.
An often seen requirement in automotive applications is the possibility to render into two or more planes, which are
automatically blended together by the underlying hardware. One reason of this is to optimize performance by
rendering content with individual frequency and/or to implement safety features.
Typical minimal render planes setup in automotive
Handling all these planes is normally a proprietary solution depending on the underlying hardware and operating
system. For HMI engine developers, besides implementing and maintaining code for different platforms, this is also
big testing effort. With the described extension, this provided with the driver.
By handling the planes, it is possible now to query for example, the amount of planes, if planes can be reordered or blended,
with a platform independent interface. Furthermore, these parameters can be set during the creation of a plane
Further information on this can be found in the official Vulkan specification, in the “C.2 Window System
Integration (WSI) Extensions” section.