Not Just for Gaming: Vulkan Graphics Brings Maximum Performance to Automotive

If you’re a game developer, you’ve probably heard of Vulkan. If you’re an automotive instrument cluster or infotainment developer, maybe not so much. But you soon will. Vulkan was released in February 2016 and as a low-overhead graphics and compute API, and it’s for more than just for mobile and desktop computer games.

Vulkan “display” window system integration extension in automotive applications

Automotive suppliers like Continental and NXP participated in the Vulkan specification. We believe that Vulkan will become one of “the” graphics APIs in automotive.

In just less than a year after it was released, you can find several articles, tutorials and videos about Vulkan on the web. However, the content goes partly into deep depth and it’s usually focused on computer games. Here’s where I want to fill the gap with this blog, by providing information about the potential impact and use of the Vulkan graphics API in automotive. (I am using it in automotive for Human Machine Interface, Head-Up Display and Infotainment System software applications.)

If you have read about Vulkan, you’ve probably heard about its optimization potential, like memory consumption, rendering performance and explicit execution control. I’ll explain how automotive applications can take advantage of the Vulkan API. From architecture to source code snippets related to Vulkan in automotive are provided in this and following blogs.

About the Vulkan extension in automotive
Vulkan in its core is specified without any windowing system. Any windowing system is added through several platform dependent extensions. This approach allows more flexibility for defining future or enhancing current windowing systems for Vulkan. Furthermore, it allows windowless Vulkan applications for compute only without violating the Vulkan specification.

If something needs to be shown on a display, at the point of writing, the window system integration (WSI) extensions VK_KHR_swapchain and VK_KHR_surface are required for any operating system. Basically, the swap chain provides the functionality to bring rendering results done with the Vulkan core API on a surface. The above surface extension does not define a function to create one. This is defined in several windowing system dependent extensions. For the Microsoft windows platform, there is the WSI extension VK_KHR_win32_surface, for Linux there are several ones for example, VK_KHR_xlib_surface.

In automotive applications, there are setups like instrument clusters, where either a windowing system is not provided by the operating system or the overhead of a windowing system is not wanted at all. In case a Wayland backend has to be used, the WSI extension VK_KHR_wayland_surface does exist.

I want to have a closer look at the VK_KHR_display extension, as it provides features, which are useful and/or requested in automotive.

The other WSI extensions normally do take a windowing system dependent argument to create a surface for example, HWND on Windows. With the discussed extension, no operating system specific types do need to be passed.

This is an advantage for target development:

For example, if the R&D for the target device is done on embedded Linux and later the production code has to run on another automotive operating system like Integrity or QNX, no adaption has to be done from application side. The HMI just calls the VK_KHR_display extension functions. The rest is done by the Vulkan driver.

Vulkan WSI extension avoids platform specific code

Usually the development of an HMI is not directly done on the target. Instead, the GUI and other elements are designed on a desktop system. Typically, Microsoft Windows is used. Unfortunately, even technically possible, the Vulkan drivers for desktop systems do not provide the VK_KHR_display extension at the moment. In this case, the native surface extension of the desktop windowing system has to be used. Also, NXP will provide a custom layer, which is simulating the VK_KHR_display extension on desktop GPUs.

An often seen requirement in automotive applications is the possibility to render into two or more planes, which are automatically blended together by the underlying hardware. One reason of this is to optimize performance by rendering content with individual frequency and/or to implement safety features.

Typical minimal render planes setup in automotive

Handling all these planes is normally a proprietary solution depending on the underlying hardware and operating system. For HMI engine developers, besides implementing and maintaining code for different platforms, this is also big testing effort. With the described extension, this provided with the driver.

By handling the planes, it is possible now to query for example, the amount of planes, if planes can be reordered or blended, with a platform independent interface. Furthermore, these parameters can be set during the creation of a plane surface.

Further information on this can be found in the official Vulkan specification, in the “C.2 Window System Integration (WSI) Extensions” section.