Linux Graphics Stack - An Overview

Hi all, So a lot has happened in recent times, I got a new job at Qualcomm. Yayy! I suppose. I will be working on something rather out of my comfort zone, maybe too much out of it. I will be working on a bit of GPU drivers, primarily with User Mode Drivers (UMDs) and the linux graphics stack and drop into the unknown territory like compositors and using mesa drivers etc. So I thought I should make a little self-learning post on how various libraries are used and how the linux graphics stack looks from a high-level persepctive. All this new stuff is very fun and exciting to learn especially the Adreno GPUs. Also one more thing… I’m finally moving back to my hometown Hyderabad, yeah I know I will be missing Bangalore a lot, especially the people and my Sony ex-colleagues and the amazing firnedships I made over the last few years. Idk I felt like I needed some change to really push myself and I think it’s a nice memory/experience to get to work in your hometown and ofc all things aside the work seems hella lot of fun so I finally decied to pull the trigger to move here for a while. Let’s see how this chapter goes. Now, shall we dive in?

Overview

The linux graphics stack is very complex and for someone like me coming from a API level it can seem very daunting but it get’s acquaintable as you spend more time with and take time to understand what each layer does and what different libs and technologies are available as alternatives. Let’s go through each later from the App to GPU HW and see what the stack contains.

1. App Level

At the very top of the stack we have is the applications we run, they can be the games we play, browsers for “Netflix and Chill” and GUI Apps like calculators, Blender etc. They use components like Graphics APIs (Opengl, Vulkan etc.), GUIs libraries (GTK, QT etc.) to render.

2. Desktop Shell/UI

One level down we have desktop environments like GNOME, KDE, XFCE, and others (depending on your Linux distribution). These environments provide the user interface shell, including panels, docks, and application launchers. They typically use window managers to handle the positioning, resizing, and stacking of application windows.

3. Display Servers, Compositors, and WSI

Display servers: They handle communication between applications and the display hardware. They provide a protocol for Window System Integration (WSI) that allows multiple windows to be managed and displayed. The two major display server protocols in Linux are:

Compositors: combine the contents of multiple windows into a final image for display. They handle effects like transparency, animations, and window transitions. Some notable examples:

Libraries that facilitate this communication include:

Misc:

4. Graphics APIs

This layer includes the 3D accelerated graphics APIs:

These libraries are just wrappers that call into the User Mode Driver (UMD) implementations.

5. UMD - User Mode Drivers

This is the user space driver component.

UMD implementations include:

Note: Mesa uses Gallium3D as an abstraction layer for most drivers, Exception: Turnip (Qualcomm Vulkan driver) is written from scratch

6. KMD - Kernal Mode Drivers

This is the kernel space driver component manages direct GPU HW access.

DRM KMS and GBM

The Direct Rendering Manager (DRM) is a kernel subsystem that manages access to GPU hardware.

Kernel Mode Setting (KMS) is a mechanism that allows the kernel to set display modes (resolution, color depth, refresh rate).

The Generic Buffer Manager (GBM) is an API that provides a mechanism for allocating buffers for graphics rendering. It works with DRM/KMS to manage display buffers and facilitate zero-copy rendering paths.

7. GPU HW

Last but not the least we have the GPU hardware itself with their own ISA as per the vendor.

Closing Notes

I hope this gives a very high-level insights into how different linux graphics components intreract to render the beautiful pixels we see. Also stay tuned for the wild stories from hyderabad diaries. I will try to slip in some life updates from time to time.

Until they see ya!

References

  1. The DRM/KMS subsystem from a newbie’s point of view
  2. Wayland Architecture
  3. Mesa3D Documentation
  4. Vulkan WSI (Window System Integration) Extensions
  5. Intro to Linux Graphics Stack
  6. X Window System core protocol Wikipedia