Introduction

Every pixel on your screen is a number in memory. A 1920×1080 display at 32 bits per pixel is 8,294,400 bytes — about 8 megabytes. Sixty times per second, a piece of hardware reads those bytes and converts them into light. That is all a display does.

Everything else — windows, buttons, 3D worlds, web pages, fonts, transparency, animations — is layers of software between your code and that 8 MB buffer. This article traces the full path, from the electrical signals hitting your monitor to the <div> tags in your browser. At every level, there is a code example you can study, and often run.

I. The Hardware: What a Display Actually Is

The Framebuffer Concept

A framebuffer is a region of memory where each address maps to a pixel on screen. The concept is simple: pixel (0,0) is at byte offset 0, pixel (1,0) is at byte offset 4 (for 32-bit color), and so on across each row. The display hardware reads this memory sequentially and converts each value into a color.

For a 1920×1080 display with 4 bytes per pixel (RGBA8888):

Pixel Formats

Different systems use different pixel formats, trading color depth for memory:

Pixels are stored in row-major order: all pixels of row 0, then all pixels of row 1, and so on. The stride (or pitch) is the number of bytes per row. Stride may be larger than width × bpp because hardware often requires rows to be aligned to 64-byte or 256-byte boundaries for efficient DMA transfer.

GPU Architecture and Scanout

Modern GPUs have their own dedicated memory (VRAM) connected to the display controller via a high-bandwidth internal bus. The display controller is a dedicated hardware unit that reads from a specific region of VRAM — the scanout buffer — and converts those pixel values into a timed signal for the monitor.

  CPU ←──PCIe──→ GPU
   ↕                ↕
  RAM             VRAM
                    ↕
              Display Controller
                    ↕
            DisplayPort / HDMI
                    ↕
                 Monitor

The display controller reads the scanout buffer pixel by pixel, row by row, at the display’s refresh rate. For a 60 Hz display, it reads the entire buffer 60 times per second. This sequential reading is called scanning out — a term inherited from CRT monitors, where an electron beam physically scanned across the screen.

VSync, Tearing, and Double Buffering

Tearing happens when the CPU/GPU writes to the framebuffer while the display controller is reading from it. The top half of the screen shows the old frame and the bottom half shows the new frame, creating a visible horizontal seam.

The solution is double buffering: maintain two buffers. The display controller reads from the front buffer while the GPU draws to the back buffer. When the frame is complete, the buffers swap. VSync synchronizes this swap to the monitor’s vertical blanking interval — the brief pause when the display controller finishes one frame and starts the next — so the swap happens between frames, never during one.

Triple buffering adds a third buffer so the GPU can start the next frame immediately without waiting for VSync, reducing latency while still preventing tearing.

II. The Display Stack: From Kernel to Compositor

Kernel Graphics Drivers

The kernel manages hardware, including the GPU. Each operating system has its own graphics driver subsystem:

On Linux, the DRM subsystem provides a device file (/dev/dri/card0) for GPU access and KMS handles display mode setting (resolution, refresh rate). An older, simpler interface — the Linux framebuffer device — gives even more direct access.

Linux Framebuffer: /dev/fb0

On Linux, /dev/fb0 is a character device that represents the framebuffer directly. You can open() it, mmap() it into your process’s address space, and write pixels. No GPU driver, no window system, no libraries. This is the closest you can get to raw pixel access on a modern operating system.

This writes directly to the display hardware. No window system, no compositor, no GPU acceleration. Just bytes in memory becoming pixels on screen. This is as close to the metal as userspace code gets.

Display Servers and Compositors

Raw framebuffer access is fine for a single fullscreen application. But modern computing requires multiple windows, overlapping content, transparency, and smooth animations. This is the job of the display server and compositor.

The compositor takes each application’s rendered buffer, applies transformations (position, transparency, blur, shadows), and composites them into a single final image that gets sent to the display controller for scanout.

The macOS Question: Can You Write Directly to Pixels?

III. The GPU Rendering Pipeline

Why GPUs Exist

A 1080p display has 2,073,600 pixels. At 60 fps, that is 124 million pixels per second. For each pixel, you might need to compute lighting, texture sampling, transparency, and shadows. A CPU processes instructions one at a time (or a few at a time with superscalar execution). A GPU processes thousands simultaneously.

The key insight: graphics computations are embarrassingly parallel. The color of pixel (100, 200) does not depend on the color of pixel (500, 300). Each vertex transformation is independent. Each pixel shading is independent. GPUs exploit this by having thousands of small cores, each running the same program on different data. An NVIDIA RTX 4090 has 16,384 CUDA cores. They are individually slower than a CPU core, but there are thousands of them.

The Rendering Pipeline

Modern GPU rendering follows a pipeline. Data flows through stages, some programmable (shaders), some fixed-function (rasterization):

  Vertices (3D coordinates)
       ↓
  ┌─────────────────────┐
  │   Vertex Shader     │  ← Programmable: transform each vertex
  │   (per-vertex)      │     (model → world → view → clip space)
  └─────────┬───────────┘
            ↓
  ┌─────────────────────┐
  │ Primitive Assembly   │  ← Fixed: group vertices into triangles
  └─────────┬───────────┘
            ↓
  ┌─────────────────────┐
  │   Rasterization     │  ← Fixed: determine which pixels each
  │                     │     triangle covers, interpolate attributes
  └─────────┬───────────┘
            ↓
  ┌─────────────────────┐
  │  Fragment Shader    │  ← Programmable: compute color for each
  │  (per-pixel)        │     pixel (lighting, textures, effects)
  └─────────┬───────────┘
            ↓
  ┌─────────────────────┐
  │ Depth Test & Blend  │  ← Fixed: depth testing, alpha blending
  └─────────┬───────────┘
            ↓
       Framebuffer

The vertex shader runs once per vertex. If you draw a triangle with 3 vertices, the vertex shader runs 3 times. The fragment shader runs once per pixel the triangle covers. A triangle covering 10,000 pixels runs the fragment shader 10,000 times — all in parallel across the GPU’s cores.

Shaders

Shaders are small programs that run on the GPU. Different APIs use different shading languages:

The vertex shader transforms each vertex from model space into clip space (the coordinate system the GPU uses for clipping and rasterization). The fragment shader produces the final color for each pixel. Between them, the rasterizer determines which pixels each triangle covers and interpolates any varying data (colors, texture coordinates) across the surface.

IV. The Landscape: Graphics APIs by Abstraction Level

There are many graphics APIs. They exist at different levels of abstraction, from raw pixel buffers to declarative layout systems. Understanding the landscape means understanding what each level gives you and what it takes away.

  Level 6  ┌──────────────────────────────┐  matplotlib, D3.js, ggplot2
           │    Domain-Specific Viz       │  "plot this data"
  Level 5  ├──────────────────────────────┤  HTML+CSS, SwiftUI, Flutter
           │    Declarative / Layout      │  "describe desired state"
  Level 4  ├──────────────────────────────┤  GTK, Qt, Cocoa, Swing
           │    Widget Toolkits           │  "buttons, text fields, windows"
  Level 3  ├──────────────────────────────┤  Canvas, Skia, Cairo, SDL2
           │    2D Graphics Libraries     │  "draw shapes, lines, text"
  Level 2  ├──────────────────────────────┤  OpenGL, WebGL, DirectX 11
           │    Managed GPU APIs          │  "draw triangles with shaders"
  Level 1  ├──────────────────────────────┤  Vulkan, Metal, DirectX 12
           │    Low-Level GPU APIs        │  "manage GPU memory & commands"
  Level 0  ├──────────────────────────────┤  /dev/fb0
           │    Direct Framebuffer        │  "write bytes to pixel memory"
           └──────────────────────────────┘

Level 0 — Direct Framebuffer

We covered this in Section II: open /dev/fb0, mmap it, write pixels. No GPU, no acceleration, no compositing. Available on Linux only. Useful for embedded systems, boot splash screens, and understanding what is really happening at the bottom of the stack.

Level 1 — Low-Level GPU APIs (Vulkan, Metal, DirectX 12)

These APIs give you explicit control over the GPU. You manage memory allocation, command buffer recording, synchronization between CPU and GPU, pipeline state objects, descriptor sets, and render passes. The driver does almost nothing for you — which is the point. You can optimize exactly for your workload.

The cost is complexity. Drawing a single colored triangle in Vulkan requires approximately 800–1200 lines of C code. Here is a simplified view of the initialization steps:

This is why Vulkan is not for casual use. It exists for game engines, professional renderers, and performance-critical applications where the overhead of a managed API is unacceptable. Vulkan is cross-platform (Windows, Linux, Android). Metal is Apple-only. DirectX 12 is Windows/Xbox-only.

Level 2 — Managed GPU APIs (OpenGL, WebGL, DirectX 11)

These APIs manage GPU state for you. The driver handles memory allocation, synchronization, and pipeline state behind the scenes. You get a much simpler programming model at the cost of less control and more driver overhead.

OpenGL uses a global state machine: you set state (bind a texture, set the blend mode, activate a shader), then draw. The driver figures out the rest. A colored triangle takes about 45 lines of C instead of 1000.

Level 3 — 2D Graphics Libraries (Canvas, Skia, Cairo, SDL2)

These libraries provide 2D drawing primitives: lines, rectangles, circles, paths, fills, strokes, text rendering, and image blitting. Under the hood, they may use the GPU (Skia uses OpenGL/Vulkan/Metal; Canvas uses Skia through the browser), but you work with a painter’s model: draw shape A, then draw shape B on top of it.

Notice the pattern: set style, then draw. This is immediate mode 2D — there is no retained scene graph. If you want to animate, you clear the canvas and redraw everything each frame.

SDL2 gives you a cross-platform pixel buffer that works everywhere. Under the hood, it uses the native graphics API (Metal on macOS, Direct3D on Windows, OpenGL/Vulkan on Linux) to upload your pixel buffer as a texture and display it. This is the practical solution when you want to write individual pixels without dealing with GPU APIs directly.

Level 4 — Widget Toolkits (GTK, Qt, Cocoa, Swing)

Widget toolkits give you pre-built UI components: buttons, text fields, scroll views, menus, dialogs. You build a tree of widgets, the toolkit handles layout, event dispatch, and rendering. This is retained mode: you create objects that persist, and the framework manages their state and appearance.

Level 5 — Declarative / Layout Systems (HTML+CSS, SwiftUI, Flutter)

Declarative systems let you describe the desired state of your UI, and the framework figures out how to render it. You do not issue draw commands. You do not manage widget objects directly. You declare what the UI should look like, and the framework handles layout, rendering, and updates when state changes.

HTML+CSS is the most successful declarative UI system ever built. You describe the structure (HTML) and styling (CSS), and the browser’s layout engine computes positions, the paint engine renders pixels, and the compositor assembles layers for the GPU. No code example needed here — this article is itself an HTML+CSS document.

SwiftUI and Flutter follow the same philosophy: declare what you want, the framework diffs old state vs. new state, and efficiently updates only what changed. React brought this pattern to the web with its virtual DOM.

Level 6 — Domain-Specific Visualization (matplotlib, D3.js, ggplot2)

At the highest level, you do not think about pixels, shapes, or even layouts. You think about data. You hand an array of numbers to a library and get a chart. The library handles axes, labels, colors, legends, tick marks, and layout — all the visual design decisions that would take hundreds of lines at a lower level.

Those 15 lines of Python produce a publication-quality scatter plot with color mapping, axes, labels, and a colorbar. To do this with raw OpenGL, you would need to implement text rendering (which alone is hundreds of lines), axis drawing, tick computation, color interpolation, and anti-aliased point rendering. That is the value of abstraction.

V. The Three Paradigms

Across all these levels, there are three fundamental approaches to building graphical interfaces. Understanding them is more important than knowing any specific API, because every API you encounter will fit one of these paradigms.

Immediate Mode

In immediate mode, you redraw everything every frame. There is no persistent scene graph, no retained objects. Each frame, your code issues draw commands from scratch. The API is stateless between frames (or nearly so).

Examples: OpenGL, HTML5 Canvas, SDL2 rendering, Dear ImGui, video games. The advantage is simplicity — no state synchronization between your data and the UI. The cost is that you must redraw everything, though GPU acceleration makes this fast for most workloads.

Retained Mode

In retained mode, you build a tree of persistent objects that the framework manages. You create a button once, and the framework draws it, handles events, and redraws it when needed. You modify the tree to update the UI.

Examples: the browser DOM, Java Swing, Qt, GTK, Cocoa. The advantage is that the framework can optimize redraws (only repaint what changed). The cost is state synchronization — your data model and the UI tree can get out of sync, leading to stale state bugs.

Declarative

Declarative systems are a refinement of retained mode. You describe the desired state as a function of your data, and the framework diffs the old and new descriptions, computing the minimal set of changes. This solves the state synchronization problem of traditional retained mode.

Examples: React (virtual DOM diffing), SwiftUI, Flutter, Jetpack Compose. You write UI = f(state) and the framework handles the rest.

VI. Tying It All Together

Now we can trace the full path from code to pixels for two common scenarios.

Trace: How a Web Page Reaches Your Screen

  HTML + CSS + JavaScript
       ↓
  Browser parses HTML → builds DOM tree
       ↓
  Browser parses CSS → builds CSSOM
       ↓
  DOM + CSSOM → Render tree (only visible elements)
       ↓
  Layout engine computes positions and sizes
       ↓
  Paint: generate display list of draw commands
       ↓
  Rasterization: Skia converts draw commands to pixels
       ↓
  Compositing: GPU composites layers (transforms, opacity)
       ↓
  OS compositor (WindowServer / DWM / Wayland)
       ↓
  Display controller reads final framebuffer
       ↓
  DisplayPort/HDMI signal → Monitor → Photons → Your eyes

The browser is a retained-mode system (the DOM is a persistent tree) running on top of an immediate-mode 2D graphics library (Skia redraws affected regions), which uses a managed GPU API (OpenGL/Vulkan/Metal), which talks to the GPU through kernel drivers (DRM/WDDM/IOKit), which writes to a framebuffer that the display controller scans out.

Trace: How a Game Frame Reaches Your Screen

  Game loop: process input → update physics → AI → game state
       ↓
  Scene graph traversal → generate draw calls
       ↓
  For each object: bind textures, set uniforms, draw triangles
       ↓
  GPU vertex shaders transform vertices to screen space
       ↓
  Rasterizer determines which pixels each triangle covers
       ↓
  GPU fragment shaders compute per-pixel color
       ↓
  Depth test, alpha blending → back buffer
       ↓
  VSync: swap back buffer → front buffer
       ↓
  OS compositor (may be bypassed in exclusive fullscreen)
       ↓
  Display controller scans out → Monitor

Games are immediate mode — the entire scene is redrawn every frame. The game engine generates GPU commands (Vulkan/Metal/DirectX 12), the GPU executes them in parallel across thousands of cores, the result goes to the back buffer, and VSync swaps it to the front for scanout.

The Common Thread

Every graphical application, from a matplotlib chart to a AAA game to this web page, ends the same way: a region of memory is filled with pixel color values, a display controller reads that memory at the refresh rate, and a cable carries the signal to a panel that emits light. The question is never whether pixels end up in a framebuffer — they always do. The question is how many layers of abstraction sit between your code and those pixels, and what each layer buys you.

A framebuffer write on Linux is zero layers: your code writes the bytes directly. SDL2 is one layer: it handles the window and the pixel format. OpenGL is two layers: you describe geometry and shaders, the GPU fills the pixels. The browser is six or seven layers deep: HTML, CSS, DOM, layout, paint, Skia, the GPU, the compositor, and the display controller.

More layers means more convenience, more portability, and more protection from hardware details. Fewer layers means more control, more performance, and a better understanding of what is actually happening. The right level depends on what you are building. But now you know what every level is doing, and what it is hiding.