Graphics is an OS problem, compute is a hardware problem

... and WebGPU is stuck in the middle.

Rendering graphics quickly and correctly has always been a software problem riding on a hardware problem. Rendering is handled by an operating system (OS) and it typically uses a hardware (GPU) to perform heavy compute and render the pixels. Operating systems want portability and cohesiveness. Hardware systems want specilization and tight integrations. Developers want both. These goals are inherently in tension which has led to a high impedance mismatch.

OS-for-any-hardware vs hardware-for-any-OS

Graphics is an OS problem while compute is a hardware problem. OS vendors provide APIs which aim to run any GPU on a specific OS - DirectX by Windows and Metal by Apple in particular. Hardware vendors provide APIs like CUDA and ROCm which aim to run on any operating system.

Nvidia GPUs are not supported on Apple devices. AMD's GPUs are mostly Linux-centric. So it is wrong to claim that they run on any operating system. However the rhetorical framing highlights that hardware strategy has been heavily focused on silicon optimization with equal (dis)regard for the OS the device is running on.

Someone working on dev tooling has to find a good balance between the two opposing forces.

Vendor lock-ins

Both OS and GPU vendors have effectively used lock-ins as a business strategy. DirectX is tightly coupled with Windows OS components and tooling ecosystem, and Metal with Apple OS. Nvidia's lock-in with CUDA is a huge factor behind its success and has enabled strong vertical integration in the hardware space.

While both Windows and Apple have vendor lock-ins, their approaches are a bit different. For DirectX, GPU vendors have traditionally created the drivers themselves. Any patch or driver update needs to be installed by the user individually. For Metal, Apple controls and ships the drivers as a part of its OS. Any patch or update is pushed as a native OS update. This leads to a tightly integrated, cohesive ecosystem with Apple in complete control.

Breaking the lock-ins

Vulkan - a cross-platform open standard maintained by Khronos - is aiming to break the OS vendor lock-in. There are quite a few projects aiming to break the hardware vendor lock-in like OpenXLA, UXL and ONNX Runtime that I know of. This portability comes with a tradeoff of slower compute. As mentioned later in this article, the priority right now is faster compute.

Enter the WebGPU

Where does WebGPU fit into this picture? It sits on top of OS-level APIs like DirectX, Vulkan and Metal. One can think of it as an any-OS-for-any-hardware paradigm. It is a lofty goal and a great technical achievement.

Why does it feel much more complex than writing a CUDA kernel?

Because it lives in a much more volatile environment.

• The screen constraint: WebGPU is a successor of WebGL which was primarily used to calculate and render pixels on a <canvas> element. WebGPU, while allowing general purpose compute, still covers the role of WebGL.
• The browser constraint: Security is a huge concern - running raw assembly code in a browser can be risky and is not allowed.
• The GPU constraint: It should work for an integrated GPU, a standalone device or a software-GPU. This versatility makes it hard to use a hardware-specific software like CUDA.

The guardrails addressing these constraints are present in the aforementioned OS-level APIs, and it makes sense to build on top of them.

In CUDA, a main function running on CPU handles memory management and data transfer. A __global__ kernel function is defined separately and launched from inside the main function using <<< ... >>> brackets. All the heavy lifting behind launching a kernel is taken care of by nvcc.

In WebGPU, a JS/TS code plays the role of the main function. It allocates memory and copies data. A rust-like WGSL code plays the role of a kernel function. To "launch a kernel" however there is no nvcc like compiler to do all the heavy-lifting. It has to be done by the programmer themselves. This is what makes it more complex and bloated.

Trading portability and dev-ex for efficiency

Any-OS-for-any-hardware is great for dev-ex. However most of the recent innovation in the space of GPUs has been heavily focused on compute and precision with little focus on cross-device portability. Given the undivided attention massive LLMs are getting, the stakes for fast compute are higher. Portability seems to have taken a backseat.

The common thread

However in both these cases one theme is common - giving more control to the developer. E.g. WebGPU exposes explicit command encoders/buffers which used to be hidden from the developer in WebGL. CUDA exposes __shared__ on-chip memory, letting the developer explicitly stage and reuse data that a CPU's cache would otherwise manage invisibly.

What to expect from WebGPU

Most of this is speculative. I expect a three.js-like counterpart to further simplify using WebGPU. For ML models, it will mostly serve as an inference backend. Its biggest advantage comes from bringing a cohesive experience for computer vision (CV). Being rooted in a browser with access to camera and canvas provides a cleaner surface for interacting with CV models. We are seeing some good examples like Gaussian splatting and segmentation already.

I am using it for visualizations and simulations e.g. vector fields but haven't really pushed it to the limit yet.

Despite my comment on WebGPU mostly serving as an inference backend, one can find cases of WebGPU being used for optimization here, like this one or this one. These are however much smaller in scope.