3D Gaussian Splatting

Photography captures a frozen moment, but what if you could walk into it? That is the promise of radiance fields: data structures that encode how light fills a volume so a renderer can synthesize any viewpoint on demand.

The dominant approach for years was Neural Radiance Fields (NeRF), which trains a neural network to answer "how bright and opaque is this point in space?" Stunning results, but slow — a single frame can take seconds or minutes to render.

In 2023, Bernhard Kerbl and colleagues at INRIA published 3D Gaussian Splatting (3DGS): represent the scene as a set of 3-D Gaussians — fuzzy ellipsoids, each with a position, shape, opacity, and color — and project ("splat") them onto the screen. No rays, no network queries per pixel. The result: real-time, photorealistic rendering at 30–100 fps on consumer hardware.

The key insight is that a Gaussian is closed under projection: a 3-D Gaussian seen from any angle is still a 2-D Gaussian on the screen, cheap to rasterize. Blend millions of them back-to-front (alpha compositing) and you reconstruct the radiance field.

The demo below is a 2-D analogy of how Gaussian Splatting builds an image. Each splat is a 2-D Gaussian: a colored blob with a position, a radius, and an opacity. They are sorted back-to-front and composited with standard alpha blending — exactly the operation 3DGS performs after projecting 3-D Gaussians onto the screen.

<!-- {{c_html_intro}} -->
<div class="controls">
  <button id="addBtn" type="button">{{btn_add}}</button>
  <button id="clearBtn" type="button" class="ghost">{{btn_clear}}</button>
</div>
<canvas id="canvas" width="480" height="300" aria-label="{{canvas_aria}}"></canvas>
<div class="splat-controls" id="splatControls" style="display:none">
  <div class="ctrl-row">
    <label for="radiusSlider">{{lbl_radius}}</label>
    <input id="radiusSlider" type="range" min="10" max="120" value="40">
    <span id="radiusVal">40</span>
  </div>
  <div class="ctrl-row">
    <label for="opacitySlider">{{lbl_opacity}}</label>
    <input id="opacitySlider" type="range" min="5" max="100" value="70">
    <span id="opacityVal">70%</span>
  </div>
  <div class="ctrl-row">
    <label for="colorPicker">{{lbl_color}}</label>
    <input id="colorPicker" type="color" value="#4a90d9">
  </div>
  <p class="formula">{{formula_label}}: C = &sum; &alpha;<sub>i</sub> c<sub>i</sub> &prod;<sub>j&lt;i</sub>(1&minus;&alpha;<sub>j</sub>)</p>
</div>
<p class="info" id="info">{{info_empty}}</p>

/* {{c_css_intro}} */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }
.controls { display: flex; gap: .5rem; margin-bottom: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui; padding: .4rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
button:hover { opacity: .88; }
#canvas { display: block; border-radius: 10px; border: 1px solid #cdd9e3;
          background: #0a0f14; cursor: crosshair; max-width: 100%; }
.splat-controls { margin-top: .6rem; background: #f4f7fa; border-radius: 8px;
                  padding: .6rem .8rem; }
.ctrl-row { display: flex; align-items: center; gap: .5rem; margin-bottom: .35rem; }
.ctrl-row label { width: 70px; font-size: .85rem; color: #444; }
.ctrl-row input[type=range] { flex: 1; }
.ctrl-row span { width: 40px; font-size: .85rem; font-variant-numeric: tabular-nums; }
#colorPicker { width: 36px; height: 28px; border: none; padding: 0; cursor: pointer;
               border-radius: 4px; }
.formula { font-size: .82rem; color: #555; margin: .3rem 0 0; font-style: italic; }
.info { font-size: .85rem; color: #666; margin: .4rem 0 0; min-height: 1.2em; }

// Code not found

Click Add splat to drop a new Gaussian at a random position. Adjust its radius and opacity sliders, then pick a color. Notice how overlapping splats blend smoothly — no hard edges, no polygons. Clear removes everything and lets you start over. The transparency math shown is the same $\alpha$-compositing formula used in the real algorithm: $C = \sum_i \alpha_i c_i \prod_{j<i}(1-\alpha_j)$.

How expensive is Gaussian Splatting compared with its predecessors?

• Triangle rasterization (classic games): $O(T)$ per frame where $T$ is triangle count. Fast but hard to capture volumetric, fuzzy effects like hair or fog.
• NeRF: shoots $R$ rays, each sampled at $S$ points, each querying a network — roughly $O(R \cdot S \cdot \text{network})$. Photorealistic but painfully slow at inference.
• Gaussian Splatting: $N$ splats are sorted by depth — $O(N \log N)$ — then rasterized front-to-back in parallel on the GPU. With tiled rendering (the screen is split into $16 \times 16$ tiles; each tile independently composites only the splats that touch it), the practical cost per frame is close to $O(N_{\text{visible}} / \text{tiles})$.

A typical outdoor scene uses 3–6 million Gaussians. Sorting costs dominate setup, but rasterization is massively parallel, so modern GPUs hit 60+ fps. The memory footprint is large (several GB for a complex scene) but training takes only minutes to hours on a single GPU — far faster than NeRF.

The dimensionality reduction analogy is apt: Gaussian Splatting compresses the infinite-dimensional light field into a finite, differentiable set of parameters optimized end-to-end by minimizing the difference between rendered and ground-truth images.

Real-time photorealistic rendering is not a toy problem. Gaussian Splatting has already found its way into:

• VR/AR telepresence: record a person with multiple cameras, fit a Gaussian scene in minutes, stream it to a headset — the viewer walks around a life-size volumetric clone.
• Game-asset creation: artists photograph a real object, reconstruct it as a Gaussian scene, and export it — faster than hand-sculpting and more realistic than photogrammetry meshes.
• Autonomous driving simulation: synthetic sensor data for training self-driving models requires photorealistic, real-time scenes. Gaussian Splatting delivers both.
• Medical imaging: CT and MRI volumes can be approximated as Gaussian fields, enabling interactive visualization of complex anatomy at surgical-planning speeds.
• Cultural heritage: scanning fragile artifacts with multi-view cameras and splatting them preserves every detail at sub-millimeter accuracy.

The technique also inspired research into dynamic Gaussian Splatting (scenes with moving objects) and compressed Gaussian representations that cut memory use by an order of magnitude — an active frontier in computer graphics and neural network training.

3D Gaussian Splatting is an elegant inversion of intuition: instead of tracing how light travels, it asks where does the light come from and answers with a cloud of glowing blobs. Each blob is just a position, a shape, a color, and an opacity — yet millions together reconstruct photographic reality at interactive speeds.

The mathematics is classical (Gaussians, alpha compositing, gradient descent), the engineering is clever (tiled rasterization, CUDA radix sort), and the result is a step change in what a single GPU can render. Whether you care about virtual worlds, surgical planning, or the future of digital twins, the blob revolution has already arrived.