Nebula · R&D overview · updated 2026-07-20 · Tiếng Việt →

Building a city-scale environment from real, license-clean data

A report of what we tried for reconstructing the world around a hero capture — the roads, blocks, and buildings around a scene — and the results.

The brief that started this (Jonathan): “Make one detailed, color 3D building at a time with AI, drop them into Blender, then pre-render or convert to Gaussian Splats — and figure out how to do it for an entire city across hundreds of machines. Only free/rights-clean material; Google/Apple imagery as reference only.”

01Two approaches we tried

A · Generate each building with AI

Photo(s) + Apple/Google reference + OSM → an image-to-3D model (TRELLIS.2 / Pixal3D) → textured GLB → LODs → Blender. Ran end-to-end on real buildings. Result: usable for a single object; per-building at scale it produced arbitrary orientation/scale, thin/degenerate geometry, and each building is a separate GPU job.

B · Reconstruct from captured data

Public USGS 3DEP LiDAR + NAIP aerial color + OSM footprints → one continuous mesh or splat covering thousands of buildings at once. Result: continuous, recognizable, license-clean, no floaters. Sections 02+ cover this path.

Approach A results — TRELLIS.2 / Pixal3D (image → 3D)

One reference image → a textured GLB mesh (~1M triangles each), run on a rented Vast GPU. Outputs below rendered fresh on hardware WebGPU.

Input reference photo
Input — one photo of the RangerOne glass towers.
TRELLIS output for the towers
Output — textured mesh (0.74×0.93×0.78, ~970k tris). Glass facade reads, but geometry warps and drips; only the photographed side reconstructs.
TRELLIS output for an ornate landmark
Ornate landmark (separate test) — dome, tower, window detail all reconstruct cleanly. A single distinctive building is where the method works best.
TRELLIS output from an aerial photo
From an aerial photo — output is a thin flat relief (0.96×0.87×0.11), a billboard not a 3D volume. Aerial input has no canonical up-axis.

02The data foundation — one district, fully measured

Everything starts from three free, commercially-clean sources for the RangerOne district (Arlington, TX): USGS 3DEP LiDAR (real 3D points), USDA NAIP aerial imagery (color), and OpenStreetMap footprints (2,722 buildings). No Google/Apple pixels touch the output — they're reference only.

03From points to a solid, continuous world

We first tried turning the environment into a Gaussian Splat (same medium as the hero). Across four retrains, the splat kept smearing buildings into vertical streaks — not a tuning problem, but a data ceiling: 2018 aerial LiDAR has dense roofs/ground and weak walls, so free-floating gaussians have nothing to anchor to.

Environment Gaussian Splat, wide oblique
The trained environment, as a Gaussian Splat — the whole district rendered from env3.csplod (2.6M gaussians, WebGPU 3DGS renderer): stadium bowl, terrain, trees, river.
Env splat close oblique
Close up — soft, volumetric gaussians (the characteristic splat look). Buildings smear and floaters drift: the 2018 aerial LiDAR has weak walls, so the gaussians have nothing to anchor to.
Env splat stadium
The stadium in 3D — reads clearly, but water becomes black holes (no LiDAR return). Four retrains couldn't fix the smear — a data ceiling, not a tuning one.

The fix was to stop fighting it: mesh the LiDAR into a continuous surface (like Cesium terrain). A surface can't float, so the smear disappears entirely — and the points already carry aerial color, so it's textured for free.

See it live — interactive 3D in your browser

🗺️
Detailed mesh · ±1.4 kmDSM mesh around RangerOne, NAIP color, despiked. Orbit / zoom.
🌆
Wide mesh · ±4.5 kmThe whole ~9 km district: 2 stadiums, roads, neighborhoods.
🏙️
Core + detail · GLBTighter core mesh around the hero, higher texel density.

These are real WebGPU meshes generated from the LiDAR — no Google/Apple geometry. Continuous surface = zero floaters, fully recognizable stadiums/roads/ponds/trees.

04The hard part: facades (walls)

Aerial capture sees roofs and ground but almost no walls. On a raw LiDAR mesh, buildings "melt" (roof slopes straight to the ground with no vertical face). The approach we used: take the OSM footprint, take the roof height from LiDAR, and extrude vertical walls between them (City3D / LoD2).

Facade extrusion on the mesh
Facade fill — extruding vertical walls from footprints + LiDAR roof heights turns melted blobs into buildings with real edges. Wall texture (what the surface looks like) is the remaining choice: sample dominant color, procedural, or AI-synthesized — since no real wall photography exists for most buildings.

05AI enhancement — what we measured

We tested AI image enhancement (Gemini, Qwen-Image-Edit) on the renders before training. Enhancing each rendered view independently: splat trained to mush, measured PSNR ~9.5. Enhancing a single shared 3D surface (texture-bake) then rendering consistent views from it: PSNR ~22.

06Google/Overture photoreal 3D tiles

We also stood up Google/Overture photorealistic 3D tiles with a streaming LOD system (90% less download for an overview).

Photorealistic 3D tiles of the district
Photorealistic 3D tiles of the same district.