Splatting Notes 5: Spatial Foveation, Maybe?

Gaussian Splat scenes are built through a training run, where splats are placed and optimized to produce results that match sets of reference images. I describe splats as 3D pixels, and liken the training process to growing crystals. Training typically consists of 30k steps, with the front half focused on placing the splats, and the back half on refining them. We use Brush for the training.

Claude informed me that, due to the specs of my current gear, I can produce scenes of up to three million splats. That’s fine by me, since it’s not clear to me that more individual splats would actually be beneficial in my case. My gallery isn’t all that large and, also I want to display these on my website, so the file sizes need to be reasonable.

The idea that we had a limited budget of splats led me to the idea of spatial foveation, where you concentrate resolution on the important bits of an image. This turned out to be one of those cases where a statement made somewhat early in the process stuck in the LLM’s mind and had unexpected and not altogether welcome consequences. I had said something along the lines of, “the artwork is what’s important. The walls and the rest can have less detail.” This led to the machine continually discounting trouble we encountered with making the flat, white gallery walls appear white and flat and I eventually explicitly rescinded that instruction. 

A prerequisite for foveation is separating the artwork from the non-artwork, and our first attempt at this failed utterly. We enlisted some image classification model off the street and told it to “find the artwork.” It could not. Not at all. In the end, I manually pulled images from the capture and told Claude, “These are the pieces. Incidentally, identifying the artwork proved very useful for the workflow, as Claude and I now agreed on what the artworks were and could reference what we were seeing.

Brush has a max resolution setting, where it downsamples all images it receives to that number. We tried various settings for this, finding that larger crashes our machine and anyway it isn’t always better. (Claude: “960 beat 1920 on the scores and trains in a third of the time; 1440 hung the driver during growth, and raising 960 to 1920 mid-run causes panics.”)

A way (the way?) that foeviation is manifesting in our project is in the processing of our set of input images. Using the “what is art” guidance I provided, we prepare custom crops of the artwork images, Brush has the detail it needs to make the artwork look good. We’re also calibrating the percentage of art-closeup crops vs as-they-come room views we’re including in the dataset we send to Brush. The remaining work is to apply the pre-processing/cropping not only to the art images, but also to the room images, which will give us granular control over our ratios. This is part of the Flat White Walls work stream, but waits patiently behind some other ideas in the queue.

At this point I’m pretty happy with the quality of artwork detail. I would go so far as to say the strongest part of our outputs so far is looking at artwork head-on, which is a necessity for our use case. 

Posted October 2026