Nearly every 3D scene-generation system starts from an explicit visual intent — a text prompt, a reference image, a sketch. EchoForge starts from a spatial audio recording, on the premise that sound already carries a hidden scene description. Unlike text, audio also arrives with direction and time attached: something is louder on one side, moves, feels distant or enclosed.

It was built by two ESGI master’s students, presented at SIGGRAPH 2026, and took first place in the ACM Student Research Competition’s undergraduate category. The pipeline:

  • Spatial cues from the recording estimate coarse source directions, active time segments and confidence values.
  • YAMNet labels semantic sound events from AudioSet classes — birds, ducks, insects, wind, rustling vegetation, water.
  • Semantic and spatial evidence are fused into higher-level families (forest, water, wind, bird vocalisation, waterfowl, insect ambience) instead of being treated as independent labels.
  • The central representation is an audio scene graph: inferred environment, candidate sources, confidences, spatial evidence, directional changes over time, relationships, consistency checks.
  • The graph converts into Unity layout instructions — terrain, vegetation clusters, water regions, source zones, ambience, representative objects.

The design choices are where the transferable lessons live. Direction comes from the recording; scale and visual form come from procedural rules, not acoustic measurement. A bird vocalisation activates trees and an elevated perch zone; wind stays ambience rather than becoming a localised object. Evidence that is weak or diffuse deliberately stays environmental context instead of being forced into a specific prop. The graph exists so any visual decision can be traced back to either the audio or an explicit rule — you can see where the system is guessing.

They are explicit that this is not reconstruction, and not forensic: audio can’t reveal geometry, and the value is a plausible, inspectable hypothesis rather than a claimed rebuild. The stated bottleneck is honest too — recognising a sound is useless if the procedural vocabulary can’t represent it. Their cricket-at-dusk example failed not because detection missed it, but because nothing in the generator knew how to draw it.

That framing opens the most interesting use case: once a soundscape is a spatial graph, describing it in space is nearly free, which is what deaf and hard-of-hearing users in immersive environments currently lack. The accessibility angle reads like a consequence of the representation rather than a feature bolted on afterwards.