3D Model Anything Turns a Phone Scan of Your Room Into Code You Can Edit

A trending repo from Wes Bos uses an AI agent to rebuild a real room in Three.js, and the way it works says more than the demo does.

Many photo-to-3D tools give you a model you cannot touch. A dense mesh, a cloud of points, a single baked file that looks fine from one angle and falls apart the moment you want to move a wall. It is impressive for about a minute, and then it is a dead end, because you cannot edit a thing you do not understand.

3D Model Anything, a project from the web developer Wes Bos that showed up on the trending boards this week, takes a different route. You scan a room with your phone and you get back a Three.js scene, the kind of editable 3D code a web developer already knows how to work with. Three.js is a widely used JavaScript library for rendering 3D in a browser, so the output is not a mystery blob. It is a scene file with named objects you can open, move, and rebuild.

But the output is not the interesting part. The interesting part is how it gets there: an AI agent builds a draft of the room, renders it, compares that render against your actual photos, and fixes what does not match, over and over. That draft-check-correct loop is a small, honest example of a way to use an AI agent that is worth copying even if you never scan a room.

What it actually is

The repo, MIT licensed and sitting around 1.2k stars as of late October, has four pieces: an iPhone capture app, a browser viewer, an agent skill called scan-to-threejs, and a demo of Wes's own office.

The capture app uses your phone's LiDAR, the little depth sensor on recent iPhones and iPads that measures distance by bouncing light off surfaces, plus regular photos. You need a LiDAR-equipped device to scan; older phones can open and look at scans other people made but cannot create them. The app is source-available, which means you can read and build the code, but there is no App Store download yet, so getting it onto your phone means building it yourself in Xcode.

Once you have a scan, you export it, drop it into a folder in the repo, and ask an agent to rebuild the room. That is where the real idea lives.

The agent does the part that is usually a black box

Here is the move that makes this project worth studying. The reconstruction is not a model spitting out a finished scene in one shot. It is an agent working the way a careful person would.

The scan-to-threejs skill has the agent measure the room from the scan, build it as an editable Three.js scene, and then do something most generation tools skip: it lines up a virtual camera to match where one of your real photos was taken, renders the scene from that exact spot, and compares the render against the photo. Where they disagree, it adjusts the model and checks again. It iterates.

Think about what that loop is actually doing. A one-shot image or model generator produces an answer and has no idea whether the answer is right. This agent produces a draft, then grades the draft against ground truth it can see, the original photo, and uses the gap to fix the next draft. The photo is the answer key. The agent is not trusted to be correct; it is made to check.

The matched-camera trick is the clever bit, and it is simple once you see it. The scan records roughly where each photo was taken. So the agent can place a virtual camera at that same spot in its reconstructed scene and render exactly what the real camera should have seen. Now it has two images of the same corner of the room, one real and one made, side by side. A chair that is too far left, a wall that is too short, a window in the wrong place: all of it shows up as a visible difference between the two pictures, and a difference it can see is a difference it can fix.

That is the same shape as the good version of any agent workflow. An agent that writes code and then runs the tests is more trustworthy than one that just writes code. An agent that drafts a reconstruction and then compares it to a photograph is more trustworthy than one that just reconstructs. The verification is not a nice extra. It is the whole reason the output is worth anything. The README is honest that this makes it "an iterative reconstruction workflow, not a one-click scan-to-perfect-model converter," and that honesty is the point, not a disclaimer.

What you get to do with the result

Because the output is a real Three.js scene and not a frozen mesh, the demo office can be viewed as a dollhouse, walked through in first person, seen from overhead, rendered as wireframe, or laid out knolling-style, with the objects pulled apart and arranged flat like a tidy overhead product photo. Those are not separate exports. They are views of one editable scene, which is exactly what you cannot do with a baked model.

For a web developer, that is the thing the headline tools never delivered: a captured real space that behaves like code, that you can drop into a site, light differently, measure against, or hand to another tool. The room stops being a picture and becomes a thing you can program.

Try it without buying anything

You can get most of the way into this project without a LiDAR phone, because the office demo ships with the repo. Here is the honest, low-friction path.

First, look before you build. Clone the repo and run the viewer:

npm ci
npm run dev

Then open the viewer at http://localhost:5188 and walk through Wes's office in the different modes. You need Node.js 22.12 or newer. This costs you nothing, needs no phone, and shows you what an editable captured room feels like. It is the fastest way to decide whether the rest is worth your time.

Two modes in the office demo, Scan and Overlay, are turned off, and the reason is worth noting because it is a limitation you will hit with your own rooms too. Those modes need the original scan files and photo collections, and Wes does not distribute his, for privacy. They come back on for your own rooms when you supply your own reference assets.

If you do have a LiDAR iPhone or iPad and want the full loop, the path is: build the capture app in Xcode, scan a room, export it through the share sheet, extract it into data/captures-in/<room>/, then open the repo in an agent that supports local skills (reusable instruction files the agent loads from the project when a task matches) and ask it plainly, something like "Use scan-to-threejs to rebuild the room in data/captures-in/my-room." The agent writes the scene into a demos folder and registers it so the viewer can load it. The skill's helper scripts want Python 3 and Pillow for the photo comparisons, with NumPy and Open3D if you turn on the optional depth reconstruction, the extra step that rebuilds 3D shape from the scan's depth data.

Expect the agent to take a while and to talk to itself as it goes. This is not a progress bar that fills to 100% and hands you a perfect room. You will watch it measure, place objects, render a comparison, notice something is off, and go again. The first pass will look roughly right and wrong in specific ways, and the iterations are where it tightens. That rhythm is the honest cost of the verification loop, and it is also the part that makes the result trustworthy. If you want a one-click result, this is the wrong tool. If you want a room you can actually edit and a model that checked its own work, the wait is the price.

The honest limitations

This is a real tool with real edges, and the README does not hide them.

It is interiors, for now. The current workflow is built around rooms and interiors, not objects or outdoor spaces, so despite the name it is not a general "3D model anything" yet.

Scanning needs LiDAR hardware. You need a LiDAR-equipped iPhone or iPad running iOS 17 or later; without one you are limited to importing and reviewing scans rather than making them. And the only example that ships with the repo is Wes's reconstructed office. The original scan bundle and photo collection behind it are deliberately not distributed, so you are looking at the finished reconstruction, not the raw capture that produced it.

The iPhone app is a build-it-yourself affair with no App Store release, which is a real barrier if you are not already set up with Xcode and an Apple developer workflow. For a lot of web developers, that alone will be the wall.

Licensing has a split worth reading before you reuse anything. The code and the skill are MIT, which is permissive. But the office images and textures have separate terms in a dedicated assets-license file, and third-party marks and products shown in the scene are not covered by the code license. If you are tempted to lift the demo assets into your own work, read that file first.

And the output quality rides on your scan and on the iteration. The README frames this as iterative on purpose. A rushed scan and one pass will give you a rough room. The verification loop is powerful, but it is not magic, and it does not invent detail that your capture never recorded.

Why a room scanner is worth your attention

If you never scan a single room, the thing to take from 3D Model Anything is the loop, not the lidar.

The reason this project produces something editable and trustworthy is that it does not trust the model to be right. It makes the agent produce a draft, check it against something real, and correct itself. That is the pattern that separates agent workflows that hold up from the ones that produce confident nonsense, and it is the same pattern whether the ground truth is a photograph, a passing test suite, a type checker, or a database query that either returns the right row or does not.

So the real question the repo poses is not "can my phone rebuild my office." It is "what is the answer key in my workflow, and is my agent actually checking against it." Most agent setups have no answer key at all. They generate, and they hope. This one points a camera at the truth and compares. Go find the camera in your own work.

Sources: wesbos/3d-model-anything on GitHub and its README (reviewed at the main branch, Oct 24 2026; MIT license for code and skill, separate assets license); Three.js. Star count (~1.2k as of Oct 24 2026) read from the live GitHub page and is approximate.