Look at your keyboard. It's roughly rectangular (or maybe you have one of those curvy ergonomic ones), but it's not just a rectangle, is it?

You can flip it over and see the little feet that prop it up at an angle, small holes for screws, maybe a sticker or two. On the front, each key protrudes from the surface, and they probably have a slight curve on the top of each one that you barely notice unless you look right at it.

If your keyboard sat in the background of a TV show, you wouldn't notice those little details. You wouldn't notice them if it sat in the background of a VR environment either, so I think you'd be annoyed if that keyboard sitting in the background were the reason your VR experience stuttered as it tries to render every little detail.

There are scanning tools already out there which can look at a real room and digitise it. Now we have VR to take a digital room and make it look real. The missing link is taking that massive raw scan with too much detail everywhere, and cut out the minutiae so your computer won't faint when it tries to process it. At the moment, this usually involves getting an artist to sit down and retopo - remake the topology/model so it captures the general shape more efficiently.

Now it looks like Microsoft has decided to throw their weight behind this problem by acquiring Simplygon, which is a state-of-the-art automatic retopo/model simplification tool. It's cool and it works, and according to this article, they're pointing that tool at AR and VR.

My prediction? Look forward to seeing more scans of real-world things in VR and AR!