file format – How are 360° photos stored and processed?

There are no finer details.

Let us start from the beginning.

A file

You have a camera with a lens and the file stores what was projected on the sensor.

You can change the lens, for example to a wide angle lens and the file just do the same. No black magic there.

You can then store an image taken with a fisheye lens. The file itself has no idea and stores the image exactly the same way.

A file format has no way to differentiate what is stored in it.


Ok. Now let’s take a look at some types of “360” images.

A 360 image for start is one that contains information on what is around you. Behind you, in front, to the left, to the right.

The most primitive way to capture this information is by taking a photo of a metallic reflective sphere. This contains information about (almost) all your surroundings.

360 images from a spherical reflective ball

This is a “Probe Map” projection. But there are several others. Cubic projection, Spherical projection, Cylindrical projection.

What now has popularized these images is the ability to compute these transformations and unwrap them again.

So this simple Jpg with a distorted projection is projected into a 3D “solid” from the inside and then you see a section of it, and you can move around.

This solid can be a sphere or a cube, depending on the application. (Cube is the most common)

Cubic projection

Here is a link to a simple paper I wrote some years ago about the basics of different projections for environment maps.

The viewing

The trick is that we have now software that can interact with this projection, either on a 3D rendering software for a video or animation, or in real-time like a video game, a VR application, or a simple panorama on a website.