The depth map is actually stored as an image: PhotoStructure accidentally used them as preview images several versions back (when depth maps were fairly novel) and I had some interesting discussions with users complaining about "ghosts" in their library:

https://forum.photostructure.com/t/i-see-ghosts/41