Creating stereoscopic panoramas using slit-scan photography

[phort99]

Hey all! I thought I'd share something I've been thinking about and working on for the last month or so.

Ever since finding this set of stereoscopic panoramas by Robert Kooima I've been wanting to create my own panoramas for the Oculus Rift.

For my approach I'm approximating slit-scan photography by taking a video with a crappy cell phone camera with a $10 wide angle lens attached, and extracting narrow columns from the video after some processing in After Effects to cancel the lens distortion.

Here's the basic principle:

shitty explanation.png

At the top of the image is basically how the "gigapan" rig from Robert Kooima's page works. It automatically takes two high res panoramas, and uses very narrow sections from each photo meaning it has to take many more photos than you would need for an ordinary 360° panorama.

In my technique, as the camera rotates and takes video, each vertical column of pixels is a different inter-ocular separation. After the video is captured you can then process the video to put all the 1px columns side by side to create the slit-scan image. The drawback is that your horizontal resolution of the panorama is limited by the framerate of your camera and your rate of rotation. That means if you want a panorama that's 4096px wide, you need to make a 2.3 minute rotation at 30fps, or 34 seconds at 120fps.

In my technique, since the camera can just keep rotating, you can call each revolution one frame of video and make a time lapse video! Potentially if you had a high enough framerate sensor and could get it spinning fast enough, you could capture real-time video in this way. The only thing I've come across that seems capable of capturing slit-scan images at a high enough framerate are Photo Finish cameras like the ones used for horse race tracks. You'd have to be pretty daring to try revolving one of those at 1800rpm though!

Here's the result of what I've managed to capture with my own flimsy setup (My camera mount is built out of Lego Mindstorms!), it seems to have a lot of promise:
Left image: http://www.flyingbreakfast.com/misc/left.png
Right image: http://www.flyingbreakfast.com/misc/right.png
I've actually used 4px wide columns instead of 1px because the rig was rotating a bit faster than I thought but you can just resize this to 1/4 width in photoshop using nearest neighbor. I had some issues with the camera wobbling since lego isn't real rigid structurally. That's why the straight lines are a bit wobbly in places.

It seems like a GoPro camera would be ideal for this sort of panorama capture because they support high framerates (60fps at 1080p, higher at lower resolutions) plus they have wide angle lenses built in!

I'm no expert on hardware or modding, I'm more familiar with programming and video production, so all I have are these preliminary experiments. I would be honored if someone with more hardware experience (and a better camera (and money)) would take up the mantle.

[mediavr]

>>>It seems like a GoPro camera would be ideal for this sort of panorama capture because they support high framerates (60fps at 1080p, higher at lower resolutions) plus they have wide angle lenses built in!

I'm no expert on hardware or modding, I'm more familiar with programming and video production, so all I have are these preliminary experiments. I would be honored if someone with more hardware experience (and a better camera (and money)) would take up the mantle.>>>

Yes Gopros are good for stereo panoramas especially with the Gopro Hero2s with the 3d Kit (there is no 3d kit for the Hero3 yet -- or ever maybe?). I have been making some with the 3d kit here
http://www.stereopanoramas.com/blog
using regular stitching software and using video capture. I will write a detailed tutorial -- soon I hope! The problem with very fast Gopro rotations is that the rolling shutter distortions become severe with anything faster than about 8 seconds for 360. There are slit assembly tools in Stereophoto Maker for stereo panoramas btw.

Peter M

[phort99]

The problem with very fast Gopro rotations is that the rolling shutter distortions become severe with anything faster than about 8 seconds for 360. There are slit assembly tools in Stereophoto Maker for stereo panoramas btw.

Wow, I didn't know about that! I was doing everything the hard way.

What do you use to rotate the camera? Do you think it might be possible to just apply a counter skew to the image to account for the rolling shutter? Does it perform better if you record at high framerate with low resolution? (or does hero 2 not support high framerates)

[relaxman]

Great topic!

I make 3d stereoscopic videos, and a few days ago i was amazed by the Oculus rift. I have gopro3 blacks, and my question how can i make 360x180 3D videos and play back on the rift with headtracking?

There are many problems: panorama stitching in 3D, time difference, exposure difference, neither cameras rotate around the NPP (no parallax point), fisheye distorsion, what player support this, etc.

Somebody tried such weird thing yet?: )