1. That is bloody amazing, especially the demo with the car! Great Job!!

Posted at 6:07PM on Nov 15th 2007 by simon

2. That is quite an impressive demonstration there.

Posted at 6:09PM on Nov 15th 2007 by Jake

3. damn, that is sweet. universities and moderate sized businesses would kill for this sort of setup

Posted at 6:12PM on Nov 15th 2007 by zigzagjoe

4. thats friggin sweet

Posted at 6:13PM on Nov 15th 2007 by spiderwebby

5. I rarely ever comment, but man, that's awesome.

Posted at 6:22PM on Nov 15th 2007 by Mike

6. Wow, I can't believe how well that works!

http://digg.com/hardware/Amazing_automated_projector_calibration

Posted at 6:27PM on Nov 15th 2007 by Eliot

7. I wonder why nobody thought of this before... but the problem with the car demo is, that it still requires the geometry to be known beforehand in order to wrap the image correctly.
And how do they want to achieve "interactive" framerates? they have to project log2(width) + log2(height) stripes, which would require 500 projections/sec for VGA, if you refresh the geometry at 25Hz. One could use a special projector that projects the stripes in infrared and at the same time the image that's for the user to see...
But wouldn't it be easier and more flexible, at the expense of 1:1 pixel precision, to point a camera at the projection surface (from the users point of view). you could then project images that look flat from a certain viewpoint onto complex unknown surfaces... but I think that would solve another problem...
btw, the setup they use looks quite cheap, just hook up some sensors to the IO-ins of your favorite programmable USB-chip...
If I just had a projector :D

Posted at 7:05PM on Nov 15th 2007 by pascal

8. THAT'S FARKING AWESOME.

I want one.

Posted at 7:11PM on Nov 15th 2007 by Shadyman

9. It must be possible to do the calibration in way less time.

Consider the projector has about a million pixels.

To uniquely identify each pixel, they must each be given a code about 20 bits (frames) long. The codes would be arranged so two adjacent pixels are different in only one frame. (basicly gray code with an extra dimension, so X and Y can be calculated simultaniously). That means positions can be calculated with only 20 frames, and with a 60Hz refresh rate it should only take a third of a second to identify a pixel (ie. 3Hz). That means it becomes practical to track moving objects using an IR projector (because you can do motion prediction and also you can presume the area certain pixels are in by their previous location, reducing the number of frames required to deduce a location). For fast-moving items, accuracy is also less important again reducing the number of frames.

I guess it would be possible to then track an item to about 10 Hz, which could have some interesting uses.

It would be possible to further speed up the process by using all 3 colour channels seperately and have detectors for all 3 colours on the ends of the fibers.

Another interesting ofshoot is it might be possible to do sub-pixel positions by measuring intensity.

Posted at 7:39PM on Nov 15th 2007 by O Mattos

10. Very impressive. Earlier today, I had an idea conveniently somewhat-relevant to the model car demo, cheap sort-of-invisibility:

Suppose a camera was added close to the projector, and the background was a random, inconsistent image, rather than a solid-gray one.

First, the camera takes a picture of the background and remembers it. Then the camera records the binary pattern, from which the offset relative to the projector can be determined; the background should not be a problem to calibration because a previously-recorded image may be used to filter out the graphical background noise.

Next, the still-flat-gray model car with the optical sensors is placed on the background. The calibration does its thing like in the demo so that the colors can be projected correctly onto it; however, this should also provide enough information to project a piece of the background over the car such that it becomes sort-of-invisible from the point of view of the view of the camera, or if multiple projector-cameras are used, sort-of-invisible from a single arbitrary point of view.

Posted at 8:12PM on Nov 15th 2007 by Samodelkin

11. This is probably the most amazing mod I have ever seen. These men are brilliant. I am speechless

Posted at 8:16PM on Nov 15th 2007 by Jordan

12. Love this post. I can't help but dig up an older article from Technology Review (late '04) that focused on one group's work with hand held projectors, aiming to shrink them enough to replace/augment cellphone/PDA displays. Some of their tricks included adaptive projecting onto uneven surfaces, keeping a steady image despite moving the projector (so you could tilt and point the projector like a mouse/laser-pointer), and automatically unifying several projectors for an even larger image. So that was baseline already pretty cool. But they took it a bit further and got RFID tags involved that each had a photodiode sensor. They then ran a similar scanlines projection (like this project uses) across a scene, and identically they could exactly locate where the sensor was. Their working example was a storage shelf with different cardboard boxes and embedded tags + sensors, and they would project graphical information for each box onto the side of each box.

Now, having recently reformatted my lappy, I'm suddenly lacking the link to the online copy of the article. I've found a similar copy, but it lacks the images that went with each number. *grumble* http://www.technologyreview.com/Infotech/13958/?a=f Also, I'm not sure if this is related, but they seem to be doing very similar things: http://www.merl.com/people/raskar/Sig04/ Ditto: http://www.trnmag.com/Stories/2004/081104/Projector_lights_radio_tags_081104.html

Although the original focused on what you could do with a palm sized ("pencil sized" they hoped) projector, I'd always been rather been intrigued with the RFID part. These days, we've got the multitouch demos and cameras tracking markers in realtime, but the light detector + projector combo still seems like a promising combination. Just wanted to share/connect this to some older research. (...and if they're the same group, I'm going to go find a wall for my forehead.)

Posted at 8:21PM on Nov 15th 2007 by Filip

13. Incidentally, this seems to be the same guy who came up with the hack of using the Wii remote's IR sensor and an array of IR LEDs to track finger positions.

Posted at 8:38PM on Nov 15th 2007 by David

14. For us who hate opening PDF's:
http://view.samurajdata.se/psview.php?id=e3af520f&page=1

Posted at 8:56PM on Nov 15th 2007 by aten

15. Neat stuff.

@7,9
It gets better, a DMD can slam out binary frames at 14,300 Hz and the sequence is short (on the order of 20 frames) so you can get a fix in about 1.4 milliseconds and after you have an absolute fix you can use coding tricks to drop the number of frames to do relative tracking, I'd put it at about 8 frames with a periodic 20 frame burst to cancel out cumulative errors.

The actual reprojection is achieved by mapping the camera space to the projector space and fitting the point positions to identical virtual models. But it can get tricky without overdetermined sampling.

This also burns resolution like nobody's business. The actual resolutions with a close up are pretty miserable. Lots of anti aliasing going on too.

Also check out this honest to goodness projector hack by the same guys:
http://www.merl.com/papers/docs/TR2005-141.pdf

Posted at 9:22PM on Nov 15th 2007 by monopole

16. Auto calibration is not a new idea.

BARCO (Belgian-American Radio Co.) uses it in their projectors, with no need for sensors, just an embedded camera

http://www.barco.com/corporate/en/products/product.asp?gennr=123

Posted at 10:46PM on Nov 15th 2007 by DrHands

17. The only problem I see with this approach is that you loose a huge amount of the available number of pixels from the projector... He is using a screen that is about 6" on a side in a field of view that is about 30 inches on a side, which is 'wasting' about 95% of the available pixels... When he turns the screen on its side it further complicates the problem because the screens footprint becomes more along the lines of 2" by 6", reducing the pixel utilisation to a meager 1%

Also, you still have to focus the projector onto the target, and your depth of field is still limited by the lens on your projector...

This could get really big when laser projectors (using moving mirror galvos to steer the beam) become affordable, as you can then adjust the field of view of the projector on the fly, and the images are always in focus.

Aside from that, that is an awesome setup. I envision some code to allow beryl/compiz (sorry windows/mac users) to have several small (or large for that matter) panels that can be mapped to an individual window. So you can drag/rotate/etc each of your windows on say a physical desk.

Posted at 12:05AM on Nov 16th 2007 by Peter

18. Some more comments to add to the discussion:

1). For moving targets you don't have to use the same search scheme as you already know roughly where the corners of the targers are (or were). You just need to be able to search those specific areas within a radius.

2). There is no reason why multiple targets could not be placed in projection view, once calibrated each could have a different image projected onto each of them.

3). The target screen are currently teathered to a compute, again not needed. The electronics could be fully self contained (not that difficult to code) and result sent to projector(s) system wirelessly (Bluetooth for example).

4). In the same manner that they use two projectors onto a wide target with 6 sensor, you could make a modular screen built from square panels where you just assemble the size screen you need and point a barrage of projectors at it to get some really (!!!) high resolutions.

A fantastically simple idea. Good hack!
Simon.

Posted at 2:04AM on Nov 16th 2007 by simon

19. VERY IMPRESSIVE!
I see great future for this..

Posted at 3:01AM on Nov 16th 2007 by NNM

20. Embedded optical sensors seems somewhat complex. Wonder why they don't use reflecting targets attached to the screen? Put a 2-d barcode on the target and pick up the reflected light with a single sensor at the projector to identify which target it is as well as the precise position. (Cube-corner retro-reflecting targets might help.)

Posted at 4:46AM on Nov 16th 2007 by Andrew Yeomans