First successful experiments with a multi-touch box

2
84

Multitouch BoxAfter a bit of trial and error I got the multi-touch box working! With this homebrew multitouch box I can place and slide multiple fingertips across the top of a box and the computer tracks my every move. The setup is shown in the image at right. It’s so cool. What a terrific home project. Yesterday, we had a family get-together and all had a blast trying it out.

You can build your own in about 30 minutes or so. All you need is a computer (running Windows XP or Vista (32-bit or 64-bit)), a USB camera, and well, a box. Seth Sandler provides a great how-to on his blog here.

I used a small, 7″x7″ box with the top flaps folded up and a sheet of white paper taped to the top. I also taped a Microsoft NLX-6000 webcam facing up to the bottom of the box. Any reasonably good webcam should work. After you have made the box, you’ll want to plug in the boxed webcam, download some sample apps and binaries that Seth points to on his blog, and run the demo_smoke.exe demo. That’s the best sample app to get you going.

When you launch the demo_smoke program a command window like this:

multitouchcmdwindow.PNG

and a camera config dialog box will appear if all goes well.

multitouchcameraconfig.PNG

All you need to do is click OK button in this window titled “Property Sheet Properties”, which I’ve called the camera config window.

If you don’t have a camera connected properly you’ll get the following error dialog box:

multitouchnocamera.png

After a couple seconds you’ll then see a third window titled “Demo Smoke.” This is the app that you’re really interested in. It displays colorful smoke trails that follow your fingertips as you slide them across the top of the box.

smokedemo.PNG

If all goes well, you’ll see small white circles where your fingertips are and as you move your fingertips around you’ll see trails of “smoke”.

A couple tips:

* Although you don’t need to calibrate anything to get this app running, you’ll want to make sure that the camera’s field of view covers the paper on the top of the box. I checked this before I taped the paper to the top of the box by connecting up the camera to the computer first and checking to see what I could see. This is a good idea anyway, because it gives you a chance to make sure the camera is working at all.

* You’ll also need enough light outside of the box. The setup works because your fingertips occlude the light or at least cast a strong shadow on the paper. Normal daylight room lighting worked for me and at night I simply had to turn on the room lights. No special set up.

* For best finger tracking lift your palm up away from the paper. The extra shadow cast by your hand may confuse the tracking.

* Depending on how light tight your box is you may see extra smoke around the edges. If so, there’s probably light leaking in.

* So far I’ve tested this on a Vista 64-bit system (Mac Pro running Bootcamp), a Lenovo X61 Tablet PC running Vista, and a MacBookPro running Bootcamp and Vista. I’ve been unable to get it to work on my Toshiba M700 Tablet PC. I think it’s because the built-in camera is conflicting with the USB webcam in the app, but I haven’t dug any deeper to see if I can get it working (outside of disabling the built-in webcam, but that didn’t help).

What about the software? There is source code available which is being archived at Google. I pointed to it the other day. The code is primarily written in C++ and built off of Intel’s OpenCV (Open Computer Vision) library. I can see where one of goals of the coders was support multi-platform, however, it does add a bit of complexity that a large chunk of the market is not going to need. Don’t get me wrong, the approach isn’t bad, however, a little brittleness and install issues may avoided by let’s say supporting primarily Windows and the largest language platform choice for many .NET. So….I’m thinking through how to incrementally port the code. It doesn’t look hard. I could start at the beginning and write my own segmentation routines in CSharp or maybe wrap the OpenCV calls needed for multi-touch, or maybe wrap the high level finger-movement event notifications in the current TouchLib. My current thought is to start near the top and work my way down–but that’s just because I can keep a lot of things running in the meantime. I have to think through this a bit more and figure out how much time I really have to do this :-).

Anyway, if you don’t think all of this is that wild or practical, don’t forget this only takes a few minutes to build and costs next to nothing. This is no Microsoft Surface computer. But hey, if this doesn’t get you thinking about building a full-sized multi-touch system, nothing will. You can guess what we’re up to now!

2 Comments
Inline Feedbacks
View all comments
trackback

[…] had a blast building a multi-touch box and am now on the way to constructing a full-sized multi-touch system. Others on YouTube have been […]

trackback

[…] here’s the funny part: As a developer interested in Tablet PCs, in pushing the state of the art in user interfaces, with a passion for Robotics, with an addiction to tracking online news and community conversations […]