Kinect Touchscreen Prototype

Kinect Touchscreen Prototype

In the attempt to build a touchscreen using Kinect and Processing, I’ve constructed a screen with material that stretches when touched. The change in depth of the screen is picked up by the Kinect camera and is displayed in an ‘Average Point Tracking’ Processing sketch.

This period of work focused on the integration of the Kinect sensor and figuring out how to go about building a rear-projection kinect touchscreen. The biggest challenge was getting the kinect to work with Processing, as the Open Kinect library was producing errors and overall, not cooperating. Once the library was setup correctly, I was able to play around with the features of the kinect sensor, such as the depth data and thresholds, and integrate the code into my existing Twitter program. My biggest achievement during this period of work was successfully getting the kinect to work with my program, as well as setting up a working prototype of the touchscreen.

The video above shows a working prototype of the touchscreen the project will be displayed on. The kinect camera wasn’t perfectly aligned with the projection, so the blue cursor isn’t yet in the same spot at the users finger.

I borrowed a regular projector from AV and set it up so it projects the image of my Twitter program onto the prototype nylon touchscreen. It worked fine just projecting the image directly onto the screen, but the necessary placement of the kinect camera was obstructing the projection. To fix this problem, I bounced the projection off of a mirror onto the touchscreen so that the kinect camera wasn’t in the way. The main annoyance was caused by a bright glare in one area due to the reflection of the projector bulb. During the grad show, I’d ideally like to use the short throw projector that sits on the ground. This would eliminate the glare problem as well as the placement of the kinect camera.

From the side - This is how the touchscreen will be setup during the grad show. The projector and the kinect will be behind the screen so they aren't visible.
From the side – This is how the touchscreen will be setup during the grad show. The projector and the kinect will be behind the screen so they aren’t visible.
From above - My grad project will be setup in a corner, allowing for maximum use of space. This also provides a stronger support structure for the touchscreen.
From above – My grad project will be setup in a corner, allowing for maximum use of space. This also provides a stronger support structure for the touchscreen.

Up next, I’ll be adjusting the final elements of my program. Currently, when someone’s finger is near a cluster of two or more tweets, the content of both will appear and overlap. I’ve been working on trying to solve this problem, but haven’t yet figured it out. I also will be doing additional research into which fabrics will be most beneficial for the touchscreen, although the current nylon screen is working quite well. Since I plan on using the short throw projector, I will be checking it out and getting measurements for the ideal screen dimensions for the grad show.

Update:

I’ve outlined how to build a rear projection touch screen, and have successfully used it to display Tweelo, my thesis project at Emily Carr University of Art and Design.


Leave a Reply