06 Tech Tuesday – Interactive Floors
Interactive floors have been around for at least 15 years and vary in resolution and sensing functionality. Commercially they are used for three purposes that I know of,
Commercial Applications
- safety mats (large piece of machinery stops working if you step on or off of a mat). These tend to be super durable low resolution switches with no visual feedback
- Walking gait analysis, which are made of a variety of different technologies such as fsr (sensing pressure), switch matrices, (I made a a mat of 1024 switches to make dinosaur footprints where kids ran across a mat for AMNH)
- Arguably not an interactive floor, but Automated Follow Spots, spot lights that automatically follow a performer are an interesting example of giving someone a unique ID and tracking their location in space. Also made by Martin I believe that these work using two technologies; all performers where a transmitter that is listening on an RF wireless channel, the system sends out a message with a specific address that has been assigned to a person, when that persons transmitter receives that message on RF it then turns on an IR (infrared) light that a camera detects and tells the spot light where to focus…
Output Types
For visual output there are for the most part 3 camps
- LED – medium to low resolution and adds complexity to any floor based sensing technology
- UVA – Contact using computer vision for tracking with, I believe, 10mm LED
- Sensacell makes an interesting product which incorporates a low resolution LED floor with capacitive sensing (the technology used in your smartphone screen). An interesting thing about capacitive sensing technology is that it will sense your hand from up to 8″ or so away so you can sense a bit of z axis.
- Youtube will provide an endlessly entertaining number of examples of LED floors if you search for “LED floor”. Below I have embedded a good one that shows a 6mm floor with video tracking. You can see that it is really fast and has a great output, but it still doesn’t know where your feet are.
- Front Projection – Which allows for very high resolution images and can be easily implemented in locations that are not as controlled (like the subway or Toys R Us) but users will cast a shadow where they are standing.
- GestureTek uses a projector paired with an IR illuminator and IR camera to get accurate video tracking that is used as a pretty robust solution for commercial applications.
- SnibbeInteractive uses a similar setup which is often paired with a reflective floor to create a clean background for easy tracking, you can experience this a the New York Hall of Science in Queens.
- Ateliers-numerique in Canada developed a capacitive sensing tile which is front projected on and can be seen in the Cirque Du Soleil show “Ka” as battles unfold on a vertical wall and a projection responds in real-time to dancers impact.
- Rear-Projection – Allows for high resolution image and NO Shadows! But it needs a large space underneath and significant infrastructure and rear-projection material to support the weight of all the people on top. I have only seen this a research not commercially… but it isn’t long…
- iGameFloor is an example of a rear projection floor that was used to make a number of simple games like pong. You can find a paper under the title “iGameFloor – a Platform for Co-Located Collaborative Games”. It tracks limbs from above and below the floor so that foot-fall contact should be possible.
- Building on that research, a student team at Hasso Plattner Institute have created multi toe and are using technology found on a lot of large rear projected multi-touch screens but put on the floor. This gives a very high resolution image.
Sensing Technology
There are a variety of different sensing technologies to sense where walking, some detect where a foot touched the ground, some use computer vision to detect where people are but don’t know where their feet are touching, and a few are starting to be able to associate a unique ID to each person/foot. I will go through an overview of some of them:
- Above ground Video tracking – often paired with an IR emitter and IR camera for a robust system
Pros – Very accurate, getting fast, can do some unique IDs,
Cons – is not good at tracking exact moment and location of footfall - Switches and FSR (force sensing resistors) – Accurate for detecting moment and location of foot fall in 2 – 8″ resolution but interfere with any floor-based display system like LEDs. Switches are mechanical and prone to failure when they need to be made at that density and there are some limitations to size.
- ZTiles was research using grids of pressure sensitive tiles
- Capacitive Sensing – Using a similar technology as the one on your smart-phone screen these sense where you are between a 2″ – 12″ resolution. Capacitive sensing can also sense when you are 8″+ away from the surface giving you a bit of z-axis, this is good for removing any latency as the system can predict a foot-fall and respond. Capacitive floors can work through solid non-conductive material so they are great for dance floors and solid walls that are projected upon. They can be a little finicky to get setup, but tend to be pretty stable as long as there aren’t any major changes in the electrical environment (like touching some chunk of metal).
- Ateliers-numerique – super durable and can have multiple touches. The inventor was working on using the sensor grid as an RFID antenna so that dancers with RFID on their shoes would be able to be uniquely identified. The sensor is somewhat opaque so that light can’t really come through and front projection must be used.
- Sensacell uses capacitive sensing built into the same board that holds white or RGB LEDs that are medium-low resolution and can work as both sensor and light output
- Diamond Touch is an interesting technology which attaches users to a series of different antennas that uniquely identify the user when they touch the surface. I believe that this would require users to be tethered but might be able to be developed using the idea of having a unique signal transmitted into the floor.
- Rear-projected Multi-touch IR – Using a technique that we have seen on large multi-touch rear-projected displays where an acrylic sheet with a piece of rear-projection material on top is used as a light-guide for infra-red LEDs, when an item touches the surface it is illuminated with infra-red light and is very easy to track
- multi toe uses this system on the floor laid on top of a structural piece of glass to track users unique shoe sole pattern. This gives a high level of accuracy and uniquely identifies each foot and can tell which part of the foot has the most pressure applied. It would be interesting to see how fast the tracking system works.