Haptography: Digitizing our sense of touch – Katherine Kuchenbecker
Translator: Andrea McDonough
Reviewer: Morton Bast I’m a mechanical engineering professor at the University of Pennsylvania, and my favorite hobby is photography. And as I travel around the world, I love taking photographs like these, so I can remember all
the beautiful and interesting things that I’ve seen. But what I can’t do is record and share how these objects feel to touch. And that’s kind of surprising, because your sense of touch
is really important. It’s involved in every physical
interaction you do every day, every manipulation task,
anything you do in the world. So the sense of touch
is actually pretty interesting. It has two main components. The first is tactile sensations, things you feel in your skin. And the second is kinesthetic sensations. This has to do with the position
of your body and how it’s moving, and the forces you encounter. And you’re really good at incorporating both of these types of sensations together to understand the physical interactions
you have with the world and understand as you touch a surface: is it a rock, is it a cat,
is it a bunny, what is it? And so, as an engineer, I’m really fascinated and I have a lot of respect
for how good people are with their hands. And I’m intrigued and curious about whether we could
make technology better by doing a better job at leveraging the human capability
with the sense of touch. Could I improve the interfaces
to computers and machines by letting you take
advantage of your hands? And indeed, I think we can, and that’s at the core
of a field called haptics, and this is the area that I work in. It’s all about interactive
touch technology. And the way it works is, as you move your body through the world, if, as an engineer, I can make a system
that can measure that motion, and then present to you
sensations over time that kind of make sense, that match up with what you might feel
in the real world, I can fool you into thinking
you’re touching something even though there’s nothing there. So here are three examples and these are all done
from research in my lab at Penn. The first one is all about that same
problem that I was showing you: how can we capture how objects feel and recreate those experiences? So the way we solve this problem is by creating a hand-held tool that has many different sensors inside. It has a force sensor, so we can tell
how hard you’re pushing; it has motion tracking, so we can tell
exactly where you’ve moved it; and it has a vibration sensor,
an accelerometer, inside, that detects the shaking
back and forth of the tool that lets you know
that’s a piece of canvas and not a piece of silk or something else. Then we take the data we record
from these interactions. Here’s ten seconds of data. You can see how the vibrations
get larger and smaller, depending on how you move. And we make a mathematical
model of those relationships and program them into a tablet computer so that when you take the stylus and go and touch the screen, that voice-coil actuator
in the white bracket plays vibrations to give you the illusion that you’re touching the real surface, just like if you touched, dragged
back and forth, on the real canvas. We can create very compelling illusions. We can do this for all kinds of surfaces
and it’s really a lot of fun. We call it haptography —
haptic photography. And I think it has potential
benefits in all sorts of areas like online shopping,
maybe interactive museum exhibits, where you’re not supposed
to touch the precious artifacts, but you always want to. The second example I want to tell you
about comes from a collaboration I have with Dr. Margrit Maggio
at the Penn Dental School. Part of her job is to teach
dental students how to tell where in a patient’s mouth
there are cavities. Of course they look at X-rays, but a large part of this clinical judgment
comes from what they feel when they touch your teeth
with a dental explorer. You’ve all had this happen,
they go across. What they’re feeling for is if the tooth
is really hard, then it’s healthy, but if it’s kind of soft and sticky, that’s a signal that the enamel
is starting to decay. These types of judgments are hard
for a new dental student to make, because they haven’t touched
a lot of teeth yet. And you want them to learn this
before they start practicing on real human patients. So what we do is add an accelerometer
on to the dental explorer, and then we record what Dr. Maggio feels as she touches different extracted teeth. And we can play it back for you as a video with a touch track — not just a sound track,
but also a touch track, that you can feel by holding
that repeating tool. You feel the same things the dentist
felt when they did the recording, and practice making judgments. So here’s a sample one. Here’s a tooth that looks
kind of suspicious, right? It has all those brown stains. You might be thinking, “We should
definitely put a filling in this tooth.” But if you pay attention to how it feels, all the surfaces of this tooth
are hard and healthy, so this patient does not need a filling. And these are exactly the kind
of judgments doctors make every day and I think this technology
we’ve invented has a lot of potential for many different things in medical
training, because it’s really simple and it does a great job at recreating
what people feel through tools. I think it could also help make games
more interactive and fun and more realistic
in the sensations that you feel. The last example I want to tell you about
is again about human movement. So if any of you have ever learned sports, how do you get good
at something like surfing? You practice. You practice some more and more, right? Making small corrections,
maybe getting some input from a coach, learning how to improve your motions. I think we could use computers to help make that process
more efficient and more fun. And so here, for example,
if I have six different arm movements that I want you to learn, you come into my lab at Penn
and try out our system. We use a Kinect to measure your motions,
we show graphics on the screen, and then we also give you touch cues, haptic feedback on your arm, delivered by these haptic arm bands
which have motors inside, and guide you as you move. So, if we put it together, as you’re trying to track this motion, if you deviate — say, maybe,
your arm is a little too high — we turn on the motors
right there on the skin to let you know you should move down, almost like a coach gently guiding you and helping you master
these movements more quickly and make more precise corrections. We developed this system
for use in stroke rehabilitation, but I think there are a lot
of applications, like maybe dance training
or all sorts of sports training as well. So now you know a little bit
about the field of haptics, which I think you’ll hear more
about in the coming years. I’ve shown you three examples. I just want to take a moment to acknowledge the great students
who work with me in my lab at Penn and my collaborators. They’re a great group. I also want to thank you
for your kind attention. (Applause)
Web Designing Course in Madhapur | web development courses | web design classes | learn web design
design a simple bootstrap website in 30 mins
Vera Modugno – Corso Web Design
About Author
kewl
I am blown away by the cool things technology is starting to develop. I can't wait to see how all this progresses.
Amazing
awesome technology!
Penny learned science
What about PORN?! This + Oculus rift <3
She is smart as well as beautiful !
I am very skeptical about the practicality of most haptics. I researched this last summer, and honestly, the expense is usually not worth the cost. The original Phantom (haptics device) was made at MIT in the 90s (did you "invent it"?), but I would not say its use is common. Moreover, Novint (which had the videogame console shown in the video) is sorely out of business. Look up the ticker symbol NVNT. Please don't try "researching haptics." Focus 99.9% on the applications of what already exists.
And to add, put the focus not on haptics. The focus – if money means anything – should be about integrating small bits of haptics into things people use everyday. And then use haptics sparingly.
The haptic devices in cell phones and tablets have a market. The haptic devices in narrow fields in the medical world, barely have a market. Please focus on bringing the cost down to a few dollars for simple devices rather than massive complex ones. But good luck with your research nonetheless.
Please deinterlace videos before uploading kthxbye. If you view these videos in 1080p you get combs when viewed in 1080 and even worse waves if you view it another resolution.
To make the point a little more clear. Consider the dentist example. It would be so much cheaper to simply injection mold fake teeth that exhibit the characteristics you want (softness for cavities). As for dance, haptics might be slightly useful, but I doubt it will catch on. An instant reply with a visual (or just the live visual) suffices. And if someone had the money for all the fancy haptic equipment, they would have just gotten a personal trainer. I would be happy to hear your response.
I thought she was going to say "as an engineer, I built a rock-cat-bunny."
first up, the porn industry. its so damn wet…
Also she got my hopes up with the force detector.. Also, she hasn't heard of the N64 Rumble Pack? This technology is pretty old, but good to see it finally being used in the medical industry.
damn bro, slow down, smoke a joint and take a stroll thru nature.
what about liquids gas… and semi solid ??? GAS?? anyways good for us in many ways, overall haptics is cool tech.
hahahahaa lol
I couldn't watch past the dentist part… fuck dentists.
I love the parrot on your team!
Also love the tech direction…
what about a computer mouse that can adapt do different touch feelings
So now we can record an orgasm and feel it all the time 😀
If you can simulate painting on a canvas surface or something with a pen device you should market that to fund all your other projects. I think a lot of artists are in halves about using smart pads to paint because they don't like the sensation of slick surfaces.
After noticing all the other comments I had to remark respectfully pornography / sexual aids isn't a bad idea either in all seriousness. Pornography funded the internet, sold VCR and DVD players, and I have heard some theorize the drive for better pornography fuels technology, and of coarse all the money.
This interesting subject is about something that enables us to work in the dark too.
………,……………….no just no
Still hope that a nerve gear is possible… if you know what i'm talking about you feel the same way. recording vibrations and how things feel would be very important to making the device work.
I'm so happy I have my dad's teeth, he got his first cavity at the age of 45, I have yet to have anything wrong with mine. That tooth they showed looked disgusting.
…………………yes, just yes.
That is very interesting.
Uhm, is that a parrot among the students in the top right corner at 6:16?
Oh we're gonna do that as soon as we get the chance, people are already working on it.
Probably a class pet.
already exists.. fleshlights
woah 5:11 what the hells wrong with his legs
What is the parrot's job?
Made me think of the Spoke word by Marshall Davis Jones : "Touchscreen"
"Maybe one day our technology will be advanced enough to make us Human again."
Here lies the beginning of the Matrix.
So awesome!
You're touching upon a very fascinating topic, that many people are unaware of. We tend to focus on sights and words, and often completely ignore the tactile/kinaesthetic side of life.
I've had this interest for a while, and it has helped me a lot with dancing, and feeling much better in my body. It never occurred to me that you can use it with technology.
This woman is hella hot.
this is wonderfull, the applications are endless…..and porn
Fascinating!
That it cool :)! Welldone!
here, not available download option, before it's available, presently remove the option why?
please it can updated this option
MARVELOUS!!!
here, take this Medal!
One can only imagine the possibilities for laparoscopic/robotic surgery. And porn.
I assume you mean this not like the pen does, but via electrical stimulation on the brain??
Hmm… I wonder how it would feel if you were to link up a man to a woman's body and vice versa
i love this woman 🙂
So with this everyone can be a great guitarist or pianist in near future?
Me too : )
Marry me? Hahaha
far from it, while this technology helps with getting used to the surface texture, guitars and pianos require positional data which cant be learnt without practice. all it might do is make learning instrument tuning easier, since you relate the tension of the string to the sound
even so, i digress, the above hypothesis is based on the demo and current tech, most likely over time the simulations become more accurate and include more sensations outside of texture
do
{
fapp()
} until come;
fapp()
{
stroke_penis up()
stroke_penis_down()
}
Free handjobs for everybody
i wonder if you could develop tele-haptography, where one uses light (generally em waves) and pulses of light to record textures and a materials hardness
could this system be equivalent to that pen she showed at the beggining?
remmeber that scene from daredevil where the girl is in the rain and the daredevil guy can see her face from the sound of the water hitting her face? maybe one could use sound too
awesome
but i guess, our mind plays more important role in preparing itself , even before we touch anything…an example is when we touch anything that is from opposite sex…or anything that we perceive as dangerous…not very good examples through
How much coffee did she drink?
Cool! That can make 4D movies not just spill some water on your face.
Doing a study on students and GPA factors, all submissions will help. https://docs.google.com/forms/d/1f4RhkiQn31yl3TQLvaInHSP9iYaoRtFeW6NsLF1ZiHM/viewform
The future of weapons and sex toys… and weaponized sex toys.
Amazing!
Matrix here we come..
this is awesome. It great potential also for the vision impaired as the sense of touch is an extremely important sense
This is new and primitive like computers were in the 60's-70's.. 40 years from now that will be the way we interact with the world no doubt about it.. too bad ill be like 60 wont be able to enjoy the awesome entertainment this will bring..
Porn any one. ?
THAT IS SO COOL!!!
So can you feel metallic surfaces with this?
nintendo nx va atener esta tecnologia
Amazing idea…
This is a truly game changing technology which will play an integral role in the further development of VR and shrinking the gap between the physical world and the virtual world
How about smell?
Is it only me or is there others annoyed by her insincerity.
wtf is going on with her hands/ the camera? xD
the possibilitys are truly magnificent with this kind of technology! keep it up Katherine and students
Awesome and very important. …
Instead of sequestering this information from me.
Until after i teach myself about this through numerous mistakes.
You could have posted this years ago.
And I'd have known what to have looked for
wow amazing