Wednesday, February 15, 2012

Why Do We Need NUIs? Assistive Technology

If I had to pick one reason why we need natural user interfaces (NUIs) it would be for use in assistive technology. Technology for increased convenience is all well and good. Technology for improving our ability to multi-task may or may not be good. A growing body of literature demonstrates the more we multi-task the less productive and efficient we actually are. Disturbing to say the least.

However, there are people for whom NUIs have the potential to be life transforming. Jonathan Josephson of Quantum Interface (introduced in my last post) told me about an incredibly moving experience he had observing a quadriplegic trying out a motion based NUI prototype. Apparently, many partially paralyzed people have limited range of motion in their forearm but not their upper arm. After studying what the natural motions were in this situation, QI designed an interface that enabled this individual to interact with his computer using his arm even though he could not lift it. Consider: prior to that moment, this man had been forced to use a straw held in his mouth. Put yourself in his shoes and imagine what that would be like. 
How would you feel?

Jonathan's voice choked up as he told me how the user and his family members were virtually in tears. To them, this interface represented freedom and autonomy.

What is holding back full scale development and deployment of this type of NUI? Sensor technology for one thing. We need highly precise 3D pinpointing sensors to locate and track motions, and to enable fine tuned feedback. Fortunately, says Jonathan, these are on the horizon.

So when people get into discussions about what is "natural", there is no single, simple, answer. As pointed out by a LinkedIn reader in response to my last post, there will be cultural differences. There will be differences based upon ability and impaired ability. As we become increasingly sophisticated, and some inventors move into the realm of AI, cognitive issues will become increasingly important. For example, what is "natural" will be different for victims of head trauma.

Our society is undeniably digital and the more people who can access technology easily and naturally, the better. 

There is a ways to go before the assistive technology Jonathan envisions can be produced and marketed widely and affordably, but when it happens the societal impact has enormous potential. At the end of the day, this potential is what drives Jonathan in his NUI work.


  1. Thank you for this post. Will pass it on to my friends who teach Life Skill classes in public school. Look forward to hearing more.
    Abigale Barnabas

  2. Linda, I just came across your blog today.

    'm a school psychologist who works mainly at two schools. One is a high school for technology and the arts, and the other is a program for students up to age 22 who have multiple or significant disabilities, including autism. Both schools are technology-rich.

    I agree with interdisciplinary approach to computing-partly because I am an "interdisciplinary" person, with university work in psychology, social science, stats, research, ed tech, special educaton, research, computer programming, art, music, dance, etc. Every discipline needs computing in some way, and also needs a few people who have computational thinking skills and an understanding of other "schools of thought".

    Most of the computer/tech people I've come across have talent, training, and/or experience in other disciplines, including the arts. I've been a member of the NUI-group since 2007, I was more active with the group when I was in graduate school taking computer courses (a mid-life venture). I'm happy to say that many of the NUI group members I met at the time had an interdisciplinary background.