Navigating Without a Joystick
Wheelchair users with severe motor impairments often face a frustrating paradox: they may need the most sophisticated mobility technology but have the least physical ability to operate standard joystick controls. Research teams at the German Research Center for Artificial Intelligence (DFKI) in Bremen believe AI can close this gap. At the CSUN Assistive Technology Conference in Anaheim, California, DFKI senior researcher Christian Mandel and his colleague Serge Autexier presented prototype wheelchairs that navigate environments both semi-autonomously and fully autonomously — responding not to joystick input but to spoken natural-language commands like "drive me to the coffee machine."
How the System Works
The DFKI prototypes integrate multiple sensor modalities: two lidar sensors providing 360-degree obstacle detection, a 3D depth camera for close-range spatial awareness, wheel odometers for position estimation, and an embedded computer running real-time processing. The system also interfaces with room-level infrastructure including drone-mounted cameras that provide a bird's-eye view of the environment.
In semi-autonomous mode, the user drives with a joystick while the system monitors surroundings and intervenes to prevent collisions. In fully autonomous mode, the user speaks a destination, confirms the intended path, and the wheelchair navigates there using the open-source ROS2 Nav2 navigation stack with simultaneous localization and mapping for real-time map building and obstacle avoidance. The system does not require pre-mapped environments, which is crucial for real-world usability in spaces that change daily.
The Cost and Reliability Problem
Pooja Viswanathan, CEO of Toronto-based Braze Mobility, identifies cost as the primary obstacle. Power wheelchairs already cost tens of thousands of dollars, and adding lidar and computing hardware could add significant cost before integration labor is counted. Funding systems were designed around conventional wheelchairs and are not equipped to evaluate or reimburse advanced AI systems.
Reliability presents an equally serious challenge. A wheelchair is not a convenience tool — for its user, it is the primary means of independence. Louise Devinge, a biomedical research engineer at IRISA in France, frames the reliability challenge clearly: "The more sensing, computation, and autonomy you add, the harder it becomes to ensure robust performance across the full range of real-world environments that wheelchair users encounter."
The Collaboration Philosophy
A recurring theme at CSUN was designing AI wheelchair systems as collaborators rather than replacements. Many wheelchair users already navigate with remarkable skill and would find fully autonomous systems disempowering. The technology should amplify capability in specific scenarios where impairment creates genuine barriers, not impose complexity on users who have developed effective compensation strategies.
Mandel describes a moment early in his career watching a user with severe impairments navigate a narrow passage with skill that surpassed his smart wheelchair system's capability. "Never underestimate what wheelchair users can do without it," he says. He estimates mainstream-ready smart wheelchairs are roughly ten years away, a timeline consistent with typical medical device development and regulatory approval cycles.
This article is based on reporting by IEEE Spectrum. Read the original article.
