If you’re like me, you’ve probably been fascinated by movies like I, Robot, Steven Spielberg’s A.I. Artificial Intelligence, or Jon Favreau’s first Iron Man. The coolest part about all three movies was, let’s face it, the robots – yes, I know the Iron Man suit is technically a powered exoskeleton, but it can operate on its own by means of AI (J.A.R.V.I.S. was my favorite). These movies also left me daydreaming about when we’ll see those types of machines in real life.
With exciting developments in the humanoid robotics space from companies like Boston Dynamics, Figure AI, and AgiBot, it seems like we have cracked two of the three most important challenges: movement and vision. The remaining challenge is touch.
A Blast From The Past
When the movie I, Robot came out, Will Smith slapping Chris Rock and AI-powered robots with ultra-intelligence seemed equally unlikely to happen.
Sure, we had ASIMO launching in Y2K (a far cry from the apocalyptic machine uprising that many feared would happen). Honda’s first-generation ASIMO was a ground-stomping development in bipedal robotics. Launched after 14 years of development, with 26 degrees of movement, it was the first robot that could walk smoothly without falling. While it still required an actual person to tell it where to go via a controller, ASIMO did work out how to shift its weight and move its legs on its own. This was done through an impressive combination of programming and sensors.
While its performance was incredible at the time, we cannot call ASIMO fully intelligent (sorry buddy). At least not to the extent of the AI robots foreshadowed in film.
Back To The Future
Making a robot walk on two legs was one of the hardest problems to solve in the past. Today, humanoid robots can perform backflips, handstands, and complex parkour routines. Now researchers are publishing low-cost, open-source biped platforms for teaching and testing walking controllers.
As we speak, Boston Dynamics is preparing to deploy its Atlas humanoid robot for industrial work, and it comes with the promise of an autonomous mode.
In a recent demo, Atlas has shown the ability to independently move car engine covers from supplier containers onto a mobile sequencing dolly. When fixtures moved, or an insertion failed, it detected the change and adjusted its actions. ASIMO walked so that Atlas could autonomously work in heavy industrial, manufacturing, and logistics environments.
Atlas has 56 degrees of freedom (30 more than the first ASIMO, for those counting at home) and joints designed for a wide range of motions. It also has cameras and vision models to help identify parts and storage fixtures, estimate their position in a 3D space, and track an object as the robot moves it. Its vision system can even notice that a fixture has been moved and update its task accordingly.
Broadly, these movement and vision advancements are consistent throughout the humanoid robotics space (Figure 03 or Agility’s Digit, for example). Yet the extraordinary dexterity of the human hand remains beyond their reach: touching, feeling, and handling objects with the same precision and adaptability is still a formidable challenge.
Seeing a cup is vastly different from knowing how firmly to grip it without it sliding out of your hand, or without crushing it. Researchers have built grippers that detect slip and handle delicate items, but grip force control with tactile feedback is an active focus for many companies planning innovations in this space. The challenge is making that adaptability reliable across many objects and tasks.
“All Your Sensors Are Belong To Us” – A Centralized Way To Process Sensor Intake
Traditionally, the best way to solve an issue was to throw more compute at it. In real life, this would be done via centralized computing inside the robot, where cameras, grippers, and other sensors would send data to a large model running on a central GPU, which in turn sends instructions back to the motors. Atlas for example, is integrating NVIDIA’s Jetson Thor alongside CPUs and other processing hardware to run complex AI models that have a system-wide impact.
However, this centralized approach has two main downsides:
- Power: Processing all sensor data through a large model consumes significant energy, which matters a lot for a battery-powered robot
- Latency: When the object that a robot is holding starts to slip, the hand will need to react immediately, with tiny adjustments. Sending every tiny adjustment through the central processor makes that control loop harder to run quickly and efficiently.
A better way to do this is to bring the control loop closer to the hand.
The Matter at Hand – Local Signal Processing Loops
We can generalize robot decision-making as binary: thinking and acting. The central processing can decide it’s time to pick up an object, while the local control loop can adjust the grip to make sure the object is not slipping or being crushed. In a more complex robot, this local processing would happen adaptively, reducing the need to send data to a central processor and wait for a response to handle every small correction. It would also leave bandwidth for the central processing to focus on different tasks.
Neuromorphic microcontrollers like Pulsar are ideal at that local level. They can process signals from a vast array of sensors (including motion, pressure, radar, microphone, etc.) to recognize object slippage or density and adjust grip in real time.
In fact, Pulsar has already been deployed by Analog Devices in a robotics tactile finger application earlier this year. The finger uses a microphone and an inertial measurement unit (IMU) to capture signals, sending them to Pulsar, which runs a spiking neural network (SSN) of roughly 200 neurons close to the finger. As the finger moved across materials such as paper, plastic, foam, and wood, Pulsar classified the surface in real time with 98% accuracy. It did so with a reported 25x improvement in power efficiency compared to more traditional NPUs.
It’s now clear that processing data from a fingertip doesn’t inherently require power-hungry, central AI to interpret the signal. A dedicated processor such as Pulsar can classify specific sensor signals at milliwatt or sub-milliwatt power. Onboard GPUs like the Jetson Thor in Atlas operate at tens to hundreds of watts to handle much broader workloads.
How To Train Your Models
We’ve managed to localize intelligence for sensing. The next question is what model do we use to turn that signal into an action? Engineers can start with what’s familiar: classical controller programming.
This approach would begin with deterministic conditions and actions written by the engineers. Sensors would measure grip force and signs of movement. Then the software compares those readings with a target: is the grip secure, and am I using more force than necessary? If the robot detects slipping, it commands the motor to tighten a little, measure again, and repeat.
These rules can be hard to tune. A smooth cup, a soft sponge, and a heavy box respond differently to touch. Scaling this process to different objects is hard, because there are too many variables to anticipate, such as surfaces, density, and fragility.
Learning from examples can reduce the need to write rules for every object. But bringing that intelligence to new hardware requires software that makes models straightforward to train and deploy.
The Jump to Neuromorphic
A neuromorphic SDK provides that route. Built around familiar tools such as PyTorch, it helps engineers train spiking neural networks (SNNs) and deploy them on neuromorphic chips, lowering the barrier to entry through workflows they already know.
For a focused job, like detecting touch or slip near a fingertip, that offers a route to even faster responses with little data movement and lower energy use, as demonstrated in the Analog Devices application. SNNs, by their nature, hold incredible potential for localized, efficient, always-on processing.
Future is Now
At the beginning, we said that walking and vision in robotics, which previously were complex programming undertakings, have now become more accessible than ever. One challenge that remains is how robots interact and adapt physically to objects and their surroundings, and how we can transfer that learning across both actions and machines.
Brute force approaches, such as throwing more compute at the challenge, unfortunately come with a high cost of power and latency.
It’s time to take a leaf from the sci-fi movies that left their mark on us decades ago and dare to dream of the future. Just because something is done a specific way doesn’t mean it’s the only way through. High-performance centralized compute and cloud compute have been the status quo for the longest time. But nature offers a different blueprint: the human brain, capable of extraordinary perception and rapid reactions on remarkably little power. Neuromorphic computing takes inspiration from that blueprint, opening up possibilities we are only beginning to imagine.
