TECH
Smartwatches could learn to read your hands
Northwestern University engineers are investigating new ways to tell what the hands are doing — including peering beneath the skin to watch muscles and tendons move. The new methods don’t use high-tech cameras, medical scans or advanced imaging techniques. Instead, they use a common smartwatch.
In one project, engineers designed a new smartwatch system to glean activities from the arm and hands. In another, electrodes built into the back of a watch can track more subtle hand movements and gestures, such as a thumbs up or clenched fist.
By tracking these movements, researchers could one day help improve the control of bionic prosthetics or monitor patients in specialized care settings, including elder care. Beyond health care, the systems also could make smartwatches easier to use, much like facial recognition has simplified smartphone access, or extend to monitoring everyday activities, such as helping dental hygienists track how many teeth they clean in a week.
The researchers presented both studies, named TransfHAR and EITWatch, today (Nov. 3) at the Association for Computing Machinery’s Symposium on User Interface Software and Technology (UIST). The full symposium will be held Nov. 2–5, 2026, in Detroit.
“Your hands are how you act on the world, but your smartwatch knows almost nothing about what they’re doing,” said Northwestern’s Karan Ahuja, the principal investigator on both papers. “These two projects show how much the wrist alone can reveal — one by sensing the muscles and tendons beneath the skin, the other by learning how the arm moves. We can’t recognize every movement yet, but it’s a strong starting point.”
Ahuja is the Lisa Wissner-Slivka and Benjamin Slivka Assistant Professor in Computer Science at Northwestern’s McCormick School of Engineering, where he directs the Sensing, Perception, Interactive Computing and Experiences (SPICE) Lab.
TransfHAR was co-authored by Aidan Bradshaw, a Ph.D. student in the SPICE lab, Riku Arakawa of Carnegie Mellon University, and Xin Liu, a senior research scientist at Google Health Research. EITWatch was co-authored by SPICE lab members Xuanyou Liu and Novel Alam, who contributed equally and led the work.
Smartwatches have advanced well beyond the basics, like counting steps or displaying notifications. Today's models can monitor heart rate, blood oxygen levels, sleep quality, and even perform electrocardiograms. Researchers are now exploring another capability that could eventually make smartwatches much more intuitive to use: recognizing hand gestures without requiring cameras or bulky wristbands.
A team at Northwestern University has just reported on EITWatch, a smartwatch-integrated gesture recognition system that uses electrical impedance tomography (EIT) to sense muscle and tendon movement beneath the skin. Rather than relying on optical sensors or inertial measurement units, the prototype sends tiny electrical currents through the wrist and measures how those signals change as muscles contract. Because every gesture subtly alters the electrical properties of the tissue, the watch can identify different hand movements based on their unique electrical signatures.
This isn’t the first EIT-based gesture recognition study. However, past attempts typically required electrodes wrapped all the way around the wrist along with external electronics. EITWatch is designed around something users already wear every day: a standard smartwatch. The prototype places eight gold-plated electrodes in a 31 mm ring beneath a 40 mm watch case, allowing the entire sensing area to remain confined to the watch's contact patch. An onboard ESP32-S3 microcontroller captures impedance measurements at 48 Hz while sequencing the electrodes through multiple measurement paths.
Since a flat electrode array cannot surround the wrist the way earlier systems could, the researchers developed what they call a multi-depth scanning technique to compensate. Instead of always sending current between neighboring electrodes, the system varies the distance between the source and return electrodes. Longer current paths penetrate deeper into the wrist, allowing the watch to sample information from multiple tissue depths and muscle groups during each scan. In testing, this approach outperformed conventional adjacent-electrode measurements by 15.1 percentage points for larger gestures and 10.4 percentage points for smaller ones.
The researchers evaluated EITWatch with 12 participants performing two gesture vocabularies. One consisted of six larger static hand poses, while the other included five subtle micro-gestures along with a relaxed resting position. Within a single session, the system achieved recognition accuracies of 92.5% for the larger gestures and 91.5% for the micro-gestures, showing that a smartwatch-sized sensing area can capture enough information for reliable classification.
Keeping watch of the hand...EITWatch takes advantage of a smartwatch’s constant contact with the wrist. Using a technique called Electrical Impedance Tomography (EIT), eight small electrodes on the back of the watch send tiny electrical signals through the wrist to map the muscles and tendons beneath the skin.
Each hand gesture — from pointing a finger to giving a thumbs up — produces a distinct pattern as muscles and tendons move. By learning to recognize these patterns, EITWatch can identify specific gestures and translate them into smartwatch commands. A thumbs up, for example, could send a text message, while a pointed finger could activate a step-counter during exercise.
“The part I care about most is accessibility. Touchscreens shut out many people,” Alam said. “A second input channel built into the device gives them another way in.”
Because the same wrist signals encode what the hand intends to do, the researchers are also exploring whether this kind of sensing could help control prosthetic hands. Looking inside the arm also gives scientists a new way to track the hand, bypassing the need for cameras.
“Right now, the best way to tell what the hands are doing is a camera, and that’s a privacy nightmare,” Ahuja said. “It also only works when your hand is in view. The wrist is always there.”
Unfortunately, the system is currently not very stable across time. When users put the watch on after two days without use, recognition accuracy dropped to 73.2%. Performance fell further when attempting to recognize gestures from entirely new users without personalized training data. To make the device practical, it will need to deal with small changes in watch placement and differences between individuals better.
The team plans to address these limitations using representation learning and contrastive learning techniques designed to produce gesture embeddings that remain stable across different users and repeated wearings. If successful, future smartwatches could gain a powerful new input method without adding cameras, external accessories, or changing their familiar form factor.
EITWatch is still a research prototype. In the study, it recognized six common hand gestures with over 92% accuracy and five subtler finger movements with over 91%, but it works best when calibrated for a single user. That means borrowing a family member’s smartwatch, for example, might make it more difficult for the device to accurately recognize the new user’s wrist movements.
“Going forward, we will focus on two things. For hardware, fitting the electronics inside the watch is the next push,” Alam said. “Then there’s the people problem. One person wearing the watch does way better. The goal stays simple though. Any wearer puts the watch on and the gestures work. No calibration step.”
Watching the whole arm...Although today’s smartwatches and smartphones can recognize certain gestures, those movements are typically limited to a predetermined set of motions. TransfHAR takes a more flexible approach. The system first learns from thousands of hours of public, unlabeled motion data from everyday activities such as walking and exercise. It can then learn an entirely new activity from just a few demonstrations. Users can then associate specific movements with customized commands, expanding what their devices can do.
New research from Northwestern University introduces TransfHAR and EITWatch, two smartwatch systems that track hand gestures and arm motions without using privacy-invasive cameras.
How the new systems work:
• EITWatch (Electrical Impedance Tomography): Uses tiny electrodes on the back of the smartwatch case to send electrical currents through the skin. By varying the distance between source and return electrodes, it scans multiple tissue depths and muscle groups in the wrist. In studies, it recognized 6 common hand gestures with over 92% accuracy and 5 micro-finger movements with over 91% accuracy.
• TransfHAR (Transformer-based Human Activity Recognition): Uses existing motion sensors (accelerometers and gyroscopes) already found in standard smartwatches. Pretrained on thousands of hours of movement data, it allows users to name and demonstrate a new action (like brushing teeth or lifting weights) via a quick one-minute recording, achieving roughly 90% recognition accuracy.
Key benefits and use cases:
• Privacy Protection: Replaces camera-based hand tracking with wrist-level internal muscle and motion sensing.
• Healthcare & Monitoring: Enables caregivers or nurses in eldercare facilities to passively track daily routines—such as hygiene or physical therapy exercises—without placing cameras in private rooms.
• Prosthetic Control: Translates intended hand and wrist signals into alternative control channels for prosthetic devices.
• Fitness Tracking: Automatically logs specific gym exercises and repetitive daily tasks without manual input.
Current limitations:
• Calibration Requirement: Both prototypes currently achieve peak accuracy when calibrated specifically to an individual user, meaning sharing a watch with someone else reduces recognition precision.
• Form-Factor Scaling: Engineers are still working on shrinking all necessary hardware components to fit comfortably inside commercial, everyday smartwatch bodies.
mundophone
