An innovative human-machine interface (HMI) system, which consists of flexible, multi-layered electronic skin, provides both visual and haptic feedback to users. The system can teleoperate the robot to imitate the user's actions to perform complicated tasks, including conducting COVID-19 swab tests.
The HMI system links users to robotics or computers and plays a significant role in teleoperating robotics. However, conventional HMIs are based on bulky, rigid and expensive machines, and the lack of adequate feedback to users limits their application for conducting complicated tasks.
The advanced HMI system, named Robotic VR, is an integration of visual, auditory and haptic virtual reality (VR) via skin-integrated electronics. It uses a layout of multilayer stacking, where the bottom layer of skin-tone elastomeric silicon serves as a soft adhesive interface, which can be mounted on the skin and joints of the user. The layers are interconnected with a collection of chip-scale integrated circuits and sensing components, including resistors, capacitors, a Bluetooth module, a microcontroller unit (MCU), and soft sensors and actuators developed by the team.
The sensors of the Robotic VR system can accurately detect and convert subtle human motion into electrical signals, which are processed by the MCU and wirelessly transmitted to the target robot.