A system lets robots inspect random objects and visually understand them enough to accomplish specific tasks without ever having seen them before. The system, called Dense Object Nets (DON), looks at objects as collections of points that serve as sort of visual road maps. What's also noteworthy is that none of the data was actually labeled by humans. Instead, the system is what the team calls “self-supervised,” not requiring any human annotations.
The DON system essentially creates a series of coordinates on a given object, which serve as a kind of visual road map, to give the robot a better understanding of what it needs to grasp, and where.
The team trained the system to look at objects as a series of points that make up a larger coordinate system. It can then map different points together to visualize an object's 3D shape, similar to how panoramic photos are stitched together from multiple photos. After training, if a person specifies a point on an object, the robot can take a photo of that object and identify and match points to then pick up the object at that specified point.
For more information, visit here.