181 karma · joined September 19, 2024
Custom geometries will require a new network, yes!
More than happy to answer questions about it either here or on my email as the corresponding author on the paper!
The reason we don't want to do this is that it is difficult to cover all possible characteristics. Say we do single point contact localization, and 3-axis forces prediction. What happens when we have multi-point contact? The calibration has only been used to calibrate/align in a lower dimensional space. This is primarily why not needing calibration and baking this into the hardware is a lot more appealing. The user/designer no longer needs to think about the task and the dimensions of alignment required for that task.
That being said, the exact quantities the policy depends on are hard to interpret, given the use of deep learning. This could potentially be modality agnostic, but there has been no sensor so far that has shown (1) the ability to detect intuitively relevant quantities like contact location and 3-axis forces, and (2) sufficient signal consistency for deep learning models to generalize across instances. This was a key motivating factor for AnySkin, and we found a relatively straightforward fabrication procedure that enables this for magnetic sensing.
With capacitative sensors, it is unclear from existing literature if it is possible to detect shear. Additionally, they generally operate at significantly lower frequencies.
- AnySkin expressly handles wear and gunk by being replaceable. So if it wears out, and you have a heuristic or learned model for the old skin, it will work pretty well on the new skin! We verify this through an analysis of the raw signal consistency across skins, as well as through visuotactile policies learned using behavior cloning. We found swapping skins to work for some pretty precise tasks like inserting USBs and swiping credit cards.
- Could definitely be used for part motion detection
- Soft, inflatable grippers are effective, but often passive. AnySkin is not just soft, but also offers contact information from the interaction to actively ensure that blueberry doesn't get squished!
- This sensor would be key for robots that seek to use learned ML policies in cluttered environments. Robots are very likely to encounter scenarios where they see an object they must interact with, but the object is occluded either by their own end-effector(s) or by other objects. Touch, and an understanding of touch in relation to vision becomes critical to manipulate objects in these settings.
- Industrial robots do have very sensitive motor and arm feedback. However, these systems are bulky and unsafe to integrate into household robotic technologies. Sensors like AnySkin could be used as a powerful, lightweight solution in these scenarios, potentially by integrating with some exciting recent household robotics models like Robot Utility Models.
- ReSkin, the predecessor to AnySkin, has previously been used quite effectively for fabric manipulation! (see work from David Held's group at CMU). AnySkin is more reliable as well as more consistent and could potentially improve the performance seen in prior work.
Our most exciting result: Learned visuotactile policies for precise tasks like inserting USBs and credit card swiping, that work out-of-the-box when you replace skins! To the best of our knowledge, this has never been shown before with any existing tactile sensor.
Why is this important? For the first time, you could now collect data and train models on one sensor and expect them to generalize to new copies of the sensor -- opening the door to the kind of large foundation models that have revolutionized vision and language reasoning.
Would love to hear the community's questions, thoughts and comments!