Vision Processing for Handling Garments in Unknown States

In recent years, automation by robots has been progressing in various industries. However, in some industries, such as garment manufacturing, most tasks are still predominantly manual. Garments can take on a multiple of shapes due to their flexibility and elasticity. This makes it difficult for robots to recognize and manipulate them. Our general research goal is to achieve flexible garment handling in order to automate various tasks in the garment manufacturing industry using robots. In our work, we assume that garments to be handled are initially in random states on a table. We are concurrently working on the vision-based recognition of the garments on the table and the garments grasped and hung by a dual-armed manipulator. In the former effort, we developed a method combining SLAM-based 3D point map building and 2D instance segmentation. In the latter effort, we assume that the manipulator hand grasps a random point on the garment at first. we developed a method for manipulating clothing in the air using the dual-armed manipulator so that the state of the garment can be recognized using a depth camera. Specifically, we developed a clothing handling strategy focusing on the action of lifting up a garment and regrasping the lowest part of the garment and a depth-image segmentation model to recognize the important parts of the garments such as the neck and hem parts.