Amazon launches reinforcement learning tools to manage robots’ workflows

Amazon today launched SageMaker Reinforcement Learning (RL) Kubeflow Components, a toolkit supporting the company’s AWS RoboMaker service for orchestrating robotics workflows. Amazon says that the goal is to make it faster to experiment and manage robotics workloads from perception to controls and optimization, and to create end-to-end solutions without having to rebuild them each time.

Robots are being used more widely for purposes that are increasing in sophistication, like assembly, picking and packing, last-mile delivery, environmental monitoring, search and rescue, and assisted surgery. In China, Oxford Economics anticipates 12.5 million manufacturing jobs will become automated, while in the U.S., McKinsey projects that machines will take upwards of 30% of such jobs. As for reinforcement learning, it’s an emerging AI technique that can help develop solutions for the kinds of problems that are increasingly cropping up in robotics.

SageMaker RL builds on top of Amazon’s SageMaker machine learning service, adding prepackaged toolkits designed to integrate with simulation environments. With Amazon SageMaker RL Components for Kubernetes, customers can use SageMaker RL Components in their pipelines to invoke and parallelize SageMaker training jobs and RoboMaker simulation jobs as steps in their reinforcement learning training workflow without having to worry about how it runs under the hood, according to Amazon.

Above: Ripley, Woodside’s robotics platform, takes advantage of reinforcement learning to perform manipulation tasks.

Image Credit: Amazon

Running the SageMaker RL Kubeflow Components requires an existing or new Kubernetes cluster. Customers also must install Kubeflow Pipelines on the cluster and set up identity and access management roles and permissions for SageMaker and RoboMaker, according to Amazon. The company provided step-by-step instructions to create the pipeline in a blog post.

Woodside Energy tapped RoboMaker with SageMaker Kubeflow operators to train, tune, and deploy reinforcement learning models to their robots to perform repetitive and dangerous manipulation tasks. The company engaged Australia-based consultancy Max Kelsen to assist in the development and contribution of the RoboMaker components. For example, Ripley, a robotics platform built by Woodside, was trained to perform a “double block and bleed,” a manual pump shutdown procedure that involves turning multiple valves in sequence. A reinforcement learning formulation created with RoboMaker and SageMaker uses joint states and camera views as inputs to a model that outputs optimal trajectories for manipulating the valves.

“Our team and our partners wanted to start exploring using machine learning methods for robotics manipulation,” Woodside robotics engineer Kyle Saltmarsh said in a press release. “Before we could do this effectively, we needed a framework that would allow us to train, test, tune, and deploy these models efficiently. Utilizing Kubeflow components and pipelines with SageMaker and RoboMaker provides us with this framework and we are excited to have our roboticists and data scientists focus their efforts and time on algorithms and implementation.”