At the 2023 CES held recently, NVIDIA released the latest version 2022.2 of Isaac Sim. What are the highlights of the new version? Which industry pain points of robots will be solved?

      Major innovation of Isaac Sim 2022.2: adding personnel simulation function

      According to ABI Research, the installed capacity of industrial and commercial robots will increase from 3.1 million in 2020 to 20 million in 2030, an increase of more than 6.4 times. In order to develop, verify and deploy these new AI robots, enterprises need to use simulation technology to put these robots into real scenes. In addition, the cooperation between humans and robots has become increasingly important.

      The latest version of NVDIA Isaac Sim 2022.2 focuses on improving the performance and functionality of the manufacturing and logistics robot use case.

      Isaac Sim's newly added personnel simulation function can add synthetic personnel to warehouses or production facilities, and let them complete common tasks such as stacking packages and pushing carts. The tool now supports many common behaviors, so simulating them is as simple as issuing commands.

Figure 1: Personnel simulation of Isaac Sim 2022.2

      It is not easy to realize this function! In order to minimize the gap between the results observed in the simulated world and those seen in the real world, a sensor model that conforms to physics must be available. With NVIDIA RTX technology, Isaac Sim can now render physics compliant data from sensors in real time. When using RTX to simulate lidar, ray tracing can provide more accurate sensor data under various lighting conditions or when reacting to reflective materials.

Figure 2: Isaac Sim provides RTX radar and sensor support

      Isaac Sim also offers a number of new simulated 3D assets, which are critical to building a physics compliant simulation environment. From warehouse parts to robots, developers and users can quickly start building.

      NVIDIA Isaac Sim 2022.2 helps three application scenarios

      Warehouse logistics: warehouse logistics is one of the most important areas of robot innovation. Several tools in Isaac Sim can meet the needs of simulated warehouse robots. The previous version included a warehouse building tool. Isaac Sim now includes a tool for building real conveyor belts. There is also a staff simulator that can add digital people to the warehouse environment. This will help to verify the perception and security systems of robots working near humans.

      Manufacturing: For a long time, the manufacturing environment has been using robots to automate processes. The 2022.2 version has many new functions in dealing with the motion control of the robot. The updated motion generation extension in Isaac Sim simplifies the integration and benchmarking of motion control algorithms in simulations. The included algorithm RMPFlow creates a smooth trajectory for the robot and has an intelligent anti-collision function. In addition to improving performance, this release also provides many usability improvements, including a graphical editor for Lula robot description files.

      Fleet optimization: launch the cuOpt expansion of Isaac Sim, which is the deployment of NVIDIA cuOpt engine directly integrated with Isaac Sim. NVIDIA cuOpt is an operational research optimization API that helps developers create complex real-time fleet routes. These APIs can be used to solve complex routing problems with multiple constraints, and provide new functions, such as dynamic rerouting, job scheduling, and robot path planning, while taking advantage of sub second solver response time.

      Simplify the work of robot researchers

      For robot researchers, the new version has a large number of new functions aimed at promoting reinforcement learning, collaborative robot programming and robot learning. Isaac Sim has upgraded its support for ROS 2 Humble and Windows for the huge robot operating system (ROS) developer community. All Isaac ROS software can now be used in simulations.

      Important new features designed for robot researchers include performance improvements for Isaac Gym. Isaac Cortex for collaborative robot programming has made new progress. Finally, Isaac ORBIT, a new open source tool, provides a simulated operating environment and benchmark for robot learning and motion planning. These new capabilities are very important for robot researchers.

      In addition, Isaac Sim 2022.2 can also support sensor upgrading, such as support for Isaac Core sensor extension program, support for RTX lidar and lidar model, and support for post processing of ROS/ROS2 replicator; You can also add new Sim Ready Assets tutorials; Add new robots, etc.

      Isaac Sim receives extensive ecological support

      The Isaac robot platform has also benefited from the support of a large number of ecosystems in many industries, including logistics, manufacturing, retail, energy, sustainable agriculture, etc. At present, more than one million developers are using the NVIDIA Isaac robot platform, and more than 1000 enterprises rely on one or more parts of the platform.

Figure 3: NVIDIA Isaac robot ecology

      Many enterprises have deployed physical robots developed and tested in the virtual world using Isaac Sim. For example, Telexitence has deployed beverage replenishment robots in 300 convenience stores in Japan; In order to improve safety, Deutsche Bahn is training the AI model to deal with critical but rarely unexpected extreme events in the real world, such as luggage falling on the train track; Sarcos Robotics is developing robots to select and place solar panels in renewable energy installations.

      Festo uses Isaac Cortex to simplify the programming of cooperative robots and transfer simulation skills to physical robots; Fraunhofer is using Isaac Sim's physics compliant full fidelity visualization function to develop advanced AMR; Flexiv is using Isaac Replicator to generate synthetic data for training AI models.

      Cloud access makes Isaac platform at your fingertips

      With Isaac Sim's cloud access now open, multidisciplinary teams engaged in robot projects worldwide can cooperate in testing and training virtual robots to improve accessibility, flexibility and scalability.

      When using robot systems to build new facilities or expand existing autonomous systems, the lack of sufficient training data often hinders deployment.

      Isaac Sim enables developers to create large-scale benchmark real data sets through Isaac Replicator, simulating the physical environment of the real world. After deployment, due to the expansion of automation requirements, it is necessary to operate an efficient fleet consisting of hundreds of robots through dynamic route planning. NVIDIA cuOpt is a real-time fleet task allocation and route planning engine that improves operational efficiency through automation.

      Although training robots is important, simulation also plays a key role in training human operators to work with robots and programming robots. Ready Robotics is using Isaac Sim to teach industrial robot programming; Universal Robotics is using Isaac Sim to develop a team and train terminal operators in the cloud.

      Epilogue

      As a robot simulation and synthetic data generation (SDG) tool, Isaac Sim, the NVIDIA Omniverse application, can accelerate the development, testing, training and deployment of intelligent robots. The release of the new version of NVIDIA Isaac Sim2022.2 will further empower developers, researchers and end users.

      Note: The article reprinted "Semiconductor Industry Observation"          http://www.semiinsights.com/s/ai/38/46815.shtml