NVIDIA Isaac

What is Isaac?

Isaac is the NVIDIA robotics platform that provides hardware and software tools for developing AI-powered robots. It includes:

1. Isaac Sim
2. Isaac Lab

It’s particularly useful for autonomous robots, industrial automation, and warehouse robotics.

Installing Isaac Sim (4.5 or later)

NVIDIA Isaac Sim™ is a reference application built on NVIDIA Omniverse that enables developers to develop, simulate, and test AI-driven robots in physically-based virtual environments.

Disable IOMMU

1. Verify that IOMMU is disabled if the two commands below do not print any outputs:

$ sudo dmesg | grep -e DMAR -e IOMMU

# Output:
[    0.004320] ACPI: DMAR 0x000000005CA0C000 000088 (v01 HPQOEM 8C77     00000002 HP   00040000)
[    0.004358] ACPI: Reserving DMAR table memory at [mem 0x5ca0c000-0x5ca0c087]
[    0.094400] DMAR: Host address width 39
[    0.094401] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.094406] DMAR: dmar0: reg_base_addr fed90000 ver 4:0 cap 1c0000c40660462 ecap 29a00f0505e
[    0.094408] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.094411] DMAR: dmar1: reg_base_addr fed91000 ver 5:0 cap d2008c40660462 ecap f050da
[    0.094412] DMAR: RMRR base: 0x00000064000000 end: 0x000000683fffff
[    0.094414] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.094415] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.094415] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.095950] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.322139] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[    0.849244] DMAR: No ATSR found
[    0.849244] DMAR: No SATC found
[    0.849245] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.849246] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.849247] DMAR: IOMMU feature nwfs inconsistent
[    0.849248] DMAR: IOMMU feature dit inconsistent
[    0.849248] DMAR: IOMMU feature sc_support inconsistent
[    0.849249] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.849250] DMAR: dmar0: Using Queued invalidation
[    0.849252] DMAR: dmar1: Using Queued invalidation
[    0.853680] DMAR: Intel(R) Virtualization Technology for Directed I/O

$ ls /sys/kernel/iommu_groups/

# Output:
0  1  10  11  12  13  14  15  16  17  18  19  2  20  21  22  23  24  25  26  3  4  5  6  7  8  9

2. Disable IOMMU

# AMD
$ sudo bash -c 'echo GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=off" >> /etc/default/grub'

# Intel
$ sudo bash -c 'echo GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=off" >> /etc/default/grub'

$ sudo update-grub
$ sudo reboot

Create Conda Environment

$ conda create -n isaac python=3.10
$ conda activate isaac

$ pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121

Before installing Isaac Sim, ensure the latest pip version is installed. To update pip, run

$ pip install --upgrade pip

Then, install the Isaac Sim packages.

$ pip install 'isaacsim[all,extscache]==4.5.0' --extra-index-url https://pypi.nvidia.com

Verifying the Isaac Sim installation

• Make sure that your virtual environment is activated (if applicable)
• Check that the simulator runs as expected:

# note: you can pass the argument "--help" to see all arguments possible.
$ isaacsim

It’s also possible to run with a specific experience file, run:

# experience files can be absolute path, or relative path searched in isaacsim/apps or omni/apps
$ isaacsim isaacsim.exp.full.kit

Cloud Deployment

Prerequisites

$ sudo apt update
$ sudo apt install pkg-config
$ sudo apt install ubuntu-desktop
$ sudo apt install gdm3
$ sudo systemctl restart gdm3

DCV Server Installation

$ wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY
$ gpg --import NICE-GPG-KEY
$ wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.1/Servers/nice-dcv-2023.1-17701-ubuntu2204-x86_64.tgz
$ tar -xvzf nice-dcv-2023.1-17701-ubuntu2204-x86_64.tgz && cd nice-dcv-2023.1-17701-ubuntu2204-x86_64
$ sudo apt install ./nice-dcv-server_2023.1.17701-1_amd64.ubuntu2204.deb
$ sudo apt install ./nice-dcv-web-viewer_2023.1.16388-1_amd64.ubuntu2204.deb
$ sudo usermod -aG video dcv
$ sudo apt install ./nice-xdcv_2023.1.565-1_amd64.ubuntu2204.deb
$ sudo apt install ./nice-dcv-simple-external-authenticator_2023.1.228-1_amd64.ubuntu2204.deb

Start DCV Server

$ sudo systemctl start dcvserver
$ sudo systemctl enable dcvserver
$ sudo systemctl status dcvserver
$ sudo dcv create-session isaac_session --owner ubuntu

Check activate sessions:

$ dcv list-sessions

Installing Isaac Lab

Install Git

$ sudo apt install git

Cloning Isaac Lab

# using SSH
$ git clone git@github.com:isaac-sim/IsaacLab.git ~/IsaacLab
# or using https
$ git clone https://github.com/isaac-sim/IsaacLab.git ~/IsaacLab

Install dependencies using apt (on Ubuntu):

$ sudo apt install cmake build-essential

Run the install command that iterates over all the extensions in source directory and installs them using pip (with --editable flag):

$ cd ~/IsaacLab
$ conda activate issac # make sure the issac virtual environment is activated
$ ./isaaclab.sh --install # or "./isaaclab.sh -i"

$ ./isaaclab.sh --install rl_games  # or "./isaaclab.sh -i rl_games"

Verifying the Isaac Lab installation

# Option 1: Using the isaaclab.sh executable
# note: this works for both the bundled python and the virtual environment
$ ./isaaclab.sh -p scripts/tutorials/00_sim/create_empty.py

# Option 2: Using python in your virtual environment
$ python scripts/tutorials/00_sim/create_empty.py

The above command should launch the simulator and display a window with a black viewport as shown below. You can exit the script by pressing Ctrl+C on your terminal. On Windows machines, please terminate the process from Command Prompt using Ctrl+Break or Ctrl+fn+B.

Train a Bot

You can now use Isaac Lab to train a robot through Reinforcement Learning! The quickest way to use Isaac Lab is through the predefined workflows using one of our Batteries-included robot tasks. Execute the following command to quickly train an ant to walk! We recommend adding --headless for faster training.

$ ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless

… Or a robot dog!

$ ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Velocity-Rough-Anymal-C-v0 --headless

Remove --headless if you want to see it in action.

Isaac Lab provides the tools you’ll need to create your own Tasks and Workflows for whatever your project needs may be. Take a look at our How-to Guides guides like Adding your own learning Library or Wrapping Environments for details.

Asset Caching

Assets used in Isaac Lab are hosted on AWS S3 buckets on the cloud. Asset loading time can depend on your network connection and geographical location. In some cases, it is possible that asset loading times can be long when assets are pulled from the AWS servers.

If you run into cases where assets take a few minutes to load for each run, we recommend enabling asset caching following the below steps.

First, launch the Isaac Sim app:

$ ./isaaclab.sh -s

On the top right of the Isaac Sim app, there will be an icon labelled CACHE:. There may be a message indicating HUB NOT DETECTED or NEW VERSION DETECTED.

Click on the message, which will enable Hub for asset caching. Once enabled, Hub will run automatically each time an Isaac Lab or Isaac Sim instance is run.

Note that for the first run, assets will still need to be pulled from the cloud, which could lead to longer loading times. However, subsequent runs that use the same assets will be able to use the cached files from Hub. Hub will provide better control for caching of assets used in Isaac Lab.

URDF

Install API Script

macOS

$ cd <path to fusion2urdf-ros2>
$ cp -r ./URDF_Exporter_Ros2 "$HOME/Library/Application Support/Autodesk/Autodesk Fusion 360/API/Scripts/"

Windows

C:\Users\[username]\AppData\Roaming\Autodesk\Autodesk Fusion 360\API

After placing the plugin, re-open Fusion 360 again. Navigate to the Utilities on top. Check the ADD-INS, you shoul see the URDF_Exporter_Ros2 under My Scripts.