Robotics

NVIDIA Isaac

What is Isaac?

Isaac is the NVIDIA robotics platform that provides hardware and software tools for developing AI-powered robots. It includes:

  1. Isaac Sim
  2. Isaac Lab

It’s particularly useful for autonomous robots, industrial automation, and warehouse robotics.

Installing Isaac Sim (4.5 or later)

NVIDIA Isaac Sim™ is a reference application built on NVIDIA Omniverse that enables developers to develop, simulate, and test AI-driven robots in physically-based virtual environments.

Note

Installing Isaac Sim with pip requires GLIBC 2.34+ version compatibility. To check the GLIBC version on your system, use command ldd --version.

Note

On Windows with CUDA 12, the GPU driver version 552.86 is required.

Also, on Windows, it may be necessary to enable long path support to avoid installation errors due to OS limitations.

Disable IOMMU

  1. Verify that IOMMU is disabled if the two commands below do not print any outputs:
$ sudo dmesg | grep -e DMAR -e IOMMU

# Output:
[    0.004320] ACPI: DMAR 0x000000005CA0C000 000088 (v01 HPQOEM 8C77     00000002 HP   00040000)
[    0.004358] ACPI: Reserving DMAR table memory at [mem 0x5ca0c000-0x5ca0c087]
[    0.094400] DMAR: Host address width 39
[    0.094401] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.094406] DMAR: dmar0: reg_base_addr fed90000 ver 4:0 cap 1c0000c40660462 ecap 29a00f0505e
[    0.094408] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.094411] DMAR: dmar1: reg_base_addr fed91000 ver 5:0 cap d2008c40660462 ecap f050da
[    0.094412] DMAR: RMRR base: 0x00000064000000 end: 0x000000683fffff
[    0.094414] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.094415] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.094415] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.095950] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.322139] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[    0.849244] DMAR: No ATSR found
[    0.849244] DMAR: No SATC found
[    0.849245] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.849246] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.849247] DMAR: IOMMU feature nwfs inconsistent
[    0.849248] DMAR: IOMMU feature dit inconsistent
[    0.849248] DMAR: IOMMU feature sc_support inconsistent
[    0.849249] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.849250] DMAR: dmar0: Using Queued invalidation
[    0.849252] DMAR: dmar1: Using Queued invalidation
[    0.853680] DMAR: Intel(R) Virtualization Technology for Directed I/O

$ ls /sys/kernel/iommu_groups/

# Output:
0  1  10  11  12  13  14  15  16  17  18  19  2  20  21  22  23  24  25  26  3  4  5  6  7  8  9
  1. Disable IOMMU
# AMD
$ sudo bash -c 'echo GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=off" >> /etc/default/grub'

# Intel
$ sudo bash -c 'echo GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=off" >> /etc/default/grub'

$ sudo update-grub
$ sudo reboot

Create Conda Environment

$ conda create -n isaac python=3.10
$ conda activate isaac

$ pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121

Before installing Isaac Sim, ensure the latest pip version is installed. To update pip, run

$ pip install --upgrade pip

Then, install the Isaac Sim packages.

$ pip install 'isaacsim[all,extscache]==4.5.0' --extra-index-url https://pypi.nvidia.com

Verifying the Isaac Sim installation

  • Make sure that your virtual environment is activated (if applicable)
  • Check that the simulator runs as expected:
# note: you can pass the argument "--help" to see all arguments possible.
$ isaacsim

It’s also possible to run with a specific experience file, run:

# experience files can be absolute path, or relative path searched in isaacsim/apps or omni/apps
$ isaacsim isaacsim.exp.full.kit

Note

When running Isaac Sim for the first time, all dependent extensions will be pulled from the registry. This process can take upwards of 10 minutes and is required on the first run of each experience file. Once the extensions are pulled, consecutive runs using the same experience file will use the cached extensions.

Note

The first run will prompt users to accept the NVIDIA Software License Agreement. To accept the EULA, reply Yes when prompted with the below message:

By installing or using Isaac Sim, I agree to the terms of NVIDIA SOFTWARE LICENSE AGREEMENT (EULA) in https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement

Do you accept the EULA? (Yes/No): Yes

Cloud Deployment

Prerequisites

$ sudo apt update
$ sudo apt install pkg-config
$ sudo apt install ubuntu-desktop
$ sudo apt install gdm3
$ sudo systemctl restart gdm3

DCV Server Installation

$ wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY
$ gpg --import NICE-GPG-KEY
$ wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.1/Servers/nice-dcv-2023.1-17701-ubuntu2204-x86_64.tgz
$ tar -xvzf nice-dcv-2023.1-17701-ubuntu2204-x86_64.tgz && cd nice-dcv-2023.1-17701-ubuntu2204-x86_64
$ sudo apt install ./nice-dcv-server_2023.1.17701-1_amd64.ubuntu2204.deb
$ sudo apt install ./nice-dcv-web-viewer_2023.1.16388-1_amd64.ubuntu2204.deb
$ sudo usermod -aG video dcv
$ sudo apt install ./nice-xdcv_2023.1.565-1_amd64.ubuntu2204.deb
$ sudo apt install ./nice-dcv-simple-external-authenticator_2023.1.228-1_amd64.ubuntu2204.deb

Start DCV Server

$ sudo systemctl start dcvserver
$ sudo systemctl enable dcvserver
$ sudo systemctl status dcvserver
$ sudo dcv create-session isaac_session --owner ubuntu

Check activate sessions:

$ dcv list-sessions
Previous
Isaac Lab
Next
MuJoCo