USD Composition Architecture for Isaac Sim

Purpose

Author sim-ready USD with layered payloads (base, instances, materials, physics, robot) following NVIDIA composition conventions.

Prerequisites

• Built Isaac Sim ($ISAAC_SIM_DIR or _build/linux-x86_64/release).
• NVIDIA GPU with a current driver (nvidia-smi).
• Shell env contract from isaac-sim-orchestrator: $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR.

Limitations

• Targets Isaac Sim 6 / Kit 110 unless a section states otherwise.
• Does not replace official NVIDIA documentation for unsupported edge cases.

Troubleshooting

Error / symptom	Cause	Solution
Extension or import not found	Wrong $ISAAC_SIM_DIR or stale build	Point env vars at _build/linux-x86_64/release or rebuild
Black or empty frames	Missing lights or non-RTX render mode	Add dome/key light; confirm RTX / PathTracing settings
Hang on stage load or first render	MDL compile or oversized stage	Follow isolation steps in isaac-sim-troubleshooting

When to use

• Build new robot or environment assets.
• Restructure an existing asset with the Asset Transformer.
• Optimize RL training startup time and VRAM.
• Diagnose physics edits in a USDA that aren't taking effect.
• Debug joint limits, mass, or solver parameters.
• Create variants of an existing asset (configs, materials).

Core Concept: One Binary Crate, Many USDA Layers

Isaac Sim's recommended asset structure splits a robot into one binary geometry crate plus a set of ASCII layers, composed by an interface.usda:

{robot}/
    interface.usda                         <- Final composed asset (entry point)
    payloads/
        base.usda                          <- Simulation-ready hierarchy + xforms
        geometries.usdc                    <- Mesh data ONLY (binary crate)
        instances.usda                     <- Mesh + material + collider assembly
        materials.usda                     <- Material defs (MDL bindings)
        Textures/                          <- Texture assets
        robot.usda                         <- Isaac robot schema + metadata
        Physics/
            physics.usda                   <- Neutral USD/Newton physics
            physx.usda                     <- PhysX-only tuning (sublayers physics.usda)
            mujoco.usda                    <- MuJoCo-only tuning (sublayers physics.usda)

USD payload arcs enable lazy loading — a payload is only loaded when explicitly requested. This is the key to headless RL optimization.

File Format Decision Guide

The rule is simple: binary crate (.usdc) for raw mesh data, USDA for everything else. The Asset Transformer's GeometriesRoutingRule enforces this split automatically.

Layer	Format	Why
geometries.usdc	.usdc (binary crate)	Mesh topology, points, indices — high-volume numeric data, never edited by hand
base.usda	.usda	Hierarchy and transforms — diffable, hand-editable
instances.usda	.usda	References meshes + applies materials + collision approximation choice
materials.usda	.usda	Material prims, MDL shader bindings — readable look-dev
physics.usda / physx.usda / mujoco.usda	.usda	Joint limits, masses, solver params — frequent tuning
robot.usda	.usda	Isaac robot schema metadata and relationships
interface.usda	.usda	Composition arcs (references, payloads, variants) — the entry point
Texture assets	original (PNG, JPG, EXR) + .mdl	Stored under Textures/
Archive/portable	.usdz	Single-file distribution (iOS AR)

Rationale: Mesh arrays are large and never hand-edited, so binary crate wins on size and load time. Everything else is small, frequently inspected, and benefits from being diffable in version control and editable by both humans and agents.

Producing This Structure: Asset Transformer

Use the Asset Transformer (Isaac Sim Structure profile) to convert an imported URDF/MJCF asset into the layout above. The relevant rules:

• GeometriesRoutingRule — extracts mesh prims to geometries.usdc (binary), creates instanceable references in instances.usda. Set save_base_as_usda: true to keep base ASCII.
• MaterialsRoutingRule — deduplicates materials into materials.usda, copies textures to Textures/.
• SchemaRoutingRule — splits physics, physx, mujoco, and robot schemas into their respective USDA layers.
• InterfaceConnectionRule — generates interface.usda with the composition arcs.

Refer to the Asset Transformer Rules Reference for the full pipeline.

Physics USDA Schema

physics.usda — Joint and Mass Definitions

#usda 1.0

def PhysicsRevoluteJoint "FL_hip_joint" {
    uniform token physics:axis = "X"
    float physics:lowerLimit = -46.0
    float physics:upperLimit = 46.0
    rel physics:body0 = </Robot/trunk>
    rel physics:body1 = </Robot/FL_hip>
}

def RigidBodyAPI "trunk" {
    float physics:mass = 4.713
    point3f physics:centerOfMass = (0.012, 0.002, -0.002)
    float3 physics:diagonalInertia = (0.0120, 0.0220, 0.0270)
}

physx.usda — PhysX-Only Tuning

#usda 1.0

def PhysxJointAPI "FL_hip_joint" {
    float physxJoint:maxJointVelocity = 20.0
    float physxJoint:jointFriction = 0.05
}

def PhysxRigidBodyAPI "trunk" {
    bool physxRigidBody:enableGyroscopicForces = true
    float physxRigidBody:maxDepenetrationVelocity = 10.0
    int physxRigidBody:solverPositionIterationCount = 32
    int physxRigidBody:solverVelocityIterationCount = 1
}

physx.usda typically sublayers physics.usda so PhysX-only opinions stack on top of the neutral physics definition.

Headless RL Optimization: Skip Appearance

The biggest RL training startup optimization: don't load appearance payloads.

When Isaac Lab loads a robot for RL:

1. Loads: interface.usda + base + physics layers (joints, masses, collision shapes).
2. Skips: materials.usda and Textures/ (irrelevant for physics sim).

ArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"{ASSETS_ROOT}/Robots/MyRobot/interface.usda",
        activate_contact_sensors=True,
        # Do NOT load appearance payloads for RL
        visual_material=None,
    )
)

Composition Arc Precedence

USD applies opinions in this order (last wins for most properties):

Sublayers < Reference < Payload < VariantSet < Direct opinions

If your physics USDA changes "don't take effect", check that the override opinion is in a higher-precedence layer.

Common Debugging

from pxr import Usd
# Find which layer is setting a specific property
attr = prim.GetAttribute("physics:mass")
for spec in attr.GetPropertyStack(Usd.TimeCode.Default()):
    print(f"  Layer: {spec.layer.GetDisplayName()} = {spec.default}")