[Question] How can I apply domain randomization for light of the environment?

[kimbring2]

Question

Hello there,

I trained Jetbot through the camera image. I need to apply the domain randomization for the visual part of the environment, such as light density.

I could see the tutorial for that from the link below.

• https://isaac-sim.github.io/IsaacLab/main/source/tutorials/03_envs/create_direct_rl_env.html#domain-randomization

However, there are only functions for mass and joint offset.

What is the appropriate way to implement domain randomization for light?

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: 2.3.0
• Isaac Sim Version: 5.1

[RandomOakForest]

Thank you for posting this. For Isaac Lab 2.3.0 (on Isaac Sim 5.1), the recommended way is to use the existing EventTerm domain‑randomization mechanism but call your own light‑randomization function. This is supported and consistent with the current DirectRLEnv / EventManager design.¹²³⁴⁵

Here is a summary to consider that is still under review, though it may be of help.

---

1. Create lights in _setup_scene

In Isaac Lab 2.3.0, lights are typically added via sim_utils.*LightCfg inside _setup_scene, exactly like in the cartpole‑style examples.²³

Example (single dome light + per‑env sphere lights):

import isaaclab.sim as sim_utils

def _setup_scene(self):
    # Clone envs first (Isaac Lab 2.3.0 pattern)
    self.scene.clone_environments(copy_from_source=False)

    # Shared dome light
    dome_cfg = sim_utils.DomeLightCfg(
        intensity=2000.0,
        color=(0.75, 0.75, 0.75),
    )
    dome_cfg.func("/World/Lights/DomeLight", dome_cfg)

    # Example per‑env light (if you want per-env control)
    sphere_cfg = sim_utils.SphereLightCfg(
        intensity=500.0,
        radius=0.2,
        color=(1.0, 1.0, 1.0),
    )
    for env_id in range(self.num_envs):
        light_path = f"/World/envs/env_{env_id}/Light"
        sphere_cfg.func(light_path, sphere_cfg)

This usage of DomeLightCfg and SphereLightCfg and the cfg.func(path, cfg) API is documented and used in current Isaac Lab examples.³²

---

2. Implement a light randomizer compatible with Isaac Sim 5.1

Isaac Sim 5.x still exposes USD lights through UsdLux and prim utilities such as get_prim_at_path, so you can mutate intensity and color attributes at reset.⁶⁷

Example utility (put in e.g. my_project/randomization/light_dr.py):

import torch
from omni.isaac.core.utils.prims import get_prim_at_path
from pxr import UsdLux, Gf

def randomize_lights(env, env_ids, intensity_range=(500.0, 5000.0),
                     color_range=((0.5, 0.5, 0.5), (1.0, 1.0, 1.0))):
    """Randomize per‑env lights for Isaac Lab 2.3.0 / Isaac Sim 5.1.

    Args:
        env: DirectRLEnv instance (self in your task).
        env_ids: 1D tensor of env indices to randomize.
        intensity_range: (min, max) intensity.
        color_range: ((r_min, g_min, b_min), (r_max, g_max, b_max)).
    """
    device = env.device
    env_ids = env_ids.to(device)
    num = env_ids.shape[^0]

    # Sample intensities and colors in torch
    min_I, max_I = intensity_range
    intensities = torch.rand(num, device=device) * (max_I - min_I) + min_I

    min_c = torch.tensor(color_range[^0], device=device)
    max_c = torch.tensor(color_range[^1], device=device)
    colors = torch.rand(num, 3, device=device) * (max_c - min_c) + min_c

    # Apply to each env light
    for i, env_id in enumerate(env_ids.tolist()):
        prim_path = f"/World/envs/env_{env_id}/Light"
        prim = get_prim_at_path(prim_path)
        if not prim:
            continue
        light = UsdLux.Light(prim)
        light.CreateIntensityAttr().Set(float(intensities[i].item()))
        col = colors[i]
        light.CreateColorAttr().Set(
            Gf.Vec3f(float(col[^0].item()), float(col[^1].item()), float(col[^2].item()))
        )

• get_prim_at_path and UsdLux.Light are part of Isaac Sim 5.x / Omniverse core APIs.⁷⁶
• This function signature (env, env_ids, **params) matches how EventTerm.func is called for custom DR in current Isaac Lab.⁴¹

---

3. Register the randomizer via EventTerm for DirectRLEnv

Isaac Lab 2.3.0 still uses EventTerm configs with func, mode, and params, and the events field is attached to your env config.¹²⁴

Config definitions:

from isaaclab.utils import configclass
from isaaclab.envs.mdp import EventTerm

@configclass
class JetbotEventCfg:
    # Randomize lights at every episode reset
    randomize_env_lights = EventTerm(
        func="my_project.randomization.light_dr.randomize_lights",
        mode="reset",
        params={
            "intensity_range": (500.0, 4000.0),
            "color_range": ((0.4, 0.4, 0.4), (1.0, 1.0, 1.0)),
        },
    )

@configclass
class JetbotEnvCfg(DirectRLEnvCfg):
    # ... your existing env config fields (scene, robot, sim, obs, rewards) ...
    events: JetbotEventCfg = JetbotEventCfg()

• The use of @configclass, EventTerm, and the events attribute on the env config is exactly how the official tutorials wire domain randomization terms.²¹
• mode="reset" ensures the function is invoked in _reset_idx, which is the documented DR location for DirectRLEnv.⁴

If you want mid‑episode lighting changes:

randomize_env_lights = EventTerm(
    func="my_project.randomization.light_dr.randomize_lights",
    mode="interval",
    is_global_time=False,
    interval_range_s=(2.0, 5.0),
    params={ ... },
)

The interval semantics time_s = num_steps * (decimation * dt) are described in the Isaac Lab environment tutorials.¹²

---

4. Using Replicator in Isaac Sim 5.1 (optional alternative)

Isaac Sim 5.1’s Replicator still exposes light randomization snippets (dome light, distant light, etc.), and these can be triggered on reset if you want more complex visual DR.⁶⁷

Typical pattern:

import omni.replicator.core as rep

with rep.new_layer():
    dome_light = rep.create.light(
        light_type="Dome",
        intensity=rep.distribution.uniform(200.0, 2000.0),
        color=rep.distribution.uniform((0.4, 0.4, 0.4), (1.0, 1.0, 1.0)),
    )

You can wrap such Replicator calls in a function randomize_lights_with_replicator(env, env_ids, ...) and use it instead of the pure‑USD version above, still wired through EventTerm. Replicator’s light randomization API, including intensity and color distributions for dome and distant lights, is documented for Isaac Sim 5.x.⁷⁶

---

5. JetBot‑specific guidance

• For a JetBot trained purely from camera images, start with moderate intensity and color ranges so the simulated images remain realistic and close to your real lab lighting (e.g., intensity 0.5–2× nominal, slightly warm/cool colors).⁸⁹
• Combine this with other visual DR (textures, backgrounds) as suggested in Isaac Lab’s “Visual Domain Randomization” guidelines to get robust sim‑to‑real transfer.⁹⁸
• Keep your camera intrinsics and exposure fixed initially, then later consider randomizing exposure and slight camera pose shifts through additional EventTerms once light DR is stable.⁸⁹

Footnotes

1. https://isaac-sim.github.io/IsaacLab/main/source/tutorials/03_envs/create_manager_base_env.html ↩ ↩² ↩³ ↩⁴ ↩⁵

2. https://isaac-sim.github.io/IsaacLab/main/source/tutorials/03_envs/create_direct_rl_env.html ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

3. https://isaac-sim.github.io/IsaacLab/main/source/setup/walkthrough/api_env_design.html ↩ ↩² ↩³

4. https://isaac-sim.github.io/IsaacLab/main/source/setup/walkthrough/technical_env_design.html ↩ ↩² ↩³ ↩⁴

5. https://isaac-sim.github.io/IsaacLab/main/source/api/lab/isaaclab.envs.html ↩

6. https://docs.isaacsim.omniverse.nvidia.com/5.0.0/replicator_tutorials/tutorial_replicator_isaac_randomizers.html ↩ ↩² ↩³ ↩⁴

7. https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/randomizer_details.html ↩ ↩² ↩³ ↩⁴

8. https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/latest/transferring-robot-learning-policies-from-simulation-to-reality/03-bridging-the-gap-simulation-enhancement/01-visual-domain-randomization.html ↩ ↩² ↩³

9. https://isaac-sim.github.io/IsaacLab/main/source/policy_deployment/02_gear_assembly/gear_assembly_policy.html ↩ ↩² ↩³

[kimbring2]

@RandomOakForest Hello there,

Thank you for your detailed answer. I am going to apply it to my project.