Converting EulerDiscreteScheduler (ODE) to DDPMScheduler (SDE)

[Devininthelab]

Hi, I’m trying to figure out how to convert an ODE-based scheduler to an SDE-based one.

1. Is there a proper way to map an ODE scheduler like EulerDiscreteScheduler to an SDE scheduler such as DDPMScheduler? I’m not sure how to translate the update rule while keeping the behavior consistent.

2. My model was trained using the EulerDiscreteScheduler timesteps, but I now need an SDE formulation so I can compute the log-probability p(x_{t-1}|x_t). How can I derive or adapt an SDE schedule that’s compatible with a model trained on the Euler (ODE) schedule?

Any guidance or examples would be greatly appreciated. Thanks!

[mdenitti]

Great question about scheduler conversion! This is a common challenge when adapting pre-trained models. Here's a comprehensive guide:

Understanding the Differences

ODE-based schedulers (like EulerDiscreteScheduler):

• Use deterministic ODE formulations
• No stochasticity in the sampling process
• Typically faster inference

SDE-based schedulers (like DDPMScheduler):

• Incorporate stochasticity in the diffusion process
• Include noise injection during sampling
• Can produce more diverse samples

Option 1: Direct Mathematical Conversion

For EulerDiscreteScheduler → DDPMScheduler, you need to:

1. Extract the timestep schedule from EulerDiscreteScheduler:

from diffusers import EulerDiscreteScheduler, DDPMScheduler

euler_scheduler = EulerDiscreteScheduler.from_pretrained(
    "model_id", subfolder="scheduler"
)

# Get timesteps
timesteps = euler_scheduler.timesteps
alphas = euler_scheduler.alphas_cumprod

2. Create DDPM scheduler with equivalent variance schedule:

ddpm_scheduler = DDPMScheduler(
    num_train_timesteps=euler_scheduler.config.num_train_timesteps,
    variance_type="learned_range",  # or "fixed_small", "fixed_large"
    timestep_spacing="leading",
)

Option 2: Hybrid Approach (Recommended)

Use the Euler scheduler's timestep schedule but with stochasticity:

from diffusers import EulerAncestralDiscreteScheduler

# EulerAncestralDiscreteScheduler is a middle ground:
# - Uses ODE framework
# - Adds stochasticity via ancestral sampling
# - Compatible with ODE-trained models

hybrid_scheduler = EulerAncestralDiscreteScheduler.from_config(
    euler_scheduler.config
)

Option 3: Model Adaptation

If you absolutely need DDPM formulation:

# Fine-tune briefly with SDE scheduler
model = UNet2DConditionModel.from_pretrained("model_id")
scheduler = DDPMScheduler.from_pretrained("model_id", subfolder="scheduler")

# Use DDPM with a few training steps to adapt
# This helps bridge the gap between ODE and SDE formulations

Practical Implementation

from diffusers import DiffusionPipeline, EulerAncestralDiscreteScheduler

pipeline = DiffusionPipeline.from_pretrained(
    "model_id",
    torch_dtype=torch.float16
)

# Replace with compatible SDE scheduler
pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config(
    pipeline.scheduler.config
)

# Generate with stochasticity
image = pipeline(
    prompt="your prompt",
    guidance_scale=7.5,
    generator=torch.Generator(device="cuda").manual_seed(42)
).images[0]

Key Considerations

1. Loss in quality: Direct ODE→SDE conversion may degrade quality since the model was optimized for ODE dynamics

2. Variance schedule: DDPM's fixed variance schedule may not match your ODE scheduler's implicit trajectory

3. Better alternative: Use EulerAncestralDiscreteScheduler which gives you stochasticity while respecting the ODE-trained model weights

4. Fine-tuning option: If quality is critical, consider brief fine-tuning with the SDE scheduler

Recommended Solution

For most use cases, EulerAncestralDiscreteScheduler is the best choice because:

• It's compatible with ODE-trained models
• It adds stochasticity without format conversion
• It preserves model behavior while enabling diversity
• No quality degradation from scheduler conversion

Hope this helps resolve your scheduler conversion question!

[Devininthelab]

Hi, thanks for your previous advice. I've attempted to implement your suggestions with stabilityai/stable-video-diffusion-img2vid-xt (using its default scheduler), but I'm unable to sample proper videos. It seems the issue stems from the timestep conversion between the model's expected EDM timesteps format and the DDPM scheduler. But I'm not sure how the math works out here.

FYI, here are the timestep sequences I am observing during my experiments:
Original EDM Timesteps (Model Default):

tensor([ 1.6378,  1.5755,  1.5110,  1.4440,  1.3743,  1.3018,  1.2260,  1.1469,
         1.0640,  0.9771,  0.8855,  0.7889,  0.6867,  0.5781,  0.4623,  0.3383,
         0.2048,  0.0604, -0.0971, -0.2702, -0.4622, -0.6780, -0.9242, -1.2108,
        -1.5537], device='cuda:0')

Your suggested option 1 (Switch to DDPM): (all sampled videos look like noise)

tensor([960, 920, 880, 840, 800, 760, 720, 680, 640, 600, 560, 520, 480, 440,
        400, 360, 320, 280, 240, 200, 160, 120,  80,  40,   0],
       device='cuda:0')

Your suggested option 2 (Alternative Switch to EulerAncestralDiscreteScheduler): (the video now can have some structure but still noisy)

tensor([961., 921., 881., 841., 801., 761., 721., 681., 641., 601., 561., 521.,
        481., 441., 401., 361., 321., 281., 241., 201., 161., 121.,  81.,  41.,
          1.], device='cuda:0')

(Note: I am skipping Option 3 as I currently lack resources for fine-tuning).

Question:

• Do you have any further suggestions for conversion of EDM to DDPM, specifically, for pretrained model like this stabilityai/stable-video-diffusion-img2vid-xt model?
• I also want to calculate p(x_{t-1} | x_t). It seems like the only way to calculate that without solving the SDE integral is using DDPM scheduler. So EulerAncestralDiscreteScheduler would not be useful in my case right? As it only allows me to calculate the marginal p(x_t)?