merge chroma radiance (tests, diff) by Ednaordinary · Pull Request #5 · Ednaordinary/diffusers

Ednaordinary · 2025-11-30T07:44:03Z

No description provided.

Ednaordinary · 2025-12-13T13:53:48Z

@lodestone-rock hey, I helped implement the original version of chroma into diffusers and I'm working on getting radiance in! what would you recommend for default params (both for 0.4 and x0)? thanks for making chroma!

sync

Ednaordinary · 2025-12-17T00:37:03Z

contd. huggingface#12850

lodestone-rock · 2025-12-17T01:48:40Z

the default params should be no different than chroma
atleast that's how i trained it
it's just now it's in pixel space

lodestone-rock · 2025-12-17T01:49:58Z

i'm not quite sure how to translate comfyui default params to huggingface, but in comfy there's this default workflow for chroma radiance and that's pretty good

Ednaordinary · 2025-12-17T02:03:51Z

hmm.. I've adapted best I've could (example shown in diffusers pr), but something in what I believe is likely the scheduler or the nerf model is wrong

lodestone-rock · 2025-12-17T02:14:30Z

try using simple euler instead for sampling instead of heun?

Ednaordinary · 2025-12-17T05:39:31Z

looks like at some point while debugging I removed the attention mask and didn't add it back.. getting closer, but still not there, just yet. It also seems both Euler and Heun are converging on the same image, which is interesting

* fix torchao quantizer for new torchao versions Summary: `torchao==0.16.0` (not yet released) has some bc-breaking changes, this PR fixes the diffusers repo with those changes. Specifics on the changes: 1. `UInt4Tensor` is removed: pytorch/ao#3536 2. old float8 tensors v1 are removed: pytorch/ao#3510 In this PR: 1. move the logger variable up (not sure why it was in the middle of the file before) to get better error messages 2. gate the old torchao objects by torchao version Test Plan: import diffusers objects with new versions of torchao works: ```bash > python -c "import torchao; print(torchao.__version__); from diffusers import StableDiffusionPipeline" 0.16.0.dev20251229+cu129 ``` Reviewers: Subscribers: Tasks: Tags: * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add a test to check modular index consistency * check for compulsory keys.

…sses in ModularPipeline (huggingface#13193) up Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-161-123.ec2.internal>

* updates in the pypi publication workflow. * change to 3.10

…ingface#12704) enable cpu offload test in torchao without compilation.

…g custom code (huggingface#13197) * update * update

…#13186) * update * update

* update * update * update * update * update * update

Improve docstring scheduling ipndm

update

* up * up * update * remove test --------- Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-161-123.ec2.internal> Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-160-103.ec2.internal>

@DN6

…3204) * add * update warn * add a test * updaqte * update_component with custom model * add more tests * Apply suggestion from @DN6 Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com> * up --------- Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-161-123.ec2.internal> Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

…gface#12196) * feat: implement requirements validation for custom blocks. * up * unify. * up * add tests * Apply suggestions from code review Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com> * reviewer feedback. * [docs] validation for custom blocks (huggingface#13156) validation * move to tmp_path fixture. * propagate to conditional and loopsequential blocks. * up * remove collected tests --------- Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com> Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

…ogVideoX LoRA training (huggingface#13207) * Distribute VAE video encoding across processes in CogVideoX LoRA training Signed-off-by: jiqing-feng <jiqing.feng@intel.com> * Apply style fixes --------- Signed-off-by: jiqing-feng <jiqing.feng@intel.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Implement synchronous onload for offloaded parameters Add fallback synchronous onload for conditionally-executed modules. * add test for new code path about group-offloading * Update tests/hooks/test_group_offloading.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * use unittest.skipIf and update the comment --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* [1/N] add helios * fix test * make fix-copies * change script path * fix cus script * update docs * fix documented check * update links for docs and examples * change default config * small refactor * add test * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * remove register_buffer for _scale_cache * fix non-cuda devices error * remove "handle the case when timestep is 2D" * refactor HeliosMultiTermMemoryPatch and process_input_hidden_states * Update src/diffusers/pipelines/helios/pipeline_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/pipelines/helios/pipeline_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * fix calculate_shift * Update src/diffusers/pipelines/helios/pipeline_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * rewritten `einops` in pure `torch` * fix: pass patch_size to apply_schedule_shift instead of hardcoding * remove the logics of 'vae_decode_type' * move some validation into check_inputs() * rename helios scheduler & merge all into one step() * add some details to doc * move dmd step() logics from pipeline to scheduler * change to Python 3.9+ style type * fix NoneType error * refactor DMD scheduler's set_timestep * change rope related vars name * fix stage2 sample * fix dmd sample * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * remove redundant & refactor norm_out * Update src/diffusers/pipelines/helios/pipeline_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * change "is_keep_x0" to "keep_first_frame" * use a more intuitive name * refactor dynamic_time_shifting * remove use_dynamic_shifting args * remove usage of UniPCMultistepScheduler * separate stage2 sample to HeliosPyramidPipeline * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * Update src/diffusers/models/transformers/transformer_helios.py Co-authored-by: YiYi Xu <yixu310@gmail.com> * fix transformer * use a more intuitive name * update example script * fix requirements * remove redudant attention mask * fix * optimize pipelines * make style . * update TYPE_CHECKING * change to use torch.split Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * derive memory patch sizes from patch_size multiples * remove some hardcoding * move some checks into check_inputs * refactor sample_block_noise * optimize encoding chunks logits for v2v * use num_history_latent_frames = sum(history_sizes) * Update src/diffusers/pipelines/helios/pipeline_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * remove redudant optimized_scale * Update src/diffusers/pipelines/helios/pipeline_helios_pyramid.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * use more descriptive name * optimize history_latents * remove not used "num_inference_steps" * removed redudant "pyramid_num_stages" * add "is_cfg_zero_star" and "is_distilled" to HeliosPyramidPipeline * remove redudant * change example scripts name * change example scripts name * correct docs * update example * update docs * Update tests/models/transformers/test_models_transformer_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update tests/models/transformers/test_models_transformer_helios.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * separate HeliosDMDScheduler * fix numerical stability issue: * Update src/diffusers/schedulers/scheduling_helios_dmd.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/schedulers/scheduling_helios_dmd.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/schedulers/scheduling_helios_dmd.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/schedulers/scheduling_helios_dmd.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * Update src/diffusers/schedulers/scheduling_helios_dmd.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * remove redudant * small refactor * remove use_interpolate_prompt logits * simplified model test * fallbackt to BaseModelTesterConfig * remove _maybe_expand_t2v_lora_for_i2v * fix HeliosLoraLoaderMixin * update docs * use randn_tensor for test * fix doc typo * optimize code * mark torch.compile xfail * change paper name * Make get_dummy_inputs deterministic using self.generator * Set less strict threshold for test_save_load_float16 test for Helios pipeline * make style and make quality * Preparation for merging * add torch.Generator * Fix HeliosPipelineOutput doc path * Fix Helios related (optimize docs & remove redudant) (huggingface#13210) * fix docs * remove redudant * remove redudant * fix group offload * Removed fixes for group offload --------- Co-authored-by: yuanshenghai <yuanshenghai@bytedance.com> Co-authored-by: Shenghai Yuan <140951558+SHYuanBest@users.noreply.github.com> Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: SHYuanBest <shyuan-cs@hotmail.com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

…ace#13212) fix

…ace#13209) fix zimage lora conversion to support for more lora.

adding lora to z-image controlnet pipelines

* LTX2 condition pipeline initial commit * Fix pipeline import error * Implement LTX-2-style general image conditioning * Blend denoising output and clean latents in sample space instead of velocity space * make style and make quality * make fix-copies * Rename LTX2VideoCondition image to frames * Update LTX2ConditionPipeline example * Remove support for image and video in __call__ * Put latent_idx_from_index logic inline * Improve comment on using the conditioning mask in denoising loop * Apply suggestions from code review Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com> * make fix-copies * Migrate to Python 3.9+ style type annotations without explicit typing imports * Forward kwargs from preprocess/postprocess_video to preprocess/postprocess resp. * Center crop LTX-2 conditions following original code * Duplicate video and audio position ids if using CFG * make style and make quality * Remove unused index_type arg to preprocess_conditions * Add # Copied from for _normalize_latents * Fix _normalize_latents # Copied from statement * Add LTX-2 condition pipeline docs * Remove TODOs * Support only unpacked latents (5D for video, 4D for audio) * Remove # Copied from for prepare_audio_latents --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>

* Fix Helios paper link in documentation Updated the link to the Helios paper for accuracy. * Fix reference link in HeliosTransformer3DModel documentation Updated the reference link for the Helios Transformer model paper. * Update Helios research paper link in documentation * Update Helios research paper link in documentation

…#13161) * change to updated repo and version. * fix version and force updated kernels. * propagate version.

* feat: implement three RAE encoders(dinov2, siglip2, mae) * feat: finish first version of autoencoder_rae * fix formatting * make fix-copies * initial doc * fix latent_mean / latent_var init types to accept config-friendly inputs * use mean and std convention * cleanup * add rae to diffusers script * use imports * use attention * remove unneeded class * example traiing script * input and ground truth sizes have to be the same * fix argument * move loss to training script * cleanup * simplify mixins * fix training script * fix entrypoint for instantiating the AutoencoderRAE * added encoder_image_size config * undo last change * fixes from pretrained weights * cleanups * address reviews * fix train script to use pretrained * fix conversion script review * latebt normalization buffers are now always registered with no-op defaults * Update examples/research_projects/autoencoder_rae/README.md Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * Update src/diffusers/models/autoencoders/autoencoder_rae.py Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> * use image url * Encoder is frozen * fix slow test * remove config * use ModelTesterMixin and AutoencoderTesterMixin * make quality * strip final layernorm when converting * _strip_final_layernorm_affine for training script * fix test * add dispatch forward and update conversion script * update training script * error out as soon as possible and add comments * Update src/diffusers/models/autoencoders/autoencoder_rae.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * use buffer * inline * Update src/diffusers/models/autoencoders/autoencoder_rae.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * remove optional * _noising takes a generator * Update src/diffusers/models/autoencoders/autoencoder_rae.py Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com> * fix api * rename * remove unittest * use randn_tensor * fix device map on multigpu * check if the key is missing in the original state dict and only then add to the allow_missing set * remove initialize_weights --------- Co-authored-by: wangyuqi <wangyuqi@MBP-FJDQNJTWYN-0208.local> Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com> Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>

…gface#13214) Convert tensors to float in optimized_scale function

…gface#13219)

* post release 0.37.0 * style

* fix Helios Context Parallelism * refacotr * make style and quality

optimize helios docs

Ednaordinary added 12 commits November 30, 2025 00:24

init (pipeline, transformer, nerf, etc)

ee2740a

undo regressions

e0707bd

fixes

07b368e

random tweaks

d3bf917

make saveable

1a20f53

update

ed746c5

fix img_ids

0a19b7e

remove some prints

df3ab44

updates

1a04172

get closer to fixing attn mask

a829909

fix attention

54f3bbf

make inference run

655b8e6

Ednaordinary and others added 5 commits December 13, 2025 07:01

cleanup

0191cfd

add x0, change final layer some

72748ec

Merge pull request #6 from Ednaordinary/main

ae023a3

sync

make style make quality

5a4312a

removed copied from (modified)

414c63e

fix attention mask

e3ee921

Ednaordinary and others added 5 commits December 16, 2025 23:23

updates

3814a6a

Merge branch 'main' into chroma-radiance

5e47460

add lru cache

17388e8

sync autocast logic

0770561

sayakpaul and others added 30 commits February 28, 2026 08:47

[tests] consistency tests for modular index (huggingface#13192)

9b97932

* add a test to check modular index consistency * check for compulsory keys.

[modular] fallback to default_blocks_name when loading base block cla…

3918824

…sses in ModularPipeline (huggingface#13193) up Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-161-123.ec2.internal>

[chore] updates in the pypi publication workflow. (huggingface#12805)

fcdd759

* updates in the pypi publication workflow. * change to 3.10

[tests] enable cpu offload test in torchao without compilation. (hugg…

5b54496

…ingface#12704) enable cpu offload test in torchao without compilation.

remove db utils from benchmarking (huggingface#13199)

3d90855

[AutoModel] Fix bug with subfolders and local model paths when loadin…

e7fe4ce

…g custom code (huggingface#13197) * update * update

[AutoModel] Allow registering auto_map to model config (huggingface…

3fd14f1

…#13186) * update * update

[Modular] Save Modular Pipeline weights to Hub (huggingface#13168)

84ff061

* update * update * update * update * update * update

docs: improve docstring scheduling_ipndm.py (huggingface#13198)

c2fdd2d

Improve docstring scheduling ipndm

Clean up accidental files (huggingface#13202)

47e8faf

update

[modular]Update model card to include workflow (huggingface#13195)

bbbcdd8

* up * up * update * remove test --------- Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-161-123.ec2.internal> Co-authored-by: yiyi@huggingface.co <yiyi@ip-26-0-160-103.ec2.internal>

[Z-Image] Fix more do_classifier_free_guidance thresholds (huggingf…

5570f81

…ace#13212) fix

[lora] fix zimage lora conversion to support for more lora. (huggingf…

3902145

…ace#13209) fix zimage lora conversion to support for more lora.

adding lora support to z-image controlnet pipelines (huggingface#13200)

20364fe

adding lora to z-image controlnet pipelines

[attention backends] change to updated repo and version. (huggingface…

29b9109

…#13161) * change to updated repo and version. * fix version and force updated kernels. * propagate version.

Convert tensors to float in Helios's optimized_scale function (huggin…

46bd005

…gface#13214) Convert tensors to float in optimized_scale function

Fix wrapped transformer config access in Flux2 Klein training (huggin…

e747fe4

…gface#13219)

post release 0.37.0 (huggingface#13215)

e1b5db5

* post release 0.37.0 * style

Fix Helios Context Parallelism (huggingface#13223)

9254417

* fix Helios Context Parallelism * refacotr * make style and quality

Optimize Helios docs (huggingface#13222)

bd7a7a0

optimize helios docs

Merge branch 'main' into chroma-radiance

72fc14e

remove lru

91d34bf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge chroma radiance (tests, diff)#5

merge chroma radiance (tests, diff)#5
Ednaordinary wants to merge 243 commits intomainfrom
chroma-radiance

Ednaordinary commented Nov 30, 2025

Uh oh!

Ednaordinary commented Dec 13, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

Ednaordinary commented Nov 30, 2025

Uh oh!

Ednaordinary commented Dec 13, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Ednaordinary commented Dec 17, 2025 •

edited

Loading