DreamForge DiT checkpoint

[habibian]

Thanks for the great work.

Wonder if there is any plan to release the DreamForge DiT checkpoint as reported in the paper?

[SimonGrande1]

Thank you for your interest in our work. We will release the code and checkpoint for DreamForge-DiT no later than the end of May.

[SimonGrande1]

Hi @habibian,
We’ve now released the code and model weights for DreamForge-DiT. Please refer to the video branch for details.

[habibian]

Hi @SimonGrande1 Amazing! Thanks a lot

[yzli-0910]

Hi @SimonGrande1 ,
thank you for the great project.

I wonder if we can follow DriveArena/WorldDreamer/DreamForge/README.md
to setup DreamForge-DiT as well?

Is the required torch version same as that of DreamForge?

[SimonGrande1]

Hi @yzli-0910

Thank you for your interest! The environment setup for DreamForge-DiT differs from DreamForge. Please refer to the dedicated setup guide for installation, training, and inference steps. The required PyTorch version is specified in this guide.

[yzli-0910]

Hi @SimonGrande1
Got it. Thank you very much for the fast response!

[yzli-0910]

Hi @SimonGrande1
I am trying to run WorldDreamer api with Dreamforge

cd WorldDreamer/DreamForge-DiT && python tools/run_fastapi_dit_t.py

Where a file named "canbus_infos_trainval.pkl" is required but not in my nuscenes folder.
Am I missing anything that this file is generated somewhere?

Looking forward to your reply.

[SimonGrande1]

Hi @yzli-0910 You can access the required files in the original DreamForge folder at: https://github.com/PJLab-ADG/DriveArena/tree/video/WorldDreamer/DreamForge/data

[yzli-0910]

Oh I see, thank you very much!

[yzli-0910]

hi @habibian, did you try the DreamForge-DiT?

When I am trying, I find some unexpected noises in the generated results (#64). I was wondering if this is just for my case.