how to enable 'vram_management_enabled' variable when loading model and lora?

[dongdk]

hi
how to enable 'vram_management_enabled' variable when loading model and lora?
i want to lora different lora for different images when inference.
thanks.

[Artiprocher]

@dongdk We have updated Loading LoRA in docs. See https://github.com/modelscope/DiffSynth-Studio/blob/main/docs/en/Pipeline_Usage/Model_Inference.md#loading-lora

[dongdk]

thanks，i try this way to solve it

    def _enable_lora_magic(self):
        if not self.pipeline.dit:
            return
        hotload = getattr(self.pipeline.dit, "vram_management_enabled", False)
       
        if not hotload:
            from diffsynth.core.vram.layers import enable_vram_management, AutoWrappedLinear, AutoWrappedModule
            from diffsynth.models.qwen_image_dit import RMSNorm
            dtype = next(iter(self.pipeline.dit.parameters())).dtype # torch.bfloat16
            vram_config = {"offload_dtype": dtype,
                           "offload_device": "cpu",
                           "onload_dtype": dtype,
                           "onload_device": "cpu",
                           "preparing_dtype": dtype,
                           "preparing_device": "cuda",
                           "computation_dtype": dtype,
                           "computation_device": "cuda",}
            enable_vram_management(self.pipeline.dit,
                                   module_map={torch.nn.Linear: AutoWrappedLinear,
                                               RMSNorm: AutoWrappedModule,},
                                   vram_config=vram_config,
                                   vram_limit=0,)

[dongdk]

thanks，i try this way to solve it

    def _enable_lora_magic(self):
        if not self.pipeline.dit:
            return
        hotload = getattr(self.pipeline.dit, "vram_management_enabled", False)
       
        if not hotload:
            from diffsynth.core.vram.layers import enable_vram_management, AutoWrappedLinear, AutoWrappedModule
            from diffsynth.models.qwen_image_dit import RMSNorm
            dtype = next(iter(self.pipeline.dit.parameters())).dtype # torch.bfloat16
            vram_config = {"offload_dtype": dtype,
                           "offload_device": "cpu",
                           "onload_dtype": dtype,
                           "onload_device": "cpu",
                           "preparing_dtype": dtype,
                           "preparing_device": "cuda",
                           "computation_dtype": dtype,
                           "computation_device": "cuda",}
            enable_vram_management(self.pipeline.dit,
                                   module_map={torch.nn.Linear: AutoWrappedLinear,
                                               RMSNorm: AutoWrappedModule,},
                                   vram_config=vram_config,
                                   vram_limit=0,)

but when dynamic loading and unloading lora using load_lora and clear_lora, and set vram_limit=36, in inference, the gpu memory management seems not work, the gpu memory is continue to increase and up to 51G.
is it normal?