Skip to content

[AI] Speed up AI inference in neural restore module#20802

Merged
TurboGit merged 1 commit intodarktable-org:masterfrom
andriiryzhkov:nr_opt
Apr 13, 2026
Merged

[AI] Speed up AI inference in neural restore module#20802
TurboGit merged 1 commit intodarktable-org:masterfrom
andriiryzhkov:nr_opt

Conversation

@andriiryzhkov
Copy link
Copy Markdown
Contributor

Summary

  • Specialize dynamic ONNX model sessions to a concrete tile size at load time using AddFreeDimensionOverrideByName, so JIT-compiling execution providers (MIGraphX, CoreML, TensorRT) compile the kernel once per session instead of recompiling on every inference call
  • Cache the working tile size in darktablerc (keyed by model/scale/provider) so subsequent runs skip OOM retry entirely
  • Persist AMD GPU caches (MIOpen kernel DB + MIGraphX compiled programs) and Intel OpenVINO compiled blobs to the user cache directory, eliminating multi-minute recompilation on every launch
  • Add spatial_dims field to model config.json so each model declares its symbolic dimension names

@TurboGit TurboGit added this to the 5.6 milestone Apr 13, 2026
@TurboGit TurboGit added feature: enhancement current features to improve priority: medium core features are degraded in a way that is still mostly usable, software stutters scope: performance doing everything the same but faster labels Apr 13, 2026
Copy link
Copy Markdown
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me, thanks!

@TurboGit TurboGit merged commit 7abae9d into darktable-org:master Apr 13, 2026
5 checks passed
@andriiryzhkov andriiryzhkov deleted the nr_opt branch April 13, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: enhancement current features to improve priority: medium core features are degraded in a way that is still mostly usable, software stutters scope: performance doing everything the same but faster

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants