[AI] Fix color shift in AI denoise on wide-gamut working profiles#20813
[AI] Fix color shift in AI denoise on wide-gamut working profiles#20813andriiryzhkov wants to merge 2 commits intodarktable-org:masterfrom
Conversation
src/libs/neural_restore.c
Outdated
| // gains a new value, the compiler will warn and force this list to be | ||
| // updated; DT_COLORSPACE_NONE and DT_COLORSPACE_FILE require additional | ||
| // context and are handled by the caller | ||
| static gboolean _profile_type_is_wide_gamut(dt_colorspaces_color_profile_type_t t) |
There was a problem hiding this comment.
Please move this to common/colorspaces.h/c and renamed:
dt_colorspaces_profile_is_wide_gamut(const dt_colorspaces_color_profile_type_t type);
There was a problem hiding this comment.
Done. Moved to common/colorspaces.h/c as dt_colorspaces_profile_is_wide_gamut()
src/libs/neural_restore.c
Outdated
| // outside sRGB gamut will be clipped by the AI model which operates | ||
| // in sRGB internally); for "image settings" (NONE), resolves to the | ||
| // image's actual working profile when imgid is valid | ||
| static gboolean _output_profile_is_wide_gamut(dt_imgid_t imgid) |
There was a problem hiding this comment.
Maybe this could be in common/image.h/c and renamed:
dt_image_has_wide_gammut_output_profile(const dt_imgid_t imgid)
There was a problem hiding this comment.
Done, with a small twist. The function read CONF_ICC_TYPE directly, which is neural_restore's config key - moving that to common/image.c would pull a module-specific config concept into shared code. I parameterized icc_type so the helper stays pure:
gboolean dt_image_has_wide_gamut_output_profile(const dt_imgid_t imgid,
const dt_colorspaces_color_profile_type_t icc_type);Handles all three UI cases (NONE → image's work profile, FILE → conservative TRUE, specific type → dt_colorspaces_profile_is_wide_gamut()). Callers read their own config and pass the resolved type - reusable for src/libs/export.c and any other module with the same export-style profile dropdown.
TurboGit
added
bugfix
|
I get specs using the PR that I do not get in git master in the denoised output. EDIT: in the example attached it is like green brown blobs at 100%. I suppose this is expected with the new algorithm, and the dark brown/green background is technically out of sRGB (Rec.709) gamut, but a lot of ordinary dark and bright color hues are. You can go outside sRGB gamut by lowering exposure as well. Could this not be an issue? The picture used is not extreme in any case, it is quite muted and when it comes to color. I am using this raw file Canon CR3: https://www.dpreview.com/sample-galleries/3976759896/canon-eos-r5-mark-ii-sample-gallery/5840436413 [dpreview sample gallery] Test: Import CR3 raw file into darktable. Run neural restore -> denoise at 50% This PR Without PR (git master) Gamut check using sRGB (Web safe) as soft proof profile |
What issue? |
The fact that many ordinary looking, somewhat underexposed dark pictures might have a large fraction of out of sRGB pixels. These areas will not be denoised. This might come across as unexpected, intuitively. The example picture I used does not come across as out of sRGB gamut, when it comes to the forest background, but it is. Besides that, I have tested on more pictures and I find that the PR handles out of sRGB gamut well. I can not detect any color shifts. Logically the handling in this PR is the best one can do. I do not have a better answer at this stage. What can be done to the case I am pointing out, of somewhat dark muted color areas that are actually out of sRGB and therefore will not be denosed? A tradeoff, or choice, or parameter, between color fidelity and denoising of the entire frame? |
|
I think this decisions to be made per image. I added notification about wide-gamut in preview. I think the only thing we can do to reduce out of sRGB pixels is to denoise in the sRGB. But that would be extreme case. |
|
I'm not sure if this issue is also related to the gamut handling, but I'm seeing some interesting colors going on in noisy areas, in this case a noisy sky image with stars behind clouds. Original image area: 
After denoising: 
Another area. Original: 
Denoised: 
Notice also the higher contrast. The darker shadow areas almost dropped to black. |
What "small floating point representations" mean? Denoise model is full precision F32. |
Sorry, this was just a desperate attempt to find an easy explanation for this issue 😅 |
|
The algorithm in this PR leaves a lot of noise in out of sRGB gamut color areas, because it is designed to do that. I got good results with many of my raw files using the git master algorithm, even if it has all the color fidelity issues listed in the problem statement of this PR. I could manage the color issues. Can we select algorithms in the module UI or similar between this PR and current GIT master? New algo in this PR Git master: |
4 participants
Problem
When AI denoise was applied to images in a wide-gamut working profile (Rec.2020, ProPhoto, etc.), the output had visible hue shifts – most noticeable as pink backgrounds shifting to teal/green. The denoise model was clearly working, but the colors came out wrong.
Root cause: the denoise model is trained on sRGB-primaried image data, but the pipeline was feeding it the working-profile RGB numbers directly with only a gamma-curve conversion. A pixel that means "saturated pink" in Rec.2020 primaries is interpreted as a different color when treated as sRGB. The model denoises that wrong color, and we convert the result back to Rec.2020 – producing a shifted hue.
A secondary issue: pixels outside the sRGB gamut (negative channel values, or values > 1.0 – common in wide-gamut workflows) were being clamped by the model to [0, 1], losing wide-gamut headroom and out-of-gamut color information entirely.
Fix
Two complementary changes in
dt_restore_run_patch():Primary conversion: when a working profile is set, convert pixels from working-profile linear RGB to sRGB linear via a 3×3 matrix before applying the sRGB gamma. After inference, invert the gamma and apply the inverse matrix to bring the result back to the working profile. This eliminates the hue shift for in-gamut pixels.
Per-pixel pass-through: during input conversion, record which pixels fall inside the sRGB [0, 1] range (cached as a 1-byte-per-pixel mask). On output, in-gamut pixels use the model's denoised result; out-of-gamut pixels pass through unchanged from the original working-profile input. Wide-gamut colors are preserved exactly – they're just not denoised.
The matrices are composed once per image at context setup using darktable's existing primaries/whitepoint helpers, so per-pixel cost is one matrix multiply on input plus one (conditional) on output.
Additional UI changes
wide-gamut preserved, not denoisedwide-gamut clipped(upscale has no pixel correspondence so pass-through doesn't apply)Notes & trade-offs
On the test images I tried, no significant color shift remains, though slight differences are still possible since the model touches every in-gamut pixel.
On images with a large wide-gamut region, noise in those areas may pass through unchanged – they're preserved exactly, but not denoised. That's the explicit trade-off of pass-through.
There are no publicly available denoise models that handle wide-gamut input natively, so this is the best we can do with an sRGB-trained model in the loop.