fix export overwrite if changed

[zisoft]

The function now checks against the exported file's modification time stamp instead of the export time stamp in the database. This makes a lot more sense especially when using the multi-preset export with different formats in multiple directories.

See #15404 (comment) and #19987 (comment)

[DaveInDev]

hi @zisoft , thanks for your help : I will check tomorrow on the next nightly build.

[zisoft]

@DaveInDev: This is only included in the nightly build after the PR is merged.

[DaveInDev]

@zisoft I'm not familiar with github. Do you mean that it won't appear in tomorrow's build ? Is there another way to test this change ?

[TurboGit]

Given the comment: #19987 (comment) I'll pass this as draft.

[zisoft]

@TurboGit @DaveInDev

Got it

The params for the format module were read during the running job.
Now the params are read at job creation and passed to _control_export_job_run

Steps to test:

• create two jpeg export presets, one with quality 10 and one with quality 90 and set their output to different directories or filenames
• start a multi-preset export with these two presets
• compare the different file sizes

For the overwrite if changed:

• create a preset with "on conflict": overwrite if changed
• start the export, the file should be created only once if no develop changes were made

[DaveInDev]

Thanks a lot @zisoft . As I'm not too familiar with github (I read "Changes can be cleanly merged"), would I be able to test this correction on tomorrow's nightly build ?

[zisoft]

@DaveInDev the changes need to be reviewed by @TurboGit first.
Once the PR is merged you can then test with the next nightly build afterwards.

[TurboGit]

Some style comments and possibly a memory leak.

[TurboGit]

@zisoft : Also, let me know if you feel this safe enough for 5.4.1. It seems to me, so I'm in favor of a merge in 5.4.1 but you'll have the final word.

[zisoft]

I made a lot of tests with different scenarios. Should be safe for 5.4.1

[zisoft]

Release notes:

Fixed wrong handling of overwrite if changed in export.

Fixed images exported with wrong settings when using multi-preset export.

[DaveInDev]

hi @zisoft , I saw this "[force-pushed]", but I don't know if your modifications are in the latest nightly build of january 03 ?

[zisoft]

@DaveInDev : No, as long as this PR is not merged the changes are not included in the nightly build.

[TurboGit]

Thanks!

[zisoft]

@DaveInDev : Now the PR is merged, you can test with the next nightly build tomorrow. Please report back if there are still any issues.

[DaveInDev]

Hi guys. I made a bunch of tests and I confirm that now everything seems to work the way it should.
Thanks a lot again for your help.

[DaveInDev]

Sorry to come back, but there is still a problem.

With this example :
create two jpeg export presets, one with [quality 90% / scale x1.0] and one with [quality 80% / scale x0.5] and set their output to different directories or filenames, and also option "overwrite when changed".

Sometimes, if both destination files are missing, it creates 2 files with quality 90% and scale x1...

But I do not manage to understand in which case it does this error... Only on some of my RAW files...

Note that both presets works fine if used individually with single export.

EDIT :
Somehow I managed to trigger the problem, maybe it could help you understand :

• If I have a directory/collection of raws that I did not touch for a week (probably using later 5.4.0).
• If I delete all the exported JPEG and I multi-export again, it first creates correct JPEG 90% and 80% files, applying both presets correctly.
• But if I delete all the exported JPEG again and I multi-export again, it then creates only 90% JPEG files, using the first preset quality and scale values for the second export...
• After that, I cannot make it work correctly on the same collection...

[zisoft]

@DaveInDev : Please open a new issue with a detailed description to reproduce. I will look.

BTW: Is this a real world scenario you really use?

[DaveInDev]

Are you sure ? It's exactly the same problem as before.
The fact is that it works perfectly the first time, but not the second time...
I do not really understand how to reproduce it from scratch.
It seems that it depends on the "age" of the raw.
It seems to work fine once on old raws treated with 5.4.0, but not on subsequent exports.
And it does not work even once on new raws that I treated recently with 5.5.0 .

BTW: Is this a real world scenario you really use?

I do not totally understand your question, but if you ask to know if I use this in my daily routine : it's yes :-)
I always export my images in 2 qualities, in separate directories with different suffixes :

• one high quality full size for my archives
• on low quality half size for web publication.

That's why this multi-export if very useful for me. ;-)

[zisoft]

@DaveInDev : I have now made countless tries with the steps you described above and cannot reproduce.
Everything is working as expected on my system.

It seems to work fine once on old raws treated with 5.4.0, but not on subsequent exports.
And it does not work even once on new raws that I treated recently with 5.5.0 .

???
not reprodicible at all

[DaveInDev]

Hi @zisoft ,
Not easy to reproduce : I do not even find out the logic behind the remaining problem, because now it works fine with brand new pictures I made yesterday and treated only with v5.5.0+56 .
Very mysterious...
Forget about it. If I find something more reproductible, I'll let you know.
Thanks again for your help.

Subsidiary question : besides XMP files connected to their RAW files, is there a database somewhere that keep track of other informations on RAW files ? Anotehr way to ask it : if I delete the XMP file, will Darktable "forget" everything about the associated RAW file ?

[DaveInDev]

Hi @zisoft ,
still cannot reproduce consistently the previous bug, even if it still appears by moments.

But another little bug remains, concerning the "export overwrite if changed" option : if the preset itself has changed, the export destination file should be overwritten. For the moment it is not the case. Example : JPEG quality changed or scale changed.

[zisoft]

This in by design: "If changed" means, the develop settings of the image have changed. A change of the preset parameters is not considered as an image change.

[DaveInDev]

Ok. I can understand, even if it would be natural to overwrite the image if I choose another JPG quality for example.
For the moment, if I change the JPG quality, I have to delete manually all the previous JPG files before exporting again.