Describe the Bug
Hi, first of all huge thanks for your work on this great plugin.
I've noticed [...].actual.png files don't have FRSOURCE_CPVRD_V set, which causes problems when they're used to update the baseline images.
In our workflow we mostly use Cypress on the command line (cypress run). We "accept" a new baseline image by overwriting the old baseline with the new [...].actual.png file (see "Additional context" for why we do this).
This mostly works fine, but this .actual.png file doesn't have the FRSOURCE_CPVRD_V field set. This means that when we run the same tests later, FRSOURCE_CPVRD_V gets set by the plugin, and therefore there is a diff according to git, but not visually. This can show up later in the diff for an unrelated change, which is difficult for a reviewer to understand, and will happen repeatedly as baselines are updated over time.
Steps to reproduce
Developer A:
- Create a test and run it, generating an initial baseline
- Modify the component under test such that the test will fail
- Run the test, which fails, producing
.actual.png and .diff.png files
- Rename
.actual.png to overwrite the baseline
- Commit, push etc.
Developer B:
- Run the test, which passes, but which automatically updates the baseline to include
FRSOURCE_CPVRD_V
- Now any commits/PR etc. this developer makes may include a confusing rewrite of that image
- Any subsequent baseline updates (performed like in step 4) will unset
FRSOURCE_CPVRD_V and cause this cycle to happen again!
Expected behavior
I would expect to see no diff after step 6. The implication of step 8 is that seemingly random results are introduced into diffs, including PRs. I think setting FRSOURCE_CPVRD_V when generating the .actual.png files would solve the issue.
Additional context
I'm aware that updateImages does not have this problem, but it doesn't work for our GHA workflow: we run the tests and would like indicate which are failing (not possible with updateImages) and then automatically open a PR with new baselines. It is these PRs which contain many irrelevant diffs.
Despite our (potentially unusual) use case, I imagine all users who rename .actual.png to update their baselines are running into similar problems.
Let me know if I'm missing anything, I could look into opening a PR otherwise.
Package version
3.3.10
Device
Any
OS and version
Any
Browser and version
Any
Node version
Any
I can work on it by myself
Code of Conduct
Describe the Bug
Hi, first of all huge thanks for your work on this great plugin.
I've noticed
[...].actual.pngfiles don't haveFRSOURCE_CPVRD_Vset, which causes problems when they're used to update the baseline images.In our workflow we mostly use Cypress on the command line (
cypress run). We "accept" a new baseline image by overwriting the old baseline with the new[...].actual.pngfile (see "Additional context" for why we do this).This mostly works fine, but this
.actual.pngfile doesn't have theFRSOURCE_CPVRD_Vfield set. This means that when we run the same tests later,FRSOURCE_CPVRD_Vgets set by the plugin, and therefore there is a diff according togit, but not visually. This can show up later in the diff for an unrelated change, which is difficult for a reviewer to understand, and will happen repeatedly as baselines are updated over time.Steps to reproduce
Developer A:
.actual.pngand.diff.pngfiles.actual.pngto overwrite the baselineDeveloper B:
FRSOURCE_CPVRD_VFRSOURCE_CPVRD_Vand cause this cycle to happen again!Expected behavior
I would expect to see no diff after step 6. The implication of step 8 is that seemingly random results are introduced into diffs, including PRs. I think setting
FRSOURCE_CPVRD_Vwhen generating the.actual.pngfiles would solve the issue.Additional context
I'm aware that
updateImagesdoes not have this problem, but it doesn't work for our GHA workflow: we run the tests and would like indicate which are failing (not possible withupdateImages) and then automatically open a PR with new baselines. It is these PRs which contain many irrelevant diffs.Despite our (potentially unusual) use case, I imagine all users who rename
.actual.pngto update their baselines are running into similar problems.Let me know if I'm missing anything, I could look into opening a PR otherwise.
Package version
3.3.10
Device
Any
OS and version
Any
Browser and version
Any
Node version
Any
I can work on it by myself
Code of Conduct