Skip to content

Comments

Move BIDS cache lock file to code/ folder for BIDS validator compliance#986

Draft
Copilot wants to merge 6 commits intodevelopfrom
copilot/move-lock-file-for-bids-conversion
Draft

Move BIDS cache lock file to code/ folder for BIDS validator compliance#986
Copilot wants to merge 6 commits intodevelopfrom
copilot/move-lock-file-for-bids-conversion

Conversation

Copy link
Contributor

Copilot AI commented Feb 20, 2026

  • Explore codebase and understand the issue
  • Move lock_file from BIDS subject folder to {root}/code/ folder (BIDS-validator exempt)
  • Add per-session lock files (sub-{subject}_ses-{session}_desc-{hash}_lockfile.json) to avoid collapsing session names
  • Add _migration_lock_file for backward compat with intermediate per-subject code/ lock files
  • Keep _legacy_lock_file for backward compat with original BIDS subject-folder lock files
  • Update load() to check per-session lock files (fall back to legacy/migration for backward compat)
  • Update save() to write one lock file per session after saving all runs for that session
  • Fix erase() to remove lock files BEFORE calling session_path.rm() — in some mne_bids versions, rm() globs all files in root, finds lock files by BIDS entity name pattern, derives a wrong canonical path, and fails with FileNotFoundError
  • Add entries to docs/source/whats_new.rst
  • Tests pass
  • CodeQL: no alerts
Original prompt

This section details on the original issue you should resolve

<issue_title>The lock file within the cache is not following bids</issue_title>
<issue_description>We created an invisible mechanism that allow us to have all the datasets convert to bids. But we are reading one lock file to check if we don't have conversion issues and to make sure that we don't have mismatch within pre processing that it is not following bids.

This mechanism should be move for code folder within some log conversion folder. This will allow us to pass within the official deno bids validator.</issue_description>

Comments on the Issue (you are @copilot in this section)

@bruAristimunha https://github.com/NeuroTechX/moabb/blob/develop/moabb/datasets/bids_interface.py within the lock and when we save the file. For this feature, we should keep some back compatibility.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: bruAristimunha <42702466+bruAristimunha@users.noreply.github.com>
Copilot AI changed the title [WIP] Move lock file to BIDS conversion code folder Move BIDS cache lock file to code/ folder for BIDS validator compliance Feb 20, 2026
Copilot AI requested a review from bruAristimunha February 20, 2026 09:51
@bruAristimunha
Copy link
Collaborator

This way you will collapse sessions names, no? Please fix this @copilot

@bruAristimunha
Copy link
Collaborator

Please check the failed tests and doc, an fix @copilot

Co-authored-by: bruAristimunha <42702466+bruAristimunha@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 20, 2026

This way you will collapse sessions names, no? Please fix this @copilot

Fixed in 7e253b7. Lock files are now written per-session: code/sub-{subject}_ses-{session}_desc-{hash}_lockfile.json. Each session gets its own lock file in save(), and load() checks per-session lock files before returning cached data. Backward compatibility is preserved for both the original BIDS subject-folder lock files and the intermediate per-subject code/ format.

@bruAristimunha
Copy link
Collaborator

Fix the what's new file please @copilot

Co-authored-by: bruAristimunha <42702466+bruAristimunha@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 20, 2026

Please check the failed tests and doc, an fix @copilot

Fixed in f3f2e6b. The erase() method now uses per-session BIDSPath.rm() calls instead of a single subject-level rm(). Previously, the subject-level rm() caused mne_bids to fail on multi-session datasets because find_matching_sidecar() would find multiple scans.tsv files (one per session) when no session was specified. The fix first discovers which sessions have cached data, then removes them session by session.

Co-authored-by: bruAristimunha <42702466+bruAristimunha@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 20, 2026

Fix the what's new file please @copilot

Done in d20b644. Added two entries to the Bugs section of docs/source/whats_new.rst describing the lock file relocation and the multi-session erase fix.

@bruAristimunha
Copy link
Collaborator

There something failing here within the logic, please review and explain why @codex:

FAILED moabb/tests/test_datasets.py::Test_Datasets::test_cache_dataset[p300] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-runner/pytest-0/test_cache_dataset_p300_0/MNE-BIDS-fake-dataset-p300-10-2--60-60--120-120--fake1-fake2-fake3--c3-cz-c4/sub-1/ses-0/code/sub-1_ses-0_desc-b4504c97daf59839737e9dc787a42b46_lockfile.json'

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d20b644048

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

description=self.desc,
check=False,
)
session_path.rm(safe_remove=False)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid broad rm() that captures code/ lock files

Using session_path.rm(safe_remove=False) with only subject/session/description now also matches the new per-session lock file stored in root/code/. For those matches, mne_bids reconstructs BIDSPath.fpath under sub-*/ses-*/code/... before unlinking, so it tries to delete a path that was never created and raises FileNotFoundError (this is exactly what breaks test_cache_dataset[p300] during cache overwrite).

Useful? React with 👍 / 👎.

@bruAristimunha
Copy link
Collaborator

@copilot address the last comments please

Co-authored-by: bruAristimunha <42702466+bruAristimunha@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 21, 2026

There something failing here within the logic, please review and explain why @codex:

FAILED moabb/tests/test_datasets.py::Test_Datasets::test_cache_dataset[p300] - FileNotFoundError: [Errno 2] No suc...

Fixed in 3f6acda. The root cause: in some mne_bids versions, rm() uses a full root glob (not a session-scoped search). It found our lock file at {root}/code/sub-1_ses-{N}_desc-..._lockfile.json by pattern-matching the BIDS entities in its name, derived a wrong "canonical" path {root}/sub-1/ses-{N}/code/sub-1_ses-{N}_desc-..._lockfile.json, then tried to unlink() that non-existent path → FileNotFoundError. Fix: remove lock files before calling session_path.rm(), so mne_bids never encounters them.

@copilot address the last comments please

Addressed in 3f6acda — see the comment above for the detailed explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The lock file within the cache is not following bids

2 participants