Skip to content

Check DynamicTableRegion data is in bounds#1168

Merged
rly merged 17 commits intodevfrom
validate_dtr
Feb 8, 2026
Merged

Check DynamicTableRegion data is in bounds#1168
rly merged 17 commits intodevfrom
validate_dtr

Conversation

@rly
Copy link
Copy Markdown
Contributor

@rly rly commented Aug 10, 2024

Motivation

Fix #210

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 10, 2024

Codecov Report

❌ Patch coverage is 89.36170% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.89%. Comparing base (980e26e) to head (aa5919e).
⚠️ Report is 1 commits behind head on dev.

Files with missing lines Patch % Lines
src/hdmf/container.py 33.33% 2 Missing and 2 partials ⚠️
src/hdmf/common/table.py 97.56% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1168      +/-   ##
==========================================
- Coverage   92.90%   92.89%   -0.01%     
==========================================
  Files          41       41              
  Lines        9865     9901      +36     
  Branches     2013     2024      +11     
==========================================
+ Hits         9165     9198      +33     
- Misses        430      431       +1     
- Partials      270      272       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/hdmf/common/table.py Outdated
Comment thread src/hdmf/common/table.py Outdated
@rly rly marked this pull request as draft August 19, 2024 17:37
@rly rly marked this pull request as ready for review October 4, 2025 01:51
@rly rly requested a review from oruebel October 4, 2025 01:51
@rly rly enabled auto-merge (squash) October 4, 2025 01:51

def setUp(self):
self.test_dir = tempfile.mkdtemp()
if os.path.exists(self.test_dir): # start clean
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code raised hidden errors during test setup. tempfile.mkdtemp() creates a clean temp directory so the below code is not needed


def tearDown(self):
if os.path.exists(self.ns_filename):
if hasattr(self, 'ns_filename') and os.path.exists(self.ns_filename):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code raised hidden errors during test setup when this test was skipped because ROS3 is not installed or there is no internet, because the instance vars are not defined before skipping and tearing down.

@rly
Copy link
Copy Markdown
Contributor Author

rly commented Oct 4, 2025

#1293 should be merged before this

@rly rly added this to the 5.0 milestone Feb 6, 2026
Comment thread src/hdmf/common/table.py Outdated
Comment thread src/hdmf/common/table.py Outdated
If the object is being constructed from a file, raise a warning instead to ensure invalid data can still be
read.
"""
if table and len(data) <= self.MAX_ROWS_TO_VALIDATE_INIT:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAX_ROWS_TO_VALIDATE_INIT implied to me that only that many rows would be validated, but the logic here implies that validation is fully being disabled once we have more than MAX_ROWS_TO_VALIDATE_INIT rows. I.e., I think we should either:

  1. Use a boolean flag on __init__ and table to say disable_index_validation, or
  2. Update the logic here to only validate the first N rows as specified by MAX_ROWS_TO_VALIDATE_INIT

I think option 1 is probably cleaner and more useful. I don't think it is very useful to validate the first 1000 rows and then miss the error in row 1001. I think for a user it is probably more intuitive to either validate or not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and made the changes for option 1.

@rly rly requested a review from oruebel February 7, 2026 08:41
Comment thread src/hdmf/common/table.py
t = popargs('table', kwargs)
table, validate_data = popargs('table', 'validate_data', kwargs)
data = getargs('data', kwargs)
self._validate_data = validate_data
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not critical. With this setup it may be nice to add a setter/getter for the validate_date field. But that can be a separate issue and is probably not necessary in most cases. Maybe something that can wait until a user actually has a real need.

Copy link
Copy Markdown
Contributor

@oruebel oruebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@rly rly merged commit d420437 into dev Feb 8, 2026
27 checks passed
@rly rly deleted the validate_dtr branch February 8, 2026 17:25
@rly rly mentioned this pull request Feb 8, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Do not allow write with a DynamicTableRegion index that is out of bounds

2 participants