Skip to content

ENH: SIMPL Backwards Compatibility Test Redesign#1588

Open
imikejackson wants to merge 1 commit intoBlueQuartzSoftware:developfrom
imikejackson:topic/import_simpl_json_refactor
Open

ENH: SIMPL Backwards Compatibility Test Redesign#1588
imikejackson wants to merge 1 commit intoBlueQuartzSoftware:developfrom
imikejackson:topic/import_simpl_json_refactor

Conversation

@imikejackson
Copy link
Copy Markdown
Contributor

The original backwards compatibility test was a single 1,100-line monolithic test file (BackwardsCompatibilityTest.cpp) that validated SIMPL 6.5/6.6 pipeline conversion for all
filters at once. It relied on two large hand-maintained data structures:

  • k_ParamMap (~500 lines) — a global map of parameter type UUIDs to expected exemplar values, with custom std::any comparator lambdas for each parameter type
  • k_KeyIgnoreMap (~180 lines) — a manual list of NX-only parameter keys per filter that the test had to skip

The core maintenance problem: whenever a developer added a new parameter to any filter, they had to also update k_KeyIgnoreMap with that parameter's key, or the centralized test
would fail. This was unintuitive, error-prone, and completely disconnected from the filter being changed. The ignore list grew with every filter enhancement and provided no
signal about what it was actually protecting.
⏺ Summary: SIMPL Backwards Compatibility Test Redesign

Problem

The original backwards compatibility test was a single 1,100-line monolithic test file (BackwardsCompatibilityTest.cpp) that validated SIMPL 6.5/6.6 pipeline conversion for all
filters at once. It relied on two large hand-maintained data structures:

  • k_ParamMap (~500 lines) — a global map of parameter type UUIDs to expected exemplar values, with custom std::any comparator lambdas for each parameter type
  • k_KeyIgnoreMap (~180 lines) — a manual list of NX-only parameter keys per filter that the test had to skip

The core maintenance problem: whenever a developer added a new parameter to any filter, they had to also update k_KeyIgnoreMap with that parameter's key, or the centralized test
would fail. This was unintuitive, error-prone, and completely disconnected from the filter being changed. The ignore list grew with every filter enhancement and provided no
signal about what it was actually protecting.

Additionally, the test:

  • Called FromSIMPLJson directly, bypassing the real pipeline loading path (UUID lookup, filter resolution, type validation)
  • Required downloading a large test data archive (backwards_compatibility_test.tar.gz) just to test JSON key mapping
  • Did not test the SIMPL 6.4 code path (pipelines without UUIDs)
  • Made it difficult to identify which filter's conversion was broken when the test failed

Solution

Replace the centralized test with per-filter backwards compatibility tests embedded in each filter's existing test file. Each test:

  1. Loads a small SIMPL pipeline JSON fixture (stored in git, not in the test data archive) through Pipeline::FromSIMPLFile — the same code path the application uses
  2. Verifies the correct filter was resolved via UUID mapping
  3. Checks that conversion completed without errors
  4. Asserts the converted parameter values match expected values
  5. Tests both SIMPL 6.5 (with UUID) and 6.4 (Filter_Name only) pipeline formats via DYNAMIC_SECTION

Why this is better

Zero-maintenance ignore list: Tests only CHECK parameters that FromSIMPLJson actually converts. NX-only parameters are never mentioned, so adding new parameters to a filter
requires no test changes whatsoever.

Full pipeline loading path coverage: The original test bypassed UUID lookup, filter resolution, and the type validation in PipelineFilter::FromSIMPLJson. The new tests exercise
the exact code path a user hits when opening a legacy pipeline in DREAM3DNX. If someone breaks the UUID mapping in a plugin's LegacyUUIDMapping.hpp, the test catches it.

SIMPL 6.4 coverage: Each filter now tests the Filter_Name → k_LegacySimplFilterUuidMap fallback path, which was completely untested before.

Clear failure isolation: When a conversion breaks, the failing test names the exact filter — not "6.6 Mega Pipeline Conversion failed" with a wall of error strings to parse.

No test data download required: JSON fixtures are small text files stored in git under test/simpl_conversion/6_5/ and 6_4/. They diff cleanly in PRs, don't need tar/hash/upload
cycles, and don't require the TestFileSentinel download machinery.

Reviewable in context: When a filter's FromSIMPLJson changes, the corresponding test and fixture are right next to the filter code — reviewable in the same PR diff.

What was removed

  • test/ConversionTest/BackwardsCompatibilityTest.cpp (1,100 lines) — can be deleted
  • k_KeyIgnoreMap (~180 lines of filter UUID → ignored key mappings) — eliminated
  • k_ParamMap + InitializeMap() + 10 CreateMapInput overloads (~500 lines) — eliminated
  • The backwards_compatibility_test.tar.gz dependency — no longer needed for conversion testing

What was added

  • 406 JSON fixture files (203 filters × 2 versions), averaging ~20 lines each
  • 199 TEST_CASE blocks added to existing filter test files, averaging ~30 lines each
  • All 199 tests pass with zero regressions to existing tests

@imikejackson imikejackson requested a review from nyoungbq April 13, 2026 00:24
@imikejackson imikejackson force-pushed the topic/import_simpl_json_refactor branch from 2d92dc6 to 0d0d2d5 Compare April 13, 2026 19:30
The original backwards compatibility test was a single 1,100-line monolithic test file (BackwardsCompatibilityTest.cpp) that validated SIMPL 6.5/6.6 pipeline conversion for all
  filters at once. It relied on two large hand-maintained data structures:

  - k_ParamMap (~500 lines) — a global map of parameter type UUIDs to expected exemplar values, with custom std::any comparator lambdas for each parameter type
  - k_KeyIgnoreMap (~180 lines) — a manual list of NX-only parameter keys per filter that the test had to skip

  The core maintenance problem: whenever a developer added a new parameter to any filter, they had to also update k_KeyIgnoreMap with that parameter's key, or the centralized test
  would fail. This was unintuitive, error-prone, and completely disconnected from the filter being changed. The ignore list grew with every filter enhancement and provided no
  signal about what it was actually protecting.
@imikejackson imikejackson force-pushed the topic/import_simpl_json_refactor branch from 0d0d2d5 to 148ebe4 Compare April 13, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant