Skip to content

Add Flat encoding test coverage across all packages#7678

Merged
Unisay merged 4 commits intomasterfrom
yura/flat-test-coverage
Mar 31, 2026
Merged

Add Flat encoding test coverage across all packages#7678
Unisay merged 4 commits intomasterfrom
yura/flat-test-coverage

Conversation

@Unisay
Copy link
Copy Markdown
Contributor

@Unisay Unisay commented Mar 18, 2026

Context

  • What: Add roundtrip and stable byte encoding tests for every Flat instance across all packages
  • Why: Pay down test coverage debt before attempting to fix the Generic Flat derivation bug (512+ constructor crash). Encoding stability must be verified by tests first.
  • Related: Preparatory work for fixing the Generic derivation bug discussed in Remove Generic-based Flat derivation machinery #7662

Approach

Added two categories of tests for Flat instances:

  1. Stable byte encoding tests (a.k.a. golden/characterization tests) pin the exact byte representation of on-chain-critical types (Version, DeBruijn, DefaultFun, DefaultUni, Binder variants, UPLC Program, PIR Recursivity/Strictness). These detect any accidental encoding changes that would break on-chain script compatibility.

  2. Roundtrip property tests verify unflat (flat x) == Right x for all Flat instances including library-internal types (monoid/semigroup wrappers, containers, etc.) where encoding stability is less critical but correctness still matters.

A standalone encoding generator executable (cabal run flat-encoding-generator) is included so anyone can reproduce the expected byte constants without GHCi sessions. It regenerates test fixtures from source so they don't remain opaque blobs of bits.

BLS12_381 types are deliberately skipped (their encode/decode implementations are error/fail).

Changes

Flat library tests (flat/test/Spec.hs)

  • Stable byte encoding tests for: Maybe, Either, NonEmpty, Complex, Ratio, Set, Tree, Map, Seq, Filler, PreAligned
  • Roundtrip QC tests for: Tree, Set, Identity, All, Any, Dual, Sum, Product, Min, Max, First, Last, Alt, Fixed, PreAligned, String

TPLC tests (plutus-core/test/Flat/Spec.hs — new module)

  • Stable byte encoding for: Version, Name, Kind, DeBruijn, NamedDeBruijn, Index, SrcSpan, SrcSpans, DefaultFun (AddInteger, SubtractInteger), SomeTypeIn DefaultUniInteger
  • Roundtrip tests for: SrcSpan, SrcSpans, NamedDeBruijn, Version, Name, Kind
  • Newtype wrapper delegation tests: TyName, TyDeBruijn, NamedTyDeBruijn, FakeNamedDeBruijn

UPLC tests (untyped-plutus-core/testlib/Flat/Spec.hs)

  • Binder stable encoding: zero-cost verification for Binder DeBruijn and Binder FakeNamedDeBruijn, delegation checks for Binder Name/TyName/NamedDeBruijn/NamedTyDeBruijn
  • Binder newtype roundtrip tests
  • Minimal UPLC Program roundtrip and stable byte test: (program 1.1.0 (con integer 0))
  • Wired test_flat into the UPLC test entry point (was previously defined but not imported)

PIR tests (plutus-ir/test/PlutusIR/Core/Tests.hs)

  • Stable byte encoding for Recursivity (NonRec/Rec) and Strictness (NonStrict/Strict)

Value tests (plutus-core/test/Value/Spec.hs)

  • K encodes identically to ByteString
  • Quantity encodes identically to Integer

Infrastructure

  • Encoding generator executable (plutus-core/tools/GenerateEncodings.hs)
  • Added FOURMOLU_DISABLE to flat/test/Spec.hs (CPP-heavy file that fourmolu cannot parse)

@Unisay Unisay self-assigned this Mar 18, 2026
@Unisay Unisay requested a review from zliu41 March 18, 2026 17:14
Comment thread plutus-core/flat/test/Spec.hs Outdated
-- | Stable byte encoding tests for flat library container/composite types.
-- Wrapper types (Identity, All, Any, Dual, etc.) only have roundtrip tests
-- since their encoding stability is not critical (they are never on-chain).
testNewEncodings = testGroup "stable byte encodings"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not introducing any new encodings, so this should just be called "testEncodings"

Copy link
Copy Markdown
Contributor Author

@Unisay Unisay Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to testEncodingStability. Tell me if you have another idea of a better name.

Comment thread plutus-core/plutus-core.cabal Outdated
-- | Reproducible encoding generator for Flat test coverage.
-- Run: cabal run flat-encoding-generator
-- Prints expected byte sequences for all Flat test values.
executable flat-encoding-generator
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this, and in what scenario would someone want to use this executable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replied by changing the code comment:

-- Golden file generator for Flat encoding stability tests.
--
-- The encoding stability tests compare encoded values against expected
-- byte sequences captured as test fixtures. This tool regenerates those
-- fixtures from source, so they don't remain opaque blobs of bits.
--
-- Example: if a bug in a Flat instance is found and the encoding must
-- change in a backward-incompatible way, the stability tests will fail.
-- Run this tool to see the new wire format, update the test fixtures,
-- and confirm the encoding change was intentional.
--
-- See: https://en.wikipedia.org/wiki/Characterization_test
-- Usage: cabal run flat-encoding-generator

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it not sufficient to use the golden test mechanism, where one can use --accept to update the golden files? In this executable you are including a specific set of test cases, which I suppose need to mirror the test cases in the stabilit tests; if we add a new test case, it also needs to be added to the executable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're dealing with binary files here, so existing golden testing mechanism IMO is not the best fit, i.e. there is no point in diffing binary files when showing an error to a user. OTOH, I see your point that with the --accept functionality its easier to re-generate individual failing tests.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you summarize how the current tests work - in particular, the comment above says

The encoding stability tests compare encoded values against expected byte sequences captured as test fixtures

Where are the expected byte sequences stored, if not in a golden file?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. The byte sequences were hardcoded inline in the test assertions:

encRaw (Nothing :: Maybe Bool) [0]
encRaw (Just True :: Maybe Bool) [192]

The generator was a separate executable that printed the same encodings to stdout for manual comparison. But you're right that --accept makes it redundant. I've reworked these into golden tests now. Each stability group produces a text file like:

Nothing :: Maybe Bool = [0]
Just True :: Maybe Bool = [192]
Right () :: Either Bool () = [128]
...

One source of truth, readable diffs, --accept for regeneration. Generator is removed.

Comment thread plutus-core/plutus-core.cabal Outdated
@Unisay Unisay force-pushed the yura/flat-test-coverage branch from b8964e3 to 6cdd889 Compare March 20, 2026 11:52
@Unisay Unisay requested a review from zliu41 March 20, 2026 11:52
@Unisay Unisay force-pushed the yura/flat-test-coverage branch 2 times, most recently from 0aad87c to d3cc240 Compare March 20, 2026 16:06
@Unisay Unisay closed this Mar 25, 2026
@Unisay Unisay deleted the yura/flat-test-coverage branch March 25, 2026 10:45
@Unisay Unisay restored the yura/flat-test-coverage branch March 25, 2026 10:46
@Unisay Unisay reopened this Mar 25, 2026
Unisay added 2 commits March 25, 2026 11:46
Pay down test coverage debt for Flat serialization instances.
This ensures encoding stability is verified by tests before
we attempt to fix the Generic derivation bug in a follow-up PR.

Adds roundtrip and stable byte encoding tests for:
- Flat library types (Maybe, Either, NonEmpty, Complex, Ratio,
  Set, Tree, Map, Seq, DList, Filler, PreAligned, and
  monoid/semigroup wrappers)
- TPLC types (Version, Name, Kind, DeBruijn, NamedDeBruijn,
  SrcSpan, SrcSpans, DefaultFun, DefaultUni, and all newtype
  wrappers including TyName, Unique, TyDeBruijn, etc.)
- PIR types (Recursivity, Strictness)
- UPLC types (Binder variants, FakeNamedDeBruijn, minimal
  Program encoding)
- Value types (K encodes as ByteString, Quantity as Integer)

Also adds a standalone encoding generator executable
(cabal run flat-encoding-generator) for reproducing expected
byte constants.
Rename testNewEncodings to testEncodingStability for clarity.
Fix plutus-core ^>=1.59 version bound to ^>=1.60. Remove
unnecessary -Wno-* flags and unused text dependency from
flat-encoding-generator. Improve executable comment with
golden testing context and Wikipedia reference.
@Unisay Unisay force-pushed the yura/flat-test-coverage branch from d3cc240 to 140f473 Compare March 25, 2026 10:47
Unisay added 2 commits March 30, 2026 19:01
The encoding stability tests now use tasty-golden instead of
hardcoded inline byte sequences. This addresses review feedback
about the generator executable being redundant when --accept
can regenerate golden files directly.

- Convert testEncodingStability (flat/test) to goldenVsStringDiff
- Convert test_flatStaticEncoding (TPLC) to goldenVsStringDiff
- Remove flat-encoding-generator executable
- Add tasty-golden dependency to flat-test
Both golden files now use encoding-stability.golden.
@Unisay Unisay merged commit 57a912c into master Mar 31, 2026
9 checks passed
@Unisay Unisay deleted the yura/flat-test-coverage branch March 31, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants