Remove Integration Monad #6346

Jimbo4350 · 2025-10-21T17:42:45Z

Description

Remove Integration monad from functions that we also want to use outside of property tests.
Introduce annotated exceptions (via liftIOAnnotated) where the Integration monad has been removed in order to preserve debugging information.

┃ hprop_dump_config :: H.Property
    38 ┃ hprop_dump_config = integrationRetryWorkspace 2 "dump-config-files" $ \tmpDir -> H.runWithDefaultWatchdog_ $ do
       ┃ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       ┃ │ ━━━ Exception (AnnotatedException SomeException) ━━━
       ┃ │ ! AnnotatedException !
       ┃ │ Underlying exception type: IOException
       ┃ │ 
       ┃ │ unsupported operation (Operation is not supported)
       ┃ │ 
       ┃ │ CallStack (from HasCallStack):
       ┃ │   liftIOCallStack, called at src/Testnet/Components/Configuration.hs:186:24 in cardano-testnet-10.0.1-inplace:Testnet.Components.Configuration
       ┃ │   createSPOGenesisAndFiles, called at src/Testnet/Start/Cardano.hs:133:8 in cardano-testnet-10.0.1-inplace:Testnet.Start.Cardano
       ┃ │   createTestnetEnv, called at test/cardano-testnet-test/Cardano/Testnet/Test/DumpConfig.hs:48:23 in cardano-testnet-10.0.1-inplace-cardano-testnet-test:Cardano.Testnet.Test.DumpConfig
       ┃ │   testCase, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:166:66 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   a given constraint, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:159:26 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   testCase, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:144:11 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithWatchdog, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:159:26 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithDefaultWatchdog, called at src/Hedgehog/Extras/Test/TestWatchdog.hs:166:36 in hedgehog-extras-0.8.0.0-26dc8622927009e7df9ba3764930f3c64dfd8a2e158af1686ffeb231b1a5219b:Hedgehog.Extras.Test.TestWatchdog
       ┃ │   runWithDefaultWatchdog_, called at test/cardano-testnet-test/Cardano/Testnet/Test/DumpConfig.hs:38:82 in cardano-testnet-10.0.1-inplace-cardano-testnet-test:Cardano.Testnet.Test.DumpConfig

Checklist

Commit sequence broadly makes sense and commits have useful messages
New tests are added if needed and existing tests are updated. These may include:
- golden tests
- property tests
- roundtrip tests
- integration tests
  See Runnings tests for more details
Any changes are noted in the CHANGELOG.md for affected package
The version bounds in .cabal files are updated
CI passes. See note on CI. The following CI checks are required:
- Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
- Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
- Code builds on Linux, MacOS and Windows for ghc-9.6 and ghc-9.12
Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

cardano-testnet/src/Testnet/Orphans.hs

cardano-testnet/src/Testnet/Process/RunIO.hs

carbolymer · 2025-11-04T16:35:25Z

cardano-testnet/src/Testnet/Process/RunIO.hs

+
+liftIOAnnotated :: (HasCallStack, MonadIO m) => IO a -> m a 
+liftIOAnnotated action = GHC.withFrozenCallStack $
+  liftIO $ action `catch` (\(e :: SomeException) -> throwM $ exceptionWithCallStack e)


This is catch from Prelude right? So this should work with watchdog exceptions I think. Could you add a comment here that we're catching async exceptions here as well intentionally?

carbolymer · 2025-11-04T17:10:19Z

cardano-testnet/src/Testnet/Process/RunIO.hs

I'm not a fan of copying code downstream. Have you given a thought how we could reconcile your changes with hedgehog-extras, so possibly we could avoid the duplication?

My main issue with that is having to fix issues in both places, because we're using similar code in cardano-cli as well.

I'll open a PR.

carbolymer · 2025-11-04T17:22:34Z

cardano-testnet/src/Testnet/Runtime.hs

    -- The port number if it is obtained using 'H.randomPort', it is firstly bound to and then closed. The closing
    -- and release in the operating system is done asynchronously and can be slow. Here we wait until the port
-    -- is out of CLOSING state.
-    H.note_ $ "Waiting for port " <> show port <> " to be available before starting node"


This was a useful trace. I don't remember if we weren't experiencing hangups long time ago at this place. In general it would be nice to have a tracer available here so we could emit traces about the progress of the node startup.

cardano-testnet/src/Testnet/Runtime.hs

cardano-testnet/src/Testnet/Start/Cardano.hs

cardano-testnet/src/Testnet/Start/Types.hs

cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/SanityCheck.hs

carbolymer · 2025-11-05T12:05:27Z

cardano-testnet/test/cardano-testnet-test/Cardano/Testnet/Test/SanityCheck.hs

+        (_,asyncA) <- asyncRegister_ (threadDelay 10_000_000)
+        let tId = asyncThreadId asyncA
+        return (s,tId)
+     afterForkedThread <- getCurrentTime


FYI You can use https://hackage-content.haskell.org/package/hedgehog-extras-0.10.1.0/docs/Hedgehog-Extras-Test-TestWatchdog.html to verify if code paths were reached.

createEnvOptions This removes the unnecessary Integration monad

Add Testnet.Process.RunIO which exposes execCli functions without the MonadTest constraint

This differs from hedgehog-extras's asyncRegister function in that the thread is linked before it is cancelled.

cardanoTestnet

…teLogging

CI fails to create the log file without this change

palas

Nice! It looks good and I don't see any issues in the logic. I've written some style suggestions. And I think the changed files could use a formatter, because I've seen several spaces at the end of line, maybe stylish-haskell would be enough

cardano-testnet/src/Testnet/Property/Assert.hs

cardano-testnet/src/Testnet/Runtime.hs

cardano-testnet/src/Testnet/Process/RunIO.hs

cardano-testnet/src/Testnet/Start/Cardano.hs

cardano-testnet/src/Testnet/Components/Configuration.hs

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 3 times, most recently from b99bb37 to 0bf6d79 Compare October 22, 2025 13:55

Jimbo4350 force-pushed the jordan/remove-integration-monad branch 19 times, most recently from 0f61855 to d7b60ca Compare November 3, 2025 15:26

Jimbo4350 marked this pull request as ready for review November 4, 2025 12:08

Jimbo4350 requested a review from a team as a code owner November 4, 2025 12:08

Jimbo4350 changed the title ~~Remove Integration Monad~~ Do not merge: Remove Integration Monad Nov 4, 2025

carbolymer reviewed Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Orphans.hs Outdated Show resolved Hide resolved

carbolymer reviewed Nov 4, 2025

View reviewed changes

carbolymer requested changes Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Runtime.hs Outdated Show resolved Hide resolved

carbolymer reviewed Nov 4, 2025

View reviewed changes

cardano-testnet/src/Testnet/Runtime.hs Show resolved Hide resolved

carbolymer reviewed Nov 5, 2025

View reviewed changes

Jimbo4350 and others added 20 commits December 10, 2025 10:27

Implement mkConfig and mkConfigAbs

4de166b

Remove runTestnet usage in runCardanoOptions and testnetRoutine from

eb8a918

createEnvOptions This removes the unnecessary Integration monad

Implement liftIOAnnotated which annotates exceptions with a callstack

e4b06ca

Add Testnet.Process.RunIO which exposes execCli functions without the MonadTest constraint

Remove testnetRoutine

5c7ec3d

Implement asyncRegister_

cdfd481

This differs from hedgehog-extras's asyncRegister function in that the thread is linked before it is cancelled.

Add hprop_asyncRegister_sanity_check

f30d525

Remove Integration monad return type from createTestnetEnv and

95580a9

cardanoTestnet

Remove MonadTest constraint from createSPOGenesisAndFiles

602f1fa

Remove MonadTest constraint from startNode and startLedgerNewEpochSta…

8878fe0

…teLogging

Remove MonadTest constraints from assertion functions

1d01c41

Propagate liftToIntegration and liftIOAnnotated

e25a64a

Fix build issues

9b629d3

Review changes

5d73464

Replace liftIO with liftIOAnnotated

ca0d30a

Fix infinite loop

aa69c66

Fix liftToIntegration to report the exception location to Hedgehog

3aaad0b

Rebase changes

8a4902e

Remove unnecessary function

a89638b

Update createDirectoryIfMissingNew

b709f04

Create log directory if it does not exist

7d2962f

Jimbo4350 force-pushed the jordan/remove-integration-monad branch from fe34af3 to 824036b Compare December 10, 2025 14:28

Final fix

8ad1337

CI fails to create the log file without this change

Jimbo4350 force-pushed the jordan/remove-integration-monad branch from 824036b to 8ad1337 Compare December 10, 2025 19:12

palas approved these changes Dec 10, 2025

View reviewed changes

Review fixes

8cf34ed

Jimbo4350 enabled auto-merge December 18, 2025 15:17

Jimbo4350 added this pull request to the merge queue Dec 18, 2025

Merged via the queue into master with commit 1b26ad0 Dec 18, 2025
23 of 25 checks passed

Jimbo4350 deleted the jordan/remove-integration-monad branch December 18, 2025 16:39

Remove Integration Monad #6346

Remove Integration Monad #6346

Uh oh!

Conversation

Jimbo4350 commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Note on CI

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Jimbo4350 Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

carbolymer Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carbolymer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

palas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Jimbo4350 commented Oct 21, 2025 •

edited

Loading