Value costing: insertCoin, unionValue, scaleValue #7435

ana-pantilie · 2025-11-19T21:31:35Z

Fixes https://github.com/IntersectMBO/plutus-private/issues/1899
Fixes https://github.com/IntersectMBO/plutus-private/issues/1900
Fixes https://github.com/IntersectMBO/plutus-private/issues/2004

See https://plutus.cardano.intersectmbo.org/pr-preview/cost-models/pr-7435/ for cost model vizualizations

plutus-core/plutus-core/src/PlutusCore/Evaluation/Machine/CostingFun/SimpleJSON.hs

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

plutus-core/cost-model/data/builtinCostModelC.json

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

…s larger

github-actions · 2025-12-18T19:35:36Z

PR Preview Action v1.6.3
🚀 View preview at https://IntersectMBO.github.io/plutus/pr-preview/cost-models/pr-7435/
Built to branch `gh-pages` at 2025-12-24 12:16 UTC. Preview will be ready when the GitHub Pages deployment is complete.

plutus-core/plutus-core/src/PlutusCore/Value.hs

plutus-core/plutus-core/src/PlutusCore/Default/Builtins.hs

zliu41 · 2025-12-20T08:09:46Z

plutus-core/cost-model/data/builtinCostModelC.json

+    "memory": {
+      "arguments": {
+        "intercept": 24,
+        "slope": 21


How are the slope and intercept determined?

See the note.

plutus-core/plutus-core/src/PlutusCore/Default/Builtins.hs

zliu41 · 2025-12-22T23:42:42Z

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs

+  generateConstrainedValueWithMaxPolicyAndQuantity numPolicies tokensPerPolicy maxBound
+
+-- | Generate constrained Value with information about max-size policy and quantity
+generateConstrainedValueWithMaxPolicyAndQuantity


Did you intend to use this somewhere, perhaps in scaleValueArgs? Otherwise there's not much point factoring this out.

I think this is a remnant of the experimentation which happened on this branch. I'll revert the change it if I have time.

zliu41 · 2025-12-22T23:59:04Z

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

+  - 2 words for the key 'Int' closure
+  - 2 words for the value 'Int' closure
+Putting this all together, we arrive at the following memory usage function:
+    1*n + 4 + 7 + 1


It's unclear what the "4", "7" and "1" refer to. Please make it more obvious.

They come from me drawing out the closures from my understanding of ghc's memory allocation and counting the number of words (which are mentioned in the note). I don't think this methodology is the best, so we might end up removing this note in the end.

The note is useful, just organize it better and make it easier to follow. There's a lot of "1 word for this", "2 words for that", etc etc., and it is non-obvious what the "4" "7" "1" correspond to.

zliu41 · 2025-12-22T23:59:36Z

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

+  ... and the allocations for the 'IntMap' and nested 'Map' themselves.
+The 'Map' and 'IntMap' are implemented as a self-balancing binary tree and a big-endian patricia tree,
+respectively. Since we are interested in the worst-case memory usage, we assume that
+for each application of the builtin function both of the trees have to be completely reallocated.


This only applies to unionValue, since in insertCoin and scaleValue, there's only one tree.

Also, insertCoin never completely allocates the whole tree, even in the worst case.

zliu41 · 2025-12-23T00:01:34Z

plutus-core/cost-model/data/builtinCostModelC.json

+        "intercept": 45,
+        "slope": 21
+      },
+      "type": "linear_in_w"


I'd still like to know if insertCoin is CPU bound or memory bound.

I don't know, I haven't had time to check this yet.

It's important to know this because if it exhausts memory 10 times faster than it exhausts CPU, it indicates that the memory cost is nonsensical and can severely limit the usefulness of Value.

I had time today to do this, these are the results. I created and ran the following test:

compiledInsertCoin :: CompiledCode BuiltinValue compiledInsertCoin = $$(compile [||insertCoin||]) `unsafeApplyCode` liftCodeDef "foo" `unsafeApplyCode` liftCodeDef "bar" `unsafeApplyCode` liftCodeDef 10 `unsafeApplyCode` liftCodeDef (mkValueInData 1000000) mkValueInData :: Int -> BuiltinValue mkValueInData n = let currSymbols = (\x -> B.mkB $ "cS" <> B.encodeUtf8 (B.toBuiltin (Text.pack (show x)))) <$> take n [1 ..] tokN = B.mkB "tN" entries = B.mkMap [ ( cS , B.mkMap [(tokN, B.mkI 100)] ) | cS <- currSymbols ] in B.unsafeDataAsValue entries

The resulting budget is:

CPU: 935_697 Memory: 1_786 AST Size: 12 Flat Size: 17_888_928

I would say that's definitely not memory bound, unless I didn't construct the example properly.

You can do a rough calculation like this: when you insertCoin into a Value of max depth 10:

CPU = (18413 * 10 + 356924) / 10000000000 = 0.00541054% of the transaction CPU limit Memory = (21 * 10 + 45) / 14000000 = 0.001821428% of the transaction memory limit

So we are good.

zliu41 · 2025-12-23T00:03:59Z

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

+respectively. Since we are interested in the worst-case memory usage, we assume that
+for each application of the builtin function both of the trees have to be completely reallocated.
+Since we are dealing with a nested 'Map' with the structure 'Map CurrencySymbol (Map TokenName Integer)',
+we can consider only the worst-case scenario where the map is flattened, i.e., each currency symbol maps


I don't know why this is the worst case memory-wise, and I highly doubt this is the worst case for insertCoin. When inserting a node into a Map, usually the deeper the Map is, the more allocation is needed. I imagine a deep Map also makes rebalancing trickier.

I don't know why this is the worst case memory-wise, and I highly doubt this is the worst case for insertCoin.

It's not, I was mistaken. @kwxm, @Unisay and I had a meeting a few days ago where we discussed this. I think the structure of the map doesn't matter when it comes to how much memory is allocated (for one map), only the number of nodes does. But that doesn't really matter much because we care about the new memory being allocated after the builtin run, not the total memory.

I imagine a deep Map also makes rebalancing trickier.

You're probably right, but it's tricky to figure out what the worst case is and how much memory is allocated.

I think we need to approach memory costing for the Value builtins similar to the way we approach cpu costing: by experimentally finding the worst case and generating the data which we can base predictions on. I don't think we currently do this for the memory costing of any of the other builtins, so we need to also add the required code infrastructure to generate the models using R.

At the same time, @kwxm said we gave up on doing accurate memory costing a while ago, but I don't know the reason behind this.

It is almost always sufficient to determine the amount of allocation by understanding how the algorithm works. I don't even know how you can accurately measure allocations from a specific builtin call - seems quite tricky.

For insertCoin, the key is to be aware that rebalancing allocates at most O(logn) nodes. I think it's more than enough to assume that it allocates ValueMaxDepth number of nodes and be done with it. If you want to be more precise, feel free to read the paper to determine the exact worst case (which should still be far easier than trying to measure it accurately), but for memory cost we don't need to be that precise.

zliu41

The Value builtins should now be added to PlutusLedgerApi.Common.Versions.

zliu41 · 2025-12-24T00:25:22Z

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

+  - 2 words for the key 'Int' closure
+  - 2 words for the value 'Int' closure
+Putting this all together, we arrive at the following memory usage function:
+    1*n + 4 + 7 + 1


The note is useful, just organize it better and make it easier to follow. There's a lot of "1 word for this", "2 words for that", etc etc., and it is non-obvious what the "4" "7" "1" correspond to.

zliu41 · 2025-12-24T00:56:50Z

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs

+respectively. Since we are interested in the worst-case memory usage, we assume that
+for each application of the builtin function both of the trees have to be completely reallocated.
+Since we are dealing with a nested 'Map' with the structure 'Map CurrencySymbol (Map TokenName Integer)',
+we can consider only the worst-case scenario where the map is flattened, i.e., each currency symbol maps


It is almost always sufficient to determine the amount of allocation by understanding how the algorithm works. I don't even know how you can accurately measure allocations from a specific builtin call - seems quite tricky.

For insertCoin, the key is to be aware that rebalancing allocates at most O(logn) nodes. I think it's more than enough to assume that it allocates ValueMaxDepth number of nodes and be done with it. If you want to be more precise, feel free to read the paper to determine the exact worst case (which should still be far easier than trying to measure it accurately), but for memory cost we don't need to be that precise.

zliu41 · 2025-12-24T01:02:48Z

plutus-core/cost-model/data/builtinCostModelC.json

+        "intercept": 45,
+        "slope": 21
+      },
+      "type": "linear_in_w"


It's important to know this because if it exhausts memory 10 times faster than it exhausts CPU, it indicates that the memory cost is nonsensical and can severely limit the usefulness of Value.

…inal

zliu41

Looks good but we need @kwxm to sign off on this.

zliu41 · 2025-12-24T23:48:52Z

plutus-core/cost-model/data/builtinCostModelC.json

+        "intercept": 45,
+        "slope": 21
+      },
+      "type": "linear_in_w"


You can do a rough calculation like this: when you insertCoin into a Value of max depth 10:

CPU = (18413 * 10 + 356924) / 10000000000 = 0.00541054% of the transaction CPU limit Memory = (21 * 10 + 45) / 14000000 = 0.001821428% of the transaction memory limit

So we are good.

This was referenced Nov 19, 2025

Costing builtin value: insertCoin, unionValue, scaleValue #7423

Closed

BuiltinValue: insertCoin and unionValue and scaleValue costing #7384

Closed

This comment was marked as outdated.

Sign in to view

ana-pantilie changed the title ~~Value costing: insertCoin, unionValue~~ Value costing: insertCoin, unionValue, scaleValue Nov 20, 2025

This comment was marked as outdated.

Sign in to view

ana-pantilie force-pushed the ana/value-costing-final branch 3 times, most recently from 1e44958 to abdc6ed Compare December 5, 2025 14:15

zliu41 reviewed Dec 8, 2025

View reviewed changes

plutus-core/plutus-core/src/PlutusCore/Evaluation/Machine/CostingFun/SimpleJSON.hs Outdated Show resolved Hide resolved

plutus-core/cost-model/budgeting-bench/Benchmarks/Values.hs Outdated Show resolved Hide resolved

ana-pantilie commented Dec 15, 2025

View reviewed changes

plutus-core/cost-model/data/builtinCostModelC.json Outdated Show resolved Hide resolved

ana-pantilie commented Dec 16, 2025

View reviewed changes

plutus-core/cost-model/create-cost-model/BuiltinMemoryModels.hs Outdated Show resolved Hide resolved

ana-pantilie added 18 commits December 18, 2025 18:59

Add insertCoin and unionValue costing skeleton

d897a75

Add benchmarking code and costing model for insertCoin

3ed4321

Add benchmarking code and costing model for unionValue

9b566bd

New models for insertCoin and unionValue; regenerate data

ab1312b

Fix model parsing and printing, wip scaleValue

54557fe

Fix issues with rebase

61d398b

Temp: experimenting with different inputs for unionValue

c81b534

Fix unionValue: shortcircuit if one is empty, flip args when second i…

a464a1c

…s larger

Add alternative implementation for unionValue

b74c996

Add new benchmarking scenarios

7d79538

Tmp: scaleValue costing

a4a95bb

UnionValue + ScaleValue generators, changes to unionValue builtin

bf67123

Data frames scaleVal unionValFInal

1fe564a

Add model for unionValue

9afbe70

Add cost models for insertCoin and unionValue

fcb377d

Remove experimental csv files and frames

0f47947

Fix cost model json

ed707b0

Fix cost models

c702815

ana-pantilie added 3 commits December 18, 2025 21:00

Fix

fed80d8

Fix

58c608b

Changelogs

202ad33

ana-pantilie marked this pull request as ready for review December 18, 2025 19:09

ana-pantilie requested review from Unisay, kwxm and zliu41 December 18, 2025 19:10

Add new functions to vizualization website

1fe82f1

ana-pantilie temporarily deployed to github-pages December 18, 2025 19:34 — with GitHub Actions Inactive

Fixes

0889b78

ana-pantilie temporarily deployed to github-pages December 18, 2025 19:49 — with GitHub Actions Inactive

Fix?

2b15ad5

ana-pantilie temporarily deployed to github-pages December 18, 2025 20:10 — with GitHub Actions Inactive

Add newtypes to denotations and regenerate golden files

c9fc008

ana-pantilie temporarily deployed to github-pages December 19, 2025 15:05 — with GitHub Actions Inactive

zliu41 reviewed Dec 20, 2025

View reviewed changes

Address review comments

d529ba8

ana-pantilie temporarily deployed to github-pages December 20, 2025 13:36 — with GitHub Actions Inactive

ana-pantilie requested a review from zliu41 December 20, 2025 13:38

zliu41 reviewed Dec 23, 2025

View reviewed changes

zliu41 reviewed Dec 24, 2025

View reviewed changes

Merge remote-tracking branch 'origin/master' into ana/value-costing-f…

78b9e7f

…inal

ana-pantilie temporarily deployed to github-pages December 24, 2025 11:17 — with GitHub Actions Inactive

Add Value builtins to batch 6

85d1aa9

ana-pantilie temporarily deployed to github-pages December 24, 2025 11:41 — with GitHub Actions Inactive

Address review comments

c411ba4

ana-pantilie temporarily deployed to github-pages December 24, 2025 12:15 — with GitHub Actions Inactive

ana-pantilie requested a review from zliu41 December 24, 2025 13:07

zliu41 approved these changes Dec 25, 2025

View reviewed changes

Value costing: insertCoin, unionValue, scaleValue #7435

Are you sure you want to change the base?

Value costing: insertCoin, unionValue, scaleValue #7435

Conversation

ana-pantilie commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Built to branch gh-pages at 2025-12-24 12:16 UTC. Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ana-pantilie Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ana-pantilie Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zliu41 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zliu41 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ana-pantilie commented Nov 19, 2025 •

edited

Loading

github-actions bot commented Dec 18, 2025 •

edited

Loading

Built to branch `gh-pages` at 2025-12-24 12:16 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

ana-pantilie Dec 23, 2025 •

edited

Loading

ana-pantilie Dec 23, 2025 •

edited

Loading