Implement PDP schedule execution path and config wiring by anjor · Pull Request #621 · data-preservation-programs/singularity

anjor · 2026-02-20T14:24:41Z

Summary

implement an executable PDP scheduling path in deal-pusher (runPDPSchedule) with sequencing: proof-set ensure -> root queue -> tx confirmation -> deal persistence
add configurable PDP scheduling parameters to run deal-pusher flags: pdp-batch-size, pdp-gas-limit, pdp-confirmation-depth, pdp-poll-interval
add DealPusher option hooks for PDP dependencies/config and switch default schedule deal-type inference to delegated provider addresses
expand wiring tests to validate PDP scheduling behavior and default deal-type inference

Validation

go test ./service/dealpusher
go test ./cmd/run

parkan · 2026-02-20T17:28:12Z

based on test failure let's park this momentarily while I push up the work related to wallet extraction from db, early next week

parkan · 2026-02-20T17:33:05Z

see #622 and data-preservation-programs/go-synapse#4

both need work but I need to sign off for the week

parkan · 2026-02-24T16:25:13Z

a few changes needed here to cleanly separate the Actor and Wallet concepts:

schedule.Preparation.Wallets → schedule.Preparation.Actors (or add a separate Wallet association on Preparation for PDP)
EnsureProofSet / QueueAddRoots now take signer.EVMSigner — caller resolves wallet→keystore→EVMSigner before calling
model.Wallet no longer has PrivateKey inline — use keystore.EVMSigner(ks, wallet) to get a signer
The WalletChooser picks actors (for market deals); PDP path needs a different resolution to get a wallet+signer

these are basically the changes from #621 (comment) that I mentioned as critical

parkan · 2026-02-25T11:41:48Z

I'll do a basic rebase fixup here with obvious changes that don't require deliberation then leave it in your hands

…tor semantics (#622) ## Problem Singularity used go-filsigner for Filecoin message signing, with key material stored inline in the database (wallets.private_key column). The Wallet model was actually an on-chain actor identity (f0 address) with an embedded private key -- confusing naming that also coupled key management to database access and left a big security hole/private keys exposed in any db backup or access violation For PDP deals, we need EVM transaction signing (secp256k1 ECDSA), which go-filsigner doesn't support. The existing model also couldn't represent the real relationship: a wallet (private key) exists before any on-chain actor, and PDP deals don't need an actor at all. ## Solution ### New keystore abstraction (util/keystore/): - KeyStore interface with Put/Get/List/Delete/Has — filesystem-backed, keys stored as lotus-export-format files named by address (for now) - Signer(ks, wallet) / EVMSigner(ks, wallet) — loads key from keystore, returns go-synapse Signer or EVMSigner - Replaces direct database reads for key material ### Model rename — old Wallet becomes Actor: - The old Wallet struct (f0 actor ID + address + private key) is renamed to Actor — that's what it always was - Actor.PrivateKey column is orphaned (will be dropped by a future export-keys migration command) - GORM TableName() renames wallets → actors on auto-migrate ### New Wallet model (model/wallet.go): - Represents a private key in an external keystore: KeyPath, KeyStore, Address - Optional ActorID foreign key — links to on-chain actor when one exists - Invariant: any deal we make requires a Wallet. Market deals also require an Actor (for ClientID). PDP deals need only a Wallet. Imported/tracked deals may have neither. ### Preparation associates with Wallet, not Actor: - Preparation.Actors []Actor → Preparation.Wallets []Wallet (many2many wallet_assignments) - All handlers updated: attach/detach/listattached operate on Wallets - WalletChooser picks from []model.Wallet, DatacapWalletChooser dereferences wallet.ActorID for datacap lookup - Dealpusher market deal path: choose wallet → require ActorID → load actor → sign with wallet → pass actor to MakeDeal - Association to Actor can be loaded on demand, see below ### MakeDeal takes ProposalSigner instead of db + keystore: - type ProposalSigner func([]byte) (*crypto.Signature, error) - Caller constructs the signer, MakeDeal just calls it — cleaner separation ### Lazy actor resolution (handler/wallet/sign.go): - GetOrCreateActor() — on first market deal, queries Lotus for the wallet's on-chain actor, creates Actor record, links it - Supports the workflow: import wallet offline → fund externally → first deal auto-discovers actor ### DealPusher options pattern (service/dealpusher/options.go): - WithPDPProofSetManager, WithPDPTransactionConfirmer, WithPDPSchedulingConfig, WithScheduleDealTypeResolver - Replaces growing constructor parameter list ### Dependency changes - Added: go-synapse updated (b27d67ac9095) — provides signer.FromLotusExport, signer.Signer, signer.EVMSigner, signer.AsEVM - Removed: go-filsigner and its transitive deps (dchest/blake2b, drand/kyber, drand/kyber-bls12381) ### What's NOT in this PR - Key export/migration command (exporting actors.private_key → keystore files) — separate PR - Dropping the private_key column — blocked on export command - PDP deal-making wiring (this PR provides the signing infra, actual PDP scheduling is #621)

parkan · 2026-02-25T12:06:58Z

actually rebase/force push over your code is rude, redoing as merge commit so you can review it clearly

resolve conflicts in dealpusher: adapt PDP scheduling to new Wallet/Actor model and keystore-based EVMSigner from #622

service/dealpusher/options.go

parkan · 2026-02-25T13:02:48Z

service/dealpusher/pdp_schedule.go

+			return model.ScheduleCompleted, nil
+		}
+
+		walletObj, err := d.walletChooser.Choose(ctx, schedule.Preparation.Wallets)


hmm I think we should remove random wallet chooser or parametrize it such that specific properties are met (e.g. non-BLS key) -- I don't really see the utility here; if we want to round-robin across wallets for some reason we can make the parameter multivalued and let the user choose, but that seems like a bonus not core

parkan · 2026-02-25T13:03:29Z

service/dealpusher/pdp_schedule.go

+		if err := baseQuery().Find(&cars).Error; err != nil {
+			return nil, errors.Wrap(err, "failed to find PDP cars")
+		}
+		filtered := make([]model.Car, 0, limit)


why not limit during db read?

parkan · 2026-02-25T13:08:49Z

service/dealpusher/pdp_schedule.go

+			}
+
+			if err := database.DoRetry(ctx, func() error { return db.Create(dealModel).Error }); err != nil {
+				return model.ScheduleError, errors.Wrap(err, "failed to create PDP deal")


we will likely want to do something more sophisticated if deal lands on chain but DB write fails or we crash etc; right now this is recoverable if noticed using --full-sync so not a blocker but I would add a TODO for more resilient lifecycle management

parkan · 2026-02-25T13:10:04Z

I would move the filter limit into DB scan unless you have conflicting plans, the other two are just TODOs for later

approved so you can merge (if you disagree about DB add a comment explaining why)

anjor requested a review from parkan February 20, 2026 14:25

anjor force-pushed the feat/pdp-scheduling-runner branch 2 times, most recently from c93e4e2 to 1641a4e Compare February 24, 2026 14:18

anjor added 3 commits February 24, 2026 16:35

Implement PDP schedule execution path and config wiring

2e9a5fa

Fix staticcheck unused PDP wiring test mocks

254284c

Fix PDP wiring test wallet ID length

c03ce04

anjor force-pushed the feat/pdp-scheduling-runner branch from 1641a4e to c03ce04 Compare February 24, 2026 16:35

Update deal-pusher CLI docs for PDP flags

dbfba3a

parkan mentioned this pull request Feb 25, 2026

wire go-synapse signer into keystore (vs go-filsigner), fix Wallet/Actor semantics #622

Merged

merge main (including #622) into feat/pdp-scheduling-runner

c8c4405

resolve conflicts in dealpusher: adapt PDP scheduling to new Wallet/Actor model and keystore-based EVMSigner from #622

parkan reviewed Feb 25, 2026

View reviewed changes

service/dealpusher/options.go Outdated Show resolved Hide resolved

Fix PDP schedule client actor resolution and test actor id

079efe2

parkan reviewed Feb 25, 2026

View reviewed changes

parkan approved these changes Feb 25, 2026

View reviewed changes

anjor merged commit 90dd8d2 into main Feb 25, 2026
3 checks passed

anjor deleted the feat/pdp-scheduling-runner branch February 25, 2026 13:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement PDP schedule execution path and config wiring#621

Implement PDP schedule execution path and config wiring#621
anjor merged 6 commits intomainfrom
feat/pdp-scheduling-runner

anjor commented Feb 20, 2026

Uh oh!

parkan commented Feb 20, 2026

Uh oh!

parkan commented Feb 20, 2026

Uh oh!

parkan commented Feb 24, 2026

Uh oh!

parkan commented Feb 25, 2026

Uh oh!

parkan commented Feb 25, 2026

Uh oh!

Uh oh!

parkan Feb 25, 2026

Uh oh!

parkan Feb 25, 2026

Uh oh!

parkan Feb 25, 2026

Uh oh!

parkan commented Feb 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anjor commented Feb 20, 2026

Summary

Validation

Uh oh!

parkan commented Feb 20, 2026

Uh oh!

parkan commented Feb 20, 2026

Uh oh!

parkan commented Feb 24, 2026

Uh oh!

parkan commented Feb 25, 2026

Uh oh!

parkan commented Feb 25, 2026

Uh oh!

Uh oh!

parkan Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

parkan Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

parkan Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

parkan commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

parkan commented Feb 25, 2026 •

edited

Loading