✨ Export reducer internals via experimental and label infrastructure effects#1127
✨ Export reducer internals via experimental and label infrastructure effects#1127taras wants to merge 5 commits intothefrontside:v4-1-alphafrom
Conversation
One of the most powerful patterns that we've uncovered in the past couple of years of writing Effection code in production is the ability to contextualize an API so that they can be decorated in order to alter its behavior at any point. In other circles, this capability is known as "Algebraic Effects" and "Contextual Effects". With them, we can build all manner of constructs with a single primitive that would otherwise require many unique mechanisms. These include things like: - dependency injection - mocking inside tests with test doubles - adding instrumentation such as OTEL spans and metrics colleciton to - existing interfaces - wrapping stuff in database transactions This functionality was available as an external extension (https://frontside.com/effection/x/context-api), but the pattern has proven so powerful that we're bringing it directly into Effection core. Among other things, this will allow us to provide the type of orthogonal observability that we need to build the Effection inspector without having to change the library itself in order to accomodate it. This change brings the context API functionality directly into Effection. To create an API, call `createApi()` with the "core" functionality, where the "core" is how it will behave without any modification. ```ts // logging.ts export const Logging = createApi("Logging", { *log(...values: unknown[]) { console.log(...values); } }) // export member operations so they can be use standalone export const { log } = Logging.operations; ``` ```ts import { log } from "./logging.ts" export function* example() { // do stuff yield* log("just did stuff"); } ``` ```ts // Override it contextually only inside this scope yield* logging.around({ *log(args, next) { yield* next(...args.map(v => `[CUSTOM] ${v}`)); } }); ``` Apis can be enhanced directly from a `Scope` as well: ```ts import { Logging } from "./logging.ts"; function enhanceLogging(scope: Scope) { scope.around(Logging, { *log(args, next) { yield* next(...args.map(v => `[CUSTOM] ${v}`)); } }); } ``` As an example, and as the api necessary to implement the inspector, this provides a Scope api which provides instrumentation for core Effection operations. Such as create(), destroy(), set(), and delete() for scopes.
commit: |
|
This should be solved with and API/middleware. |
|
Can you say more? Do you expect this to already be solvable with middleware? or is something missing that would make it possible with middleware? |
|
Extensions to the core should always be done with an |
|
@cowboyd we already have a reducer context and the reducer is an interface so that makes sense. we'll need to see what the performance implications are. |
taras
force-pushed
the
feat/experimental-reducer-exports
branch
from
9675d87 to
26afaed
Compare
Add api.Reducer as a middleware interception point for effect reduction, enabling external packages (e.g., @effectionx/durably) to intercept instruction processing without forking Effection. Changes: - lib/api.ts: Define ReducerApi interface and api.Reducer with createApi - lib/coroutine.ts: Route reduce calls through reducerApi.invoke() instead of directly calling ReducerContext, using routine.scope for correct middleware chain resolution - lib/scope-internal.ts: Install stock Reducer as base-layer api.Reducer middleware on the global scope - experimental.ts: Export InstructionQueue, Instruction type, and DelimiterContext for custom reducer implementations - lib/callcc.ts, lib/delimiter.ts, lib/each.ts, lib/future.ts, lib/scope-internal.ts: Add description labels to withResolvers() calls so custom reducers can distinguish infrastructure effects from user-facing effects
taras
force-pushed
the
feat/experimental-reducer-exports
branch
from
26afaed to
bc757a0
Compare
cowboyd
force-pushed
the
v4-1-alpha
branch
2 times, most recently
from
9fe00aa to
9dbdf93
Compare
|
hey! just wanted to express interest in this landing :) +1 this seems like a useful addition and it's not something we can easily do via patching as things break transitively |
|
@jackyzha0 I haven't pushed this forward because I was able to do everything I wanted to do without it. I point to some of my solutions if it aligns with your use case. Can you tell me how you want to use this? |
|
I've also been working on runtime tricky that needs access to the Reducer object. This work would be quite useful and would save us from having to commit import crimes. I mean, there are all sorts of workarounds for not having this export, ranging from ugly to Lovecraftian, but since this is a useful extension point and pretty (IMHO) feasible to support with forward compatibility, the best option is to land it after all. |
|
@taras i'd be curious what you came up with! we basically needed access to Priority, ReducerContext, and ErrorContext, and the Reducer class the context ones we were able to fish out by taking advantage of the stringly-keyed context slots but got a bit stick on Reducer and ended up doing some import patching hackery with |
|
I was using this branch for thefrontside/effectionx#179 but I was able to simplify my implementation not to need these. However, we do intend to expose some of the Effection internals via middleware in 4.1. We can do a write up about what's coming to see if it'll find both of your use cases. |
|
@jackyzha0 @dcolascione +1 on hearing your use-cases. We definitely are all about exposing the internals of Effection. We just want to make sure we do it in the right way since it means moving more things into the public API, and we wanted to craft it in just the right shape. |
4 participants
Motivation
External packages (like
@effectionx/durable) need to build custom reducers that intercept effect processing — for example, to record effect resolutions to a durable stream and replay them on resume. Currently, the types and contexts required to build a custom reducer (ReducerContext,Instruction,InstructionQueue,DelimiterContext) are internal to Effection and not accessible through any public entrypoint.Additionally, custom reducers need to distinguish infrastructure effects (like
useCoroutine(),useScope(), scope destruction) from user-facing effects (likesleep(),call(),action()). Infrastructure effects that usewithResolvers()currently have no description label, making them indistinguishable from user code.This PR makes durable execution (and other custom reducer patterns) completely additive to Effection — no fork required.
Approach
1. Export reducer internals via
experimental.ts(7 lines added)Re-export
Reducer,ReducerContext,InstructionQueue, and theInstructiontype fromlib/reducer.ts, plusDelimiterContextfromlib/delimiter.ts, through the existingexperimentalentry point. This follows the established pattern —experimental.tsalready exportsapifor the same reason (enabling external middleware).2. Add
exporttoInstructionandInstructionQueue(2 lines changed inlib/reducer.ts)These were module-private. Making them
exportallows the re-export inexperimental.ts.3. Label infrastructure
withResolvers()calls (5 files, 1 line each)Add description strings to
withResolvers()calls in core infrastructure:lib/callcc.ts:"await callcc"lib/delimiter.ts:"await delimiter"lib/each.ts:"await each done","await each context"lib/future.ts:"await future"lib/scope-internal.ts:"await destruction"The
descriptionparameter already exists onwithResolvers()and is already used elsewhere. These labels are purely additive — they make infrastructure effects identifiable by description, which is useful for debugging, inspection, and custom reducers.Zero behavior changes. All 121 test steps across 15 core test suites pass unchanged.