Identify start function in CM `thread.spawn` with an index

[abrown]

When designing this proposal's integration with the component model (CM), we decided to pass the start function as a reference to the CM's thread.spawn intrinsic. IIRC, this avoided the issue of forcing the creation of a shared table to contain the start function. But, while doing pathfinding implementation to vet these design choices, I discovered that there are missing pieces in wasm-tools that make this "pass start as a reference" very difficult.

@alexcrichton and @fitzgen can correct me here if I'm not explaining this correctly, but my take is that the implementations in wasm-tools and Wasmtime don't have the necessary plumbing to refer to a CM core type from within a separate CM core module. No other CM intrinsics besides thread.spawn refer to a type declared elsewhere (see $ft here); if we are to pass a reference, we need a type for that reference and that type has an index, but not necessarily the same index, at different stages in the toolchain. @alexcrichton and @fitzgen tell me that adding the necessary index conversions for this new intrinsic in wasm-tools and Wasmtime is too much work; their recommendation is to switch the thread.spawn intrinsic to receive an i32 instead.

At this point, this issue is a roadblock before implementing enough to experiment with different TLS design options, which is what I believe needs more focused attention. The necessary bits are implemented in wasm-tools but this problem of "how to pass the start function" must to be resolved to have a working implementation in Wasmtime. Here are some options to resolve this:

1. figure out some way to implement the missing plumbing between wasm-tools and Wasmtime; @alexcrichton and @fitzgen tell me this is the least desirable choice
2. replace the signature of the CM thread.spawn: [f: (ref null $f) n:i32 c:i32] would become [f: i32 n: i32, c: i32]
3. add an additional CM intrinsic, thread.spawn_indirect, with the new signature [f: i32 n: i32, c: i32]

Note that I am attempting to get this all working in wasi-libc, where one would expect that C function pointers would be placed in a table anyways. It's not clear to me that if we adopted 3 how we might implement the current version of thread.spawn and who would be using it, given how most projects tend to build on top wasi-libc.

[tlively]

that type has an index, but not necessarily the same index, at different stages in the toolchain.

Can you elaborate on this part? What are the different relevant stages and why does the type index change? There is a relocation R_WASM_TYPE_INDEX_LEB that would seemingly solve this problem if the toolchain could emit it in the correct location.

I suspect that part of the problem is that core Wasm doesn't support type parameters at all, and the component model probably doesn't want to be in the business of introducing them.

Another option would be to make the type of the parameter funcref and spec that the intrinsic performs a cast to the intended function type.

In general the idea of creating thread.spawn_indirect for now and adding a thread.spawn_ref later when GC langages are more prevalent on WASI makes sense to me.

[rossberg]

Re 2: Wouldn't this require a table index immediate? Why is that easier to handle than a type index?

Re 3: Wouldn't spawn_indirect need a type index immediate as well? Similarly any solution using funcref?

AFAICS, somehow the intended type has to be (statically) derivable, so some form of immediate appears to be unavoidable.

[alexcrichton]

To clarify a bit on why things are hard right now for wasmtime/wasm-tools, the precise reasons are pretty specific to the design of the two libraries themselves. The exact details are a bit much to put in this thread, but the general gist is that the full matrix of interactions of the GC proposal of WebAssembly and the component model have not all been fleshed out previously in Wasmtime. That's being stressed here with today's thread.spawn intrinsic because it generates a function where the parameter of that function is a typed function. This of course should work just fine in Wasmtime, and one day we'll of course have to do the work to get all that fixed. For the purposes of testing shared-everything-threads today, however, it's quite a lot for Andrew to take on as the reason things aren't working are basically unrelated to threads.

The other rationale we have for not using a function-reference-based-approach is that LLVM-based tooling basically isn't ready for that. AFAIK there's no way, today, to import the thread.spawn function with the right signature from C/Rust/etc. That means that actually binding it would involve wrappers/shims/etc. All of this is of course possible, but again for the purpose of testing shared-everything-threads this is quite a lot to take on.

I believe Nick and I were advocating for (3) here where we're not saying that thread.spawn should go away, only that in the meantime there's also thread.spawn_indirect. Wasmtime won't implement thread.spawn in this evaluation phase but we'd of course come back as this proposal goes through the design phases.

And yes, spawn_indirect would have an immediate as to which table it's referencing to load the index from.

[rossberg]

Well, call_indirect has both a type and a table index. It's not clear to me how spawn_indirect could get around that?

[tlively]

spawn_indirect only needs to take a type index if it needs to support different function types. If there's just one hard-coded function type, then all it would need is a table index immediate. I'm not sure that makes things significantly easier, since LLVM also supports type index immediates, but it might.

[rossberg]

@tlively, well yes, but isn't that orthogonal, and you could do the same with a ref? Not sure I follow how call_indirect helps.

[tlively]

Right, that's why I suggested that another short-term solution would be to use funcref rather than a typed function reference or a spawn_indirect variant.

[rossberg]

Even if the parameter itself was funcref, you'd need to annotate its expected type somehow, otherwise the expected operand count and types wouldn't be known to the validator. Unless they're hardcoded in the semantics, but then there's no point in using funcref either. So again, I can't see how any of these options solves anything.

[tlively]

but then there's no point in using funcref either.

Why not? It still lets you dynamically choose which function to spawn the thread with, as long as it has the expected signature.

[rossberg]

Yes, but what's the advantage over the status quo using a typed reference?

[tlively]

LLVM-based tooling would have a much easier time dealing with a funcref than dealing with a typed function reference.

[rossberg]

@alexcrichton, I can't quite tell from your comment whether you are referring to function references in the signature generally being problematic for LLVM, or typed function references in particular. Can you clarify?

[alexcrichton]

Personally I find both problematic. While LLVM tooling handles funcref there's at least no way in Rust for example to use funcref. IIRC the solution for C/C++ was to have a special compiler-defined type which has special rules about it (e.g. you can't take the address of it) but that doesn't work for Rust. This means that if the intrinsic here takes a funcref as an argument there's no way to actually call it from Rust. Rust will be forced to do the indirect table thing no matter what basically.

I'll reiterate I understand why the spec-level definition of this function takes a typed-function or a generic funcref, and I think that makes sense. Getting things working for Rust (and C/C++? unsure) can be considered "just a tooling problem go figure it out". I think that's reasonable too. @abrown's goal here is evaluation though as opposed to finishing a design, and in my opinion it would be far easier to evaluate spawn_indirect which takes a table index as an i32 and the intrinsic definition has a static immediate of which table to reference.

[tlively]

That sounds perfectly fine to me 👍