What refactoring the TSRM gets us. I think?

[withinboredom]

I’m working on a major refactoring of PHP’s Thread Safe Resource Manager (TSRM) that’ll eventually enable FrankenPHP to run PHP requests as native Go goroutines. Here’s where I’m at, along with some pie-in-the-sky.

I’m nearly 80–90% through with refactoring TSRM and about to start playing with it on FrankenPHP. One thing to note is that thread-pools can go away, eventually. Instead of a specific thread, you pass PHP a context handle and it’ll do the thing. That being said, this is only the beginning of a larger plan.

TSRM changes

For the most part, TSRM remains almost the same. I’ve got a working compatibility layer so that for existing extensions/SAPIs, nothing has to change. For FrankenPHP, however, we’ll use the new stuff. This basically amounts to:

// Create a new PHP context (not tied to any thread)
zend_tsrm_context_handle ctx = zend_tsrm_context_create();

// Activate context on current thread (works on **ANY** thread)
zend_tsrm_context_set_current(ctx);

// Execute PHP code
php_execute_script(...);

// Deactivate and clean up
zend_tsrm_context_set_current(NULL);
zend_tsrm_context_destroy(ctx);

The key difference here is that the context (globals/CG/EG/etc) can actually migrate between threads. No more allocating special worker pools, or dealing with num workers. Instead, you have a pool of threads that you allocate contexts to as needed. You can have more contexts than threads (so, 15 workerA contexts, 10 workerB contexts, but only 15 threads) and then only need to spin up new threads when required to handle the load. This is waaay cheaper than what we currently have (I hope).

All that being said, little will change unless we have ...

Go-based signal handling

Once we have a TSRM that can migrate between threads, then we need a way to deliver signals so they can actually migrate between them as well. This means we’ll have to have some way to dispatch these signals from Go to PHP. This will also have to introduce an API into PHP where we can "interrupt" PHP contexts on-demand.

I’ve identified several safe points where this can be done:

• function boundaries
• backward jumps (aka, loops)
• i/o operations (we’re waiting anyway, but someone needs to introduce proper i/o hooks)

This also conveniently gives us preemption in PHP... and this doesn’t need the TSRM bit at all. This is a bit tricky, though, because other extensions might use signals. So, we’ll need to figure out some sort of API for extensions or something. (Yes, I know the "or something" is carrying a lot of weight here)

We can start connecting up a preemptive scheduler that works with the go scheduler to run php preemptively, essentially making thread pools a thing of the past and having a single Go thread handling hundreds (if not thousands) of requests fairly, and concurrently; all across a much smaller number of threads. PHP requests basically could become just another goroutine.

When

This will take awhile. Maybe PHP 9.x? This is mostly pie-in-the-sky thinking right now, and glosses over the technical (and political) details and challenges (there are a ton). But, I wanted to share my vision and see what you think.

[henderkes]

I mean I'd love it, it would essentially level the playing field with swoole. But it would also condemn us to maintain the same i/o extension hooks and we'd need very strict testing to ensure we're still 1:1 with php-fpm in classic mode (in behaviour, outside of php's thread bugs).

[withinboredom]

Yeah, I'm quite excited about @bukka's work around i/o, and hopefully it will help here. It's really hard to say at this point. This also heavily relies on the Fibers C-api to be able to "jump out" of the stack to pause PHP execution (hence the safe points, things look pretty stable around there as far as the stack goes). I've only done rudimentary testing for feasibility, so there may be dragons I'm unaware of yet.

I have no idea what the performance implications will be, but I think even if we take a small perf hit at first, it'll be worth it in the end. I do expect there to be a negative performance impact to the TSRM, but not much (±2-5%?) since we get rid of thread-local variables and have a couple of new indirections on every global lookup. I might be able to do something about that, but gotta get it working first.

[arnaud-lb]

You may be able to integrate with @EdmondDantes's work on Async, too.

I do expect there to be a negative performance impact to the TSRM, but not much (±2-5%?) since we get rid of thread-local variables and have a couple of new indirections on every global lookup. I might be able to do something about that, but gotta get it working first.

You can avoid getting rid of thread-local variables if you update them in zend_tsrm_context_set_current(). The thread-local variables were key to make TLS builds fast.

This could be achieved like this:

• Each TSRMLS_CACHE_DEFINE() should define a function that sets TSRMLS_CACHE, like static void tsrm_cache_set(void *cache) { TSRMLS_CACHE = cache; } or static void tsrm_cache_update(void) { TSRMLS_CACHE_UPDATE(); }.
• Register these functions with TSRM
• Call them in zend_tsrm_context_set_current()

[withinboredom]

That's a pretty slick idea and would work pretty well I think.

[AlliBalliBaba]

Nice 👍 good to see that you've already made some progress.
I've considered some scenarios, let me know what the end-goal is in your mind:

Scenario 1: PHP runs on C threads (blocking)

This is the current scenario. We need a worker thread for each concurrent request. Worker threads are spawned dynamically if needed. Each thread has its own dedicated connection to e.g a database.

Scenario 2: PHP runs on go threads (blocking)

If PHP runs on go threads instead, but IO is still blocking, we'd basically have to lock the go OS thread when serving requests. That would save having to cross the following thread boundaries:

• request arrives go thread -> c thread
• request is finished: c thread -> go thread

That would definitely save some friction. I suspect though that having to spawn additional Go OS threads (basically 1 for every concurrent requests) would outweigh that friction.

Scenario 3: PHP runs on go threads (non-blocking)

I think it's more interesting if PHP actually runs on go threads while PHP IO is non-blocking. Now we only need the standard amount of go threads and save on having to cross the boundaries between go and C. This might potentially also unlock some other optimizations, but we have to deal with all the downsides that come from having 2 runtimes on the same thread.

And each 'worker instance' will only be able to handle one request at a time, not sharing its memory with other instances.

Scenario 4: PHP runs on C threads (async non-blocking)

Async runtimes have the advantage that you can re-use memory across concurrent requests. This allows for many potential optimizations like connection pools. The downside is of course that you can re-use memory across concurrent requests 😅 , introducing a new potential source for bugs.

PHP being async is actually the Swoole model. In our case workers run in threads instead of processes.
Replicating the Swoole model would probably be easy to do if the true-async RFC passes. We'd just need to provide a way to inject the request into the PHP event loop. The PHP events loop would be independent of the go event loop.

Scenario 5: PHP runs on go threads (async, non-blocking)

Additionally to the Swoole async model, PHP can also run directly on go threads. Go would in that case be the async backend for PHP and we don't have to cross thread boundaries. Each go thread has access to multiple async PHP workers, seamlessly switching between them. We would only need num_cpu go threads and we would only need num_cpu worker 'instances'. Probably opens up more optimizations, with the difficulty of having 2 runtimes on the go thread.

Scenario 6: PHP runs on go threads (async, non-blocking, single instance)

Even if PHP runs asynchronously, it would not fully reflect the go execution model. Every global go variable is global per-process, not per-instance or per-thread.

It might be possible to just start a single PHP worker instance and have its globals be fully unsafe across the whole process. PHP would then handle requests asynchronously and in-parallel, exactly like go. That would be the 'true go mode', but it would also require devs to lock globals and PHP's memory management is very far from being safe in such a scenario.

[henderkes]

PHP being async is actually the Swoole model. In our case workers run in threads instead of processes.

Actually, swoole has a thread model with ZTS builds! Very similar to what we'd be doing (if we keep php execution in C land).

Php running directly in Goroutines sounds great, but we'd need to pin them to OS threads as far as I can imagine. And how to keep both runtimes from colliding, no idea. My understanding of the Go runtime is limited to "it doesn't do things the same way that other native runtimes work".

[dunglas]

Why would they collide? The main issues are signals, that are used mostly for timeouts PHP-side, but @arnaud-lb created a POC of signal-less timeouts that we could maybe reuse.

[AlliBalliBaba]

I think the other potential issue could be go potentially switching the stack when scheduling. Currently unsure when this might become a problem. It previously was a problem with fibers at least.

[withinboredom]

The issue with fibers was to do with Go -> C -> fiber stack -> Go. Specifcally, IIRC, the Go stack was "too far away" from the top of the stack, and Go couldn't resume — or something like that. As in, it couldn't even print out an error message. We won't have this problem here.

[arnaud-lb]

This means we’ll have to have some way to dispatch these signals from Go to PHP. This will also have to introduce an API into PHP where we can "interrupt" PHP contexts on-demand.

This description fits the existing VM interrupt mechanism, and how signals are delivered by pcntl currently :) EG(vm_interrupt) is checked at the points you have listed. When it's set, the VM jumps to zend_interrupt_helper, which handles signals (via zend_interrupt_function) and timeouts.

[EdmondDantes]

Hello everyone.

So, we've detached ZTS from the thread, and now the PHP VM can hop between threads like a butterfly.
What does this give us?
If we use a mechanism to interrupt the VM, then the Go scheduler will be able to stop PHP execution at any point.

This is great.
But PHP is still tied to a thread just like before.... only now it's a virtual thread.

And if we want to bind a specific PHP coroutine to a specific Go coroutine, that likely won’t be possible, because PHP code won’t be able to jump from one VM to another by crossing virtual-thread boundaries.

On the other hand, we can implement this scheme:
one Goroutine ⇒ many connections ⇒ one virtual thread ⇒ many PHP coroutines.
(I don’t know if this can be implemented on the Go side, but it probably should be possible)

I don’t see any conflicts here.
With this setup, we probably don’t even need to interrupt PHP, because the threads will be so heavily loaded that they won’t have idle time anyway. And if there is idle time, the scheduler can yield control back to Go on its own.

[EdmondDantes]

On that note: was it ever an idea to invert the 'context' approach? In other words, all global variables are still bound to the current request, but there is some kind of overreaching 'thread context' that is shared across requests. That would allow PHP applications to access globals like before, and slowly shift stuff into the overreaching 'thread context' for optimization.

The issue isn’t just about global variables. You can pass a single local variable by reference into two closures, and it becomes a problem as well. The asynchronous approach in all languages equally requires the developer to use memory correctly.

[EdmondDantes]

I think the strategy going forward would be to further simplify the initial async implementation and target a single use case, maybe remove suspension and cancellation entirely. I understand though if you are too frustrated to create another RFC version.

The future of asynchrony in PHP today depends on whether the PHP Foundation and everyone involved can unite around a single vision of the task. For several years PHP has been trying to move forward while being split into multiple groups. This brings nothing good to the language, because PHP has few developers. They shouldn’t be divided. If people can come together, everything will work out.

[withinboredom]

So, we've detached ZTS from the thread, and now the PHP VM can hop between threads like a butterfly.
What does this give us?

Nothing, currently. It wouldn’t even allow multiple requests being served on the same thread. Signals still need to be delivered to the correct context (the second orthogonal problem I presented above). This is a multi-step problem.

Even then, there’s still a long way to go. This is specifically why I’m targeting PHP 9.x. It won’t be ready by even 8.6, probably.

With this setup, we probably don’t even need to interrupt PHP, because the threads will be so heavily loaded that they won’t have idle time anyway. And if there is idle time, the scheduler can yield control back to Go on its own.

The end goal isn’t cooperative yielding (Go sends a signal when it wants to interrupt). Eventually, I’d like to have PHP interrupts tied to the Go scheduler, so that Go can interrupt a PHP runtime, shift it to another thread (if it wants to), etc. This won’t be cooperative scheduling running in a preemptive environment, but full preemption. Eventually.

Some concrete advantages even if PHP has (or doesn’t have) a userland async model:

• If a request is burning CPU, we can preempt it and let other PHP contexts run on the same thread, which results in better fairness and lower tail latency.
• When a request is cancelled, and the dev wants to continue the request, we can deprioritize the workload. Basically, a real policy-driven, preemptive scheduler.
• Eventually, this can open the door to background tasks at the VM level, such as a concurrent mark-and-sweep GC for long-running workers.

The change to TSRM doesn’t unlock any of this by itself. But without this step, none of that is even possible. This is just laying a foundation...

Like I said, we’re years from there (unless I can get my company to sponsor more than 40 hours per year + my free time to work on FrankenPHP).

[withinboredom]

@EdmondDantes you may be interested in #1981 (reply in thread) -- this is why documenting queue ordering is important (from the RFC discussion). ;)

[EdmondDantes]

@EdmondDantes you may be interested in #1981 (reply in thread) -- this is why documenting queue ordering is important (from the RFC discussion). ;)

Thank you.

As I understand it, this is about the specifics of preemptive multitasking, which PHP does not have. In this case, the programmer becomes a hostage of Go’s scheduling algorithms and is forced to take them into account, because a multithreaded environment has more variability than a single-threaded one. Meaning… the abstraction of multithreaded programming is “worse” than single-threaded, because it sits on top of a more complex system that is difficult to abstract without losing performance.

TrueAsync also has weak spots in its abstraction. For example, when trying to handle a large number of sockets using regular PHP functions, performance drops by about 10-20% compared to what could be achieved. When writing complex code, this will have to be taken into account. But this is not about coroutine ordering. It’s more about how the internal interactions are structured.

P.S.
As I understand it, in this situation you can have as many coroutines as there are workers. For example, 16.
Then only 16 coroutines will compete for the 500 connections, which changes the situation significantly.

[AlliBalliBaba]

Combining 'virtual threads' with coroutines also has its merits. This is how a 'maximum backwards compatibility' approach could look like:

• every request runs in a coroutine with its own globals
• switching the request coroutine also switches globals
• Memory across coroutines is shared explicitly via a thread-global $_SHARED_MEMORY

Won't solve all the additional complexity from async, but would be compatible with current PHP code.