Support scheduling systems with non-Send resources

[maaku]

First of all, thank you for putting together a truly pleasant, and performant (the real P in APECS) entity-component-system implementation.

Has there been any thought as to how to support resources which do not implement the Send trait, and the systems which use them (that, by implication, must be run on the main thread)?

Some context: I'm coming from the Bevy ecosystem, but looking for a roll-my-own solution to state management in a non-game setting where Bevy's opinionated design makes less sense. In Bevy's ECS you can add a resource which does not implement Send (or Sync) using the World::insert_non_send_resource method, and then access it as a system parameter using the NonSend<R> or NonSendMut<R> traits. Any system which accesses non-Send resources has to run on the application's main thread, and this is of course enforced by the compiler.

Why is this useful? Well there are in fact some commonly used types which do not implement Send. Most notably, the event loop of a winit program must be created and run from the main thread, as must also some calls to the operating system / display manager on macOS and Windows. Allowing non-Send resources means you can just write a system that uses these resources, and the schedule runner automatically makes sure these run on the main thread only.

As I need to implement a winit program that needs to access non-Send resources from its input event and window management systems, I've been looking into how to do this with apecs. As near as I can tell apecs does not support this. It's easy enough to add a separate non-Send TypeMap (I used the anymap crate) to store these resources, but I wasn't sure how to handle the scheduling to run these systems on the main thread. I admit that moongraph is still a bit of a mystery to me at this point.

I'm happy to continue hacking along, but I'm wondering if anyone has considered this use case and how it might best be implemented in apecs. Maybe there's a better, easier solution that was just not obvious to me?

[schell]

Thank you for the kind words! I'm glad you're getting some use out of apecs.

Unfortunately !Send resources can't be packed into the World. But your use-case might be served well by World::visit, which allows you run a "one-off" system that visits the world with an Edges type (which are the Send resources your system can borrow from the World) and the !Send resources can be referenced from the visiting closure.

I know it's a bummer to have to store the !Send resources outside of the World but there's no way to put them in the world without either breaking the Send guarantees or erring at runtime.

[schell]

I've updated the docs for World::visit to explain a little bit about !Send systems.

[schell]

@maaku did this end up helping you at all? If so I'll close the ticket :)

[maaku]

No, it doesn’t resolve the issue. How do you have systems which takes both Send and non-Send resources, while having the ones which take only Send resources run in parallel but any with non-Send run on the main thread in strict sequence? How do you properly manage dependencies between the two types of systems, such that you can have Send systems which depend on outputs of non-Send systems, and vice versa? Bevy’s ECS handles this. I’m pretty sure the same capability within apecs would require first class support by the crate.For the moment my application is going to continue with bevy-ecs. I like apecs and would consider switching if this wasn’t such a blocker for us.On Aug 27, 2024, at 9:56 PM, Schell Carl Scivally ***@***.***> wrote: @maaku did this end up helping you at all? If so I'll close the ticket :) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[schell]

Thanks, I'll check out bevy to see how it accomplishes this.

[schell]

But just FYI, the approach that I personally use is to keep my !Send and/or !Sync resources outside of the World, and make my main loop something like this:

world.tick();

let non_send_resources = borrow_my_non_send_resources_etc();
world.visit(|send_resources| { 
    non_send_tick_fn_one(send_resources, non_send_resources);
    non_send_tick_fn_two(send_resources, non_send_resources);
    // etc.
});

So running the sync systems after world.tick() is explicit.

It works well enough, but I agree it's a bit of a wart.

[schell]

@maaku maybe you could tell me some more about your use case so I can understand why World::visit does not solve your issue? Just for example - I think you could be served well by something like this:

let non_send_resources = borrow_my_non_send_resources_etc();

let rez_a = world.visit(|send_resources| { 
    non_send_tick_fn_one(send_resources, non_send_resources)
}).unwrap();
world.add_resource(rez_a);

let rez_b = world.visit(|send_resources| { 
    non_send_tick_fn_two(send_resources, non_send_resources)
}).unwrap();
world.add_resource(rez_b);

world.tick().unwrap();