Skip to content

Conversation

@spirali
Copy link
Collaborator

@spirali spirali commented Dec 22, 2025

This PR introduces ResourceRqId (and LocalResourceRqId).

Overall idea is to replace ResourceRequestVariants (self-contained resource request description) by ResourceRqId (u32) that is an index into global table of resources (the table is updated when a new type resource request occurs; we assume that is a relatively low number of different resource requests).

To make submits self-contained, the task graph submit uses LocalResourceRqId, that is the same idea but ID is valid only within the submit. The resource request map to (local) ids is attached to the submit of the task graph. It reduce size of submits (ResourceRequests are not repeated) and when the task is attached to the server we need to resolve local ids to global ids only once, not for each task).

It is mainly preparation for the new scheduler that heavily relies on grouping by resource requests.

Advantages for the current implementation:

  • Resource requests is not copied all overall place but just u32.
  • Skipping serialization/deserialization of resource request when task is assigned to a worker.
  • Smaller submits in case of TaskGraphs

Future open questions:

resource_rq_id is now part of server Task structure (so it is slightly bigger) instead of SharedData. On the other hand it allows to share SharedData when resource request is different and it saves one dereference to share data when only resource request is needed. The server needs to access resource requests much often compared to other "shared data". But I feel a kind of schizophrenia between ResrouceRqs and SharedData.

Copy link
Member

@Kobzol Kobzol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through it commit-by-commit, so some things might be out-of-date. Looks like a great change!

@spirali spirali merged commit e84a9a2 into main Jan 2, 2026
9 checks passed
@spirali spirali deleted the resource-rq-id branch January 2, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants