Skip to content

Implement taskprocessing:worker occ command#59015

Open
Copilot wants to merge 4 commits intomasterfrom
copilot/add-taskprocessing-worker-command
Open

Implement taskprocessing:worker occ command#59015
Copilot wants to merge 4 commits intomasterfrom
copilot/add-taskprocessing-worker-command

Conversation

Copy link

Copilot AI commented Mar 17, 2026

Summary

adds a dedicated OCC worker loop for synchronous TaskProcessing providers, plus unit tests and command registration.

What changed (files)

  • Added core/Command/TaskProcessing/WorkerCommand.php
  • Modified core/register_command.php (registers the new command)
  • Modified (generated) lib/composer/composer/autoload_classmap.php
  • Modified (generated) lib/composer/composer/autoload_static.php
  • Added tests/Core/Command/TaskProcessing/WorkerCommandTest.php

What the new command does

Introduces:

  • occ taskprocessing:worker

It runs a loop that repeatedly tries to process exactly one scheduled TaskProcessing task per iteration (one task at a time per process), using the same “preferred synchronous provider” logic as OC\TaskProcessing\SynchronousBackgroundJob, but without scheduling/re-adding background jobs.

Core behavior:

  • Fetch providers via IManager::getProviders()
  • Consider only ISynchronousProvider
  • For each provider:
    • optionally filter by a task-type whitelist
    • check it’s the preferred provider for that task type (getPreferredProvider($taskTypeId), compare provider IDs)
    • fetch the next scheduled task for that task type via getNextScheduledTask([$taskTypeId])
      • NotFoundException: no task → continue
      • other TaskProcessing Exception: log error → continue
    • process it via processTask($task, $provider)
    • then return (so the loop iteration processed one task)

If no task was processed in an iteration, it sleeps for the configured interval.

It also calls abortIfInterrupted() (from OC\Core\Command\Base) to stop cleanly on SIGTERM/SIGINT, printing “Task processing worker stopped”.

Verbose output prints lines like:

  • “Processing task of type with provider ”
  • “Finished processing task ”

CLI options

  • --timeout / -t (default 0): stop after N seconds; 0 means run indefinitely.
  • --interval / -i (default 1): sleep time (seconds) when no task was processed.
  • --once: process at most one task then exit (handy for testing/one-shot runs).
  • --taskTypes (repeatable array): only process tasks for the given task type IDs; providers whose getTaskTypeId() isn’t in the list are skipped.

Tests added

tests/Core/Command/TaskProcessing/WorkerCommandTest.php adds a fairly comprehensive unit-test suite around the command loop behavior using mocks for IManager, LoggerInterface, and provider interfaces, including:

  • --once exits when no providers/tasks
  • --once processes a single task end-to-end
  • skips non-ISynchronousProvider providers
  • skips non-preferred providers
  • handles NotFoundException (no task) without calling processTask
  • logs and continues on generic Exception
  • timeout exits the loop
  • --taskTypes whitelist filtering cases (match-only, no match, empty whitelist)

Notes / things you may want to review

  • The command currently ignores the return value of processTask() (it calls it but doesn’t branch on true/false). SBJ uses the boolean to decide whether to reschedule; for this worker it may be fine, but it’s a behavioral difference worth validating.
  • Autoloader files were regenerated and included (expected in Nextcloud PRs when adding classes).

If you want, I can also point out a couple of small potential improvements (argument validation for negative interval/timeout, printing timeout message only in verbose, etc.), but the above is what the PR currently implements.

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI
Original prompt

Create a PR on nextcloud/server (base branch: master) adding a new OCC command taskprocessing:worker that behaves like a dedicated worker for TaskProcessing synchronous providers, without relying on the background job queue.

Context

  • Existing background job: lib/private/TaskProcessing/SynchronousBackgroundJob.php (commit shown in conversation) processes TaskProcessing tasks for providers that implement OCP\TaskProcessing\ISynchronousProvider, but as a QueuedJob it is removed from the jobs table during execution and re-added via IJobList::add(), which can lead to timing-dependent parallelism when using occ background-job:worker with multiple workers.
  • Goal: implement a dedicated worker command that directly pulls scheduled tasks and processes them, similar to what SynchronousBackgroundJob::run() does, running in a loop.

Requirements

  1. Add a new Symfony console command under core/ named occ taskprocessing:worker.
  2. The command must process one task at a time per process (no internal parallelism). Users can run multiple processes for parallelism.
  3. The command must run until:
    • the process is killed, OR
    • a timeout specified by a CLI parameter is reached.
  4. CLI options:
    • --timeout=<seconds>: integer seconds. 0 (or omitted) means run indefinitely.
    • --interval=<seconds>: sleep duration when no task was processed in an iteration; default should be reasonable (e.g. 1). Should be integer seconds.
    • Optional: --once to process at most one task then exit (nice to have, but not required unless it helps testing).
  5. Processing logic must mirror OC\TaskProcessing\SynchronousBackgroundJob::run($argument):
    • Retrieve providers via IManager::getProviders().
    • Consider only providers that are ISynchronousProvider.
    • For each such provider, consider it only if it is the preferred provider for its task type (IManager::getPreferredProvider($taskTypeId) and compare IDs).
    • Attempt to fetch the next scheduled task for that provider/task type with IManager::getNextScheduledTask([$taskTypeId]).
      • If NotFoundException: no task for that task type; continue.
      • If generic OCP\TaskProcessing\Exception\Exception: log error and continue.
    • Process it using IManager::processTask($task, $provider).
    • Do not schedule/re-add background jobs in this command.
  6. When no task is found/processed for any provider in an iteration, sleep for --interval seconds to avoid a busy loop.
  7. Logging and output:
    • Use injected Psr\Log\LoggerInterface for errors similar to SBJ.
    • Print to console (OutputInterface) at least which task/provider is being processed when in verbose mode.
    • Handle SIGTERM/SIGINT gracefully if feasible (optional); at minimum, the timeout and interval loop must be correct.
  8. Tests:
    • Add unit/integration tests where feasible (e.g., mocking IManager, providers, and verifying loop behavior / timeout / interval / preferred provider filtering). Keep tests fast.

Implementation hints

  • Look at existing worker loop implementation in core/Command/Background/JobWorker.php for structure (timeout/interval/once patterns). Reuse similar patterns.
  • Command should be registered as other core commands are.

Deliverables

  • New command class in core/Command/TaskProcessing/ (or similar appropriate location) with name taskprocessing:worker.
  • Wiring/registration so occ taskprocessing:worker is available.
  • Tests added/updated.
  • Documentation note in appropriate admin/occ docs if the repo has a place for it (optional but preferred).

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: As I see it, the synchronous background job is added to the job list in the taskprocessing manager upon scheduling a task, but can there be multiple entries of the same backgroundjob class in the job list? If not then, the second task will never be executed, I think. Is that correct?

Assistant: No — your conclusion (“if not, then the second task will never be executed”) is not correct for Nextcloud’s background job system as implemented here.

Can there be multiple entries of the same backgroundjob class in the job list?

Not for the same (class, argument) pair.

In lib/private/BackgroundJob/JobList.php, IJobList::add() does this:

  • It computes:
    • $class (the job class name)
    • $argumentJson = json_encode($argument)
    • an argument_hash = hash('sha256', $argumentJson)
  • Then it checks has($job, $argument):
    • has() queries the jobs table for a row with same class and same argument_hash.
  • If it does not exist: it inserts a new row.
  • If it does exist: it updates the existing row (not inserting another), resetting fields like reserved_at, last_checked, last_run.

So “enqueue again” typically means “make sure there is a job row present and runnable”, not “add a second copy”.

Does that mean a newly scheduled “...

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Copilot AI changed the title [WIP] Add OCC command for task processing worker Add occ taskprocessing:worker dedicated synchronous task processing command Mar 17, 2026
Copilot AI requested a review from marcelklehr March 17, 2026 18:37
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Copilot AI changed the title Add occ taskprocessing:worker dedicated synchronous task processing command Add --taskTypes whitelist option to taskprocessing:worker Mar 17, 2026
Co-authored-by: marcelklehr <986878+marcelklehr@users.noreply.github.com>
Copilot AI changed the title Add --taskTypes whitelist option to taskprocessing:worker taskprocessing:worker: add --taskTypes whitelist option Mar 17, 2026
declare(strict_types=1);

/**
* SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
* SPDX-FileCopyrightText: 2026 Nextcloud GmbH and Nextcloud contributors

declare(strict_types=1);

/**
* SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* SPDX-FileCopyrightText: 2024 Nextcloud GmbH and Nextcloud contributors
* SPDX-FileCopyrightText: 2026 Nextcloud GmbH and Nextcloud contributors

@marcelklehr marcelklehr requested review from icewind1991, leftybournes, nfebe and salmart-dev and removed request for a team March 17, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants