fix issue 9424: MultiQC report handling for AWS batch with S3 workDir

[SZhengP]

MultiQC report handling for AWS batch with S3 workDir

Fix #9424

PR checklist

Closes #XXX

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the versions.yml file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • For modules:
    • nf-core modules test <MODULE> --profile docker
    • nf-core modules test <MODULE> --profile singularity
    • nf-core modules test <MODULE> --profile conda
  • For subworkflows:
    • nf-core subworkflows test <SUBWORKFLOW> --profile docker
    • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
    • nf-core subworkflows test <SUBWORKFLOW> --profile conda

[SPPearce]

@SZhengP , is this still needed?

[SZhengP]

Hi @SPPearce ,

Yes. This is still needed.
But I am hesitating which way to go to fix the nf-core lint issue.

It looks like that for some subworks (like utils_nfcore_pipeline and utils_nextflow_pipeline) the tests folder for nf-test always has:

• main.function.nf.test
• main.workflow.nf.test

For modules, pipelines and other subworkslow, the tests folder always has main.nf.test.

We can either add main.nf.test for the subworkflows or modify nf-core tools lint logic.

It is a design question.
Would like to hear some suggestions here @mashehu @mirpedrol.

[SPPearce]

I think @maxulysse is keen to remove these subworkflows entirely in favour of a plugin, does the plugin work correctly?

[maxulysse]

I've been using the plugin in question in rnavar and no issues encountered so far

[SZhengP]

The plug-in solution looks like a cleaner solution.
I took a look of the plug-in codes, which have the same issue.
I could make PR for the plug-in too.

But I wonder is there an annoucement or roadmap to switch from subworkflows to one plug-in?
As for most of nf-core pipelines, they are still using the subworkflows.
We had to made the changes internally to multiple pipelines to make sending email work with AWSbatch & S3 workDir.

I feel that before the switch is made in nf-core tool pipeline template, we do need the changes in the subworkflows too.