Preserve input request when adding job to broker

[EddyCMWF]

This should address the problem that the user request is modified by the normalise_request which resulted in issues related to how the request was displayed in the web-portal, and some of the integration tests @malmans2 had set up in ecmwf-datastores-client.

As far as I'm aware, we repeat all the normalise steps in the adaptor.retrieve method, therfore we can store the original request in the database with no issue.

(I looked into making the change in cads-adaptors, but that would mean that the normalise_request function has to return the input request, which is not terrbile it just means the name of the function was gramatically misleading so decided on this option)

[malmans2]

This change to the retrieve API looks OK, but it would need to be coupled with a change in the worker. At the moment, the request passed to the broker is used as input for the retrieve method run by the workers, and therefore determines the hash used for caching. As a result, if this PR is merged, the worker would need to run normalise_request (the main goal of normalise_request is to optimise the cache).

The only downside is that if normalise_request produces non-deterministic results, it would be impossible to determine the exact request that was passed to the retrieve method. I don't know the adaptors code well enough to be sure whether logic like this would conflict with the change:
https://github.com/ecmwf-projects/cads-adaptors/blob/e8f47a28f79f61374a14843cb574ca83481439b5/cads_adaptors/adaptors/cds.py#L343

Let me know if you want to go ahead with this, and we'll apply the change to the worker.

Side note: @mcucchi9 suggests that, in the long term, we might want to allow the adaptors to build and customise the snippet shown in the web portal. This would be a fairly large change and would need some thought. We can explore this when we open another round of discussion on new features and/or system changes.

[EddyCMWF]

This change to the retrieve API looks OK, but it would need to be coupled with a change in the worker. At the moment, the request passed to the broker is used as input for the retrieve method run by the workers, and therefore determines the hash used for caching. As a result, if this PR is merged, the worker would need to run normalise_request (the main goal of normalise_request is to optimise the cache).

The only downside is that if normalise_request produces non-deterministic results, it would be impossible to determine the exact request that was passed to the retrieve method. I don't know the adaptors code well enough to be sure whether logic like this would conflict with the change: https://github.com/ecmwf-projects/cads-adaptors/blob/e8f47a28f79f61374a14843cb574ca83481439b5/cads_adaptors/adaptors/cds.py#L343

Let me know if you want to go ahead with this, and we'll apply the change to the worker.

Side note: @mcucchi9 suggests that, in the long term, we might want to allow the adaptors to build and customise the snippet shown in the web portal. This would be a fairly large change and would need some thought. We can explore this when we open another round of discussion on new features and/or system changes.

Thanks @malmans2 , you're right. I'm going to transfer this conversation to Teams as there are other related aspects that could be addressed together.