Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 30 additions & 14 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
.. image:: https://img.shields.io/pypi/pyversions/viresclient
:target: https://pypi.org/project/viresclient/
:alt: Python Version

.. image:: https://img.shields.io/pypi/v/viresclient
:target: https://pypi.org/project/viresclient/
Expand All @@ -14,27 +17,36 @@
.. image:: https://zenodo.org/badge/138034133.svg
:target: https://zenodo.org/badge/latestdoi/138034133

::

pip install viresclient

::

conda install -c conda-forge viresclient

viresclient_ is a Python package which connects to a VirES server, of which there are two: *VirES for Swarm* (https://vires.services) and *VirES for Aeolus* (https://aeolus.services), through the WPS_ interface. This package handles product requests and downloads, enabling easy access to data and models from ESA's Earth Explorer missions, Swarm_ and Aeolus_. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk (for Swarm data) or `raise an issue on GitHub`_.
``viresclient`` is a Python package which provides access to products (including on-demand processing) from two of ESA's Earth Explorer missions: `Swarm`_ and `Aeolus`_. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk or `raise an issue on GitHub`_.

.. _viresclient: https://github.com/ESA-VirES/VirES-Python-Client
.. _WPS: http://www.opengeospatial.org/standards/wps
.. _Swarm: https://earth.esa.int/eogateway/missions/swarm
.. _Aeolus: https://earth.esa.int/eogateway/missions/aeolus
.. _EOX: https://eox.at/category/vires/
.. _EOX: https://eox.at/tag/vires/
.. _`raise an issue on GitHub`: https://github.com/ESA-VirES/VirES-Python-Client/issues

For code recipes and more, see `Swarm Notebooks`_ & `Aeolus Notebooks`_. To start experimenting right away, *viresclient* is installed on the "Virtual Research Environment" (VRE), which is a managed Jupyter-based system provided for ESA by EOX. The service is free and open to all, accessible through your VirES account - check the notebooks to read more and get started.
There are two VirES services (Virtual environments for Earth Scientists) which *viresclient* can communicate with:

.. _`Swarm Notebooks`: https://notebooks.vires.services
.. _`Aeolus Notebooks`: https://notebooks.aeolus.services
- *VirES for Swarm:*

- Interact with the `VirES for Swarm graphical interface (web client) <https://vires.services>`_
- Browse code recipes: `Swarm Notebooks <https://notebooks.vires.services>`_
- JupyterHub: `Swarm VRE (Virtual Research Environment) <https://vre.vires.services>`_
- Swarm data documentation: `Swarm handbook <https://swarmhandbook.earth.esa.int>`_
- *Note that this service is not only for Swarm*:

- Multi-mission products including magnetometry from CHAMP, CryoSat-2, and more
- INTERMAGNET ground magnetometers via the ``AUX_OBS`` collection
- Custom geomagnetic model evaluation

- Read more about the ecosystem: `Python tools for ESA's Swarm mission: VirES for Swarm and surrounding ecosystem <https://doi.org/10.3389/fspas.2022.1002697>`_

- *VirES for Aeolus:*

- Interact with the `VirES for Aeolus graphical interface (web client) <https://aeolus.services>`_
- Browse code recipes: `Aeolus Notebooks <https://notebooks.aeolus.services>`_
- JupterHub: `Aeolus VRE <https://vre.aeolus.services>`_
- `Aeolus data documentation <https://earth.esa.int/eogateway/missions/aeolus/data>`_

Data and models are processed on demand on the VirES server - a combination of measurements from any time interval can be accessed. These are the same data that can be accessed by the VirES GUI. *viresclient* handles the returned data to allow direct loading as a single pandas.DataFrame_, or xarray.Dataset_.

Expand Down Expand Up @@ -109,3 +121,7 @@ You can reference *viresclient* directly using the DOI of our zenodo_ record. Vi
| "We use the Python package, viresclient [1], to access [...] from ESA's VirES for Swarm service [2]"
| [1] https://doi.org/10.5281/zenodo.2554162
| [2] https://vires.services

You can also cite this paper:

| Smith A.R.A., Pačes M. and Swarm DISC (2022) Python tools for ESA’s Swarm mission: VirES for Swarm and surrounding ecosystem. Front. Astron. Space Sci. 9:1002697. doi: 10.3389/fspas.2022.1002697
2 changes: 2 additions & 0 deletions docs/available_parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,8 @@ NB: When using model names containing a hyphen (``-``) then extra single (``'``)

----

.. _Swarm auxiliaries:

``auxiliaries``
---------------

Expand Down
19 changes: 15 additions & 4 deletions docs/capabilities.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,22 @@ VirES provides more than just *access* to data. Some operations can be peformed
| :py:meth:`viresclient.SwarmRequest.get_orbit_number`
| :py:meth:`viresclient.SwarmRequest.get_times_for_orbits`
| :py:meth:`viresclient.SwarmRequest.get_collection_info`
**Auxiliary information**
| e.g. geomagnetic coordinates (``QDLat``), orbit ascending/descending (``OrbitDirection``), geomagnetic activity (``Kp, Dst, dDst``), solar position (``SunZenithAngle``), ...
| See :ref:`Swarm auxiliaries<Swarm auxiliaries>`
**Geomagnetic model evaluation**
| Forwards evaluation of magnetic field models when a magnetic dataset is selected (e.g. ``MAGx_LR``). For more detail, see :ref:`Geomagnetic model handling`.
| :py:meth:`viresclient.SwarmRequest.available_models`
| :py:meth:`viresclient.SwarmRequest.get_model_info`
| `models` option in :py:meth:`viresclient.SwarmRequest.set_products`

- Forward evaluation of magnetic field models when a magnetic dataset is selected (e.g. ``MAGx_LR``) and using the `models` option in :py:meth:`viresclient.SwarmRequest.set_products`
- Model evaluation at user-provided coordinates. See:

- :py:meth:`viresclient.SwarmRequest.eval_model`
- :py:meth:`viresclient.SwarmRequest.eval_model_for_cdf_file`

- For more detail, see :ref:`Geomagnetic model handling`, and:

- :py:meth:`viresclient.SwarmRequest.available_models`
- :py:meth:`viresclient.SwarmRequest.get_model_info`

**Identifying conjunctions between spacecraft**
| :py:meth:`viresclient.SwarmRequest.get_conjunctions`
**Synchronous and asynchronous processing**
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
# -- Project information -----------------------------------------------------

project = "viresclient"
copyright = "2018, Ashley Smith"
copyright = "2025, Ashley Smith"
author = "Ashley Smith"


Expand Down
9 changes: 6 additions & 3 deletions docs/geomagnetic_models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ The geomagnetic models provided by VirES are all based on spherical harmonics, t

In VirES, we provide model evaluations calculated at the same sample points as the data products. This means that the spherical harmonic expansion is made at the time and location of each datapoint (e.g. in the case of ``MAGx_LR``, at every second). The main purpose is to provide the data-model residuals, or magnetic perturbations, useful for studying the external magnetic field, typically ionospheric in origin.

*Pending:* In a future VirES release, some interpolation will be used to provide the magnetic model predictions along the 50Hz ``MAGx_HR`` products (to improve speed). The predictions at the 1Hz (``MAGx_LR``) locations will be used and a cubic interpolation performed to provide the predictions at the 50Hz locations.
The magnetic data products provide the magnetic field vector as the parameter ``B_NEC`` in the geocentric NEC (North, East, Centre) frame, as well as the magnetic field intensity/magnitude (scalar), ``F``. When requesting models from VirES (by supplying the ``models`` kwarg to :py:meth:`viresclient.SwarmRequest.set_products`), the corresponding model predictions will be returned in parameters named ``B_NEC_<model-name>`` or ``F_<model-name>``. Alternatively, the data-model residual alone, named ``B_NEC_res_<model-name>`` or ``F_res_<model-name>`` can be returned directly by also supplying the kwarg ``residuals=True``. Models should be provided as a list, like ``models=["CHAOS", "IGRF"]``.

The magnetic data products provide the magnetic field vector as the parameter ``B_NEC`` in the NEC (North, East, Centre) frame, as well as the magnetic field intensity/magnitude (scalar), ``F``. When also requesting models from VirES (by supplying the ``models`` kwarg to :py:meth:`viresclient.SwarmRequest.set_products`), the corresponding model predictions will be returned in parameters named ``B_NEC_<model-name>`` or ``F_<model-name>``. Alternatively, the data-model residual alone, named ``B_NEC_res_<model-name>`` or ``F_res_<model-name>`` can be returned directly by also supplying the kwarg ``residuals=True``. Models should be provided as a list, like ``models=["CHAOS", "IGRF"]``.
When using high rate 50Hz ``MAGx_HR`` products, interpolatation is used to improve speed. The predictions at the 1Hz (``MAGx_LR``) locations are used and a cubic interpolation performed to provide the predictions at the 50Hz locations. This should be accurate enough for applications that we are aware of. To disable this interpolation, use ``do_not_interpolate_models=True`` in :py:meth:`viresclient.SwarmRequest.set_products`.

To evaluate models at arbitrary times and locations, see: :py:meth:`viresclient.SwarmRequest.eval_model` and :py:meth:`viresclient.SwarmRequest.eval_model_for_cdf_file`.

Available models
----------------
Expand Down Expand Up @@ -88,7 +90,8 @@ The predictions for these models are cached only at the positions and times defi

The logic describing when the cache is used is as follows:

.. image:: https://mermaid.ink/img/pako:eNqFkk1v2zAMhv8KoVMLND20PRlDiyYxugKxW6wekCAeBkZiEgGyFEhUt6Dpf5_8kSw7zQdDfvnyISnzQ0inSGRibdwvuUXPUE1rC-l5vChSyIB0dq030SNrZx8uhyCMRveHN0ar0KsDjM_lSQzsmgMsenF8UemGjLaUWO_kScFqDxLllh6-rPz9pD21ZRi1DWCQKTA0bfFjuXHHXVA4wPxcKd2pSv-eL78H6tk_emWxzN_RxATtkUkerC0B5jfD183fLpWjANYx0G-UbPbQIMvtWcd0vblOBjsqHp9-zr7BzjsVJUMgQ5JJXR6ZXZMvvCV_9KQBbv-JtoivJ0RLf83L6XP5lEH5UsEir-C5eJ3lRV5W-TSl3w3pt8tZahY9aMvkd850P6gFxKDtBvrehluY3y0ncaUlhF034v9yxJVoyDeoVdqNj5ZRizRGQ7XI0lHRGqPhWtT2M1njTqX7zZVm50W2RhPoSmBk97a3UmTsIx1NU40bj83JRV1S0S9ht4uffwB28dYw?type=png
.. image:: https://mermaid.ink/img/pako:eNqFUstu2zAQ_BWCpwSQDT0jWwgSxE7QHuIGSFLAjlUUNLmWCEikwUca1_a_l3ql7qk6EOTM7sysyAOmkgHO8LaSv2hJlEGv97lA7ru7WDiqQlSKLS-sIoZLcXvZk2g0ujm-GCIYUeyIZufw3Goj6yNadeDs4pXXUHEBTusdFDC02SNKaAm31xt1M292jY0hXGhUEQPaoLoxH-xmre4K9BEtz5Fv8tOlW5fr7xo67R8dslo_vJPKOtFO0sF9aaOAlmF_Cv-mZBI0EtIg-CDUVHtUE0PLs8QwLsauQIwWd19-Pj6jnZLMUoM0VEANsMtBsw35ZEpQQ40bIPqHbSS-Pp_RcU9H60cXhijEhQG1k1V7AY291VwUqPPup1zG67ndcIr0rh3hfz3YwzWomnDm7v7QaOTYxawhx5nbMtgSW5kc5-LkSok18mUvKM6MsuBhJW1R4mxLKu1Odsfc373npFCk_kR3RLxJWQ8thWqs-nYQDNRcWmFwduW3tTg74A-cBUE6TtMoTdJpFCepHwUe3js49cdX08T30yiMw2CSJicP_27lHRH7gR8nwXQyjf0wnngYGDdSLbqX3T7wIeVDy_QhT38AUx3zUQ?type=png
https://mermaid.live/edit#pako:
:alt: Flowchart showing the cache usage logic

*Custom* configured models, e.g. ``CHAOS-Static(max_degree=80)``, are not cached and must be evaluated directly.
Expand Down
137 changes: 123 additions & 14 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,123 @@ Installation and First Usage

.. note:: For VRE users (it's free! read more: `Swarm <https://notebooks.vires.services>`_, `Aeolus <https://notebooks.aeolus.services>`_), viresclient is already installed and configured so skip these steps

1. Installation
---------------
1. Installation and updating
----------------------------

Python ≥ 3.6 is required. Testing is primarily on Linux, but macOS and Windows should also work. Available through both pip and conda (conda-forge).
The package is available through both pip (PyPI) and conda (conda-forge). To add viresclient to an existing environment, use your preferred tool:

.. tabs::

.. group-tab:: pip

To install:

.. code-block:: sh

python -m pip install viresclient

To update:

.. code-block:: sh

python -m pip install --upgrade viresclient

.. tip::

If you are using an online notebook (e.g. Jupyter, Google Colab), you can run (in a notebook cell):

.. code-block:: sh

%pip install --upgrade viresclient

.. group-tab:: uv

To install:

(Assuming you use a `uv project <https://docs.astral.sh/uv/guides/projects/>`_)

.. code-block:: sh

pip install viresclient
uv add viresclient

To update:

.. code-block:: sh

uv sync --upgrade-package viresclient

.. group-tab:: conda

To install:

.. code-block:: sh

conda install --channel conda-forge viresclient

To update:

.. code-block:: sh

conda update --channel conda-forge viresclient

.. group-tab:: mamba

To install:

.. code-block:: sh

mamba install --channel conda-forge viresclient

To update:

.. code-block:: sh

mamba update --channel conda-forge viresclient


Recommended setup if starting without Python already
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There are many ways to work with Python. We recommend using conda/mamba to manage your programming environment because of the availability of many data science packages through conda.
There are many ways to work with Python. Depending on your computing environment, we recommend using either:

- `uv <https://docs.astral.sh/uv/>`_: if your project relies specifically on Python packages (available via `PyPI <https://pypi.org/>`_) and you can use a system package manager (e.g. apt) to manage non-Python dependencies
- `Mamba <https://mamba.readthedocs.io/>`_ or `conda <https://docs.conda.io/projects/conda/>`_: which also manage non-Python dependencies (from `conda-forge <https://conda-forge.org/>`_)

- Use mamba preferably. It is a drop-in replacement for conda and is separate from Anaconda Inc.
- Use conda if you have an existing Anaconda or Miniconda installation
- `Pixi <https://pixi.sh/>`_: a newer tool that provides reproducible environments with lock files; works with both conda and pip packages (integrates with uv)

This guidance is appropriate for unix-like platforms (macOS & Linux & WSL). If you are using Windows, we recommend using `Windows Subsystem for Linux (WSL) <https://learn.microsoft.com/en-us/windows/wsl/>`_. While viresclient is compatible with Windows, you will get a more consistent experience with the scientific Python ecosystem by using WSL.

.. tabs::

.. group-tab:: uv

1. `Install uv <https://docs.astral.sh/uv/getting-started/installation>`_
2. Create a new `project <https://docs.astral.sh/uv/guides/projects/>`_::

uv init --python 3.12 my-science-project

3. Enter the project and add some packages (including viresclient)::

cd my-science-project
uv add viresclient scipy matplotlib cartopy

4. Run code within the uv-managed environment. Either:

- Use `uv run <https://docs.astral.sh/uv/guides/scripts/>`_ to run scripts
- Activate the environment manually (``source .venv/bin/activate``)
- Configure your IDE to use this environment
- `Launch JupyterLab together with your project's dependencies <https://docs.astral.sh/uv/guides/integration/jupyter/>`_::

cd my-science-project
uv run --with jupyter jupyter lab

You should avoid using pip to install or update packages. By instead using ``uv add <package>``, or ``uv sync --upgrade-package <package>``, this lets uv track your project's dependencies (via the ``pyproject.toml`` file) and the exact versions used (via the ``uv.lock`` file) to help with safer updates and additions to the environment, and to ensure reproducibility.

.. group-tab:: conda

1. Install Miniconda: https://docs.conda.io/en/latest/miniconda.html
1. (If you do not have conda already) Install Miniconda: https://docs.conda.io/en/latest/miniconda.html

2. Set the conda-forge channel as the priority to install packages from::

Expand All @@ -46,28 +128,55 @@ There are many ways to work with Python. We recommend using conda/mamba to manag

You should do this to avoid mixing packages from the anaconda channel (which can result in broken environments), and try to get all packages from conda-forge where available for consistency.

3. Create a new conda environment with some recommended packages, including viresclient::
3. Create a new environment with some packages (including viresclient)::

conda create --name myenv python=3.10 jupyterlab scipy matplotlib pandas xarray cartopy h5py netCDF4 pytables ipywidgets viresclient
conda create --name myenv python=3.12 jupyterlab scipy matplotlib cartopy viresclient

4. Activate the new environment (you do this each time you want to use it)::

conda activate myenv

.. group-tab:: mamba
5. Or run a command directly from within the environment (without needing to activate it), e.g.::

conda run -n myenv jupyter lab

`Mamba <https://mamba.readthedocs.io/>`_ is a drop-in replacement for conda. You can install it into an existing (base) conda environment (``conda install -c conda-forge mamba``) and then just use ``mamba`` in place of ``conda`` in any commands - mamba is significantly faster. You can also install *mambaforge* directly to get mamba and conda-forge immediately configured in the base environment.
.. group-tab:: mamba

1. Download and install the `mambaforge installer <https://github.com/conda-forge/miniforge#mambaforge>`_ or check the `mamba documentation <https://mamba.readthedocs.io/en/latest/installation.html>`_
1. Download and install `Miniforge <https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html>`_

2. Create a new environment for your development work::
2. Create a new environment with some packages (including viresclient)::

mamba create --name myenv python=3.10 jupyterlab scipy matplotlib pandas xarray cartopy h5py netCDF4 pytables ipywidgets viresclient
mamba create --name myenv python=3.12 jupyterlab scipy matplotlib cartopy viresclient

3. Activate it to use it::
3. Activate the new environment (you do this each time you want to use it)::

mamba activate myenv

4. Or run a command directly from within the environment (without needing to activate it), e.g.::

mamba run -n myenv jupyter lab


.. group-tab:: docker

viresclient is available in several publicly available (experimental) Docker images:

- `Swarm DISC SwarmPAL processor (swarmpal-processor) <https://github.com/Swarm-DISC/SwarmPAL-processor/pkgs/container/swarmpal-processor>`_
- `Python in Heliophysics Community environment (pyhc-environment) <https://hub.docker.com/r/spolson/pyhc-environment>`_

.. group-tab:: cloud / online

Online services provide access to a range of scientific software within a Jupyter environment. viresclient may be pre-installed or can be added as above.

- `ESA Swarm VRE <https://vre.vires.services/>`_

- Run by us in connection with VirES (free self-signup, included with your VirES account)
- `ESA Datalabs <https://datalabs.esa.int/>`_

- Requires an authorised ESA Cosmos account
- `HelioCloud <https://heliocloud.org/>`_

- In development

2. First usage / Configuration
------------------------------
Expand Down
Loading
Loading