Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ repos:
- id: check-added-large-files

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: 'v0.14.7'
rev: "v0.14.7"
hooks:
- id: ruff
- id: ruff-check
args: ["--fix", "--show-fixes"]
- id: ruff-format

Expand All @@ -51,28 +51,34 @@ repos:
- boto3-stubs[essential]
exclude: ^(diracx-client/src/diracx/client/_generated|diracx-[a-z]+/tests/|diracx-testing/|build|extensions/gubbins/gubbins-client/src/gubbins/client/_generated)

- repo: https://github.com/executablebooks/mdformat
- repo: https://github.com/hukkin/mdformat
rev: 1.0.0
hooks:
- id: mdformat
args: ["--number"]
additional_dependencies:
- mdformat-mkdocs
- mdformat-gfm
- mdformat-black
- id: mdformat
args: ["--number"]
additional_dependencies:
- mdformat-mkdocs
- mdformat-gfm

- repo: https://github.com/codespell-project/codespell
rev: v2.4.1
hooks:
- id: codespell
args: ["-w"]
- id: codespell
args: ["-w"]

- repo: https://github.com/adamchainz/blacken-docs
rev: 1.20.0
hooks:
- id: blacken-docs
additional_dependencies:
- black==22.12.0

- repo: local
hooks:
- id: check-lollygag
name: check for lollygag
types: [python]
entry: '(?i)lollygag'
entry: "(?i)lollygag"
language: pygrep
files: ^diracx

Expand All @@ -83,4 +89,4 @@ repos:
entry: pixi run -e default python .github/workflows/generate_pixi_tasks_doc.py
language: system
pass_filenames: false
files: ^pixi\.toml$|^pixi\.lock$ # only run if pixi files change
files: ^pixi\.toml$|^pixi\.lock$ # only run if pixi files change
4 changes: 2 additions & 2 deletions diracx-client/src/diracx/client/patches/jobs/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@

import json
from io import BytesIO, IOBase
from typing import Any, IO, Dict, TypedDict, Union, Unpack, cast, Literal

from typing import Any, IO, Dict, Union, Unpack, cast, Literal
from typing_extensions import TypedDict
from diracx.core.models import SearchSpec


Expand Down
3 changes: 2 additions & 1 deletion diracx-core/src/diracx/core/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@

import asyncio
import base64
from typing import TYPE_CHECKING, TypedDict, cast
from typing import TYPE_CHECKING, cast

from botocore.errorfactory import ClientError
from typing_extensions import TypedDict

from .models import ChecksumAlgorithm

Expand Down
3 changes: 3 additions & 0 deletions diracx-db/src/diracx/db/os/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,12 @@ class BaseOSDB(metaclass=ABCMeta):
MyDBClass = BaseOSDB.available_implementations(db_name)[0]

db = MyDBClass(conn_params)
await db.init()

async with db.client_context:
async with db:
# Do something with the OpenSearch client
pass
```
"""

Expand Down
1 change: 1 addition & 0 deletions diracx-db/src/diracx/db/sql/utils/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ class BaseSQLDB(metaclass=ABCMeta):
async with db:
# Do something in the first transaction
# Commit will be called automatically
pass

async with db:
# This transaction will be rolled back due to the exception
Expand Down
3 changes: 2 additions & 1 deletion docs/dev/explanations/components/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ from .utils import with_client


@with_client
async def create_sandbox(paths: list[Path], *, client: AsyncDiracClient) -> str: ...
async def create_sandbox(paths: list[Path], *, client: AsyncDiracClient) -> str:
...
```

In this example, `paths` are the parameters of the API. The `@with_client` decorator allows the method to be called without manually managing the client:
Expand Down
5 changes: 3 additions & 2 deletions docs/dev/explanations/components/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,11 @@ TODO: WRONG
- To associate the command with `dirac`, import the module in `src/diracx/__init__.py`:

```python
from . import command
from . import my_command

...

app.add_typer(<command name>.app, name="<command name>")
app.add_typer(my_command.app, name="my_command")
```

Users can then call the CLI:
Expand Down
31 changes: 23 additions & 8 deletions docs/dev/explanations/components/routes.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ Example route definition:

```python
@router.post("/search", responses=EXAMPLE_RESPONSES)
def search():
pass
```

## Dependency Injection
Expand Down Expand Up @@ -66,7 +68,8 @@ Usage example:

```python
@router.get("/openid-configuration")
async def get_openid_configuration(settings: AuthSettings): ...
async def get_openid_configuration(settings: AuthSettings):
...
```

### User Info
Expand All @@ -77,7 +80,12 @@ To retrieve information about the current user, depend on `AuthorizedUserInfo`.
@router.get("/userinfo")
async def userinfo(
user_info: Annotated[AuthorizedUserInfo, Depends(verify_dirac_access_token)],
): ...
):
...
```

```
pass
```

**TODO:** Consider avoiding the need to manually specify the annotation.
Expand All @@ -88,7 +96,8 @@ To extract information from the central DIRAC configuration:

```python
@router.post("/summary")
async def summary(config: Annotated[Config, Depends(ConfigSource.create)]): ...
async def summary(config: Annotated[Config, Depends(ConfigSource.create)]):
...
```

The `Config` object is cached efficiently between requests and automatically refreshed. It is strongly typed and immutable for the duration of a request.
Expand All @@ -106,17 +115,18 @@ from diracx.routers.dependencies import JobDB, JobLoggingDB


@router.delete("/{job_id}")
async def delete_single_job(job_db: JobDB, job_logging_db: JobLoggingDB): ...
async def delete_single_job(job_db: JobDB, job_logging_db: JobLoggingDB):
...
```

There are advanced and uncommon scenarios where committing a transaction is necessary even when returning an error response (e.g., revoking tokens in the database and returning an error to a potentially malicious user). In such cases, explicitly committing the transaction before raising an exception is crucial. Without this explicit commit, the intended changes would be rolled back along with the transaction, leading to unintended consequences:

```python
from diracx.routers.dependencies import AuthDB


@router.post("/token")
async def token(auth_db: AuthDB, ...)
...
async def token(auth_db: AuthDB):
if refresh_token_attributes["status"] == RefreshTokenStatus.REVOKED:
# Revoke all the user tokens associated with the subject
await auth_db.revoke_user_refresh_tokens(sub)
Expand All @@ -129,6 +139,8 @@ async def token(auth_db: AuthDB, ...)
raise HTTPException(status_code=401)
```

````

Refer to the [SQLAlchemy documentation](https://docs.sqlalchemy.org/en/20/core/pooling.html) for more details.

### OpenSearch Databases
Expand All @@ -142,7 +154,8 @@ from diracx.routers.dependencies import JobParametersDB


@router.post("/search", responses=EXAMPLE_RESPONSES)
async def search(job_parameters_db: JobParametersDB): ...
async def search(job_parameters_db: JobParametersDB):
...
```

## Permission Management
Expand Down Expand Up @@ -183,7 +196,8 @@ from .access_policies import open_access

@open_access
@router.get("/")
async def serve_config(): ...
async def serve_config():
...
```

Implementing a new `AccessPolicy` is done by:
Expand All @@ -206,3 +220,4 @@ To ensure consistency, the following rules must be followed:
- All routers must be tagged and the first tag becomes the name of the sub-client.
- The name of the route becomes the name of the client method.
- Uses of `fastapi.Form` must specify a `description`.
````
6 changes: 4 additions & 2 deletions docs/dev/reference/coding-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ import pytest


@pytest.fixture
def my_ficture(): ...
def my_ficture():
...
```

</td>
Expand All @@ -44,7 +45,8 @@ from pytest import fixture


@fixture
def my_ficture(): ...
def my_ficture():
...
```

</td>
Expand Down
2 changes: 1 addition & 1 deletion docs/dev/reference/dependency-injection.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ For advanced scenarios requiring explicit transaction commits (e.g., revoking to

```python
@router.post("/token")
async def token(auth_db: AuthDB, ...):
async def token(auth_db: AuthDB):
if refresh_token_attributes["status"] == RefreshTokenStatus.REVOKED:
# Revoke all the user tokens associated with the subject
await auth_db.revoke_user_refresh_tokens(sub)
Expand Down
Loading
Loading