Merge remote-tracking branch 'origin/claude/drop-legacy-support-rFhLT' into claude/add-type-aliases-6uN3E

Author: claude

.github/workflows/post_draft_release_published.yaml

@@ -23,7 +23,7 @@ jobs:
     strategy:
       matrix:
         include:
-          - py_ver: "3.9"
+          - py_ver: "3.10"
     runs-on: ubuntu-latest
     env:
       PY_VER: ${{matrix.py_ver}}
@@ -40,14 +40,14 @@ jobs:
       - name: Update version.py
         run: |
           VERSION=$(echo "${{ github.event.release.name }}" | grep -oP '\d+\.\d+\.\d+')
-          sed -i "s/^__version__ = .*/__version__ = \"$VERSION\"/" datajoint/version.py
-          cat datajoint/version.py
+          sed -i "s/^__version__ = .*/__version__ = \"$VERSION\"/" src/datajoint/version.py
+          cat src/datajoint/version.py
           # Commit the changes
           BRANCH_NAME="update-version-$VERSION"
           git switch -c $BRANCH_NAME
           git config --global user.name "github-actions"
           git config --global user.email "github-actions@github.com"
-          git add datajoint/version.py
+          git add src/datajoint/version.py
           git commit -m "Update version.py to $VERSION"
           echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
       - name: Update README.md badge

.github/workflows/test.yaml

@@ -21,11 +21,8 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        py_ver: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+        py_ver: ["3.10", "3.11", "3.12", "3.13"]
         mysql_ver: ["8.0"]
-        include:
-          - py_ver: "3.9"
-            mysql_ver: "5.7"
     steps:
       - uses: actions/checkout@v4
       - name: Set up Python ${{matrix.py_ver}}

docker-compose.yaml

@@ -41,7 +41,7 @@ services:
       context: .
       dockerfile: Dockerfile
       args:
-        PY_VER: ${PY_VER:-3.9}
+        PY_VER: ${PY_VER:-3.10}
         HOST_UID: ${HOST_UID:-1000}
     depends_on:
       db:

docs/src/client/credentials.md

@@ -1,46 +1,82 @@
 # Credentials
 
-Configure the connection through DataJoint's `config` object:
+Database credentials should never be stored in config files. Use environment variables or a secrets directory instead.
 
-```python
-> import datajoint as dj
-DataJoint 0.4.9 (February 1, 2017)
-No configuration found. Use `dj.config` to configure and save the configuration.
+## Environment Variables (Recommended)
+
+Set the following environment variables:
+
+```bash
+export DJ_HOST=db.example.com
+export DJ_USER=alice
+export DJ_PASS=secret
 ```
 
-You may now set the database credentials:
+These take priority over all other configuration sources.
+
+## Secrets Directory
+
+Create a `.secrets/` directory next to your `datajoint.json`:
 
-```python
-dj.config['database.host'] = "alicelab.datajoint.io"
-dj.config['database.user'] = "alice"
-dj.config['database.password'] = "haha not my real password"
+```
+myproject/
+├── datajoint.json
+└── .secrets/
+    ├── database.user      # Contains: alice
+    └── database.password  # Contains: secret
 ```
 
-Skip setting the password to make DataJoint prompt to enter the password every time.
+Each file contains a single secret value (no JSON, just the raw value).
 
-You may save the configuration in the local work directory with
-`dj.config.save_local()` or for all your projects in `dj.config.save_global()`.
-Configuration changes should be made through the `dj.config` interface; the config file
-should not be modified directly by the user.
+Add `.secrets/` to your `.gitignore`:
 
-You may leave the user or the password as `None`, in which case you will be prompted to
-enter them manually for every session.
-Setting the password as an empty string allows access without a password.
+```
+# .gitignore
+.secrets/
+```
 
-Note that the system environment variables `DJ_HOST`, `DJ_USER`, and `DJ_PASS` will
-overwrite the settings in the config file.
-You can use them to set the connection credentials instead of config files.
+## Docker / Kubernetes
 
-To change the password, the `dj.set_password` function will walk you through the
-process:
+Mount secrets at `/run/secrets/datajoint/`:
+
+```yaml
+# docker-compose.yml
+services:
+  app:
+    volumes:
+      - ./secrets:/run/secrets/datajoint:ro
+```
+
+## Interactive Prompt
+
+If credentials are not provided via environment variables or secrets, DataJoint will prompt for them when connecting:
 
 ```python
-dj.set_password()
+>>> import datajoint as dj
+>>> dj.conn()
+Please enter DataJoint username: alice
+Please enter DataJoint password:
 ```
 
-After that, update the password in the configuration and save it as described above:
+## Programmatic Access
+
+You can also set credentials in Python (useful for testing):
 
 ```python
-dj.config['database.password'] = 'my#cool!new*psswrd'
-dj.config.save_local()   # or dj.config.save_global()
+import datajoint as dj
+
+dj.config.database.user = "alice"
+dj.config.database.password = "secret"
+```
+
+Note that `password` uses `SecretStr` internally, so it will be masked in logs and repr output.
+
+## Changing Database Password
+
+To change your database password, use your database's native tools:
+
+```sql
+ALTER USER 'alice'@'%' IDENTIFIED BY 'new_password';
 ```
+
+Then update your environment variables or secrets file accordingly.

docs/src/client/install.md

@@ -1,6 +1,6 @@
 # Install and Connect
 
-DataJoint is implemented for Python 3.4+.
+DataJoint is implemented for Python 3.10+.
 You may install it from [PyPI](https://pypi.python.org/pypi/datajoint):
 
 ```bash
@@ -25,7 +25,7 @@ to connect to DataJoint pipelines.
 
 Quick install steps for advanced users are as follows:
 
-- Install latest Python 3.x and ensure it is in `PATH` (3.6.3 current at time of writing)
+- Install latest Python 3.x and ensure it is in `PATH` (3.10+ required)
   ```bash
   pip install datajoint
   ```
@@ -46,7 +46,7 @@ Python for Windows is available from:
 
 https://www.python.org/downloads/windows
 
-The latest 64 bit 3.x version, currently 3.6.3, is available from the [Python site](https://www.python.org/ftp/python/3.6.3/python-3.6.3-amd64.exe).
+The latest 64 bit 3.x version (3.10 or later required) is available from the [Python site](https://www.python.org/downloads/windows/).
 
 From here run the installer to install Python.
 

docs/src/client/settings.md

@@ -1,11 +1,166 @@
 # Configuration Settings
 
-If you are not using DataJoint on your own, or are setting up a DataJoint
-system for other users, some additional configuration options may be required
-to support [TLS](#tls-configuration) or
-[external storage](../sysadmin/external-store.md).
+DataJoint uses a strongly-typed configuration system built on [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/).
+
+## Configuration Sources
+
+Settings are loaded from the following sources (in priority order):
+
+1. **Environment variables** (`DJ_*`)
+2. **Secrets directory** (`.secrets/` or `/run/secrets/datajoint/`)
+3. **Project config file** (`datajoint.json`, searched recursively)
+4. **Default values**
+
+## Project Structure
+
+```
+myproject/
+├── .git/
+├── datajoint.json      # Project config (commit this)
+├── .secrets/           # Local secrets (add to .gitignore)
+│   ├── database.password
+│   └── aws.secret_access_key
+└── src/
+    └── analysis.py     # Config found via parent search
+```
+
+## Config File
+
+Create a `datajoint.json` file in your project root:
+
+```json
+{
+    "database": {
+        "host": "db.example.com",
+        "port": 3306
+    },
+    "stores": {
+        "raw": {
+            "protocol": "file",
+            "location": "/data/raw"
+        }
+    },
+    "display": {
+        "limit": 20
+    },
+    "safemode": true
+}
+```
+
+DataJoint searches for this file starting from the current directory and moving up through parent directories, stopping at the first `.git` or `.hg` directory (project boundary) or filesystem root.
+
+## Credentials
+
+**Never store credentials in config files.** Use one of these methods:
+
+### Environment Variables (Recommended)
+
+```bash
+export DJ_USER=alice
+export DJ_PASS=secret
+export DJ_HOST=db.example.com
+```
+
+### Secrets Directory
+
+Create files in `.secrets/` next to your `datajoint.json`:
+
+```
+.secrets/
+├── database.password    # Contains: secret
+├── database.user        # Contains: alice
+├── aws.access_key_id
+└── aws.secret_access_key
+```
+
+Add `.secrets/` to your `.gitignore`.
+
+For Docker/Kubernetes, secrets can be mounted at `/run/secrets/datajoint/`.
+
+## Accessing Settings
+
+```python
+import datajoint as dj
+
+# Attribute access (preferred)
+dj.config.database.host
+dj.config.safemode
+
+# Dict-style access
+dj.config["database.host"]
+dj.config["safemode"]
+```
+
+## Temporary Overrides
+
+Use the context manager for temporary changes:
+
+```python
+with dj.config.override(safemode=False):
+    # safemode is False here
+    table.delete()
+# safemode is restored
+```
+
+For nested settings, use double underscores:
+
+```python
+with dj.config.override(database__host="test.example.com"):
+    # database.host is temporarily changed
+    pass
+```
+
+## Available Settings
+
+### Database Connection
+
+| Setting | Environment Variable | Default | Description |
+|---------|---------------------|---------|-------------|
+| `database.host` | `DJ_HOST` | `localhost` | Database server hostname |
+| `database.port` | `DJ_PORT` | `3306` | Database server port |
+| `database.user` | `DJ_USER` | `None` | Database username |
+| `database.password` | `DJ_PASS` | `None` | Database password (use env/secrets) |
+| `database.reconnect` | — | `True` | Auto-reconnect on connection loss |
+| `database.use_tls` | — | `None` | TLS mode: `True`, `False`, or `None` (auto) |
+
+### Display
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `display.limit` | `12` | Max rows to display in previews |
+| `display.width` | `14` | Column width in previews |
+| `display.show_tuple_count` | `True` | Show total count in previews |
+
+### Other Settings
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `safemode` | `True` | Prompt before destructive operations |
+| `loglevel` | `INFO` | Logging level |
+| `fetch_format` | `array` | Default fetch format (`array` or `frame`) |
+| `enable_python_native_blobs` | `True` | Use Python-native blob serialization |
 
 ## TLS Configuration
 
-Starting with v0.12, DataJoint will by default use TLS if it is available. TLS can be
-forced on or off with the boolean `dj.config['database.use_tls']`.
+DataJoint uses TLS by default if available. Control this with:
+
+```python
+dj.config.database.use_tls = True   # Require TLS
+dj.config.database.use_tls = False  # Disable TLS
+dj.config.database.use_tls = None   # Auto (default)
+```
+
+## External Storage
+
+Configure external stores in the `stores` section. See [External Storage](../sysadmin/external-store.md) for details.
+
+```json
+{
+    "stores": {
+        "raw": {
+            "protocol": "file",
+            "location": "/data/external"
+        }
+    }
+}
+```

docs/src/develop.md

@@ -43,15 +43,15 @@ git clone https://github.com/datajoint/datajoint-python.git
 ### With Virtual Environment
 
 ```bash
-# Check if you have Python 3.9 or higher, if not please upgrade
+# Check if you have Python 3.10 or higher, if not please upgrade
 python --version
 # Create a virtual environment with venv
 python -m venv .venv
 source .venv/bin/activate
 pip install -e .[dev]
 
 # Or create a virtual environment with conda
-conda create -n dj python=3.13 # any 3.9+ is fine
+conda create -n dj python=3.13 # any 3.10+ is fine
 conda activate dj
 pip install -e .[dev]
 ```
@@ -81,7 +81,7 @@ Here are some options that provide a great developer experience:
   - Ensure you have [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
   - Ensure you have [Docker](https://docs.docker.com/get-docker/)
   - `git clone` the codebase repository and open it in VSCode
-  - Issue the following command in the terminal to build and run the Docker container: `HOST_UID=$(id -u) PY_VER=3.11 DJ_VERSION=$(grep -oP '\d+\.\d+\.\d+' datajoint/version.py) docker compose --profile test run --rm -it djtest -- sh -c 'pip install -qe ".[dev]" && bash'`
+  - Issue the following command in the terminal to build and run the Docker container: `HOST_UID=$(id -u) PY_VER=3.11 DJ_VERSION=$(grep -oP '\d+\.\d+\.\d+' src/datajoint/version.py) docker compose --profile test run --rm -it djtest -- sh -c 'pip install -qe ".[dev]" && bash'`
   - Issue the following command in the terminal to stop the Docker compose stack: `docker compose --profile test down`
 
 [Back to top](#table-of-contents)