Rewrite

.flake8

@@ -1,20 +1,17 @@
 [flake8]
-max_line_length = 120
 ignore =
-    # allow empty line at end of file
+    # Allow block comments to start without a space after #
+    E265,
+    # Allow empty line at end of file
     W391,
-    # break before binary operator - allow either style
-    W503,
-    # break after binary operator - allow either style
-    W504,
-    # missing whitespace around arithmetic operator
-    E226,
+    # Allow breaks before and after binary operators
+    W503, W504,
+    # Allow assigning a lambda expression instead of using a def
+    E731,
     # Allow I, l and O as variable names
     E741,
-
-
 exclude=
     .git,
     venv,
     test_data,
-    test_output,
+    pcp1,

.github/workflows/tests.yml

@@ -14,10 +14,10 @@ jobs:
     if: github.event.pull_request.draft == false
 
     env:
-        python-latest: 3.14
+        python-latest: '3.14'
     strategy:
       matrix:
-        python-version: ['3.10', 3.11, 3.12, 3.13, 3.14]
+        python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
 
     steps:
     - name: Checkout repository
@@ -31,12 +31,17 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install -e .[test]
+        python -m pip install -e .[dev]
 
     - name: Check code style with flake8
       if: ${{ matrix.python-version == env.python-latest }}
       run: |
-        python -m flake8 pcpostprocess
+        python -m flake8 -j4
+
+    - name: Check typing with mypy
+      if: ${{ matrix.python-version == env.python-latest }}
+      run: |
+        python -m mypy
 
     - name: Extract test data
       run: |

.gitignore

@@ -1,14 +1,11 @@
-# Autogenerated by setuptools-wcm
+# Autogenerated by setuptools-wcm during (developer) install
 pcpostprocess/_version.py
 
 # Tests and test data
 .coverage
 /test_data
-/test_output
-/output
 
 # Compiled python
-*.pyc
 __pycache__
 
 # Installation files
@@ -19,10 +16,7 @@ __pycache__
 
 # Virtual environments
 venv
-env
 
-# DS Store
+# Annoyances
 .DS_Store
-
-# VS code config
 .vscode

CONTRIBUTING.md

@@ -42,7 +42,7 @@ When making changes, we try to follow the procedure below.
 **TODO: Once there is a "user" way to install, move the git clone etc. information here [#105](https://github.com/CardiacModelling/pcpostprocess/issues/105).**
 
 ```
-pip install -e .[test]
+pip install -e .[dev]
 ```
 
 ## Style guidelines
@@ -62,6 +62,17 @@ In addition to the rules checked by flake8, we try to use single quotes (`'`) fo
 
 Class, method, and argument names are in UK english.
 
+### Type hints
+
+We'll try to use [type hints](https://docs.python.org/3/library/typing.html), checked with [mypy](https://www.mypy-lang.org/).
+
+To run locally, use
+```
+$ mypy
+```
+
+Mypy is configured in `pyproject.toml`.
+
 ## Documentation
 
 Every method and every class should have a [docstring](https://www.python.org/dev/peps/pep-0257/) that describes in plain terms what it does, and what the expected input and output is.
@@ -70,6 +81,48 @@ The only exception are unit test methods starting with `test_` - unit test class
 Each docstring should start with a one-line explanation.
 If more explanation is needed, this one-liner is followed by a blank line and more information in the following paragraphs.
 
+Pcpostprocess uses the [Numpy docstring syntax](https://numpydoc.readthedocs.io/en/latest/format.html) for parameters and return values.
+For example:
+
+```
+Single line explanation.
+
+Detailed, multi-line explanation.
+Possibly with ``code`` examples or even latex :math:`\sqrt{x}`.
+
+Parameters
+----------
+x : int
+    Description of parameter ``x``.
+y : float, default=1
+    Description of the optional parameter ``y``. If not set, the default value
+    1 will be used
+z : str, optional
+    Description of the optional parameter ``z``. If not given, ``z`` won't be
+    used.
+
+Returns
+-------
+float
+    Description of the returnved value
+```
+
+Return values can also be named, which is especially useful when there are multiple
+
+```
+Returns
+-------
+g: float
+    The estimated conductance, in nS.
+E : float
+    The estimated reversal potential, in mV.
+```
+
+(Note that the numpy documentation currently suggests using single backticks ``` `x` ``` when describing parameters, but Sphinx doesn't fully support this yet.
+ Until this is fully supported, we will use double backticks as in the example above).
+
+
+
 **TODO: READTHEDOCS [#60](https://github.com/CardiacModelling/pcpostprocess/issues/60)**
 
 **TODO: SYNTAX, RUNNING LOCALLY, ETC**

LICENSE

@@ -1,6 +1,6 @@
 BSD 3-Clause License
 
-Copyright (c) 2024-2025, University of Nottingham
+Copyright (c) 2024-2026, University of Nottingham
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:

README.md

@@ -5,7 +5,7 @@ This repository contains a python package and scripts for handling time-series d
 The package has been tested with data from a SyncroPatch 384, but may be adapted to work with data in other formats.
 It can also be used to perform quality control (QC) as described in [Lei et al. (2019)](https://doi.org/10.1016%2Fj.bpj.2019.07.029).
 
-This package is tested on Ubuntu with Python 3.10, 3.11, 3.12, 3.13 and 3.14.
+This package is tested on Ubuntu with Python 3.10 to 3.14.
 
 ## Getting Started
 
@@ -27,7 +27,7 @@ Then install the package with `pip`.
 
 ```sh
 python3 -m pip install --upgrade pip
-python3 -m pip install -e .'[test]'
+python3 -m pip install -e .'[dev]'
 ```
 
 To run the tests you must first download some test data.
@@ -79,72 +79,6 @@ First, all four staircase protocols are checked in all wells using the Lei et al
 Any wells not rejected are then tested using a smaller set of (less protocol specific) criteria detailed in Shuttleworth et al.
 Only wells passing all tests on all protocols are retained.
 
-### Writing an export_config.py
-
-Prior to performing QC and exporting, an `export_config.py` file should be added to the root of the data directory.
-An example file is available [here](./example_config.py).
-This should contain three entries:
-
-1. A `saveID` variable, providing a name to use in exported data. 
-2. A dictionary variable `Q2S_DC` indicating the name of the staircase protocol, and mapping it onto an more user-friendly name used in export.
-3. A dictionary variable `D2S` indicating names of other protocols to export, again mapping onto names for export.
-
-For example, in the test data (see "Getting Started") above, we have six subdirectories:
-- `staircaseramp (2)_2kHz_15.01.07`, staircase run as first protocol before E-4031
-- `StaircaseInStaircaseramp (2)_2kHz_15.01.51`, non-QC protocol
-- `staircaseramp (2)_2kHz_15.06.53`, staircase run as last protocol before E-4031
-- `staircaseramp (2)_2kHz_15.11.33`, staircase run as first protocol after E-4031 addition
-- `StaircaseInStaircaseramp (2)_2kHz_15.12.17`, non-QC protocol
-- `staircaseramp (2)_2kHz_15.17.19`, staircase run as final protocol
-
-Here each directory name is a protocol name followed by a timestamp.
-Corresponding dictionaries could be `Q2S_DC = {'staircaseramp (2)_2kHz': 'staircase'}` (indicating the staircase protocol) and `D2S = {'StaircaseInStaircaseramp (2)_2kHz': 'sis'}`.
-The saveID could be any string, but in our example we use `saveID = '13112023_MW2'`
-
-### Running
-
-To run, we call `run_herg_qc` with:
-```sh
-pcpostprocess run_herg_qc test_data/13112023_MW2_FF -o output --output_traces -w A01 A02 A03
-```
-Here:
-- `test_data/13112023_MW2_FF` is the path to the test data,
-- `-o output` tells the script to store all results in the folder `output`
-- `--output_traces` tells the script to export the data (instead of running QC only), and
-
-The last part, `-w A01 A02 A03`, tells the script to only check the first three wells.
-This speeds things up for testing, but would be omitted in normal use.
-
-To see the full set of options, use 
-
-```sh
-$ pcpostprocess run_herg_qc --help
-```
-
-### Interpreting the output
-
-After running the command above on the test data provided, the directory `output` will contain the subdirectories:
-
-- `-120mV time constant` Plots illustrating the fits performed when estimating time constants. These are output but not used in QC.
-- `leak_correction` Plots illustrating the _linear_ leak correction process in all wells passing staircase QC.
-- `qc3-bookend` Plots illustrating the "QC3 bookend" criterion used in Shuttleworth et al.
-- `reversal_plots` Plots illustrating the reversal potential estimation
-- `subtraction_plots` Plots illustrating the drug subtracted trace leak removal.
-- `traces` The exported traces (times, voltages, currents)
-
-and the files:
-
-- `chrono.txt` Lists the order in which protocols were run. For the staircase, the repeat at the end of the sequence is indicated by an added `_2`. 
-- `passed_wells.txt` Lists the wells that **passed all QC**.
-- `pcpostprocess_info.txt` Information on the run that generated this output (date, time, command used etc.)
-- `QC-13112023_MW2.csv` A CSV representation of the pass/fail results for each individual criterion
-- `QC-13112023_MW2.json` A JSON representation of the same
-- `qc_table.tex` A table in Tex format with the above.
-- `qc_vals_df.csv` Numerical values used in the final QC.
-- `selected-13112023_MW2.txt` **Preliminary QC selection** based on staircase protocol 1 (before other protocols) and 2 (after other protocols)
-- `selected-13112023_MW2-staircaseramp.txt` Preliminary selection based on staircase 1
-- `selected-13112023_MW2-staircaseramp_2.txt` Preliminary selection based on staircase 2
-- `subtraction_qc.csv` Numerical values used in the final QC and leak subtraction.
 
 ## Contributing
 

cases/proc-lei-2019.md

@@ -0,0 +1,67 @@
+# Lei 2019 procedure
+
+## Per well
+
+1. `estimate_gE_leak_ramp` (staircase) or `estimate_gE_leak_step` (validation signals)
+   - linear regression
+   - **Figure S2B** in the publication (Rapid I)
+2. `correct_gE_leak_ramp`
+    Having separate from estimation would allow estimate to be re-used on traces without ramps or suitable steps
+3. E-4031 subtraction (0.5uM E-4031 vs dimethyl vehicle)
+   **Not sure where to implement**
+   - **Figure S2A**
+4. `estimate_reversal_potential_ramp`
+   - Third order poylnomial
+   - **Figure S2C** or **Figure 10A**
+   - Output: EK
+   - In Chon's code `fit_EK_poly` in `qc/analyse-ek.py`
+5. Correct leak correction (**validation protocols only (except act/inact?**)
+   - "These leak corrections can overcorrect or undercorrect."
+   - "IKr should only be negative when the voltage is below its reversal potential, approximately -85.2 mV (NERNST)"
+   - "If the leak-corrected current showed a negative current at voltages **substantially larger** than NERNST, we concluded we overestimated leak."
+   - Most noticeable during highest V in staircase (max I leak)
+   - **For each validation proto, we specified a time window during which we believe IKr should be almost zero (please refer to our GitHub repository for detail).**
+   - To rectify the over- or undercorrection, we re-estimated the leak correction by adding an extra linear leak current of the form `g * (V + 80)`, with `g` chosen to make `mean(I_window) = 0`
+
+## QC - before time series
+
+1. `QC No cell`: Not in paper: Well not listed, or `R_seal`, `C_m`, or `R_series` is `None`
+2. `QC1.Rseal`: R_seal in [.1, 1000] GOhm
+   - Needs trace that provides R_seal, needs lower and upper bound
+3. `QC1.Cm`: C_m in [1, 100] pF
+   - Needs trace that provides C_m, needs lower and upper bound
+4. `QC1.Rseries`: R_series in [1, 25] MOhm
+   - Needs trace that provides R_series, needs lower and upper bound
+
+## QC - before subtraction
+
+1. `QC2.raw` as `snr = (np.std(c) / np.std(c[:200]))**2` and `snr < 25 or not np.isfinite(np.std(c))`
+
+
+
+
+
+
+
+## Aggregate
+
+1. EK histogram of 124 accepted wells
+   - **Figure 10B**
+
+
+
+
+
+
+X. Bar plots of number filtered out on each QC crit
+
+
+
+
+
+## Maybe
+
+- `estimate_max_conductance`
+  In Supplement to Rapid 2, showing a single exp fit to a tail current and
+  using the max as max current
+the max value

pcpostprocess/__init__.py

@@ -0,0 +1,14 @@
+#
+# This file is part of pcpostprocess.
+# See https://github.com/CardiacModelling/pcpostprocess for copyright, sharing,
+# and licensing details.
+#
+
+from ._trace import (
+    Trace,
+)
+
+from ._leak import (
+    estimate_gE_leak_ramp,
+    estimate_gE_leak_step,
+)