From f6e32943d00acf532cfbe2da2260fc04dd5d1367 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 20 Feb 2025 08:34:13 +1300 Subject: [PATCH 01/29] add StatsModels.jl to readme --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 2554142e..d5fa1c9d 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,8 @@ support such algorithms. - [StatisticalMeasures.jl](https://github.com/JuliaAI/StatisticalMeasures.jl): Package providing metrics, compatible with LearnAPI.jl +- [StatsModels.jl](https://github.com/JuliaStats/StatsModels.jl): Provides the R-style formula implementation of data preprocessing handled by [LearnDataFrontEnds.jl]((https://github.com/JuliaAI/LearnDataFrontEnds.jl) + ### Selected packages providing alternative API's The following alphabetical list of packages provide public base API's. Some provide From d06706899af70d38e308be46b052762b7593ab3f Mon Sep 17 00:00:00 2001 From: "Anthony Blaom, PhD" Date: Fri, 21 Feb 2025 11:36:25 +1300 Subject: [PATCH 02/29] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d5fa1c9d..66baf8af 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Here `learner` specifies the configuration the algorithm (the hyperparameters) w `model` stores learned parameters and any byproducts of algorithm execution. LearnAPI.jl is mostly method stubs and lots of documentation. It does not provide -meta-algorithms, such as cross-validation or hyperparameter optimization, but does aim to +meta-algorithms, such as cross-validation, hyperparameter optimization, or model composition, but does aim to support such algorithms. ## Related packages From 22d5af438bb9d170fb7be3c68f7719e4847e17a9 Mon Sep 17 00:00:00 2001 From: "Anthony Blaom, PhD" Date: Fri, 21 Feb 2025 11:38:17 +1300 Subject: [PATCH 03/29] fix formatting of link in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 66baf8af..3c0d3693 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ support such algorithms. - [StatisticalMeasures.jl](https://github.com/JuliaAI/StatisticalMeasures.jl): Package providing metrics, compatible with LearnAPI.jl -- [StatsModels.jl](https://github.com/JuliaStats/StatsModels.jl): Provides the R-style formula implementation of data preprocessing handled by [LearnDataFrontEnds.jl]((https://github.com/JuliaAI/LearnDataFrontEnds.jl) +- [StatsModels.jl](https://github.com/JuliaStats/StatsModels.jl): Provides the R-style formula implementation of data preprocessing handled by [LearnDataFrontEnds.jl](https://github.com/JuliaAI/LearnDataFrontEnds.jl) ### Selected packages providing alternative API's From 18c154dc96dd44b5135bcbc1694261f1cd949f99 Mon Sep 17 00:00:00 2001 From: "Anthony Blaom, PhD" Date: Fri, 21 Feb 2025 11:39:07 +1300 Subject: [PATCH 04/29] Update ROADMAP.md --- ROADMAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 98de84fe..0d66e90f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ "Common Implementation Patterns". As real-world implementations roll out, we could increasingly point to those instead, to conserve effort - [x] regression - - [ ] classification + - [x] classification - [ ] clustering - [x] gradient descent - [x] iterative algorithms From b406b6908ba61dfdaf702920811a0e552ed61391 Mon Sep 17 00:00:00 2001 From: anthony Date: Sat, 15 Mar 2025 10:10:46 +1300 Subject: [PATCH 05/29] minor doc tweak --- docs/src/examples.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/src/examples.md b/docs/src/examples.md index 49932084..2bde3a59 100644 --- a/docs/src/examples.md +++ b/docs/src/examples.md @@ -24,7 +24,6 @@ end Instantiate a ridge regression learner, with regularization of `lambda`. """ Ridge(; lambda=0.1) = Ridge(lambda) -LearnAPI.constructor(::Ridge) = Ridge # struct for output of `fit` struct RidgeFitted{T,F} From da305f5f14df2cd6b8a6e6c0ac81f59ee5e272cb Mon Sep 17 00:00:00 2001 From: anthony Date: Sat, 15 Mar 2025 10:18:55 +1300 Subject: [PATCH 06/29] update actions/cache to julia-actions/cache --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d71082e6..10293c22 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -29,7 +29,7 @@ jobs: with: version: ${{ matrix.version }} arch: ${{ matrix.arch }} - - uses: actions/cache@v1 + - uses: julia-actions/cache@v1 env: cache-name: cache-artifacts with: From f89288bbc3103d0c4d22c896133eacce64947239 Mon Sep 17 00:00:00 2001 From: anthony Date: Mon, 7 Apr 2025 11:00:17 +1200 Subject: [PATCH 07/29] fix a bad link --- docs/src/anatomy_of_an_implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 338f61b8..9572676c 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -414,7 +414,7 @@ The [`obs`](@ref) methods exist to: !!! important - While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnAPI.jl/dev/), we + While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we focus here on a self-contained implementation of `obs` for the ridge example above, to show how it works. From 283de3fd5192ad7c97f58babbeb14c9093ba78dd Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Tue, 13 May 2025 15:41:28 +1000 Subject: [PATCH 08/29] spelling --- docs/src/anatomy_of_an_implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 9572676c..986e2f75 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -573,7 +573,7 @@ LearnAPI.target(learner::Ridge, data) = LearnAPI.target(learner, obs(learner, da Since LearnAPI.jl provides fallbacks for `obs` that simply return the unadulterated data argument, overloading `obs` is optional. This is provided data in publicized -`fit`/`predict` signatures already consists only of objects implement the +`fit`/`predict` signatures already consists only of objects implementing the [`LearnAPI.RandomAccess`](@ref) interface (most tables¹, arrays³, and tuples thereof). To opt out of supporting the MLCore.jl interface altogether, an implementation must From 920f7a58fad16ad2fcf4af0adbf55c03bddffaf4 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Tue, 29 Jul 2025 20:13:51 +1200 Subject: [PATCH 09/29] add sees_features trait --- docs/src/kinds_of_target_proxy.md | 6 ---- docs/src/patterns/density_estimation.md | 21 +++++++++++++ docs/src/traits.md | 36 +++++++++++----------- src/features_target_weights.jl | 2 +- src/fit_update.jl | 15 ++++++---- src/traits.jl | 40 +++++++++++++++++++++++++ 6 files changed, 91 insertions(+), 29 deletions(-) diff --git a/docs/src/kinds_of_target_proxy.md b/docs/src/kinds_of_target_proxy.md index ff9d3f4b..d90095a8 100644 --- a/docs/src/kinds_of_target_proxy.md +++ b/docs/src/kinds_of_target_proxy.md @@ -14,12 +14,6 @@ LearnAPI.KindOfProxy LearnAPI.IID ``` -## Proxies for density estimation algorithms - -```@docs -LearnAPI.Single -``` - ## Joint probability distributions ```@docs diff --git a/docs/src/patterns/density_estimation.md b/docs/src/patterns/density_estimation.md index 74cad18f..a374821b 100644 --- a/docs/src/patterns/density_estimation.md +++ b/docs/src/patterns/density_estimation.md @@ -1,5 +1,26 @@ # Density Estimation +In density estimators, `fit` is trained only on [target data](@ref proxy), and +`predict(model, kind_of_proxy)` consumes no data at all. Typically `predict` returns a +single probability density/mass function (`kind_of_proxy = Distribution()`). + +Here's a sample workflow: + +```julia +model = fit(learner, y) # no features +predict(model) # shortcut for `predict(model, SingleDistribution())`, or similar +``` + +A one-liner will typically be implemented as well: + +```julia +predict(learner, y) +``` + +A density estimator, `learner`, will need to arrange that +[`LearnAPI.features(learner, data)`](@ref) always returns `nothing` and +[`LearnAPI.sees_features(learner)`](@ref) returns `false`. + See these examples from the JuliaTestAPI.jl test suite: - [normal distribution estimator](https://github.com/JuliaAI/LearnTestAPI.jl/blob/dev/src/learners/incremental_algorithms.jl) diff --git a/docs/src/traits.md b/docs/src/traits.md index 890a53f9..4ba58ec0 100644 --- a/docs/src/traits.md +++ b/docs/src/traits.md @@ -13,24 +13,25 @@ training). They may also record more mundane information, such as a package lice In the examples column of the table below, `Continuous` is a name owned the package [ScientificTypesBase.jl](https://github.com/JuliaAI/ScientificTypesBase.jl/). -| trait | return value | fallback value | example | -|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------|:---------------------------------------------------------------| -| [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | -| [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | -| [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | -| [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | -| [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | -| [`LearnAPI.pkg_name`](@ref)`(learner)` | name of package providing core code (may be different from package providing LearnAPI.jl implementation) | `"unknown"` | `"DecisionTree"` | -| [`LearnAPI.pkg_license`](@ref)`(learner)` | name of license of package providing core code | `"unknown"` | `"MIT"` | -| [`LearnAPI.doc_url`](@ref)`(learner)` | url providing documentation of the core code | `"unknown"` | `"https://en.wikipedia.org/wiki/Decision_tree_learning"` | -| [`LearnAPI.load_path`](@ref)`(learner)` | string locating name returned by `LearnAPI.constructor(learner)`, beginning with a package name | `"unknown"` | `FastTrees.LearnAPI.DecisionTreeClassifier` | -| [`LearnAPI.nonlearners`](@ref)`(learner)` | properties *not* corresponding to other learners | all properties | `(:K, :leafsize, :metric,)` | -| [`LearnAPI.human_name`](@ref)`(learner)` | human name for the learner; should be a noun | type name with spaces | "elastic net regressor" | -| [`LearnAPI.iteration_parameter`](@ref)`(learner)` | symbolic name of an iteration parameter | `nothing` | :epochs | +| trait | return value | fallback value | example | +|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------|:---------------------------------------------------------------| +| [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | +| [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | +| [`LearnAPI.sees_features`](@ref)`(learner)` | `true` unless `fit` only sees target data and `predict`/`transform` consume no data. | `true` | `false` | +| [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | +| [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | +| [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | +| [`LearnAPI.pkg_name`](@ref)`(learner)` | name of package providing core code (may be different from package providing LearnAPI.jl implementation) | `"unknown"` | `"DecisionTree"` | +| [`LearnAPI.pkg_license`](@ref)`(learner)` | name of license of package providing core code | `"unknown"` | `"MIT"` | +| [`LearnAPI.doc_url`](@ref)`(learner)` | url providing documentation of the core code | `"unknown"` | `"https://en.wikipedia.org/wiki/Decision_tree_learning"` | +| [`LearnAPI.load_path`](@ref)`(learner)` | string locating name returned by `LearnAPI.constructor(learner)`, beginning with a package name | `"unknown"` | `FastTrees.LearnAPI.DecisionTreeClassifier` | +| [`LearnAPI.nonlearners`](@ref)`(learner)` | properties *not* corresponding to other learners | all properties | `(:K, :leafsize, :metric,)` | +| [`LearnAPI.human_name`](@ref)`(learner)` | human name for the learner; should be a noun | type name with spaces | "elastic net regressor" | +| [`LearnAPI.iteration_parameter`](@ref)`(learner)` | symbolic name of an iteration parameter | `nothing` | :epochs | | [`LearnAPI.data_interface`](@ref)`(learner)` | Interface implemented by objects returned by [`obs`](@ref) | `Base.HasLength()` (supports `MLCore.getobs/numobs`) | `Base.SizeUnknown()` (supports `iterate`) | -| [`LearnAPI.fit_scitype`](@ref)`(learner)` | upper bound on `scitype(data)` ensuring `fit(learner, data)` works | `Union{}` | `Tuple{AbstractVector{Continuous}, Continuous}` | -| [`LearnAPI.target_observation_scitype`](@ref)`(learner)` | upper bound on the scitype of each observation of the targget | `Any` | `Continuous` | -| [`LearnAPI.is_static`](@ref)`(learner)` | `true` if `fit` consumes no data | `false` | `true` | +| [`LearnAPI.fit_scitype`](@ref)`(learner)` | upper bound on `scitype(data)` ensuring `fit(learner, data)` works | `Union{}` | `Tuple{AbstractVector{Continuous}, Continuous}` | +| [`LearnAPI.target_observation_scitype`](@ref)`(learner)` | upper bound on the scitype of each observation of the targget | `Any` | `Continuous` | +| [`LearnAPI.is_static`](@ref)`(learner)` | `true` if `fit` consumes no data | `false` | `true` | ### Derived Traits @@ -94,6 +95,7 @@ informative (as in `LearnAPI.target_observation_scitype(learner) = Any`). ```@docs LearnAPI.constructor LearnAPI.functions +LearnAPI.sees_features LearnAPI.kinds_of_proxy LearnAPI.tags LearnAPI.is_pure_julia diff --git a/src/features_target_weights.jl b/src/features_target_weights.jl index 578772fa..910ba32f 100644 --- a/src/features_target_weights.jl +++ b/src/features_target_weights.jl @@ -129,4 +129,4 @@ data). features(learner, data) = _first(data) _first(data) = data _first(data::Tuple) = first(data) -# note the factoring above guards against method ambiguities +# the factoring above guards against method ambiguities diff --git a/src/fit_update.jl b/src/fit_update.jl index 39f78273..5c513010 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -39,16 +39,21 @@ overloaded to return `true`. The signature must include `verbosity` with `1` as default. -If `data` encapsulates a *target* variable, as defined in LearnAPI.jl documentation, then -[`LearnAPI.target`](@ref) must be implemented. If [`predict`](@ref) or [`transform`](@ref) -are implemented and consume data, then you made need to overload -[`LearnAPI.features`](@ref). - The LearnAPI.jl specification has nothing to say regarding `fit` signatures with more than two arguments. For convenience, for example, an implementation is free to implement a slurping signature, such as `fit(learner, X, y, extras...) = fit(learner, (X, y, extras...))` but LearnAPI.jl does not guarantee such signatures are actually implemented. +## The `target`, `features` and `sees_features` methods + +If `data` encapsulates a *target* variable, as defined in LearnAPI.jl documentation, then +[`LearnAPI.target`](@ref) must be implemented. If [`predict`](@ref) or [`transform`](@ref) +are implemented and consume data, then you may need to overload +[`LearnAPI.features`](@ref). If [`predict`](@ref) or [`transform`](@ref) are implemented +and consume no data, then you must instead overload +[`LearnAPI.sees_features(learner)`](@ref) to return `false`, and overload +[`LearnAPI.features(learner, data)`](@ref) to return `nothing`. + $(DOC_DATA_INTERFACE(:fit)) """ diff --git a/src/traits.jl b/src/traits.jl index 54cdacfc..a5905e12 100644 --- a/src/traits.jl +++ b/src/traits.jl @@ -152,6 +152,46 @@ macro functions(learner) end |> esc end +""" + LearnAPI.sees_features(learner) + +Returns `false` for those learners trained only on target data, such as density +estimators; in these cases `predict(model, ...)` and `transform(model, ...)` consume no +data at all. + +More precisely, supposing `model = fit(learner, data)`, then + +- If `false` is returned, then: + + - The only possible continuations of [`predict`](@ref)`(model, ...)` are `predict(model, + ::KindOfProxy)` and `predict(model)`. + + - The only possible continuation of [`transform`](@ref)`(model, ...)` are + `transform(model)`. + + - [`LearnAPI.features(learner, data)`](@ref) returns `nothing`. + +- If `true` is returned, then: + + - The only possible continuations of [`predict`](@ref)`(model, ...)` are `predict(model, + ::KindOfProxy, data)` and `predict(model, data)`. + + - The only possible continuation of [`transform`](@ref)`(model, ...)` is + `transform(model, data)`. + + - `LearnAPI.features(learner, data)` never returns `nothing` (meaning its output is + valid data input for `predict` or `transform`, where implemented). + +See also [`LearnAPI.features`](@ref), [`fit`](@ref). + +# New implementations + +The fallback return value is `true`. Typically overloaded because `learner` is a density +estimator. + +""" +sees_features(::Any) = true + """ LearnAPI.kinds_of_proxy(learner) From 6bed975c2587320686ae0130eab33ce3e07e04e6 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Tue, 29 Jul 2025 20:14:17 +1200 Subject: [PATCH 10/29] add new KindsOfLearner trait; and forgotten tweak to IID contract --- docs/make.jl | 1 + docs/src/fit_update.md | 23 ++++- docs/src/kinds_of_learner.md | 8 ++ docs/src/reference.md | 22 ++++- src/types.jl | 185 +++++++++++++++++++++++++++-------- 5 files changed, 191 insertions(+), 48 deletions(-) create mode 100644 docs/src/kinds_of_learner.md diff --git a/docs/make.jl b/docs/make.jl index 66e71113..85b7e226 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -18,6 +18,7 @@ makedocs( "Reference" => [ "Overview" => "reference.md", "Public Names" => "list_of_public_names.md", + "Kinds of learner" => "kinds_of_learner.md", "fit/update" => "fit_update.md", "predict/transform" => "predict_transform.md", "Kinds of Target Proxy" => "kinds_of_target_proxy.md", diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index 2329d494..6caad3eb 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -1,5 +1,6 @@ # [`fit`, `update`, `update_observations`, and `update_features`](@id fit_docs) + ### Training ```julia @@ -7,9 +8,13 @@ fit(learner, data; verbosity=1) -> model fit(learner; verbosity=1) -> static_model ``` -A "static" algorithm is one that does not generalize to new observations (e.g., some -clustering algorithms); there is no training data and heavy lifting is carried out by -`predict` or `transform` which receive the data. See example below. +The first signature applies in the case `LearnAPI.kind_of(learner)` is +[`LearnAPI.Standard()`](@ref) or [`LearnAPI.Generative()`](@ref). + +The second signature applies in the case `LearnAPI.kind_of(learner) == +`[`LearnAPI.Static()`](@ref). + +Examples appear below. ### Updating @@ -20,6 +25,9 @@ update_observations(model, new_data; verbosity=..., :param1=new_value1, ...) -> update_features(model, new_data; verbosity=..., :param1=new_value1, ...) -> updated_model ``` +[`LearnAPI.Static()`](@ref) learners cannot be updated. + + ## Typical workflows ### Supervised models @@ -41,6 +49,8 @@ model = update(model; n=150) predict(model, Distribution(), X) ``` +In this case, `LearnAPI.kind_of(learner) == `[`LearnAPI.Standard()`](@ref). + See also [Classification](@ref) and [Regression](@ref). ### Transformers @@ -58,6 +68,9 @@ or, if implemented, using a single call: transform(learner, X) # `fit` implied ``` +In this case also, `LearnAPI.kind_of(learner) == `[`LearnAPI.Standard()`](@ref). + + ### [Static algorithms (no "learning")](@id static_algorithms) Suppose `learner` is some clustering algorithm that cannot be generalized to new data @@ -74,6 +87,8 @@ labels = predict(learner, X) LearnAPI.extras(model) ``` +In this case `LearnAPI.kind_of(learner) == `[`LearnAPI.Static()`](@ref). + See also [Static Algorithms](@ref) ### [Density estimation](@id density_estimation) @@ -92,6 +107,8 @@ A one-liner will typically be implemented as well: predict(learner, y) ``` +In this case `LearnAPI.kind_of(learner) == `[`LearnAPI.Generative()`](@ref). + See also [Density Estimation](@ref). diff --git a/docs/src/kinds_of_learner.md b/docs/src/kinds_of_learner.md new file mode 100644 index 00000000..58e69baf --- /dev/null +++ b/docs/src/kinds_of_learner.md @@ -0,0 +1,8 @@ +# [Kinds of learner](@id kinds_of_learner) + +```@docs +LearnAPI.KindOfLearner +LearnAPI.Standard +LearnAPI.Static +LearnAPI.Generative +``` diff --git a/docs/src/reference.md b/docs/src/reference.md index a4499e55..c0fb95ab 100644 --- a/docs/src/reference.md +++ b/docs/src/reference.md @@ -40,11 +40,11 @@ number of user-specified *hyperparameters*, such as the number of trees in a ran forest. Hyperparameters are understood in a rather broad sense. For example, one is allowed to have hyperparameters that are not data-generic. For example, a class weight dictionary, which will only make sense for a target taking values in the set of specified -dictionary keys, should be given as a hyperparameter. For simplicity and composability, -LearnAPI.jl discourages "run time" parameters (extra arguments to `fit`) such as -acceleration options (cpu/gpu/multithreading/multiprocessing). These should be included as -hyperparameters as far as possible. An exception is the compulsory `verbosity` keyword -argument of `fit`. +dictionary keys, should be given as a hyperparameter. For simplicity and easier +composability, LearnAPI.jl discourages "run time" parameters (extra arguments to `fit`) +such as acceleration options (cpu/gpu/multithreading/multiprocessing). These should be +included as hyperparameters as far as possible. An exception is the compulsory `verbosity` +keyword argument of `fit`. ### [Targets and target proxies](@id proxy) @@ -107,6 +107,18 @@ generally requires overloading `Base.==` for the struct. deep copies of RNG hyperparameters before using them in an implementation of [`fit`](@ref). + +#### Kinds of learners + +Variations in the signature patterns for `fit`/`predict`/`transform` lead to a +division of learners into three distinct kinds, articlated by the return value of the +[`LearnAPI.kind_of(learner)`](@ref) trait. For traditional supervised learners and many +transformers, this value is [`LearnAPI.Standard()`](@ref). For "one-shot" clustering +algorithms and transformers, it is [`LearnAPI.Static()`](@ref). For density estimation, +it will be [`LearnAPI.Generative()`](@ref). The precise pattern differences are detailed +[here](@ref kinds_of_learner). + + #### Composite learners (wrappers) A *composite learner* is one with at least one property that can take other learners as diff --git a/src/types.jl b/src/types.jl index ec94fdb2..0fba1ff4 100644 --- a/src/types.jl +++ b/src/types.jl @@ -1,6 +1,137 @@ +# # KIND OF LEARNER + +# see later for doc-string: +abstract type KindOfLearner end + +""" + LearnAPI.Standard + +Type with a single instance, `LearnAPI.Standard()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Standard()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner, data) +predict(model, new_data) +predict(model, kop::KindOfProxy, new_data) +transform(model, new_data) +``` + +and the one-line convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy, new_data) +transform(learner, data) +``` + +See also [`LearnAPI.Static`](@ref), [`LearnAPI.Generative`](@ref). + +""" +struct Standard <: KindOfLearner end + +""" + LearnAPI.Static + +Type with a single instance, `LearnAPI.Static()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Static()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner) # (no `data` argument) +predict(model, data) +predict(model, kop::KindOfProxy, data) +transform(model, data) +``` + +and the one-line convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy) +transform(learner, data) +``` + +See also [`LearnAPI.Standard](@ref), [`LearnAPI.Generative`](@ref). + +""" +struct Static <: KindOfLearner end + +""" + LearnAPI.Generative + +Type with a single instance, `LearnAPI.Generative()`. + +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Generative()`, then the only possible +signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing +below, or variations on these in which keyword arguments are also supplied: + +``` +model = fit(learner, data) +predict(model) +predict(model, kop::KindOfProxy) +transform(model) +``` + +and the one-liner convenience forms + +``` +predict(learner, data) +predict(learner, kop::KindOfProxy, data) +transform(learner, data) +``` + +""" +struct Generative <: KindOfLearner end + + +""" + LearnAPI.KindOfLearner + +Abstract type whose instances are the possible values of +[`LearnAPI.kind_of(learner)`](@ref). All instances of this type, and brief indications of +their interpretation, appear below. + +[`LearnAPI.Standard()`](@ref): A typical workflow looks like: + +``` +model = fit(learner, data) +predict(learner, new_data) +# or +transform(learner, new_data) +``` + +[`LearnAPI.Static()`](@ref): A typical workflow looks like: + +``` +model = fit(learner) +predict(learner, data) +# or +transform(learner, data) +``` + +[`LearnAPI.Generative()`](@ref): A typical workflow looks like: + +``` +model = fit(learner, data) +predict(learner) +# or +transform(learner) +``` + +For precise details, refer to the document strings for [`LearnAPI.Standard`](@ref), +[`LearnAPI.Static`](@ref), and [`LearnAPI.Generative`](@ref). +""" +KindOfLearner + + # # TARGET PROXIES -# see later for doc string: +# see later for doc-string: abstract type KindOfProxy end """ @@ -16,6 +147,13 @@ following must hold: - The ``j``th observation of `ŷ`, for any ``j``, depends only on the ``j``th observation of the provided `data` (no correlation between observations). +Alternatively, in the case `LearnAPI.sees_features(learner) == false` (so that +`predict(model, ...)` consumes no data, and `fit` sees only target data), one requires +only that: + +- `LearnAPI.predict(model, kind_of_proxy)` consists of a single observation (such as a + single probability distribution). + See also [`LearnAPI.KindOfProxy`](@ref). # Extended help @@ -113,40 +251,8 @@ for S in JOINT_SYMBOLS end |> eval end -""" - Single <: KindOfProxy - -Abstract subtype of [`LearnAPI.KindOfProxy`](@ref). It applies only to learners for -which `predict` has no data argument, i.e., is of the form `predict(model, -kind_of_proxy)`. An example is an algorithm learning a probability distribution from -samples, and we regard the samples as drawn from the "target" variable. If in this case, -`kind_of_proxy` is an instance of `LearnAPI.Single` then, `predict(learner)` returns a -single object representing a probability distribution. - -| type `T` | form of output of `predict(model, ::T)` | -|:--------------------------------:|:-----------------------------------------------------------------------| -| `SingleSampleable` | object that can be sampled to obtain a single target observation | -| `SingleDistribution` | explicit probability density/mass function for sampling the target | -| `SingleLogDistribution` | explicit log-probability density/mass function for sampling the target | - -""" -abstract type Single <: KindOfProxy end - -const SINGLE_SYMBOLS = [ - :SingleSampeable, - :SingleDistribution, - :SingleLogDistribution, -] - -for S in SINGLE_SYMBOLS - quote - struct $S <: Single end - end |> eval -end - const CONCRETE_TARGET_PROXY_SYMBOLS = [ IID_SYMBOLS..., - SINGLE_SYMBOLS..., JOINT_SYMBOLS..., ] @@ -166,17 +272,16 @@ are probability density/mass functions, assuming `learner = LearnAPI.learner(mod supports predictions of that form, which is true if `Distribution() in` [`LearnAPI.kinds_of_proxy(learner)`](@ref). -Proxy types are grouped under three abstract subtypes: +Proxy types are grouped under two abstract subtypes: - [`LearnAPI.IID`](@ref): The main type, for proxies consisting of uncorrelated individual - components, one for each input observation + components, one for each input observation. The type also applies to learners, such as + density estimators, that are trained on a target variable only (no features), and where + `predict` consumes no data and the returned target proxy is a single observation (e.g., + a single probability mass function) - [`LearnAPI.Joint`](@ref): For learners that predict a single probabilistic structure - encapsulating correlations between target predictions for different input observations - -- [`LearnAPI.Single`](@ref): For learners, such as density estimators, that are trained on - a target variable only (no features); `predict` consumes no data and the returned target - proxy is a single probabilistic structure. + encapsulating correlations between target predictions for different input observations. For lists of all concrete instances, refer to documentation for the relevant subtype. From 6db67eab50dedfd7b6f66ad99eb577c7c1d146d2 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 10 Aug 2025 18:55:08 +1200 Subject: [PATCH 11/29] tweak features/target/weights contracts; dump sees_features trait --- src/features_target_weights.jl | 47 ++++++++++----------- src/traits.jl | 74 ++++++++-------------------------- 2 files changed, 38 insertions(+), 83 deletions(-) diff --git a/src/features_target_weights.jl b/src/features_target_weights.jl index 910ba32f..81e90799 100644 --- a/src/features_target_weights.jl +++ b/src/features_target_weights.jl @@ -22,14 +22,16 @@ the LearnAPI.jl documentation. ## New implementations -A fallback returns `last(data)`. The method must be overloaded if [`fit`](@ref) consumes -data that includes a target variable and this fallback fails to fulfill the contract stated -above. +The method should be overloaded if [`fit`](@ref) consumes data that includes a target +variable (in the sense above). This will include both [`LearnAPI.Descriminative`](@ref) +and [`LearnAPI.Generative`](@ref) learners, but never [`LearnAPI.Static`](@ref) learners. +Implementation allows for certain meta-functionality, such as cross-validation in +supervised learning, supervised anomaly detection, and density estimation. If `obs` is being overloaded, then typically it suffices to overload `LearnAPI.target(learner, observations)` where `observations = obs(learner, data)` and `data` is any documented supported `data` in calls of the form [`fit(learner, -data)`](@ref), and to add a declaration of the form +data)`](@ref), and to then add a declaration of the form ```julia LearnAPI.target(learner, data) = LearnAPI.target(learner, obs(learner, data)) @@ -39,10 +41,10 @@ to catch all other forms of supported input `data`. Remember to ensure the return value of `LearnAPI.target` implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=true)) +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=false)) """ -target(::Any, data) = last(data) +function target end """ LearnAPI.weights(learner, data) -> weights @@ -50,9 +52,8 @@ target(::Any, data) = last(data) Return, for each form of `data` supported by the call [`fit(learner, data)`](@ref), the per-observation weights part of `data`. -The returned object has the same number of observations -as `data` has and is guaranteed to implement the data interface specified by -[`LearnAPI.data_interface(learner)`](@ref). +The returned object has the same number of observations as `data` has and is guaranteed to +implement the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). Where `nothing` is returned, weighting is understood to be uniform. @@ -60,7 +61,7 @@ Where `nothing` is returned, weighting is understood to be uniform. # New implementations -Overloading is optional. A fallback returns `nothing`. +Implementing is optional. If `obs` is being overloaded, then typically it suffices to overload `LearnAPI.weights(learner, observations)` where `observations = obs(learner, data)` and @@ -75,7 +76,7 @@ to catch all other forms of supported input `data`. Ensure the returned object, unless `nothing`, implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.weights)"; overloaded=true)) +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.weights)"; overloaded=false)) """ weights(::Any, data) = nothing @@ -86,8 +87,8 @@ weights(::Any, data) = nothing Return, for each form of `data` supported by the call [`fit(learner, data)`](@ref), the features part `X` of `data`. -While "features" will typically have the commonly understood meaning, the only -learner-generic guaranteed properties of `X` are: +While "features" will typically have the commonly understood meaning ("covariates" or +"prediuctors"), the only learner-generic guaranteed properties of `X` are: - `X` can be passed to [`predict`](@ref) or [`transform`](@ref) when these are supported by `learner`, as in the call `predict(model, X)`, where `model = fit(learner, data)`. @@ -95,18 +96,13 @@ learner-generic guaranteed properties of `X` are: - `X` has the same number of observations as `data` has and is guaranteed to implement the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). -Where `nothing` is returned, `predict` and `transform` consume no data. - # Extended help # New implementations -A fallback returns `first(data)` if `data` is a tuple, and otherwise returns `data`. The -method has no meaning for static learners (where `data` is not an argument of `fit`) and -otherwise an implementation needs to overload this method if the fallback is inadequate. - -For density estimators, whose `fit` typically consumes *only* a target variable, you -should overload this method to always return `nothing`. +Implementation of this method allows for certain meta-functionality, such as +cross-validation. It can only be implemented for [`LearnAPI.Descriminative`](@ref) +learners. If `obs` is being overloaded, then typically it suffices to overload `LearnAPI.features(learner, observations)` where `observations = obs(learner, data)` and @@ -118,15 +114,14 @@ LearnAPI.features(learner, data) = LearnAPI.features(learner, obs(learner, data) ``` to catch all other forms of supported input `data`. -Ensure the returned object, unless `nothing`, implements the data interface specified by +Ensure the returned object, implements the data interface specified by [`LearnAPI.data_interface(learner)`](@ref). `:(LearnAPI.features)` must be included in the return value of [`LearnAPI.functions(learner)`](@ref), unless the learner is static (`fit` consumes no data). +$(DOC_IMPLEMENTED_METHODS(":(LearnAPI.target)"; overloaded=false)) + """ -features(learner, data) = _first(data) -_first(data) = data -_first(data::Tuple) = first(data) -# the factoring above guards against method ambiguities +function features end diff --git a/src/traits.jl b/src/traits.jl index a5905e12..74e86bf9 100644 --- a/src/traits.jl +++ b/src/traits.jl @@ -84,23 +84,23 @@ functions owned by LearnAPI.jl. All new implementations must implement this trait. Here's a checklist for elements in the return value: -| expression | implementation compulsory? | include in returned tuple? | -|:----------------------------------|:---------------------------|:---------------------------------| -| `:(LearnAPI.fit)` | yes | yes | -| `:(LearnAPI.learner)` | yes | yes | -| `:(LearnAPI.clone)` | never overloaded | yes | -| `:(LearnAPI.strip)` | no | yes | -| `:(LearnAPI.obs)` | no | yes | -| `:(LearnAPI.features)` | no | yes, unless `learner` is static | -| `:(LearnAPI.target)` | no | only if implemented | -| `:(LearnAPI.weights)` | no | only if implemented | -| `:(LearnAPI.update)` | no | only if implemented | -| `:(LearnAPI.update_observations)` | no | only if implemented | -| `:(LearnAPI.update_features)` | no | only if implemented | -| `:(LearnAPI.predict)` | no | only if implemented | -| `:(LearnAPI.transform)` | no | only if implemented | -| `:(LearnAPI.inverse_transform)` | no | only if implemented | -| < accessor functions> | no | only if implemented | +| expression | implementation compulsory? | include in returned tuple? | +|:----------------------------------|:---------------------------|:---------------------------| +| `:(LearnAPI.fit)` | yes | yes | +| `:(LearnAPI.learner)` | yes | yes | +| `:(LearnAPI.clone)` | never overloaded | yes | +| `:(LearnAPI.strip)` | no | yes | +| `:(LearnAPI.obs)` | no | yes | +| `:(LearnAPI.features)` | no | only if implemented | +| `:(LearnAPI.target)` | no | only if implemented | +| `:(LearnAPI.weights)` | no | only if implemented | +| `:(LearnAPI.update)` | no | only if implemented | +| `:(LearnAPI.update_observations)` | no | only if implemented | +| `:(LearnAPI.update_features)` | no | only if implemented | +| `:(LearnAPI.predict)` | no | only if implemented | +| `:(LearnAPI.transform)` | no | only if implemented | +| `:(LearnAPI.inverse_transform)` | no | only if implemented | +| < accessor functions> | no | only if implemented | Also include any implemented accessor functions, both those owned by LearnaAPI.jl, and any learner-specific ones. The LearnAPI.jl accessor functions are: $ACCESSOR_FUNCTIONS_LIST @@ -152,46 +152,6 @@ macro functions(learner) end |> esc end -""" - LearnAPI.sees_features(learner) - -Returns `false` for those learners trained only on target data, such as density -estimators; in these cases `predict(model, ...)` and `transform(model, ...)` consume no -data at all. - -More precisely, supposing `model = fit(learner, data)`, then - -- If `false` is returned, then: - - - The only possible continuations of [`predict`](@ref)`(model, ...)` are `predict(model, - ::KindOfProxy)` and `predict(model)`. - - - The only possible continuation of [`transform`](@ref)`(model, ...)` are - `transform(model)`. - - - [`LearnAPI.features(learner, data)`](@ref) returns `nothing`. - -- If `true` is returned, then: - - - The only possible continuations of [`predict`](@ref)`(model, ...)` are `predict(model, - ::KindOfProxy, data)` and `predict(model, data)`. - - - The only possible continuation of [`transform`](@ref)`(model, ...)` is - `transform(model, data)`. - - - `LearnAPI.features(learner, data)` never returns `nothing` (meaning its output is - valid data input for `predict` or `transform`, where implemented). - -See also [`LearnAPI.features`](@ref), [`fit`](@ref). - -# New implementations - -The fallback return value is `true`. Typically overloaded because `learner` is a density -estimator. - -""" -sees_features(::Any) = true - """ LearnAPI.kinds_of_proxy(learner) From 35ee2673a01203f7d81c39947fcf3540e7fc5114 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 10 Aug 2025 19:34:00 +1200 Subject: [PATCH 12/29] temporarily block out LearnTestAPI docs --- docs/make.jl | 4 +- docs/src/testing_an_implementation.md | 82 ++++++++++++++------------- 2 files changed, 44 insertions(+), 42 deletions(-) diff --git a/docs/make.jl b/docs/make.jl index 85b7e226..dbe4e333 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -2,12 +2,12 @@ using Documenter using LearnAPI using ScientificTypesBase using DocumenterInterLinks -using LearnTestAPI +# using LearnTestAPI const REPO = Remotes.GitHub("JuliaAI", "LearnAPI.jl") makedocs( - modules=[LearnAPI, LearnTestAPI], + modules=[LearnAPI, ], #LearnTestAPI], format=Documenter.HTML( prettyurls = true,#get(ENV, "CI", nothing) == "true", collapselevel = 1, diff --git a/docs/src/testing_an_implementation.md b/docs/src/testing_an_implementation.md index cc0d58f6..6dd625cc 100644 --- a/docs/src/testing_an_implementation.md +++ b/docs/src/testing_an_implementation.md @@ -1,55 +1,57 @@ # Testing an Implementation -Testing is provided by the LearnTestAPI.jl package documented below. +Testing is provided by the LearnTestAPI.jl package. -## Quick start + -```@docs -LearnTestAPI -``` + -!!! warning + + + - New releases of LearnTestAPI.jl may add tests to `@testapi`, and - this may result in new failures in client package test suites, because - of previously undetected broken contracts. Adding a test to `@testapi` - is not considered a breaking change - to LearnTestAPI, unless it supports a breaking change to LearnAPI.jl. + + + + + + -## The @testapi macro -```@docs -LearnTestAPI.@testapi -``` + -## Learners for testing + + + -LearnTestAPI.jl provides some simple, tested, LearnAPI.jl implementations, which may be -useful for testing learner wrappers and meta-algorithms. + -```@docs -LearnTestAPI.Ridge -LearnTestAPI.BabyRidge -LearnTestAPI.ConstantClassifier -LearnTestAPI.TruncatedSVD -LearnTestAPI.Selector -LearnTestAPI.FancySelector -LearnTestAPI.NormalEstimator -LearnTestAPI.Ensemble -LearnTestAPI.StumpRegressor -``` + + -## Private methods + + + + + + + + + + + -For LearnTestAPI.jl developers only, and subject to breaking changes at any time: + -```@docs -LearnTestAPI.@logged_testset -LearnTestAPI.@nearly -LearnTestAPI.isnear -LearnTestAPI.learner_get -LearnTestAPI.model_get -LearnTestAPI.verb -LearnTestAPI.filter_out_verbosity -``` + + + + + + + + + + + From 146d61455bd9307e57c79f044dfa3958946c153f Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 10 Aug 2025 19:59:49 +1200 Subject: [PATCH 13/29] roll out the kind_of(learner) additions and changes --- docs/src/anatomy_of_an_implementation.md | 250 ++++++++++++----------- docs/src/examples.md | 6 +- docs/src/features_target_weights.md | 12 +- docs/src/fit_update.md | 23 ++- docs/src/kinds_of_learner.md | 2 +- docs/src/list_of_public_names.md | 9 + docs/src/patterns/density_estimation.md | 20 +- docs/src/predict_transform.md | 18 +- docs/src/reference.md | 24 +-- docs/src/traits.md | 4 +- src/fit_update.jl | 68 +++--- src/predict_transform.jl | 2 +- src/traits.jl | 13 ++ src/types.jl | 44 ++-- 14 files changed, 285 insertions(+), 210 deletions(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 986e2f75..0c23c80c 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -1,6 +1,7 @@ # Anatomy of an Implementation -The core LearnAPI.jl pattern looks like this: +LearnAPI.jl supports three core patterns. The default pattern, known as the +[`LearnAPI.Descriminative`](@ref) pattern, looks like this: ```julia model = fit(learner, data) @@ -10,38 +11,51 @@ predict(model, newdata) Here `learner` specifies [hyperparameters](@ref hyperparameters), while `model` stores learned parameters and any byproducts of algorithm execution. -Variations on this pattern: +[Transformers](@ref) ordinarily implement `transform` instead of `predict`. For more on +`predict` versus `transform`, see [Predict or transform?](@ref) -- [Transformers](@ref) ordinarily implement `transform` instead of `predict`. For more on - `predict` versus `transform`, see [Predict or transform?](@ref) +Two other `fit`/`predict`/`transform` patterns supported by LearnAPI.jl are: +[`LearnAPI.Generative`](@ref) which has the form: -- ["Static" (non-generalizing) algorithms](@ref static_algorithms), which includes some - simple transformers and some clustering algorithms, have a `fit` that consumes no - `data`. Instead `predict` or `transform` does the heavy lifting. +```julia +model = fit(learner, data) +predict(model) # a single distribution, for example +``` -- In [density estimation](@ref density_estimation), the `newdata` argument in `predict` is - missing. +and [`LearnAPI.Static`](@ref), which looks like this: + +```julia +model = fit(learner) # no `data` argument +predict(model, data) # may mutate `model` to record byproducts of computation +``` -These are the basic possibilities. +Do not read too much into the names for these patterns, which are formalized [here](@ref kinds_of_learner). They may not correspond in every case to prior conceptions. -Elaborating on the core pattern above, this tutorial details an implementation of the -LearnAPI.jl for naive [ridge regression](https://en.wikipedia.org/wiki/Ridge_regression) -with no intercept. The kind of workflow we want to enable has been previewed in [Sample -workflow](@ref). Readers can also refer to the [demonstration](@ref workflow) of the -implementation given later. +Elaborating on the very common `Descriminative` pattern above, this tutorial details an +implementation of the LearnAPI.jl for naive [ridge +regression](https://en.wikipedia.org/wiki/Ridge_regression) with no intercept. The kind of +workflow we want to enable has been previewed in [Sample workflow](@ref). Readers can also +refer to the [demonstration](@ref workflow) of the implementation given later. -## A basic implementation +!!! tip "Quick Start for new implementations" -See [here](@ref code) for code without explanations. + 1. From this tutorial, read at least "[A basic implementation](@ref)" below. + 1. Looking over the examples in "[Common Implementation Patterns](@ref patterns)", identify the apppropriate core learner pattern above for your algorithm. + 1. Implement `fit` (probably following an existing example). Read the [`fit`](@ref) document string to see what else may need to be implemented, paying particular attention to the "New implementations" section. + 3. Rinse and repeat with each new method implemented. + 4. Identify any additional [learner traits](@ref traits) that have appropriate overloadings; use the [`@trait`](@ref) macro to define these in one block. + 5. Ensure your implementation includes the compulsory method [`LearnAPI.learner`](@ref) and compulsory traits [`LearnAPI.constructor`](@ref) and [`LearnAPI.functions`](@ref). Read and apply "[Testing your implementation](@ref)". -We suppose our algorithm's `fit` method consumes data in the form `(X, y)`, where -`X` is a suitable table¹ (the features) and `y` a vector (the target). + If you get stuck, refer back to this tutorial and the [Reference](@ref reference) sections. -!!! important - Implementations wishing to support other data - patterns may need to take additional steps explained under - [Other data patterns](@ref di) below. +## A basic implementation + +See [here](@ref code) for code without explanations. + +Let us suppose our algorithm's `fit` method is to consume data in the form `(X, y)`, where +`X` is a suitable table¹ (the features, a.k.a., covariates or predictors) and `y` a vector +(the target, a.k.a., labels or response). The first line below imports the lightweight package LearnAPI.jl whose methods we will be extending. The second imports libraries needed for the core algorithm. @@ -59,7 +73,7 @@ Here's a new type whose instances specify the single ridge regression hyperparam ```@example anatomy struct Ridge{T<:Real} - lambda::T + lambda::T end nothing # hide ``` @@ -73,7 +87,7 @@ fields) that are not other learners, and we must implement ```@example anatomy """ - Ridge(; lambda=0.1) + Ridge(; lambda=0.1) Instantiate a ridge regression learner, with regularization of `lambda`. """ @@ -97,9 +111,9 @@ coefficients labelled by feature name for inspection after training: ```@example anatomy struct RidgeFitted{T,F} - learner::Ridge - coefficients::Vector{T} - named_coefficients::F + learner::Ridge + coefficients::Vector{T} + named_coefficients::F end nothing # hide ``` @@ -111,25 +125,25 @@ The implementation of `fit` looks like this: ```@example anatomy function LearnAPI.fit(learner::Ridge, data; verbosity=1) - X, y = data + X, y = data - # data preprocessing: - table = Tables.columntable(X) - names = Tables.columnnames(table) |> collect - A = Tables.matrix(table, transpose=true) + # data preprocessing: + table = Tables.columntable(X) + names = Tables.columnnames(table) |> collect + A = Tables.matrix(table, transpose=true) - lambda = learner.lambda + lambda = learner.lambda - # apply core algorithm: - coefficients = (A*A' + learner.lambda*I)\(A*y) # vector + # apply core algorithm: + coefficients = (A*A' + learner.lambda*I)\(A*y) # vector - # determine named coefficients: - named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] + # determine named coefficients: + named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] - # make some noise, if allowed: - verbosity > 0 && @info "Coefficients: $named_coefficients" + # make some noise, if allowed: + verbosity > 0 && @info "Coefficients: $named_coefficients" - return RidgeFitted(learner, coefficients, named_coefficients) + return RidgeFitted(learner, coefficients, named_coefficients) end ``` @@ -151,13 +165,29 @@ We provide this implementation for our ridge regressor: ```@example anatomy LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = - Tables.matrix(Xnew)*model.coefficients + Tables.matrix(Xnew)*model.coefficients ``` If the kind of proxy is omitted, as in `predict(model, Xnew)`, then a fallback grabs the first element of the tuple returned by [`LearnAPI.kinds_of_proxy(learner)`](@ref), which we overload appropriately below. +### Data deconstructors: `target` and `features` + +LearnAPI.jl is flexible about the form of training `data`. However, to buy into +meta-functionality, such as cross-validation, we'll need to say something about the +structure of this data. We implement [`LearnAPI.target`](@ref) to say what +part of the data consistutes a [target variable](@ref proxy), and +[`LearnAPI.features`](@ref) to say what are the features (valid `newdata` in a +`predict(model, newdata)` call): + +```@example anatomy +LearnAPI.target(learner::Ridge, (X, y)) = y +LearnAPI.features(learner::Ridge, (X, y)) = X +``` + +Another data deconstructor, for learners that support per-observation weights in training, +is [`LearnAPI.weights`](@ref). ### [Accessor functions](@id af) @@ -185,7 +215,7 @@ dump the named version of the coefficients: ```@example anatomy LearnAPI.strip(model::RidgeFitted) = - RidgeFitted(model.learner, model.coefficients, nothing) + RidgeFitted(model.learner, model.coefficients, nothing) ``` Crucially, we can still use `LearnAPI.strip(model)` in place of `model` to make new @@ -210,20 +240,20 @@ A macro provides a shortcut, convenient when multiple traits are to be defined: ```@example anatomy @trait( - Ridge, - constructor = Ridge, - kinds_of_proxy=(Point(),), - tags = ("regression",), - functions = ( - :(LearnAPI.fit), - :(LearnAPI.learner), - :(LearnAPI.clone), - :(LearnAPI.strip), - :(LearnAPI.obs), - :(LearnAPI.features), - :(LearnAPI.target), - :(LearnAPI.predict), - :(LearnAPI.coefficients), + Ridge, + constructor = Ridge, + kinds_of_proxy=(Point(),), + tags = ("regression",), + functions = ( + :(LearnAPI.fit), + :(LearnAPI.learner), + :(LearnAPI.clone), + :(LearnAPI.strip), + :(LearnAPI.obs), + :(LearnAPI.features), + :(LearnAPI.target), + :(LearnAPI.predict), + :(LearnAPI.coefficients), ) ) nothing # hide @@ -245,11 +275,7 @@ meaningfully applied to the learner or associated model, with the exception of t always include the first five you see here: `fit`, `learner`, `clone` ,`strip`, `obs`. Here [`clone`](@ref) is a utility function provided by LearnAPI that you never overload, while [`obs`](@ref) is discussed under [Providing a separate data front -end](@ref) below and is always included because it has a meaningful fallback. The -`features` method, here provided by a fallback, articulates how the features `X` can be -extracted from the training data `(X, y)`. We must also include `target` here to flag our -model as supervised; again the method itself is provided by a fallback valid in the -present case. +end](@ref) below and is always included because it has a meaningful fallback. See [`LearnAPI.functions`](@ref) for a checklist of what the `functions` trait needs to return. @@ -340,11 +366,6 @@ assumptions about data from those made above. under [Providing a separate data front end](@ref) below; or (ii) overload the trait [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. -- Where the form of data consumed by `fit` is different from that consumed by - `predict/transform` (as in classical supervised learning) it may be necessary to - explicitly overload the functions [`LearnAPI.features`](@ref) and (if supervised) - [`LearnAPI.target`](@ref). The same holds if overloading [`obs`](@ref); see below. - ## Providing a separate data front end @@ -361,31 +382,31 @@ end Ridge(; lambda=0.1) = Ridge(lambda) struct RidgeFitted{T,F} - learner::Ridge - coefficients::Vector{T} - named_coefficients::F + learner::Ridge + coefficients::Vector{T} + named_coefficients::F end LearnAPI.learner(model::RidgeFitted) = model.learner LearnAPI.coefficients(model::RidgeFitted) = model.named_coefficients LearnAPI.strip(model::RidgeFitted) = - RidgeFitted(model.learner, model.coefficients, nothing) + RidgeFitted(model.learner, model.coefficients, nothing) @trait( - Ridge, - constructor = Ridge, - kinds_of_proxy=(Point(),), - tags = ("regression",), - functions = ( - :(LearnAPI.fit), - :(LearnAPI.learner), - :(LearnAPI.clone), - :(LearnAPI.strip), - :(LearnAPI.obs), - :(LearnAPI.features), - :(LearnAPI.target), - :(LearnAPI.predict), - :(LearnAPI.coefficients), + Ridge, + constructor = Ridge, + kinds_of_proxy=(Point(),), + tags = ("regression",), + functions = ( + :(LearnAPI.fit), + :(LearnAPI.learner), + :(LearnAPI.clone), + :(LearnAPI.strip), + :(LearnAPI.obs), + :(LearnAPI.features), + :(LearnAPI.target), + :(LearnAPI.predict), + :(LearnAPI.coefficients), ) ) @@ -414,9 +435,9 @@ The [`obs`](@ref) methods exist to: !!! important - While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we - focus here on a self-contained implementation of `obs` for the ridge example above, to show - how it works. + While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we + focus here on a self-contained implementation of `obs` for the ridge example above, to show + how it works. In the typical case, where [`LearnAPI.data_interface`](@ref) is not overloaded, the alternative data representations must implement the MLCore.jl `getobs/numobs` interface @@ -459,9 +480,9 @@ introduce a new type: ```@example anatomy2 struct RidgeFitObs{T,M<:AbstractMatrix{T}} - A::M # `p` x `n` matrix - names::Vector{Symbol} # features - y::Vector{T} # target + A::M # `p` x `n` matrix + names::Vector{Symbol} # features + y::Vector{T} # target end ``` @@ -469,10 +490,10 @@ Now we overload `obs` to carry out the data preprocessing previously in `fit`, l ```@example anatomy2 function LearnAPI.obs(::Ridge, data) - X, y = data - table = Tables.columntable(X) - names = Tables.columnnames(table) |> collect - return RidgeFitObs(Tables.matrix(table)', names, y) + X, y = data + table = Tables.columntable(X) + names = Tables.columnnames(table) |> collect + return RidgeFitObs(Tables.matrix(table)', names, y) end ``` @@ -484,27 +505,27 @@ methods - one to handle "regular" input, and one to handle the pre-processed dat ```@example anatomy2 function LearnAPI.fit(learner::Ridge, observations::RidgeFitObs; verbosity=1) - lambda = learner.lambda + lambda = learner.lambda - A = observations.A - names = observations.names - y = observations.y + A = observations.A + names = observations.names + y = observations.y - # apply core learner: - coefficients = (A*A' + learner.lambda*I)\(A*y) # 1 x p matrix + # apply core learner: + coefficients = (A*A' + learner.lambda*I)\(A*y) # 1 x p matrix - # determine named coefficients: - named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] + # determine named coefficients: + named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] - # make some noise, if allowed: - verbosity > 0 && @info "Coefficients: $named_coefficients" + # make some noise, if allowed: + verbosity > 0 && @info "Coefficients: $named_coefficients" - return RidgeFitted(learner, coefficients, named_coefficients) + return RidgeFitted(learner, coefficients, named_coefficients) end LearnAPI.fit(learner::Ridge, data; kwargs...) = - fit(learner, obs(learner, data); kwargs...) + fit(learner, obs(learner, data); kwargs...) ``` ### The `obs` contract @@ -528,7 +549,7 @@ this is [`LearnAPI.RandomAccess()`](@ref) (the default) it usually suffices to o ```@example anatomy2 Base.getindex(data::RidgeFitObs, I) = - RidgeFitObs(data.A[:,I], data.names, y[I]) + RidgeFitObs(data.A[:,I], data.names, y[I]) Base.length(data::RidgeFitObs) = length(data.y) ``` @@ -539,19 +560,16 @@ LearnAPI.obs(::RidgeFitted, Xnew) = Tables.matrix(Xnew)' LearnAPI.obs(::RidgeFitted, observations::AbstractArray) = observations # involutivity LearnAPI.predict(model::RidgeFitted, ::Point, observations::AbstractMatrix) = - observations'*model.coefficients + observations'*model.coefficients LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = - predict(model, Point(), obs(model, Xnew)) + predict(model, Point(), obs(model, Xnew)) ``` -### `features` and `target` methods +### Data deconstructors: `features` and `target -Two methods [`LearnAPI.features`](@ref) and [`LearnAPI.target`](@ref) articulate how -features and target can be extracted from `data` consumed by LearnAPI.jl -methods. Fallbacks provided by LearnAPI.jl sufficed in our basic implementation -above. Here we must explicitly overload them, so that they also handle the output of -`obs(learner, data)`: +These methods must be able to handle any `data` supported by `fit`, which includes the +output of `obs(learner, data)`: ```@example anatomy2 LearnAPI.features(::Ridge, observations::RidgeFitObs) = observations.A diff --git a/docs/src/examples.md b/docs/src/examples.md index 2bde3a59..fe690ff1 100644 --- a/docs/src/examples.md +++ b/docs/src/examples.md @@ -57,6 +57,10 @@ end LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = Tables.matrix(Xnew)*model.coefficients +# data deconstructors: +LearnAPI.target(learner::Ridge, (X, y)) = y +LearnAPI.features(learner::Ridge, (X, y)) = X + # accessor functions: LearnAPI.learner(model::RidgeFitted) = model.learner LearnAPI.coefficients(model::RidgeFitted) = model.named_coefficients @@ -159,7 +163,7 @@ LearnAPI.predict(model::RidgeFitted, ::Point, observations::AbstractMatrix) = LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = predict(model, Point(), obs(model, Xnew)) -# methods to deconstruct training data: +# training data deconstructors: LearnAPI.features(::Ridge, observations::RidgeFitObs) = observations.A LearnAPI.target(::Ridge, observations::RidgeFitObs) = observations.y LearnAPI.features(learner::Ridge, data) = LearnAPI.features(learner, obs(learner, data)) diff --git a/docs/src/features_target_weights.md b/docs/src/features_target_weights.md index e2878672..c495784e 100644 --- a/docs/src/features_target_weights.md +++ b/docs/src/features_target_weights.md @@ -4,7 +4,7 @@ Methods for extracting certain parts of `data` for all supported calls of the fo [`fit(learner, data)`](@ref). ```julia -LearnAPI.features(learner, data) -> +LearnAPI.features(learner, data) -> LearnAPI.target(learner, data) -> LearnAPI.weights(learner, data) -> ``` @@ -29,11 +29,11 @@ training_loss = sum(ŷ .!= y) # Implementation guide -| method | fallback return value | compulsory? | -|:-------------------------------------------|:---------------------------------------------:|--------------------------| -| [`LearnAPI.features(learner, data)`](@ref) | `first(data)` if `data` is tuple, else `data` | if fallback insufficient | -| [`LearnAPI.target(learner, data)`](@ref) | `last(data)` | if fallback insufficient | -| [`LearnAPI.weights(learner, data)`](@ref) | `nothing` | no | +| method | fallback return value | compulsory? | +|:-------------------------------------------|:---------------------:|-------------| +| [`LearnAPI.features(learner, data)`](@ref) | no fallback | no | +| [`LearnAPI.target(learner, data)`](@ref) | no fallback | no | +| [`LearnAPI.weights(learner, data)`](@ref) | `nothing` | no | # Reference diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index 6caad3eb..40018986 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -5,14 +5,19 @@ ```julia fit(learner, data; verbosity=1) -> model -fit(learner; verbosity=1) -> static_model ``` -The first signature applies in the case `LearnAPI.kind_of(learner)` is -[`LearnAPI.Standard()`](@ref) or [`LearnAPI.Generative()`](@ref). +This is the typical `fit` pattern, applying in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns one of: + +- [`LearnAPI.Descriminative()`](@ref) +- [`LearnAPI.Generative()`](@ref) + +``` +fit(learner; verbosity=1) -> static_model +``` -The second signature applies in the case `LearnAPI.kind_of(learner) == -`[`LearnAPI.Static()`](@ref). +This pattern applies in the case [`LearnAPI.kind_of(learner)`](@ref) returns [`LearnAPI.Static()`](@ref). Examples appear below. @@ -28,7 +33,7 @@ update_features(model, new_data; verbosity=..., :param1=new_value1, ...) -> upda [`LearnAPI.Static()`](@ref) learners cannot be updated. -## Typical workflows +## [Typical workflows](@id fit_workflows) ### Supervised models @@ -49,7 +54,7 @@ model = update(model; n=150) predict(model, Distribution(), X) ``` -In this case, `LearnAPI.kind_of(learner) == `[`LearnAPI.Standard()`](@ref). +In this case, `LearnAPI.kind_of(learner) == `[`LearnAPI.Descriminative()`](@ref). See also [Classification](@ref) and [Regression](@ref). @@ -68,7 +73,7 @@ or, if implemented, using a single call: transform(learner, X) # `fit` implied ``` -In this case also, `LearnAPI.kind_of(learner) == `[`LearnAPI.Standard()`](@ref). +In this case also, `LearnAPI.kind_of(learner) == `[`LearnAPI.Descriminative()`](@ref). ### [Static algorithms (no "learning")](@id static_algorithms) @@ -98,7 +103,7 @@ which consumes no data, returns the learned density: ```julia model = fit(learner, y) # no features -predict(model) # shortcut for `predict(model, SingleDistribution())`, or similar +predict(model) # shortcut for `predict(model, Distribution())`, or similar ``` A one-liner will typically be implemented as well: diff --git a/docs/src/kinds_of_learner.md b/docs/src/kinds_of_learner.md index 58e69baf..bda42d91 100644 --- a/docs/src/kinds_of_learner.md +++ b/docs/src/kinds_of_learner.md @@ -2,7 +2,7 @@ ```@docs LearnAPI.KindOfLearner -LearnAPI.Standard +LearnAPI.Descriminative LearnAPI.Static LearnAPI.Generative ``` diff --git a/docs/src/list_of_public_names.md b/docs/src/list_of_public_names.md index 0e224fe2..f0359508 100644 --- a/docs/src/list_of_public_names.md +++ b/docs/src/list_of_public_names.md @@ -8,6 +8,8 @@ - [`update_observations`](@ref) +- [`update_features`](@ref) + - [`predict`](@ref) - [`transform`](@ref) @@ -34,6 +36,13 @@ See [here](@ref accessor_functions). See [here](@ref traits). +## Kinds of learner + +- [`LearnAPI.Descriminative`](@ref) + +- [`LearnAPI.Static`](@ref) + +- [`LearnAPI.Generative`](@ref) ## Kinds of target proxy diff --git a/docs/src/patterns/density_estimation.md b/docs/src/patterns/density_estimation.md index a374821b..0e60a3e8 100644 --- a/docs/src/patterns/density_estimation.md +++ b/docs/src/patterns/density_estimation.md @@ -1,26 +1,26 @@ # Density Estimation In density estimators, `fit` is trained only on [target data](@ref proxy), and -`predict(model, kind_of_proxy)` consumes no data at all. Typically `predict` returns a -single probability density/mass function (`kind_of_proxy = Distribution()`). +`predict(model, kind_of_proxy)` consumes no data at all, a pattern flagged by the +identities [`LearnAPI.target(learner, y)`](@ref)` == y` and +[`LearnAPI.kind_of(learner)`](@ref)` == `[`LearnAPI.Generative()`](@ref). -Here's a sample workflow: + +Typically `predict` returns a single probability density/mass function. Here's a sample +workflow: ```julia model = fit(learner, y) # no features -predict(model) # shortcut for `predict(model, SingleDistribution())`, or similar +predict(model) # shortcut for `predict(model, Distribution())`, or similar ``` -A one-liner will typically be implemented as well: +A one-line convenience method will typically be implemented as well: ```julia predict(learner, y) ``` -A density estimator, `learner`, will need to arrange that -[`LearnAPI.features(learner, data)`](@ref) always returns `nothing` and -[`LearnAPI.sees_features(learner)`](@ref) returns `false`. - -See these examples from the JuliaTestAPI.jl test suite: +However, having the multi-line workflow enables the possibility of updating the model with +new data. See this example from the JuliaTestAPI.jl test suite: - [normal distribution estimator](https://github.com/JuliaAI/LearnTestAPI.jl/blob/dev/src/learners/incremental_algorithms.jl) diff --git a/docs/src/predict_transform.md b/docs/src/predict_transform.md index 2733fc64..a7db8251 100644 --- a/docs/src/predict_transform.md +++ b/docs/src/predict_transform.md @@ -1,12 +1,26 @@ # [`predict`, `transform` and `inverse_transform`](@id operations) ```julia -predict(model, kind_of_proxy, data) +predict(model, [kind_of_proxy,] data) transform(model, data) inverse_transform(model, data) ``` -Versions without the `data` argument may apply, for example in [density +The above signatures apply in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns one of: + +- [`LearnAPI.Descriminative()`](@ref) +- [`LearnAPI.Static()`](@ref) + + +```julia +predict(model[, kind_of_proxy]) +transform(model) +inverse_transform(model) +``` + +The above signatures apply in the case that [`LearnAPI.kind_of(learner)`](@ref) +returns [`LearnAPI.Generative()`](@ref), as in [density estimation](@ref density_estimation). ## [Typical worklows](@id predict_workflow) diff --git a/docs/src/reference.md b/docs/src/reference.md index c0fb95ab..e77d8cdd 100644 --- a/docs/src/reference.md +++ b/docs/src/reference.md @@ -108,15 +108,13 @@ generally requires overloading `Base.==` for the struct. [`fit`](@ref). -#### Kinds of learners +#### Kinds of learner -Variations in the signature patterns for `fit`/`predict`/`transform` lead to a -division of learners into three distinct kinds, articlated by the return value of the -[`LearnAPI.kind_of(learner)`](@ref) trait. For traditional supervised learners and many -transformers, this value is [`LearnAPI.Standard()`](@ref). For "one-shot" clustering -algorithms and transformers, it is [`LearnAPI.Static()`](@ref). For density estimation, -it will be [`LearnAPI.Generative()`](@ref). The precise pattern differences are detailed -[here](@ref kinds_of_learner). +As previewed in [Anatomy of an Implementation](@ref), different +`fit`/`predict`/`transform` patterns lead to a division of learners into three distinct +kinds, [`LearnAPI.Descriminative()`](@ref), [`LearnAPI.Generative`](@ref), and +[`LearnAPI.Static`](@ref), which is detailed [here](@ref kinds_of_learner). See also +[these workflows](@ref fit_workflows) for concrete examples. #### Composite learners (wrappers) @@ -134,19 +132,19 @@ Below is an example of a learner type with a valid constructor: ```julia struct GradientRidgeRegressor{T<:Real} - learning_rate::T - epochs::Int - l2_regularization::T + learning_rate::T + epochs::Int + l2_regularization::T end """ - GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) + GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) Instantiate a gradient ridge regressor with the specified hyperparameters. """ GradientRidgeRegressor(; learning_rate=0.01, epochs=10, l2_regularization=0.01) = - GradientRidgeRegressor(learning_rate, epochs, l2_regularization) + GradientRidgeRegressor(learning_rate, epochs, l2_regularization) LearnAPI.constructor(::GradientRidgeRegressor) = GradientRidgeRegressor ``` diff --git a/docs/src/traits.md b/docs/src/traits.md index 4ba58ec0..b22a0462 100644 --- a/docs/src/traits.md +++ b/docs/src/traits.md @@ -17,7 +17,7 @@ In the examples column of the table below, `Continuous` is a name owned the pack |:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------|:---------------------------------------------------------------| | [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | | [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | -| [`LearnAPI.sees_features`](@ref)`(learner)` | `true` unless `fit` only sees target data and `predict`/`transform` consume no data. | `true` | `false` | +| [`LearnAPI.kind_of`](@ref)`(learner)` | the `fit`/`predict`/`transform` pattern used by `learner` | `LearnAPI.Static()` | `LearnAPI.Descriminative()` | | [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | | [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | | [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | @@ -95,7 +95,7 @@ informative (as in `LearnAPI.target_observation_scitype(learner) = Any`). ```@docs LearnAPI.constructor LearnAPI.functions -LearnAPI.sees_features +LearnAPI.kind_of LearnAPI.kinds_of_proxy LearnAPI.tags LearnAPI.is_pure_julia diff --git a/src/fit_update.jl b/src/fit_update.jl index 5c513010..b72600a4 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -4,11 +4,11 @@ fit(learner, data; verbosity=1) fit(learner; verbosity=1) -Execute the machine learning or statistical algorithm with configuration `learner` using -the provided training `data`, returning an object, `model`, on which other methods, such -as [`predict`](@ref) or [`transform`](@ref), can be dispatched. -[`LearnAPI.functions(learner)`](@ref) returns a list of methods that can be applied to -either `learner` or `model`. +In the case of the first signature, execute the machine learning or statistical algorithm +with configuration `learner` using the provided training `data`, returning an object, +`model`, on which other methods, such as [`predict`](@ref) or [`transform`](@ref), can be +dispatched. [`LearnAPI.functions(learner)`](@ref) returns a list of methods that can be +applied to either `learner` or `model`. For example, a supervised classifier might have a workflow like this: @@ -17,42 +17,53 @@ model = fit(learner, (X, y)) ŷ = predict(model, Xnew) ``` -The signature `fit(learner; verbosity=...)` (no `data`) is provided by learners that do -not generalize to new observations (called *static algorithms*). In that case, -`transform(model, data)` or `predict(model, ..., data)` carries out the actual algorithm -execution, writing any byproducts of that operation to the mutable object `model` returned -by `fit`. Inspect the value of [`LearnAPI.is_static(learner)`](@ref) to determine whether -`fit` consumes `data` or not. - Use `verbosity=0` for warnings only, and `-1` for silent training. -See also [`predict`](@ref), [`transform`](@ref), -[`inverse_transform`](@ref), [`LearnAPI.functions`](@ref), [`obs`](@ref). +This `fit` signature applies to all learners for which [`LearnAPI.kind_of(learner)`](@ref) +returns [`LearnAPI.Descriminative()`](@ref) or [`LearnAPI.Generative()`](@ref). + +# Static learners + +In the case of a learner that does not generalize to new data, the second `fit` signature +can be used to wrap the `learner` in an object called `model` that the calls +`transform(model, data)` or `predict(model, ..., data)` may mutate, so as to record +byproducts of the core algorithm specified by `learner`, before returning the outcomes of +primary interest. + +Here's a sample workflow: + +```julia +model = fit(learner) # e.g, `learner` specifies DBSCAN clustering parameters +labels = predict(model, X) # compute and return cluster labels for `X` +LearnAPI.extras(model) # return outliers in the data `X` +``` +This `fit` signature applies to all learners for which +[`LearnAPI.kind_of(learner)`](@ref)` == `[`LearnAPI.Static()`](@ref). + +See also [`predict`](@ref), [`transform`](@ref), [`inverse_transform`](@ref), +[`LearnAPI.functions`](@ref), [`obs`](@ref), [`LearnAPI.kind_of`](@ref). # Extended help # New implementations -Implementation of exactly one of the signatures is compulsory. If `fit(learner; -verbosity=...)` is implemented, then the trait [`LearnAPI.is_static`](@ref) must be -overloaded to return `true`. +Implementation of exactly one of the signatures is compulsory. Unless implementing the +[`LearnAPI.Descriminative()`](@ref) `fit`/`predict`/`transform` pattern, +[`LearnAPI.kind_of(learner)`](@ref) will need to be suitably overloaded. -The signature must include `verbosity` with `1` as default. +The `fit` signature must include `verbosity` with `1` as default. The LearnAPI.jl specification has nothing to say regarding `fit` signatures with more than two arguments. For convenience, for example, an implementation is free to implement a slurping signature, such as `fit(learner, X, y, extras...) = fit(learner, (X, y, extras...))` but LearnAPI.jl does not guarantee such signatures are actually implemented. -## The `target`, `features` and `sees_features` methods +## The `target` and `features` methods -If `data` encapsulates a *target* variable, as defined in LearnAPI.jl documentation, then -[`LearnAPI.target`](@ref) must be implemented. If [`predict`](@ref) or [`transform`](@ref) -are implemented and consume data, then you may need to overload -[`LearnAPI.features`](@ref). If [`predict`](@ref) or [`transform`](@ref) are implemented -and consume no data, then you must instead overload -[`LearnAPI.sees_features(learner)`](@ref) to return `false`, and overload -[`LearnAPI.features(learner, data)`](@ref) to return `nothing`. +If [`LearnAPI.kind_of(learner)`](@ref) returns [`LearnAPI.Descriminative()`](@ref) or +[`LearnAPI.Generative()`](@ref) then the methods [`LearnAPI.target`](@ref) and/or +[`LearnAPI.features`](@ref), which deconstruct the form of `data` consumed by `fit`, may +require overloading. Refer to their document strings for details. $(DOC_DATA_INTERFACE(:fit)) @@ -97,6 +108,7 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update)")) @@ -137,6 +149,8 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. + $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update_observations)")) See also [`LearnAPI.clone`](@ref). @@ -167,6 +181,8 @@ Implementation is optional. The signature must include `verbosity`. It should be `LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and `newlearner = LearnAPI.clone(learner, replacements...)`. +Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. + $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.update_features)")) See also [`LearnAPI.clone`](@ref). diff --git a/src/predict_transform.jl b/src/predict_transform.jl index 0a92d3f5..f097a976 100644 --- a/src/predict_transform.jl +++ b/src/predict_transform.jl @@ -46,7 +46,7 @@ DOC_DATA_INTERFACE(method) = case then an implementation must either: (i) overload [`obs`](@ref) to articulate how provided data can be transformed into a form that does support [`LearnAPI.RandomAccess`](@ref); or (ii) overload the trait - [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. Refer tbo + [`LearnAPI.data_interface`](@ref) to specify a more relaxed data API. Refer to the document strings for details. """ diff --git a/src/traits.jl b/src/traits.jl index 74e86bf9..655b2289 100644 --- a/src/traits.jl +++ b/src/traits.jl @@ -152,6 +152,19 @@ macro functions(learner) end |> esc end +""" + LearnAPI.kind_of(learner) + +Return the `fit`/`predict`/`transform` signature pattern used by `learner`. See +[`KindOfLearner`](@ref) for details. + +# New implementations + +The fallback value is [`LearnAPI.Descriminative()`]. + +""" +kind_of(learner) = Descriminative() + """ LearnAPI.kinds_of_proxy(learner) diff --git a/src/types.jl b/src/types.jl index 0fba1ff4..43dd420e 100644 --- a/src/types.jl +++ b/src/types.jl @@ -4,11 +4,11 @@ abstract type KindOfLearner end """ - LearnAPI.Standard + LearnAPI.Descriminative -Type with a single instance, `LearnAPI.Standard()`. +Type with a single instance, `LearnAPI.Descriminative()`. -If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Standard()`, then the only possible +If [`LearnAPI.kind_of(learner)`](@ref)` == LearnAPI.Descriminative()`, then the only possible signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those appearing below, or variations on these in which keyword arguments are also supplied: @@ -30,7 +30,7 @@ transform(learner, data) See also [`LearnAPI.Static`](@ref), [`LearnAPI.Generative`](@ref). """ -struct Standard <: KindOfLearner end +struct Descriminative <: KindOfLearner end """ LearnAPI.Static @@ -42,10 +42,10 @@ signatures of [`fit`](@ref), [`predict`](@ref) and [`transform`](@ref) are those below, or variations on these in which keyword arguments are also supplied: ``` -model = fit(learner) # (no `data` argument) -predict(model, data) -predict(model, kop::KindOfProxy, data) -transform(model, data) +model = fit(learner) # no `data` argument +predict(model, data) # may mutate `model` +predict(model, kop::KindOfProxy, data) # may mutate `model` +transform(model, data) # may mutate `model` ``` and the one-line convenience forms @@ -56,7 +56,7 @@ predict(learner, kop::KindOfProxy) transform(learner, data) ``` -See also [`LearnAPI.Standard](@ref), [`LearnAPI.Generative`](@ref). +See also [`LearnAPI.Descriminative`](@ref), [`LearnAPI.Generative`](@ref). """ struct Static <: KindOfLearner end @@ -72,12 +72,12 @@ below, or variations on these in which keyword arguments are also supplied: ``` model = fit(learner, data) -predict(model) -predict(model, kop::KindOfProxy) -transform(model) +predict(model) # no `newdata` argument +predict(model, kop::KindOfProxy) # no `newdata` argument +transform(model) # no `newdata` argument ``` -and the one-liner convenience forms +and the one-line convenience forms ``` predict(learner, data) @@ -85,6 +85,7 @@ predict(learner, kop::KindOfProxy, data) transform(learner, data) ``` +See also [`LearnAPI.Descriminative`](@ref), [`LearnAPI.Static`](@ref). """ struct Generative <: KindOfLearner end @@ -96,7 +97,7 @@ Abstract type whose instances are the possible values of [`LearnAPI.kind_of(learner)`](@ref). All instances of this type, and brief indications of their interpretation, appear below. -[`LearnAPI.Standard()`](@ref): A typical workflow looks like: +[`LearnAPI.Descriminative()`](@ref): A typical workflow looks like: ``` model = fit(learner, data) @@ -109,21 +110,19 @@ transform(learner, new_data) ``` model = fit(learner) -predict(learner, data) +predict(model, data) # may mutate `model` to record byproducts of computation # or -transform(learner, data) +transform(model, data) ``` [`LearnAPI.Generative()`](@ref): A typical workflow looks like: ``` model = fit(learner, data) -predict(learner) -# or -transform(learner) +predict(learner) # e.g., returns a single probability distribution ``` -For precise details, refer to the document strings for [`LearnAPI.Standard`](@ref), +For precise details, refer to the document strings for [`LearnAPI.Descriminative`](@ref), [`LearnAPI.Static`](@ref), and [`LearnAPI.Generative`](@ref). """ KindOfLearner @@ -147,9 +146,8 @@ following must hold: - The ``j``th observation of `ŷ`, for any ``j``, depends only on the ``j``th observation of the provided `data` (no correlation between observations). -Alternatively, in the case `LearnAPI.sees_features(learner) == false` (so that -`predict(model, ...)` consumes no data, and `fit` sees only target data), one requires -only that: +An exception holds in the case that [`LearnAPI.kind_of(learner)`](@ref)` == +[`LearnAPI.Generative()`](@ref): - `LearnAPI.predict(model, kind_of_proxy)` consists of a single observation (such as a single probability distribution). From 49d3fe6c5bba932c48128a2ec1fd2efe24972cae Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 10 Aug 2025 22:18:31 +1200 Subject: [PATCH 14/29] fix some whitespace --- docs/src/anatomy_of_an_implementation.md | 154 +++++++++++------------ 1 file changed, 77 insertions(+), 77 deletions(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 0c23c80c..8493237a 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -73,7 +73,7 @@ Here's a new type whose instances specify the single ridge regression hyperparam ```@example anatomy struct Ridge{T<:Real} - lambda::T + lambda::T end nothing # hide ``` @@ -87,7 +87,7 @@ fields) that are not other learners, and we must implement ```@example anatomy """ - Ridge(; lambda=0.1) + Ridge(; lambda=0.1) Instantiate a ridge regression learner, with regularization of `lambda`. """ @@ -111,9 +111,9 @@ coefficients labelled by feature name for inspection after training: ```@example anatomy struct RidgeFitted{T,F} - learner::Ridge - coefficients::Vector{T} - named_coefficients::F + learner::Ridge + coefficients::Vector{T} + named_coefficients::F end nothing # hide ``` @@ -125,25 +125,25 @@ The implementation of `fit` looks like this: ```@example anatomy function LearnAPI.fit(learner::Ridge, data; verbosity=1) - X, y = data + X, y = data - # data preprocessing: - table = Tables.columntable(X) - names = Tables.columnnames(table) |> collect - A = Tables.matrix(table, transpose=true) + # data preprocessing: + table = Tables.columntable(X) + names = Tables.columnnames(table) |> collect + A = Tables.matrix(table, transpose=true) - lambda = learner.lambda + lambda = learner.lambda - # apply core algorithm: - coefficients = (A*A' + learner.lambda*I)\(A*y) # vector + # apply core algorithm: + coefficients = (A*A' + learner.lambda*I)\(A*y) # vector - # determine named coefficients: - named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] + # determine named coefficients: + named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] - # make some noise, if allowed: - verbosity > 0 && @info "Coefficients: $named_coefficients" + # make some noise, if allowed: + verbosity > 0 && @info "Coefficients: $named_coefficients" - return RidgeFitted(learner, coefficients, named_coefficients) + return RidgeFitted(learner, coefficients, named_coefficients) end ``` @@ -165,7 +165,7 @@ We provide this implementation for our ridge regressor: ```@example anatomy LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = - Tables.matrix(Xnew)*model.coefficients + Tables.matrix(Xnew)*model.coefficients ``` If the kind of proxy is omitted, as in `predict(model, Xnew)`, then a fallback grabs the @@ -215,7 +215,7 @@ dump the named version of the coefficients: ```@example anatomy LearnAPI.strip(model::RidgeFitted) = - RidgeFitted(model.learner, model.coefficients, nothing) + RidgeFitted(model.learner, model.coefficients, nothing) ``` Crucially, we can still use `LearnAPI.strip(model)` in place of `model` to make new @@ -240,20 +240,20 @@ A macro provides a shortcut, convenient when multiple traits are to be defined: ```@example anatomy @trait( - Ridge, - constructor = Ridge, - kinds_of_proxy=(Point(),), - tags = ("regression",), - functions = ( - :(LearnAPI.fit), - :(LearnAPI.learner), - :(LearnAPI.clone), - :(LearnAPI.strip), - :(LearnAPI.obs), - :(LearnAPI.features), - :(LearnAPI.target), - :(LearnAPI.predict), - :(LearnAPI.coefficients), + Ridge, + constructor = Ridge, + kinds_of_proxy=(Point(),), + tags = ("regression",), + functions = ( + :(LearnAPI.fit), + :(LearnAPI.learner), + :(LearnAPI.clone), + :(LearnAPI.strip), + :(LearnAPI.obs), + :(LearnAPI.features), + :(LearnAPI.target), + :(LearnAPI.predict), + :(LearnAPI.coefficients), ) ) nothing # hide @@ -382,31 +382,31 @@ end Ridge(; lambda=0.1) = Ridge(lambda) struct RidgeFitted{T,F} - learner::Ridge - coefficients::Vector{T} - named_coefficients::F + learner::Ridge + coefficients::Vector{T} + named_coefficients::F end LearnAPI.learner(model::RidgeFitted) = model.learner LearnAPI.coefficients(model::RidgeFitted) = model.named_coefficients LearnAPI.strip(model::RidgeFitted) = - RidgeFitted(model.learner, model.coefficients, nothing) + RidgeFitted(model.learner, model.coefficients, nothing) @trait( - Ridge, - constructor = Ridge, - kinds_of_proxy=(Point(),), - tags = ("regression",), - functions = ( - :(LearnAPI.fit), - :(LearnAPI.learner), - :(LearnAPI.clone), - :(LearnAPI.strip), - :(LearnAPI.obs), - :(LearnAPI.features), - :(LearnAPI.target), - :(LearnAPI.predict), - :(LearnAPI.coefficients), + Ridge, + constructor = Ridge, + kinds_of_proxy=(Point(),), + tags = ("regression",), + functions = ( + :(LearnAPI.fit), + :(LearnAPI.learner), + :(LearnAPI.clone), + :(LearnAPI.strip), + :(LearnAPI.obs), + :(LearnAPI.features), + :(LearnAPI.target), + :(LearnAPI.predict), + :(LearnAPI.coefficients), ) ) @@ -435,9 +435,9 @@ The [`obs`](@ref) methods exist to: !!! important - While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we - focus here on a self-contained implementation of `obs` for the ridge example above, to show - how it works. + While many new learner implementations will want to adopt a canned data front end, such as those provided by [LearnDataFrontEnds.jl](https://juliaai.github.io/LearnDataFrontEnds.jl/dev/), we + focus here on a self-contained implementation of `obs` for the ridge example above, to show + how it works. In the typical case, where [`LearnAPI.data_interface`](@ref) is not overloaded, the alternative data representations must implement the MLCore.jl `getobs/numobs` interface @@ -480,9 +480,9 @@ introduce a new type: ```@example anatomy2 struct RidgeFitObs{T,M<:AbstractMatrix{T}} - A::M # `p` x `n` matrix - names::Vector{Symbol} # features - y::Vector{T} # target + A::M # `p` x `n` matrix + names::Vector{Symbol} # features + y::Vector{T} # target end ``` @@ -490,10 +490,10 @@ Now we overload `obs` to carry out the data preprocessing previously in `fit`, l ```@example anatomy2 function LearnAPI.obs(::Ridge, data) - X, y = data - table = Tables.columntable(X) - names = Tables.columnnames(table) |> collect - return RidgeFitObs(Tables.matrix(table)', names, y) + X, y = data + table = Tables.columntable(X) + names = Tables.columnnames(table) |> collect + return RidgeFitObs(Tables.matrix(table)', names, y) end ``` @@ -505,27 +505,27 @@ methods - one to handle "regular" input, and one to handle the pre-processed dat ```@example anatomy2 function LearnAPI.fit(learner::Ridge, observations::RidgeFitObs; verbosity=1) - lambda = learner.lambda + lambda = learner.lambda - A = observations.A - names = observations.names - y = observations.y + A = observations.A + names = observations.names + y = observations.y - # apply core learner: - coefficients = (A*A' + learner.lambda*I)\(A*y) # 1 x p matrix + # apply core learner: + coefficients = (A*A' + learner.lambda*I)\(A*y) # 1 x p matrix - # determine named coefficients: - named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] + # determine named coefficients: + named_coefficients = [names[j] => coefficients[j] for j in eachindex(names)] - # make some noise, if allowed: - verbosity > 0 && @info "Coefficients: $named_coefficients" + # make some noise, if allowed: + verbosity > 0 && @info "Coefficients: $named_coefficients" - return RidgeFitted(learner, coefficients, named_coefficients) + return RidgeFitted(learner, coefficients, named_coefficients) end LearnAPI.fit(learner::Ridge, data; kwargs...) = - fit(learner, obs(learner, data); kwargs...) + fit(learner, obs(learner, data); kwargs...) ``` ### The `obs` contract @@ -549,7 +549,7 @@ this is [`LearnAPI.RandomAccess()`](@ref) (the default) it usually suffices to o ```@example anatomy2 Base.getindex(data::RidgeFitObs, I) = - RidgeFitObs(data.A[:,I], data.names, y[I]) + RidgeFitObs(data.A[:,I], data.names, y[I]) Base.length(data::RidgeFitObs) = length(data.y) ``` @@ -560,10 +560,10 @@ LearnAPI.obs(::RidgeFitted, Xnew) = Tables.matrix(Xnew)' LearnAPI.obs(::RidgeFitted, observations::AbstractArray) = observations # involutivity LearnAPI.predict(model::RidgeFitted, ::Point, observations::AbstractMatrix) = - observations'*model.coefficients + observations'*model.coefficients LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = - predict(model, Point(), obs(model, Xnew)) + predict(model, Point(), obs(model, Xnew)) ``` ### Data deconstructors: `features` and `target From 80b3aad41aa3e841752fb088c126da03691828ca Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 21 Aug 2025 19:19:16 +1200 Subject: [PATCH 15/29] remove fallback for weights --- docs/src/features_target_weights.md | 2 +- src/features_target_weights.jl | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/features_target_weights.md b/docs/src/features_target_weights.md index c495784e..2d383d05 100644 --- a/docs/src/features_target_weights.md +++ b/docs/src/features_target_weights.md @@ -33,7 +33,7 @@ training_loss = sum(ŷ .!= y) |:-------------------------------------------|:---------------------:|-------------| | [`LearnAPI.features(learner, data)`](@ref) | no fallback | no | | [`LearnAPI.target(learner, data)`](@ref) | no fallback | no | -| [`LearnAPI.weights(learner, data)`](@ref) | `nothing` | no | +| [`LearnAPI.weights(learner, data)`](@ref) | no fallback | no | # Reference diff --git a/src/features_target_weights.jl b/src/features_target_weights.jl index 81e90799..9fae00f9 100644 --- a/src/features_target_weights.jl +++ b/src/features_target_weights.jl @@ -63,7 +63,7 @@ Where `nothing` is returned, weighting is understood to be uniform. Implementing is optional. -If `obs` is being overloaded, then typically it suffices to overload +If `obs` is being implemented, then typically it suffices to overload `LearnAPI.weights(learner, observations)` where `observations = obs(learner, data)` and `data` is any documented supported `data` in calls of the form [`fit(learner, data)`](@ref), and to add a declaration of the form @@ -79,7 +79,7 @@ Ensure the returned object, unless `nothing`, implements the data interface spec $(DOC_IMPLEMENTED_METHODS(":(LearnAPI.weights)"; overloaded=false)) """ -weights(::Any, data) = nothing +function weights end """ LearnAPI.features(learner, data) From 8b01f8672be1751771b7e325d33520ef575b090a Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 21 Aug 2025 19:25:44 +1200 Subject: [PATCH 16/29] doc updates --- docs/src/anatomy_of_an_implementation.md | 36 ++++++++++++------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 8493237a..7bab530e 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -29,9 +29,9 @@ model = fit(learner) # no `data` argument predict(model, data) # may mutate `model` to record byproducts of computation ``` -Do not read too much into the names for these patterns, which are formalized [here](@ref kinds_of_learner). They may not correspond in every case to prior conceptions. +Do not read too much into the names for these patterns, which are formalized [here](@ref kinds_of_learner). Use may not always correspond to prior associations. -Elaborating on the very common `Descriminative` pattern above, this tutorial details an +Elaborating on the common `Descriminative` pattern above, this tutorial details an implementation of the LearnAPI.jl for naive [ridge regression](https://en.wikipedia.org/wiki/Ridge_regression) with no intercept. The kind of workflow we want to enable has been previewed in [Sample workflow](@ref). Readers can also @@ -177,7 +177,7 @@ we overload appropriately below. LearnAPI.jl is flexible about the form of training `data`. However, to buy into meta-functionality, such as cross-validation, we'll need to say something about the structure of this data. We implement [`LearnAPI.target`](@ref) to say what -part of the data consistutes a [target variable](@ref proxy), and +part of the data constitutes a [target variable](@ref proxy), and [`LearnAPI.features`](@ref) to say what are the features (valid `newdata` in a `predict(model, newdata)` call): @@ -271,11 +271,11 @@ the *type* of the argument. ### The `functions` trait The last trait, `functions`, above returns a list of all LearnAPI.jl methods that can be -meaningfully applied to the learner or associated model, with the exception of traits. You -always include the first five you see here: `fit`, `learner`, `clone` ,`strip`, -`obs`. Here [`clone`](@ref) is a utility function provided by LearnAPI that you never -overload, while [`obs`](@ref) is discussed under [Providing a separate data front -end](@ref) below and is always included because it has a meaningful fallback. +meaningfully applied to the learner or the output of `fit` (denoted `model` above), with +the exception of traits. You always include the first five you see here: `fit`, `learner`, +`clone` ,`strip`, `obs`. Here [`clone`](@ref) is a utility function provided by LearnAPI +that you never overload, while [`obs`](@ref) is discussed under [Providing a separate data +front end](@ref) below and is always included because it has a meaningful fallback. See [`LearnAPI.functions`](@ref) for a checklist of what the `functions` trait needs to return. @@ -469,14 +469,14 @@ newobservations = MLCore.getobs(observations, test_indices) predict(model, newobservations) ``` -which works for any non-static learner implementing `predict`, no matter how one is -supposed to accesses the individual observations of `data` or `newdata`. See also the -demonstration [below](@ref advanced_demo). Furthermore, fallbacks ensure the above pattern -still works if we choose not to implement a front end at all, which is allowed, if -supported `data` and `newdata` already implement `getobs`/`numobs`. +which works for any [`LearnAPI.Descriminative`](@ref) learner implementing `predict`, no +matter how one is supposed to accesses the individual observations of `data` or +`newdata`. See also the demonstration [below](@ref advanced_demo). Furthermore, fallbacks +ensure the above pattern still works if we choose not to implement a front end at all, +which is allowed, if supported `data` and `newdata` already implement `getobs`/`numobs`. -Here we specifically wrap all the preprocessed data into single object, for which we -introduce a new type: +In the ridge regression example we specifically wrap all the preprocessed data into single +object, for which we introduce a new type: ```@example anatomy2 struct RidgeFitObs{T,M<:AbstractMatrix{T}} @@ -497,8 +497,8 @@ function LearnAPI.obs(::Ridge, data) end ``` -We informally refer to the output of `obs` as "observations" (see [The `obs` -contract](@ref) below). The previous core `fit` signature is now replaced with two +We informally refer to the output of `obs` as "observations" (see "[The `obs` +contract](@ref)" below). The previous core `fit` signature is now replaced with two methods - one to handle "regular" input, and one to handle the pre-processed data (observations) which appears first below: @@ -566,7 +566,7 @@ LearnAPI.predict(model::RidgeFitted, ::Point, Xnew) = predict(model, Point(), obs(model, Xnew)) ``` -### Data deconstructors: `features` and `target +### Data deconstructors: `features` and `target` These methods must be able to handle any `data` supported by `fit`, which includes the output of `obs(learner, data)`: From d925e513554d1d6bb2e5f13c57a25df8307dda81 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 21 Aug 2025 19:26:07 +1200 Subject: [PATCH 17/29] more doc updates --- docs/src/common_implementation_patterns.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/src/common_implementation_patterns.md b/docs/src/common_implementation_patterns.md index 0c57ff50..db5ff1f2 100644 --- a/docs/src/common_implementation_patterns.md +++ b/docs/src/common_implementation_patterns.md @@ -9,8 +9,10 @@ This guide is intended to be consulted after reading [Anatomy of an Implementati which introduces the main interface objects and terminology. Although an implementation is defined purely by the methods and traits it implements, many -implementations fall into one (or more) of the following informally understood patterns or -tasks: +implementations fall into one (or more) of the informally understood patterns or tasks +below. While some generally fall into one of the core `Descriminative`, `Generative` or +`Static` patterns detailed [here](@id kinds of learner), there are exceptions (such as +clustering, which has both `Descriminative` and `Static` variations). - [Regression](@ref): Supervised learners for continuous targets From 50ec2fa3b3ead8a9ffcc5d82e72966860602129e Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 21 Aug 2025 19:26:29 +1200 Subject: [PATCH 18/29] fix mistake in table of traits re `kind_of(learner)` --- docs/src/traits.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/src/traits.md b/docs/src/traits.md index b22a0462..16077a8f 100644 --- a/docs/src/traits.md +++ b/docs/src/traits.md @@ -1,4 +1,4 @@ -# [Learner Traits](@id traits) +2# [Learner Traits](@id traits) Learner traits are simply functions whose sole argument is a learner. @@ -17,7 +17,7 @@ In the examples column of the table below, `Continuous` is a name owned the pack |:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------|:---------------------------------------------------------------| | [`LearnAPI.constructor`](@ref)`(learner)` | constructor for generating new or modified versions of `learner` | (no fallback) | `RidgeRegressor` | | [`LearnAPI.functions`](@ref)`(learner)` | functions you can apply to `learner` or associated model (traits excluded) | `()` | `(:fit, :predict, :LearnAPI.strip, :(LearnAPI.learner), :obs)` | -| [`LearnAPI.kind_of`](@ref)`(learner)` | the `fit`/`predict`/`transform` pattern used by `learner` | `LearnAPI.Static()` | `LearnAPI.Descriminative()` | +| [`LearnAPI.kind_of`](@ref)`(learner)` | the `fit`/`predict`/`transform` pattern used by `learner` | `LearnAPI.Descriminative()` | `LearnAPI.Static()` | | [`LearnAPI.kinds_of_proxy`](@ref)`(learner)` | instances `kind` of `KindOfProxy` for which an implementation of `LearnAPI.predict(learner, kind, ...)` is guaranteed. | `()` | `(Distribution(), Interval())` | | [`LearnAPI.tags`](@ref)`(learner)` | lists one or more suggestive learner tags from `LearnAPI.tags()` | `()` | (:regression, :probabilistic) | | [`LearnAPI.is_pure_julia`](@ref)`(learner)` | `true` if implementation is 100% Julia code | `false` | `true` | From df811e3211010c4e8f805c8591fde4eec2ed1f70 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Thu, 21 Aug 2025 19:26:57 +1200 Subject: [PATCH 19/29] doc tweak --- docs/src/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/src/index.md b/docs/src/index.md index 6f6e1a01..a8fe9c4f 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -94,8 +94,8 @@ Some canned data front ends (implementations of [`obs`](@ref)) are provided by t ## Learning more -- [Anatomy of an Implementation](@ref): informal introduction to the main actors in a new - LearnAPI.jl implementation +- [Anatomy of an Implementation](@ref): informal tutorial introducing the main actors in a + new LearnAPI.jl implementation, including a **Quick Start** for new implementations. - [Reference](@ref reference): official specification From 4541243ff73dc3737bea52e3ffe9afda023e3335 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sat, 23 Aug 2025 12:58:35 +1200 Subject: [PATCH 20/29] bump 0.2.0 --- Project.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Project.toml b/Project.toml index 9a1351fc..44182acd 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "LearnAPI" uuid = "92ad9a40-7767-427a-9ee6-6e577f1266cb" authors = ["Anthony D. Blaom "] -version = "1.0.1" +version = "2.0.0" [compat] julia = "1.10" From 9885d5868b32b801a99bbda406804dc2779edea0 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 18:51:25 +1200 Subject: [PATCH 21/29] doc tweak --- docs/src/patterns/density_estimation.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/src/patterns/density_estimation.md b/docs/src/patterns/density_estimation.md index 0e60a3e8..e86a50ee 100644 --- a/docs/src/patterns/density_estimation.md +++ b/docs/src/patterns/density_estimation.md @@ -1,9 +1,9 @@ # Density Estimation -In density estimators, `fit` is trained only on [target data](@ref proxy), and -`predict(model, kind_of_proxy)` consumes no data at all, a pattern flagged by the -identities [`LearnAPI.target(learner, y)`](@ref)` == y` and -[`LearnAPI.kind_of(learner)`](@ref)` == `[`LearnAPI.Generative()`](@ref). +In density estimators, `fit` is trained only on [target data](@ref proxy), and `predict` +consumes no data at all, a pattern flagged by the identities [`LearnAPI.target(learner, +y)`](@ref)` == y` and [`LearnAPI.kind_of(learner)`](@ref)` == +`[`LearnAPI.Generative()`](@ref), respetively. Typically `predict` returns a single probability density/mass function. Here's a sample @@ -11,7 +11,7 @@ workflow: ```julia model = fit(learner, y) # no features -predict(model) # shortcut for `predict(model, Distribution())`, or similar +predict(model) # shortcut for `predict(model, Distribution())`, or similar ``` A one-line convenience method will typically be implemented as well: From 65041b8fd019f5ba1e43a2ca723dd07c9d830784 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 21:22:24 +1200 Subject: [PATCH 22/29] remove out-dated tests --- test/features_target_weights.jl | 11 ----------- test/runtests.jl | 1 - 2 files changed, 12 deletions(-) delete mode 100644 test/features_target_weights.jl diff --git a/test/features_target_weights.jl b/test/features_target_weights.jl deleted file mode 100644 index 4809f5df..00000000 --- a/test/features_target_weights.jl +++ /dev/null @@ -1,11 +0,0 @@ -using Test -using LearnAPI - -struct Avocado end - -@test LearnAPI.target(Avocado(), (1, 2, 3)) == 3 -@test isnothing(LearnAPI.weights(Avocado(), "salsa")) -@test LearnAPI.features(Avocado(), "salsa") == "salsa" -@test LearnAPI.features(Avocado(), (:X, :y)) == :X - -true diff --git a/test/runtests.jl b/test/runtests.jl index e8117976..4b64816c 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -7,7 +7,6 @@ test_files = [ "predict_transform.jl", "obs.jl", "accessor_functions.jl", - "features_target_weights.jl", ] files = isempty(ARGS) ? test_files : ARGS From 5d504d799586e913dede6495d2d3d38a7ec6d640 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 21:36:41 +1200 Subject: [PATCH 23/29] fix typo --- docs/src/anatomy_of_an_implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index 7bab530e..c3f58219 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -40,7 +40,7 @@ refer to the [demonstration](@ref workflow) of the implementation given later. !!! tip "Quick Start for new implementations" 1. From this tutorial, read at least "[A basic implementation](@ref)" below. - 1. Looking over the examples in "[Common Implementation Patterns](@ref patterns)", identify the apppropriate core learner pattern above for your algorithm. + 1. Looking over the examples in "[Common Implementation Patterns](@ref patterns)", identify the appropriate core learner pattern above for your algorithm. 1. Implement `fit` (probably following an existing example). Read the [`fit`](@ref) document string to see what else may need to be implemented, paying particular attention to the "New implementations" section. 3. Rinse and repeat with each new method implemented. 4. Identify any additional [learner traits](@ref traits) that have appropriate overloadings; use the [`@trait`](@ref) macro to define these in one block. From 55c550b7865f3dde2dcc33657a3432eec6f34e31 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 21:39:34 +1200 Subject: [PATCH 24/29] add a test --- test/traits.jl | 1 + 1 file changed, 1 insertion(+) diff --git a/test/traits.jl b/test/traits.jl index 0a7023dd..8eb5dd48 100644 --- a/test/traits.jl +++ b/test/traits.jl @@ -32,6 +32,7 @@ LearnAPI.learner(model::SmallLearner) = model small = SmallLearner() @test LearnAPI.constructor(small) == SmallLearner @test :(LearnAPI.learner) in LearnAPI.functions(small) +@test LearnAPI.kind_of(learner) == LearnAPI.Descriminative() @test isempty(LearnAPI.kinds_of_proxy(small)) @test isempty(LearnAPI.tags(small)) @test !LearnAPI.is_pure_julia(small) From 67a900da08de97afcdf6b0fa812028204157cf1b Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 21:41:13 +1200 Subject: [PATCH 25/29] fix mistake in test --- test/traits.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/traits.jl b/test/traits.jl index 8eb5dd48..6a99a025 100644 --- a/test/traits.jl +++ b/test/traits.jl @@ -32,7 +32,7 @@ LearnAPI.learner(model::SmallLearner) = model small = SmallLearner() @test LearnAPI.constructor(small) == SmallLearner @test :(LearnAPI.learner) in LearnAPI.functions(small) -@test LearnAPI.kind_of(learner) == LearnAPI.Descriminative() +@test LearnAPI.kind_of(small) == LearnAPI.Descriminative() @test isempty(LearnAPI.kinds_of_proxy(small)) @test isempty(LearnAPI.tags(small)) @test !LearnAPI.is_pure_julia(small) From 5e310434bf590c47788c98df9648cf3b3e455783 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 24 Aug 2025 21:46:21 +1200 Subject: [PATCH 26/29] temporarily remove LearnTestAPI from the docs/Project.toml --- docs/Project.toml | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/Project.toml b/docs/Project.toml index 4ffd503c..30f1981a 100644 --- a/docs/Project.toml +++ b/docs/Project.toml @@ -2,7 +2,6 @@ Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4" DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656" LearnAPI = "92ad9a40-7767-427a-9ee6-6e577f1266cb" -LearnTestAPI = "3111ed91-c4f2-40e7-bb19-7f6c618409b8" MLCore = "c2834f40-e789-41da-a90e-33b280584a8c" ScientificTypesBase = "30f210dd-8aff-4c5f-94ba-8e64358c1161" Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" From 69d8d3e7b2cedab1780d80221dffa33df0a0c8a0 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sat, 18 Oct 2025 21:15:35 +1300 Subject: [PATCH 27/29] add "verbosity" preference and `default_verbosity()` oops --- .gitignore | 1 + Project.toml | 8 ++++++-- src/LearnAPI.jl | 3 +++ src/preferences.jl | 30 ++++++++++++++++++++++++++++++ test/preferences.jl | 21 +++++++++++++++++++++ test/runtests.jl | 1 + 6 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 src/preferences.jl create mode 100644 test/preferences.jl diff --git a/.gitignore b/.gitignore index 0e1b98c8..004a03ea 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ sandbox/ /docs/site/ /docs/Manifest.toml .vscode +LocalPreferences.toml \ No newline at end of file diff --git a/Project.toml b/Project.toml index 44182acd..c129a5d8 100644 --- a/Project.toml +++ b/Project.toml @@ -1,13 +1,17 @@ +authors = ["Anthony D. Blaom "] name = "LearnAPI" uuid = "92ad9a40-7767-427a-9ee6-6e577f1266cb" -authors = ["Anthony D. Blaom "] version = "2.0.0" [compat] +Preferences = "1.5.0" julia = "1.10" +[deps] +Preferences = "21216c6a-2e73-6563-6e65-726566657250" + [extras] Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [targets] -test = ["Test",] +test = ["Test"] diff --git a/src/LearnAPI.jl b/src/LearnAPI.jl index c32ab3b8..c7ca3035 100644 --- a/src/LearnAPI.jl +++ b/src/LearnAPI.jl @@ -1,5 +1,8 @@ module LearnAPI +using Preferences + +include("preferences.jl") include("types.jl") include("tools.jl") include("predict_transform.jl") diff --git a/src/preferences.jl b/src/preferences.jl new file mode 100644 index 00000000..8bba6616 --- /dev/null +++ b/src/preferences.jl @@ -0,0 +1,30 @@ +const VERBOSITY = @load_preference "verbosity" 1 + +INFO_VERBOSITY_IS(verbosity) = + "Currently the baseline verbosity is $verbosity. " +INFO_VERBOSITY_WILL_BE(verbosity) = + "After restarting Julia, this will be changed to $verbosity. " + +""" + LearnAPI.default_verbosity() + +Return the default verbosity for training LearnAPI learners. + +The value is determined at compile time by a Preferences.jl-style preference, with key +"verbosity". + +""" +default_verbosity() = VERBOSITY + +""" + LearnAPI.default_verbosity(level) + +Set the default verbosity for training LearnAPI learners to `level`. Changes do not take +effect until the next Julia session. + +""" +function default_verbosity(level) + @info INFO_VERBOSITY_IS(VERBOSITY) + @set_preferences! "verbosity" => level + @info INFO_VERBOSITY_WILL_BE(level) +end diff --git a/test/preferences.jl b/test/preferences.jl new file mode 100644 index 00000000..53dc021c --- /dev/null +++ b/test/preferences.jl @@ -0,0 +1,21 @@ +using LearnAPI +using Preferences + +@testset "default_verbosity" begin + @test LearnAPI.default_verbosity() == LearnAPI.VERBOSITY + @test_logs( + (:info, LearnAPI.INFO_VERBOSITY_IS(LearnAPI.default_verbosity())), + (:info, LearnAPI.INFO_VERBOSITY_WILL_BE(3)), + LearnAPI.default_verbosity(3), + ) + @test load_preference(LearnAPI, "verbosity") == 3 + + # restore active preference: + @test_logs( + (:info, LearnAPI.INFO_VERBOSITY_IS(LearnAPI.default_verbosity())), + (:info, LearnAPI.INFO_VERBOSITY_WILL_BE(LearnAPI.VERBOSITY)), + LearnAPI.default_verbosity(LearnAPI.VERBOSITY), + ) +end + +true diff --git a/test/runtests.jl b/test/runtests.jl index 4b64816c..a78ee0b6 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -1,6 +1,7 @@ using Test test_files = [ + "preferences.jl", "tools.jl", "traits.jl", "clone.jl", From e74f1d0c788f48005545c4042cb2f64ef3a44bf4 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 19 Oct 2025 10:02:54 +1300 Subject: [PATCH 28/29] adjust docstrings/docs to reflect verbosity change --- docs/src/fit_update.md | 32 +++++++++++++++--------------- src/fit_update.jl | 45 +++++++++++++++++++++++++++--------------- src/preferences.jl | 4 +++- 3 files changed, 48 insertions(+), 33 deletions(-) diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index 40018986..91337810 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -4,7 +4,7 @@ ### Training ```julia -fit(learner, data; verbosity=1) -> model +fit(learner, data; verbosity=...) -> model ``` This is the typical `fit` pattern, applying in the case that [`LearnAPI.kind_of(learner)`](@ref) @@ -25,9 +25,9 @@ Examples appear below. ### Updating ``` -update(model, data; verbosity=..., :param1=new_value1, :param2=new_value2, ...) -> updated_model -update_observations(model, new_data; verbosity=..., :param1=new_value1, ...) -> updated_model -update_features(model, new_data; verbosity=..., :param1=new_value1, ...) -> updated_model +update(model, data, :param1=>new_value1, :param2=>new_value2, ...; verbosity=...) -> updated_model +update_observations(model, new_data, :param1=>new_value1, ...; verbosity=...) -> updated_model +update_features(model, new_data, :param1=>new_value1, ...; verbosity=...) -> updated_model ``` [`LearnAPI.Static()`](@ref) learners cannot be updated. @@ -50,7 +50,7 @@ ŷ = predict(model, Distribution(), Xnew) LearnAPI.feature_importances(model) # Add 50 iterations and predict again: -model = update(model; n=150) +model = update(model, n => 150) predict(model, Distribution(), X) ``` @@ -123,21 +123,21 @@ See also [Density Estimation](@ref). Exactly one of the following must be implemented: -| method | fallback | -|:--------------------------------------------|:---------| -| [`fit`](@ref)`(learner, data; verbosity=1)` | none | -| [`fit`](@ref)`(learner; verbosity=1)` | none | +| method | fallback | +|:-----------------------------------------------------------------------|:---------| +| [`fit`](@ref)`(learner, data; verbosity=LearnAPI.default_verbosity())` | none | +| [`fit`](@ref)`(learner; verbosity=LearnAPI.default_verbosity())` | none | ### Updating -| method | fallback | compulsory? | -|:-------------------------------------------------------------------------------------|:---------|-------------| -| [`update`](@ref)`(model, data; verbosity=1, hyperparameter_updates...)` | none | no | -| [`update_observations`](@ref)`(model, new_data; verbosity=1, hyperparameter_updates...)` | none | no | -| [`update_features`](@ref)`(model, new_data; verbosity=1, hyperparameter_updates...)` | none | no | +| method | fallback | compulsory? | +|:-------------------------------------------------------------------------------------------|:---------|-------------| +| [`update`](@ref)`(model, data, hyperparameter_updates...; verbosity=...)` | none | no | +| [`update_observations`](@ref)`(model, new_data, hyperparameter_updates...; verbosity=...)` | none | no | +| [`update_features`](@ref)`(model, new_data, hyperparameter_updates...; verbosity=...)` | none | no | -There are some contracts governing the behaviour of the update methods, as they relate to -a previous `fit` call. Consult the document strings for details. +There are contracts governing the behaviour of the update methods, as they relate to a +previous `fit` call. Consult the document strings for details. ## Reference diff --git a/src/fit_update.jl b/src/fit_update.jl index b72600a4..ab7f3ff8 100644 --- a/src/fit_update.jl +++ b/src/fit_update.jl @@ -1,8 +1,8 @@ # # FIT """ - fit(learner, data; verbosity=1) - fit(learner; verbosity=1) + fit(learner, data; verbosity=LearnAPI.default_verbosity()) + fit(learner; verbosity=LearnAPI.default_verbosity())) In the case of the first signature, execute the machine learning or statistical algorithm with configuration `learner` using the provided training `data`, returning an object, @@ -51,7 +51,8 @@ Implementation of exactly one of the signatures is compulsory. Unless implementi [`LearnAPI.Descriminative()`](@ref) `fit`/`predict`/`transform` pattern, [`LearnAPI.kind_of(learner)`](@ref) will need to be suitably overloaded. -The `fit` signature must include `verbosity` with `1` as default. +The `fit` signature must include the keyword argument `verbosity` with +`LearnAPI.default_verbosity()` as default. The LearnAPI.jl specification has nothing to say regarding `fit` signatures with more than two arguments. For convenience, for example, an implementation is free to implement a @@ -74,7 +75,7 @@ function fit end # # UPDATE AND COUSINS """ - update(model, data, param_replacements...; verbosity=1) + update(model, data, param_replacements...; verbosity=LearnAPI.default_verbosity()) Return an updated version of the `model` object returned by a previous [`fit`](@ref) or `update` call, but with the specified hyperparameter replacements, in the form `:p1 => @@ -104,9 +105,10 @@ See also [`fit`](@ref), [`update_observations`](@ref), [`update_features`](@ref) # New implementations -Implementation is optional. The signature must include `verbosity`. It should be true that -`LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and -`newlearner = LearnAPI.clone(learner, replacements...)`. +Implementation is optional. The signature must include the `verbosity` keyword +argument. It should be true that `LearnAPI.learner(newmodel) == newlearner`, where +`newmodel` is the return value and `newlearner = LearnAPI.clone(learner, +replacements...)`. Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. @@ -118,10 +120,15 @@ See also [`LearnAPI.clone`](@ref) function update end """ - update_observations(model, new_data, param_replacements...; verbosity=1) + update_observations( + model, + new_data, + param_replacements...; + verbosity=LearnAPI.default_verbosity(), + ) Return an updated version of the `model` object returned by a previous [`fit`](@ref) or -`update` call given the new observations present in `new_data`. One may additionally +`update` call, given the new observations present in `new_data`. One may additionally specify hyperparameter replacements in the form `:p1 => value1, :p2 => value2, ...`. ```julia-repl @@ -145,9 +152,10 @@ See also [`fit`](@ref), [`update`](@ref), [`update_features`](@ref). # New implementations -Implementation is optional. The signature must include `verbosity`. It should be true that -`LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and -`newlearner = LearnAPI.clone(learner, replacements...)`. +Implementation is optional. The signature must include the `verbosity` keyword +argument. It should be true that `LearnAPI.learner(newmodel) == newlearner`, where +`newmodel` is the return value and `newlearner = LearnAPI.clone(learner, +replacements...)`. Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. @@ -159,7 +167,11 @@ See also [`LearnAPI.clone`](@ref). function update_observations end """ - update_features(model, new_data, param_replacements...; verbosity=1) + update_features( + model, + new_data, + param_replacements,...; + verbosity=LearnAPI.default_verbosity(), ) Return an updated version of the `model` object returned by a previous [`fit`](@ref) or @@ -177,9 +189,10 @@ See also [`fit`](@ref), [`update`](@ref), [`update_features`](@ref). # New implementations -Implementation is optional. The signature must include `verbosity`. It should be true that -`LearnAPI.learner(newmodel) == newlearner`, where `newmodel` is the return value and -`newlearner = LearnAPI.clone(learner, replacements...)`. +Implementation is optional. The signature must include the `verbosity` keyword +argument. It should be true that `LearnAPI.learner(newmodel) == newlearner`, where +`newmodel` is the return value and `newlearner = LearnAPI.clone(learner, +replacements...)`. Cannot be implemented if [`LearnAPI.kind_of(learner)`](@ref)` == `LearnAPI.Static()`. diff --git a/src/preferences.jl b/src/preferences.jl index 8bba6616..aee264d4 100644 --- a/src/preferences.jl +++ b/src/preferences.jl @@ -8,11 +8,13 @@ INFO_VERBOSITY_WILL_BE(verbosity) = """ LearnAPI.default_verbosity() -Return the default verbosity for training LearnAPI learners. +Return the default LearnAPI verbosity level. The value is determined at compile time by a Preferences.jl-style preference, with key "verbosity". +If `verbosity=0`, only warnings are displayed; `verbosity=-1` suppresses warnings. + """ default_verbosity() = VERBOSITY From 77de4869fb1921e840a1396778d6bded48f4fc80 Mon Sep 17 00:00:00 2001 From: "Anthony D. Blaom" Date: Sun, 19 Oct 2025 10:51:12 +1300 Subject: [PATCH 29/29] update Anatomy of an Implementation re verbosity --- docs/src/anatomy_of_an_implementation.md | 4 ++-- docs/src/common_implementation_patterns.md | 2 +- docs/src/examples.md | 10 +++++++--- docs/src/fit_update.md | 6 ++++-- docs/src/reference.md | 3 ++- docs/src/traits.md | 2 +- 6 files changed, 17 insertions(+), 10 deletions(-) diff --git a/docs/src/anatomy_of_an_implementation.md b/docs/src/anatomy_of_an_implementation.md index c3f58219..806ca9bb 100644 --- a/docs/src/anatomy_of_an_implementation.md +++ b/docs/src/anatomy_of_an_implementation.md @@ -124,7 +124,7 @@ Note that we also include `learner` in the struct, for it must be possible to re The implementation of `fit` looks like this: ```@example anatomy -function LearnAPI.fit(learner::Ridge, data; verbosity=1) +function LearnAPI.fit(learner::Ridge, data; verbosity=LearnAPI.default_verbosity()) X, y = data # data preprocessing: @@ -503,7 +503,7 @@ methods - one to handle "regular" input, and one to handle the pre-processed dat (observations) which appears first below: ```@example anatomy2 -function LearnAPI.fit(learner::Ridge, observations::RidgeFitObs; verbosity=1) +function LearnAPI.fit(learner::Ridge, observations::RidgeFitObs; verbosity=LearnAPI.default_verbosity()) lambda = learner.lambda diff --git a/docs/src/common_implementation_patterns.md b/docs/src/common_implementation_patterns.md index db5ff1f2..9573c0d9 100644 --- a/docs/src/common_implementation_patterns.md +++ b/docs/src/common_implementation_patterns.md @@ -11,7 +11,7 @@ which introduces the main interface objects and terminology. Although an implementation is defined purely by the methods and traits it implements, many implementations fall into one (or more) of the informally understood patterns or tasks below. While some generally fall into one of the core `Descriminative`, `Generative` or -`Static` patterns detailed [here](@id kinds of learner), there are exceptions (such as +`Static` patterns detailed [here](@id kinds_of_learner), there are exceptions (such as clustering, which has both `Descriminative` and `Static` variations). - [Regression](@ref): Supervised learners for continuous targets diff --git a/docs/src/examples.md b/docs/src/examples.md index fe690ff1..5a20d85e 100644 --- a/docs/src/examples.md +++ b/docs/src/examples.md @@ -32,7 +32,7 @@ struct RidgeFitted{T,F} named_coefficients::F end -function LearnAPI.fit(learner::Ridge, data; verbosity=1) +function LearnAPI.fit(learner::Ridge, data; verbosity=LearnAPI.default_verbosity()) X, y = data # data preprocessing: @@ -129,7 +129,11 @@ function LearnAPI.obs(::Ridge, data) end LearnAPI.obs(::Ridge, observations::RidgeFitObs) = observations -function LearnAPI.fit(learner::Ridge, observations::RidgeFitObs; verbosity=1) +function LearnAPI.fit( + learner::Ridge, + observations::RidgeFitObs; + verbosity=LearnAPI.default_verbosity(), + ) lambda = learner.lambda @@ -226,7 +230,7 @@ frontend = FrontEnds.Saffron() LearnAPI.obs(learner::Ridge, data) = FrontEnds.fitobs(learner, data, frontend) LearnAPI.obs(model::RidgeFitted, data) = obs(model, data, frontend) -function LearnAPI.fit(learner::Ridge, observations::FrontEnds.Obs; verbosity=1) +function LearnAPI.fit(learner::Ridge, observations::FrontEnds.Obs; verbosity=LearnAPI.default_verbosity()) lambda = learner.lambda diff --git a/docs/src/fit_update.md b/docs/src/fit_update.md index 91337810..1e897c0e 100644 --- a/docs/src/fit_update.md +++ b/docs/src/fit_update.md @@ -14,10 +14,12 @@ returns one of: - [`LearnAPI.Generative()`](@ref) ``` -fit(learner; verbosity=1) -> static_model +fit(learner; verbosity=...) -> static_model ``` -This pattern applies in the case [`LearnAPI.kind_of(learner)`](@ref) returns [`LearnAPI.Static()`](@ref). +This pattern applies in the case [`LearnAPI.kind_of(learner)`](@ref) returns: + +- [`LearnAPI.Static()`](@ref) Examples appear below. diff --git a/docs/src/reference.md b/docs/src/reference.md index e77d8cdd..6cf6285d 100644 --- a/docs/src/reference.md +++ b/docs/src/reference.md @@ -217,17 +217,18 @@ Most learners will also implement [`predict`](@ref) and/or [`transform`](@ref). ## Utilities - - [`LearnAPI.is_learner`](@ref) - [`clone`](@ref): for cloning a learner with specified hyperparameter replacements. - [`@trait`](@ref): for simultaneously declaring multiple traits - [`@functions`](@ref): for listing functions available for use with a learner +- [`LearnAPI.default_verbosity`](@ref): get/reset the default verbosity ```@docs LearnAPI.is_learner clone @trait @functions +LearnAPI.default_verbosity ``` --- diff --git a/docs/src/traits.md b/docs/src/traits.md index 16077a8f..1eb49edb 100644 --- a/docs/src/traits.md +++ b/docs/src/traits.md @@ -1,4 +1,4 @@ -2# [Learner Traits](@id traits) +# [Learner Traits](@id traits) Learner traits are simply functions whose sole argument is a learner.