Reshuffle contributing sections, add more detail

_quarto.yml

@@ -108,10 +108,17 @@ website:
             - tutorials/gaussian-processes-introduction/index.qmd
             - tutorials/gaussian-process-latent-variable-models/index.qmd
 
-        - section: "Developers"
+        - section: "Contributing"
           contents:
-          - developers/contributing/index.qmd
+            - contributing/start-contributing/index.qmd
+            - contributing/documentation/index.qmd
+            - contributing/tests/index.qmd
+            - contributing/code-formatting/index.qmd
+            - contributing/pull-requests/index.qmd
+            - contributing/core-developers/index.qmd
 
+        - section: "Developers"
+          contents:
           - section: "DynamicPPL's Compiler"
             collapse-level: 1
             contents:
@@ -214,7 +221,12 @@ usage-tracking-extra-quantities: usage/tracking-extra-quantities
 usage-troubleshooting: usage/troubleshooting
 usage-varnamedtuple: usage/varnamedtuple
 
-contributing-guide: developers/contributing
+contributing-start: contributing/start-contributing
+contributing-documentation: contributing/documentation
+contributing-tests: contributing/tests
+contributing-pull-requests: contributing/pull-requests
+contributing-code-formatting: contributing/code-formatting
+contributing-core-developers: contributing/core-developers
 dev-model-manual: developers/compiler/model-manual
 contexts: developers/compiler/minituring-contexts
 minituring: developers/compiler/minituring-compiler

contributing/code-formatting/index.qmd

@@ -0,0 +1,72 @@
+---
+title: Code Formatting
+---
+
+The TuringLang ecosystem currently uses [JuliaFormatter.jl](https://github.com/domluna/JuliaFormatter.jl) to ensure consistent code style across the codebase.
+All code must be formatted before submitting a pull request, and ideally with every commit.
+
+# Basic usage
+
+We use v1 of JuliaFormatter (as of the time of writing, JuliaFormatter v2 is still somewhat unreliable).
+Make sure to install it in your *global* Julia environment (not the project environment, as adding it to the `Project.toml` would make it an invalid dependency of the project):
+
+```julia
+# Do not include `--project=...` here
+julia -e 'using Pkg; Pkg.add(name="JuliaFormatter", version="1"); Pkg.pin("JuliaFormatter")'
+```
+
+To format all Julia files in the current directory and subdirectories:
+
+```julia
+julia -e 'using JuliaFormatter; format(".")'
+```
+
+Run this command from the root of the repository before committing your changes.
+
+# Faster Formatting
+
+:::{.callout-note}
+This section is optional, but highly recommended if you are comfortable with Julia and are working on Turing codebases regularly.
+:::
+
+Running `julia -e ...`, especially on v1 of JuliaFormatter, can be very slow due to Julia's TTFX.
+To get around this, you can use [PackageCompiler.jl](https://julialang.github.io/PackageCompiler.jl/) to create a custom sysimage, which already includes cached compiled versions of JuliaFormatter's functions.
+In our experience, this has led to drastic improvements in formatting speed: for example, on DynamicPPL.jl it slashes the formatting time from around 6 seconds to 0.5 seconds.
+
+The easiest way to do so is to directly copy [the code in this GitHub Gist](https://gist.github.com/penelopeysm/9338c160eeb05437205535c2edcf80ee) into your shell configuration file (e.g. `~/.bash_profile`).
+You can of course modify this as you see fit.
+Once you have done this (and started a new shell session), you can simply use the `jf1` command to format your code, which will be *much* faster than the previous method.
+
+# Pre-commit hook
+
+We do not currently have [pre-commit hooks](https://pre-commit.com/) for code formatting, but you can include your own if you wish.
+
+To do so, add this to `.pre-commit-config.yaml` at the top level of the repository, and then run `pre-commit install` to set up the hook.
+
+```yaml
+repos:
+- repo: "https://github.com/domluna/JuliaFormatter.jl"
+  rev: "v1.0.62"
+  hooks:
+  - id: "julia-formatter"
+```
+
+:::{.callout-note}
+If you are intending to format on every commit, you will likely find that the basic `julia -e ...` method is intolerably slow!
+We therefore recommend using the PackageCompiler-enabled method for the pre-commit hook.
+:::
+
+If you are using the faster PackageCompiler-enabled method, use the following instead (adapt as necessary if you made any changes to the function):
+
+```yaml
+repos:
+- repo: local
+  hooks:
+  - id: format
+    name: format
+    language: system
+    entry: bash -c 'source ~/.bash_profile; jf1'
+```
+
+However, in either case, please make sure that you do not commit this file to the repository.
+To avoid accidentally doing so, you can add `.pre-commit-config.yaml` to `.git/info/exclude` (which is like a local `.gitignore` that is not part of source control).

contributing/core-developers/index.qmd

@@ -0,0 +1,34 @@
+---
+title: For Core Developers
+---
+
+This page contains notes that are specific to core developers of TuringLang.
+Everything on the other Contributing pages applies to core devs too; this is just the extra stuff.
+
+# Branches and PRs
+
+You don't need to make your own fork of the package you are editing.
+Just make a new branch on the main repository, usually named `your-username/change-you-are-making` (we don't strictly enforce this convention though).
+You should definitely still make a branch and a PR, and never push directly to `main` or `breaking`.
+
+All code should generally be reviewed by another core developer and pass continuous integration (CI) checks.
+Exceptions can be made in some cases though, such as ignoring failing CI checks where the cause is known and not due to the current pull request, or skipping code review when the pull request author is an experienced developer of the package and the changes are trivial.
+
+# Making a release
+
+You can make a release of the package after your work is merged into `main`.
+This is done by leaving a comment on the latest commit on `main`, saying
+
+```
+@JuliaRegistrator register
+
+Release notes:
+
+[YOUR RELEASE NOTES HERE]
+```
+
+It is often easiest to take the release notes right from the `HISTORY.md` file (which is why we ask that PRs include changelog entries).
+
+If you are making a breaking release, your release notes must also contain the string `Breaking changes` somewhere in them (this is mandated by the `@JuliaRegistrator` bot).
+
+The `@JuliaRegistrator` bot will handle creating a pull request into the Julia central package repository and tagging a new release in the repository.

contributing/documentation/index.qmd

@@ -0,0 +1,75 @@
+---
+title: Documentation
+---
+
+# Navigating Turing's documentation system
+
+Each of the packages in the Turing ecosystem (see [Libraries](/library)) has its own documentation, which is typically found in the `docs` folder of the corresponding package.
+For example, the source code for DynamicPPL's documentation can be found in [its repository](https://github.com/TuringLang/DynamicPPL.jl).
+
+On top of the library-specific documentation, we also have a general documentation repository, which is what builds the website you are currently reading!
+Anything that appears in `turinglang.org/docs` is built from the [`docs` repository](https://github.com/TuringLang/docs).
+
+Other sections of the website (anything that isn't a package, or a tutorial) – for example, the list of libraries – is built from the [`turinglang.github.io` repository](https://github.com/TuringLang/turinglang.github.io).
+
+*In general, we prefer documentation for Turing users to be written on the `docs` repository.*
+This is because it is more easily discoverable for users via the search bar and sidebar.
+
+Documentation written on the individual package repositories can be found via the main site's search bar (due to a GitHub workflow that scrapes all the packages' contents and indexes them here), but once you navigate to a package's documentation, you cannot then use the sidebar to come back to the main documentation site.
+As such, we tend to only use package-specific documentation for developer notes and API docs.
+
+## Documenting unreleased features
+
+There are sometimes cases where it is not possible to add docs to the `docs` repository.
+In particular, because the `docs` repo builds from a released version of Turing and all its dependencies, new features in unreleased versions cannot be documented here.
+However, it's always better to document things as you go along rather than to wait for a release and then play catch-up!
+
+In such instances, we recommend first adding documentation to the relevant package's internal documentation (using Documenter.jl as usual), and later copying it over to the main `docs` repository (adjusting for the Quarto format) once the new version is released.
+Note that because the `docs` repository is tied to a specific version of Turing.jl, if you have updated the documentation for a new dependency of Turing (e.g. DynamicPPL or Bijectors), you also need to ensure that there is a version of Turing.jl that is compatible with that new version.
+
+# How to contribute to the docs repo
+
+## Local development
+
+The `TuringLang/docs` repository uses Quarto.
+You can add a new page by creating a new `.qmd` file and including its filepath in the top-level `_quarto.yml` file, which defines the structure of the website.
+Generally, we prefer adding individual `index.qmd` files in new folders, as this leads to cleaner URLs.
+
+When you create a new page, you can test it locally by running `quarto render /path/to/mynewpage/index.qmd`, and then launching a HTTP server from the folder containing the rendered HTML file (for example, using `cd _site/path/to/mynewpage && python -m http.server`).
+
+Note that if you use `quarto preview`, this may take a very long time since Quarto will attempt to render the entire site!
+You can see the [repository README](https://github.com/TuringLang/docs) for some information on how to get around this using the `_freeze` folder.
+
+## The docs environment
+
+The `docs` repository is built from a single Manifest file, which contains a pinned version of Turing.jl.
+All notebooks are run with the same environment, which ensures consistency across the documentation site.
+
+In general, you should **not** add new packages to this environment which **depend** on Turing (i.e., reverse dependencies of Turing), or packages that have Turing extensions.
+The reason for this is because such packages will have compatibility bounds on Turing.
+Thus, we will be unable to update the docs to use the newest Turing version, until and unless those packages also update their compatibility bounds.
+
+Whenever a new version of Turing.jl is released, the Manifest file should ideally be updated (with a single `] up` in the Julia REPL), and the new docs published.
+This process is not yet automated, though, so sometimes the version of Turing may fall behind the latest release.
+Usually for minor versions we make sure to update the docs as soon as practicable, but less so for patch versions.
+
+## GitHub CI
+
+Each PR to the `docs` repository will trigger two GitHub Actions workflows:
+
+1. The workflow that builds the documentation. If this workflow fails, it's usually due to an error in the code, or some environment resolution issues.
+
+   :::{.callout-tip}
+   If you specifically want to demonstrate the *presence* of an error, you can add `#| error: true` to the top of the code block in question.
+   :::
+
+2. The workflow that checks that the version of Turing is the newest possible version.
+   If this workflow fails, it's because the Manifest file contains an older version of Turing.
+
+Note that the build-docs workflow utilises a lot of caching in order to speed up the build process (a full docs build can take 1.5 hours).
+However, this caching is dependent on the Manifest file not changing.
+Therefore, be aware that if you update the Manifest file, the next workflow run will require a full build!
+
+**If you are writing new docs that do not rely on a newer version of any dependency, it is often better to avoid updating the Manifest file, and instead just add the new docs.**
+In this case, it's fine to let the check-version CI workflow fail.
+You can add a Manifest update in a separate PR.

contributing/pull-requests/index.qmd

@@ -0,0 +1,80 @@
+---
+title: Pull Requests, Versions, and Releases
+---
+
+Don't worry about getting everything right on your first PR — we're happy to help you through the process.
+Even an imperfect or half-finished PR is welcome; it's often easier to discuss changes with code in front of us.
+
+Once you open a PR, someone from the dev team will review your code (if they don't, ping `@TuringLang/maintainers` in a comment to get their attention).
+If all looks good, we'll merge it. If not, we'll help you fix it and then merge it.
+
+# Which branch should I target?
+
+All TuringLang repositories follow [semantic versioning](https://semver.org/); however, the way we manage this differs between repositories.
+
+**Turing.jl and DynamicPPL.jl** retain two long-lived branches: `main` and `breaking`.
+
+When choosing a target branch for your PR, consider whether your change is breaking:
+
+- **`main`** is for non-breaking changes: bug fixes, new features that don't change existing behaviour, documentation, etc.
+- **`breaking`** is for changes that lead to incompatibilities with previous versions' public API.
+  Because we don't like introducing breaking changes too frequently, we tend to aggregate multiple breaking changes into a single release.
+  This is why we have a separate `breaking` branch: it allows us to merge breaking changes as they come in without having to wait for a release.
+
+If you're unsure about whether your change is breaking, just let us know in the PR: we can help you figure it out.
+
+**Other repositories** don't have separate `main` and `breaking` branches.
+In this case, if you have a breaking change, just target `main` and make sure to increment the appropriate version in the `Project.toml` file (see below).
+
+# Version numbers
+
+For packages which have a version number of `0.x.y`:
+
+- If the change you're making is non-breaking, you should increment the patch version (the last number) in the `Project.toml` file.
+  For example, 0.3.4 would go to 0.3.5.
+
+- If it's a breaking change, you should increment the minor version (the middle number) and reset the patch version to zero.
+  For example, 0.3.4 would go to 0.4.0.
+
+For packages which have a version number of `1.x.y` or higher:
+
+- If you are only providing a bug fix, and not introducing any new features, you can increment the patch version (the last number) in the `Project.toml` file.
+  For example, 1.2.3 would go to 1.2.4.
+
+- If you are introducing new features but not breaking any existing functionality, you should increment the minor version (the middle number) and reset the patch version to zero.
+  For example, 1.2.3 would go to 1.3.0.
+
+- If you are introducing breaking changes, you should increment the major version (the first number) and reset the minor and patch versions to zero.
+  For example, 1.2.3 would go to 2.0.0.
+
+# Changelogs
+
+When you make a PR, please add an entry to `HISTORY.md` describing the change.
+You can see existing entries for examples of the style.
+
+If you are working on a repo that does not already have a `HISTORY.md` file, please consider creating one at the root of the repository!
+A changelog that's started late is still better than no changelog at all.
+
+# What happens after I open a PR?
+
+## Code review
+
+A maintainer will review your code.
+This is a collaborative process: don't worry if you get comments or change requests, that's normal and expected.
+
+## CI checks
+
+We run the test suite in GitHub Actions for several Julia versions.
+Generally, all tests and code formatting checks should pass before merging.
+
+If the formatting check fails, please follow [the instructions on the Code Formatting page]({{< meta contributing-code-formatting >}}) to fix it.
+
+Some CI checks are allowed to fail:
+
+- **Julia prereleases:** These inform us of upcoming issues but don't block merging.
+
+- **Code coverage:** It's often helpful to inspect these, but we don't enforce specific coverage numbers.
+
+- **Upstream bugs:** Occasionally CI fails due to issues in dependencies or base Julia.
+  A good indicator is if the same test fails on the base branch too.
+  Do ping a maintainer if you're unsure about whether a CI failure is expected.