diff --git a/for-developers/git-workflows/git-crashcourse.md b/for-developers/git-workflows/git-crashcourse.md index 1718817..005ba90 100644 --- a/for-developers/git-workflows/git-crashcourse.md +++ b/for-developers/git-workflows/git-crashcourse.md @@ -11,6 +11,14 @@ For first-time Git and GitHub users, contributing to open source can feel overwh Companion is open source—anyone can view and modify the code. While you're free to customize it for your own use, maintaining software quality requires coordination. Git and GitHub are the tools we use to make this possible: GitHub hosts the source code and metadata, while Git manages version control and transfers code between repositories. +:::tip + +1. A GUI front-end such as SmartGit, GitGUI, can greatly simplify the overall process. See the tip under [Installing Git](installing-git#install-git) for our recommendations. + +2. The instructions on this page describe how to contribute to repositories for which you don't have write permission, which is typically the case when you're contributing to an existing repository. Eventually you may get write-access, for example if you become the maintainer of a Companion module, in which case the flow can be simplified...but it's still good practice to develop new features on a new branch. + +::: + ## The Fork and Clone Workflow ### Why You Can't Edit Directly diff --git a/for-developers/git-workflows/github-workflow.md b/for-developers/git-workflows/github-workflow.md index 26c5a02..5a43c55 100644 --- a/for-developers/git-workflows/github-workflow.md +++ b/for-developers/git-workflows/github-workflow.md @@ -9,9 +9,14 @@ description: Introduction to git and github workflow for beginners The following is the basic workflow for contributing to Companion (and many other open-source repositories) on GitHub. The order of operations is a key to keeping everything straight... This page assumes basic knowledge of Git and GitHub. To learn Git, check out our [Git Crashcourse](git-crashcourse), or [Git's documentation](https://git-scm.com/learn) or the many online resources. +The examples below use the core Companion repo, bitfocus/companion, for specificity. If you are contributing to a module, substitute the module's repository name for bitfocus/companion. (For example, bitfocus/companion-module-generic-http.) + :::tip -A GUI front-end such as SmartGit, GitGUI, can greatly simplify the overall process. See the tip under [Installing Git](installing-git#install-git) for our recommendations. -::: + +1. A GUI front-end such as SmartGit, GitGUI, can greatly simplify the overall process. See the tip under [Installing Git](installing-git#install-git) for our recommendations. + +2. The instructions on this page describe how to contribute to repositories for which you don't have write permission, which is typically the case when you're contributing to an existing repository. Eventually you may get write-access, for example if you become the maintainer of a Companion module, in which case the flow can be simplified...but it's still good practice to develop new features on a new branch. + ::: :::note[Expert Note] The following outline names the bitfocus remote "origin" and your remote fork "personal". This seems much clearer than the convention of naming the bitfocus repo "upstream" and your remote fork "origin" (which it is not!). The convention recommended here eliminates a step and also ensures that your pulls come from the Bitfocus repo. Many thanks to Brian Teeman for this and other suggestions. @@ -27,6 +32,8 @@ If you are cloning to a Windows computer be sure to [configure git line-endings git clone https://github.com/bitfocus/companion.git ``` +(module developers: substitute the appropriate GitHub location.) + :::note The Bitfocus repository is read-only for most users. In the next step we will create a writeable "fork" of the Bitfocus repo that will be used for sending in proposed code changes (PRs) ::: @@ -35,7 +42,7 @@ The Bitfocus repository is read-only for most users. In the next step we will cr ## 1b. Fork Companion to your GitHub account -Back on the [Companion Website](https://github.com/bitfocus/companion), click on the `Fork` button in the Code tab. This will create a repository under your Github account that you can write to. +Back on the [Companion GitHub page](https://github.com/bitfocus/companion) or the module's GitHub page, click on the `Fork` button in the Code tab. This will create a repository under your Github account that you can write to. **updating**: Once the fork has been created, use the "Sync Fork" button in GitHub to keep it up-to-date. @@ -43,7 +50,7 @@ Back on the [Companion Website](https://github.com/bitfocus/companion), click on Your local repository will need two "know" two remote (GitHub) repos -- one for keeping in sync with Bitfocus and the other for writing your code changes. We suggest naming this second one "personal" for clarity -- it is your personal fork of companion _on GitHub_: -On the GitHub page for your new fork, copy the HTTPS link to your fork using the green **<>Code** button. Then paste it on a line starting with `it remote add personal`. It should look like this: +On the GitHub page for your new fork, copy the HTTPS link to your fork using the green **<> Code ▼** button. Then paste it on a line starting with `it remote add personal`. It should look like this (for core Companion): ```bash git remote add personal https://github.com//companion.git @@ -52,7 +59,7 @@ git remote add personal https://github.com//companion.git (In a GUI frontend you would select "Add Remote..." from a menu. SmartGit, for example, will automatically pick up the URL from your clipboard when you do this.) :::note -By default, the Bitfocus remote that was cloned above, was named "origin". So "origin" refers to Bitfocus (the "origin" of your clone) and "personal" refers to your fork on GitHub. +By default, the Bitfocus remote that you cloned above, was named "origin". So "origin" refers to Bitfocus (the "origin" of your clone) and "personal" refers to your fork on GitHub. ::: ## 2. Create a new branch in your local repo @@ -61,7 +68,7 @@ By default, the Bitfocus remote that was cloned above, was named "origin". So "o git switch -c ``` -This creates and checks out the new branch. +(In a GUI frontend you would select "Add Branch..." from a menu.) This creates and checks out the new branch. Next, modify/add code for your feature of bug-fix using your favorite coding tools. @@ -71,7 +78,7 @@ When you are satisfied with some or all of your changes, commit them to the new git commit -m 'message' ``` -This is where you really want to be using a GUI front-end to git for managing you repo! +This is where you really want to be using a GUI front-end to git to keep track of everything! ## 3. Push the branch to your GitHub fork @@ -79,7 +86,7 @@ This is where you really want to be using a GUI front-end to git for managing yo git push -u personal ``` -After the first push, you can push further commits on that branch using just `git push` +(In a GUI frontend you would select "Push to..." from a menu and select "personal".) After the first push, you can push further commits on that branch using just `git push` ## 4. Submit a Pull Request (PR) to Bitfocus diff --git a/for-developers/module-development/home.md b/for-developers/module-development/home.md index 442faa1..33f4243 100644 --- a/for-developers/module-development/home.md +++ b/for-developers/module-development/home.md @@ -6,13 +6,14 @@ description: Developer environment setup specific to module development. --- So, you want to develop a module for Companion? Welcome! -This section describes what you need to know and do to make a beautiful module to control your favorite gear with Companion. + +Companion uses plug-ins to expand its capabilities, we call these plug-ins "modules". For every device you can control with Companion there is a "module" that manages the connection. This page describes how to set up your computer for developing Companion modules. Subsequent pages will provide details on the contents of the module and its lifecycle. ## Prerequisites Before you start, make sure you have [Installed the Development Tools](../setting-up-developer-environment.md) and familiarized yourself with [Git Workflows](../git-workflows/git-crashcourse.md). -This page discusses additional setup needed in order to develop or contribute to Companion modules. +This page discusses additional setup specific to Companion modules. ## Install Companion @@ -21,44 +22,61 @@ You can develop modules against any standard release or beta build of Companion. Download and install a recent version from [the website](https://bitfocus.io/user/downloads) :::tip -Starting with Companion v4.0, the module release cycle has been separated from the Companion release cycle: The timing of module version releases are determined by the module developer independently of the Companion releases. +Starting with Companion v4.0, the module release cycle has been decoupled from the Companion release cycle: The timing of module version releases are determined by the module developer independently of the Companion releases. We therefore recommend developing your module using the current stable version to ensure that users will be able to use your module right away instead of having to wait until the current beta is released. ::: -## Set up Companion for Module Development +## Set up the Development Folder -Follow the instructions in [Setting up a Development Folder](./local-modules.md) to create the folders and to setup Companion for module development. Briefly: +- Create a modules development folder somewhere on your computer. You can call it whatever you like. +- Open the companion launcher and click the cog in the top right. In versions prior to v4.1 this will reveal a 'Developer modules path' field. Starting in Companion v4.1 it will open up a settings window in which you can set the path. Set it to the development folder you just created. -- Create a `companion-module-dev` folder somewhere on your computer. You can call it whatever you like. -- Once you have the companion launcher open, click the cog in the top right. In versions prior to v4.1 this will reveal a 'Developer modules path' field. Starting in v4.1 it will open up a settings window in which you can set the path. Set it to the `companion-module-dev` folder you created earlier. +Please see the next page: [Setting up a Dev Folder](./local-modules.md) for full details. :::tip -Companion will load any modules it finds in subfolders of that folder. -When files inside those folders is changed, Companion will automatically restart any connections using that module. +Companion will load any modules that are in _subfolders_ of the modules development folder. +However, starting with v4.0, note that **your new dev module will show up in the Connections page** but not in the Modules page, since it is not yet part of the official set of Companion modules. + ::: -## Clone the module +## Clone or Copy a module from GitHub + +With over 700 published modules, you may find a module already written for you device. You can find the GitHub repository for each module by searching [Bitfocus in GitHub](https://github.com/bitfocus). + +- If you want to **contribute to an existing module**, clone it to your development folder using your [preferred git tool](../git-workflows/installing-git.md). (See [Git Crash Course](../git-workflows/git-crashcourse.md) and [The GitHub Workflow](../git-workflows/github-workflow.md) for important details). +- If you are **writing a new module**, then download and expand the zip file for the [TypeScript template module](https://github.com/bitfocus/companion-module-template-ts/archive/refs/heads/main.zip) into the module development folder (see the next section on naming your module). If you prefer, you can clone the [template repo](https://github.com/bitfocus/companion-module-template-ts), but remember to delete the remote link (`git remote remove origin`) before continuing. We encourage you to use Typescript for all new modules but acknowledge that it isn't for everyone, so we fully support [Javascript](https://github.com/bitfocus/companion-module-template-js) too! +- Alternatively for a new module, you can start by downloading the zip or cloning a module that controls a device similar to yours -- both options can be found under the the green **<> Code ▼** button for that module's GitHub repo. Beware: by using another module as a base, you could inherit some subtle misconfigurations or deviations they have made from our recommendations. As with using the template, if you cloned the repo be sure to remove the remote connection by running `git remote remove origin`, since you will be creating a new module rather than modifying the existing one. + +### Additional steps for new modules + +1. Rename your module's directory _companion-module-mymanufacturer-myproduct_, replacing _mymanufacturer-myproduct_ with appropriate names. Try to think of what is most appropriate for your device: Are there other similar devices by the manufacturer that use the same protocol that the module could support later on? If so try and name it to more easily allow for that. + +2. In at least _package.json_, _companion/manifest.json_ and _companion/HELP.md_, edit the name and description of the module to match what yours is called. The search feature in your IDE is really helpful to find all of the places the name shows up! See the [Module Configuration section](./module-config/file-structure.md) and especially the [documentation for the manifest.json file](./module-config/manifest.json.md) for further details. + +3. Please see [Module Configuration](./module-config/file-structure.md) and the other pages in that section for more details and more options on starting your own module. + +## Install the dependencies + +In a shell inside the new folder, run the following: + +1. `yarn install` This will install any dependencies needed by the module. + +:::tip -- If you are working on an existing module, clone it inside this folder using your preferred git tool. -- If you are making a new module, then clone the template module, available in [TypeScript](https://github.com/bitfocus/companion-module-template-ts) or [Javascript](https://github.com/bitfocus/companion-module-template-js), to an appropriately named folder inside of `companion-module-dev`. If you are starting a new module we encourage you to use Typescript but acknowledge that it isn't for everyone, so fully support Javascript too! +If you are using an IDE such as [VS Code](https://code.visualstudio.com/), make the clone your current project (i.e., open the repository folder), then open a terminal (shell) inside the IDE to ensure you are running yarn in the correct folder. -:::note -Starting with v4.0, note that **your new dev module will show up in the Connections page** but not in the Modules page, since it is not yet part of the official set of Companion modules. ::: ## Start working on your module -You are now ready to start developing your module. +You are now ready to start developing your module. Here are our suggested next steps: -You should notice that whenever you save a file inside your `companion-module-dev` folder, companion will restart itself. -If it does not, or you are having issues getting your module to load in then please reach out on slack and we will be happy to assist you in getting started. +- Familiarize yourself with the [Module Configuration](module-config/file-structure.md) to understand the general file structure and configuration options, especially if working on a new module. +- Read [Module development 101](./module-development-101.md) for an overview of the development lifecycle. +- Review the recommended [GitHub Workflow](../git-workflows/github-workflow.md) to learn best practices for new features to your codebase. -## Further Reading +You can develop code while the module is running in Companion: whenever you save a file inside your module development folder, Companion will automatically restart any connections using that module. -- [Module development 101](./module-development-101.md) -- [Creating a module](https://github.com/bitfocus/companion-module-base/wiki/Creating-a-module) -- [Actions](https://github.com/bitfocus/companion-module-base/wiki/Actions) -- [Feedbacks](https://github.com/bitfocus/companion-module-base/wiki/Feedbacks) -- [Variables](https://github.com/bitfocus/companion-module-base/wiki/Variables) +If it does not, or you are having issues getting your module to load in then please reach out on [Slack](https://bfoc.us/ke7e9dqgaz) and we will be happy to assist you in getting started. diff --git a/for-developers/module-development/local-modules.md b/for-developers/module-development/local-modules.md index 2483b12..8a0e2ac 100644 --- a/for-developers/module-development/local-modules.md +++ b/for-developers/module-development/local-modules.md @@ -1,8 +1,8 @@ --- title: Setting up the Development Folder -sidebar_label: Setting a Dev Folder +sidebar_label: '└─ Setting up a Dev Folder' sidebar_position: 2 -description: How to load modules during module development. +description: How to load modules in Companion during module development. --- Starting with Companion 3.0 you can develop, test and use your own module code without the need for the Core Companion development environment. Here is how you do it. @@ -34,7 +34,7 @@ Inside of this folder should be one or more folders that use the following layou - Make sure **Enable Developer Modules** is switched on. You can now close the window - Click on "Launch GUI" to open the Admin interface. In the connections list you should find the connection provided by the developer module. If the developer module is using the same internal ID as a module that is distributed with Companion, be sure to choose the "dev" version in the configuration. If you don't see the developers module, please check the log and switch on debug, maybe the module has crashed. -- You can replace a developers module or files within it while Companion is running. Companion will detect the change and restart only that module without affecting other modules. +- You can replace a developers module or files within it while Companion is running. Companion will detect the change and restart only that module without affecting other modules. You can also force a module to reload by disabling and re-enabling a connection. :::note @@ -56,13 +56,11 @@ the **Developer Module Path** is _**mydev**_ and not _mydev/module1_. ## Headless development -- If you are running Companion from source in development mode, there is a folder `module-local-dev` inside the repository that gets read for your modules. -- It follows the same rules for structuring as above. -- Any changes made to the contents of the folder will cause the affected modules to be reloaded. You can force them to reload by disabling and re-enabling a connection. - +- If you are running Companion from source in development mode, there is a folder `module-local-dev` inside the repository that gets read for your modules. It follows the same rules for structuring as above. - You can also use the command-line argument `--extra-module-path` to specify the dev folder. - Alternatively, you can set the environment variable `COMPANION_DEV_MODULES` to the dev folder. For your convenience, you can put the following in `.env` in your companion source folder: ``` COMPANION_DEV_MODULES= ``` + diff --git a/for-developers/module-development/module-config/_category_.json b/for-developers/module-development/module-config/_category_.json new file mode 100644 index 0000000..9639e8b --- /dev/null +++ b/for-developers/module-development/module-config/_category_.json @@ -0,0 +1,11 @@ +{ + "label": "Module Configuration", + "position": 10, + "link": { + "type": "generated-index", + "title": "Configuring your companion module" + }, + "customProps": { + "description": "The files and file structure necessary to configure a module repository." + } +} diff --git a/for-developers/module-development/module-config/code-quality.md b/for-developers/module-development/module-config/code-quality.md new file mode 100644 index 0000000..fea8756 --- /dev/null +++ b/for-developers/module-development/module-config/code-quality.md @@ -0,0 +1,181 @@ +--- +title: 'Preventing Common Bugs with ESLint and Prettier' +sidebar_label: 'Code Quality Configs' +sidebar_position: 4 +description: Using prettier and a linter to prevent common bugs. +--- + +We recommend using [eslint](https://eslint.org/) and [prettier](https://prettier.io/) to improve the readability and uniformity of your code and to prevent easily-overlooked coding errors. The two tools work together to enforces the formatting and coding rules. If you are using an IDE such as VS Code, we recommend installing the prettier plugin. In that plugin you can also enable 'format on save' to automate some of the formatting. + +:::important + +Our [recommended templates](./file-structure.md) come with eslint and prettier configuration files. The remainder of this page is only necessary for people who are setting up their repo manually or want to upgrade an older module's repo... or if you just want to gain a deeper understanding of the mechanics of these config files. + +::: + +## Installation + +If you are _not_ using the [recommended templates](./file-structure.md), you can configure your module for prettier and eslint as follows: + +1. Run `yarn add --dev eslint prettier` in the root folder of your repo to install it as a dev dependency. +2. If you are using TypeScript you will also need to `yarn add --dev typescript-eslint`. +3. Add the line `"prettier": "@companion-module/tools/.prettierrc.json",` to your _package.json_ file. +4. Add the `scripts` block in your `package.json` to include: + +```json +"lint:raw": "eslint", +"lint": "yarn lint:raw ." +"format": "prettier -w ." +``` + +## Prettier config + +We recommend adding a `.prettierignore` file too, containing: + +``` +package.json +pkg +``` + +You can list other files/folders here that should be ignored. We want to ignore package.json as prettier and yarn disagree on how it should be formatted. + +## ESLint config + +A new basic config should be called `eslint.config.mjs`, and could contain: + +```js +import { generateEslintConfig } from '@companion-module/tools/eslint/config.mjs' + +export default generateEslintConfig({}) +``` + +If using typescript, you should specify a `typescriptRoot` + +```js +import { generateEslintConfig } from '@companion-module/tools/eslint/config.mjs' + +export default generateEslintConfig({ + enableTypescript: true, +}) +``` + +You can now run `yarn lint` and see any linter errors. If you need any help understanding the errors or warnings you are given, the eslint docs are really helpful. + +:::note + +If you have any suggestions on changes to make to this eslint config, do [open an issue](https://github.com/bitfocus/companion-module-tools/issues) to let us know. We hope that this set of rules will evolve over time based on what is and isnt useful to module developers. + +::: + +### Enabling linting on commit + +You can set it up so that git runs the linter when you make a commit, so you can be sure that only good code is committed. + +Install the needed tools `yarn add --dev lint-staged husky`. + +Add the `scripts` block in your `package.json` to include: + +```json +"postinstall": "husky", +``` + +And a block to the `package.json`: + +```json +"lint-staged": { + "*.{css,json,md,scss}": [ + "run prettier" + ], + "*.{ts,tsx,js,jsx}": [ + "run lint:raw" + ] +}, +``` + +Create a file `.husky/pre-commit` with the content `yarn lint-staged` + +Run `yarn husky` to ensure husky is initialised. + +### Running the linter on push + +It is a good idea to setup a Github Actions workflow to run the linter, so that you don't get surprised by unexpected linter failures when running it locally. + +To do this, create a new file in the repository at `.github/workflows/lint.yaml`, with the content: + +```yaml +name: Lint + +on: + push: + branches: + - '**' + tags: + - 'v[0-9]+.[0-9]+.[0-9]+*' + pull_request: + +jobs: + lint: + name: Lint + runs-on: ubuntu-latest + timeout-minutes: 15 + + steps: + - uses: actions/checkout@v6 + - name: Use Node.js 22.x + uses: actions/setup-node@v6 + with: + node-version: 22.x + - name: Prepare Environment + run: | + corepack enable + yarn install + yarn build + env: + CI: true + - name: Run lint + run: | + yarn lint + env: + CI: true +``` + +:::tip + +Make sure to set the correct nodejs version in the workflow + +::: + +## Customising the rules + +You can easily override rules in this setup with: + +```js +import { generateEslintConfig } from '@companion-module/tools/eslint/config.mjs' + +const baseConfig = await generateEslintConfig({ + enableTypescript: true, +}) + +const customConfig = [ + ...baseConfig, + + { + rules: { + 'n/no-missing-import': 'off', + 'node/no-unpublished-import': 'off', + }, + }, +] + +export default customConfig +``` + +## Upgrading to @companion-module/tools v2.x + +If you are upgrading your config from v1.x, you will need to reconfigure eslint, due to changes in both eslint and how the tools library provides config templates. + +You can follow the steps for [ESLint Config](#eslint-config), with the following extra steps: + +Removing any existing `.eslintrc.json` or `.eslintrc.cjs`, you may want to port any custom configuration across to the new config file + +That should be it! diff --git a/for-developers/module-development/module-config/file-structure.md b/for-developers/module-development/module-config/file-structure.md new file mode 100644 index 0000000..9174bac --- /dev/null +++ b/for-developers/module-development/module-config/file-structure.md @@ -0,0 +1,114 @@ +--- +title: 'File Structure Overview' +sidebar_label: 'Essential Files' +sidebar_position: 1 +description: Introduction to the file structure of a Companion module repository +--- + +When creating a new module, we strongly recommend using our **[TypeScript module template](https://github.com/bitfocus/companion-module-template-ts)** as this will incorporate the latest module features and improvements. For those who prefer Javascript, we also have a [JavaScript template](https://github.com/bitfocus/companion-module-template). To get an idea of how a completed module looks, search for an existing module that may provide services similar to your equipment, or refer to reference modules such as [homeassistant-server](https://github.com/bitfocus/companion-module-homeassistant-server) (TypeScript) and [generic-osc](https://github.com/bitfocus/companion-module-generic-osc) (JavaScript). + +Below are the minimum files necessary to create a control module for Companion (aka Connection). You can add subfolders and other support files as needed. + +:::tip + +All of these files, and more, are included in the templates mentioned above. Using the templates will save you time in setting +things up and even more time by making sure the essential content has been included correctly -- especially for the package.json file! + +The templates also include sample source code to get you started programming the module's functionality. + +::: + +### File Structure + +The essential repository structure includes the source code (in _src_ here), libraries that are automatically loaded by yarn (in _node_modules_) and a number of configuration files as follows: + +```text +├───companion +│ ├── HELP.md +│ └── manifest.json +├───node_modules +├───src +│ ├── main.ts +│ ├── actions.ts (these files are all optional but recommended) +│ ├── feedbacks.ts +│ ├── presets.ts +│ ├── upgrades.ts +│ └── variables.ts +│ +├── .gitignore +├── LICENSE +├── package.json +└── README.md +``` + +### companion/HELP.md + +A structured 'Help' document that Companion users will see when clicking on the module help icon in the Connections page. + +Describe the module's capabilities and anything else they might need to know to use the module. + +Format the contents using [markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax). + +### companion/manifest.json + +Provides information to Companion about the module. See the [manifest.json](./manifest.json.md) page for full details. + +### node_modules folder + +When you run `yarn install`, yarn reads the _package.json_ file and installs or updates all of the listed dependencies into the _node_modules_ folder. (It will create this folder if it doesn't already exist.) + +At a minimum, for Companion modules, it should install @companion-module/base and @companion-module/tools + +### src/main.ts (or main.js, for the JavaScript template) + +This main entrypoint for the module. The file and folder can be called something else, as long as the `"runtime": { "entrypoint": }` field in `companion/manifest.json` and the `"main":` field in `package.json` are updated to match. (Older JavaScript repos skip the subdirectory altogether, though this is no longer recommended.) + +Generally you will create additional source-code files for each major module element: actions, feedback, presets, variables, user-configurations and upgrade scripts. (These files are already present in the template modules mentioned above.) + +:::note + +When TypeScript is "transpiled" into JavaScript, the code in `src/` is used to generate a `dist/` folder containing the corresponding JavaScript code. In a TypeScript repo, the manifest and package.json files therefore refer to the `dist/` folder even though your development code is in the `src/` folder. (But note that `dist/` is _not_ part of the git repository and must be listed in .gitignore file.) + +::: + +### .gitignore + +This is a standard Git file that lists files and folders to be excluded from Git version control. `node_modules/` should always be included in the `.gitignore` as should `dist/` in TypeScript modules. + +### LICENSE + +Companion is an MIT licensed project. We recommend modules released with the project are also MIT, and are open to other licenses being used if there is a good reason for it. + +In the future it might be possible to use different licenses for modules, but that is not yet certain. + +:::important + +Consult the Companion team if you wish to incorporate a dependency that does not have an MIT license. + +::: + +### package.json + +This is a standard node.js file to tell it about your project. It is required to be able to install dependencies to your module such as [@companion-module/base and and @companion-module/tools](../module-lifecycle/companion-module-library.md). + +### README.md + +This file should include any relevant developer documentation that the Companion Core and Module Development teams should be aware of, as well as any helpful information for people who wish to fork and contribute. It is only shown on github and when editing the module, so can be reasonably technical. + +As with the HELP.md file, you can format it with markdown. + +### Additional files for good practices + +In addition to the essential files we recommend several other files that help enforce good coding practices. +These files are all included in the templates recommended above and are discussed on separate pages of this section. + +- [Code Quality Enforcement](./code-quality.md): prettier and eslint configuration +- [TypeScript configuration](./typescript-config.md) if you are using TypeScript (as recommended) +- [Unit Testing Setup](./unit-testing.md) (there's not much there..) + +## Next steps + +- Set up your [_manifest.json_](./manifest.json.md) and _HELP.md_ files in the _companion/_ folder. +- Look through the optional files mentioned above +- Familiarize yourself with the [Companion Module Library](../module-lifecycle/companion-module-library.md) concept +- Read through our introduction to module development: [Module Development 101](../module-development-101.md) diff --git a/for-developers/module-development/module-config/manifest.json.md b/for-developers/module-development/module-config/manifest.json.md new file mode 100644 index 0000000..31d9b2e --- /dev/null +++ b/for-developers/module-development/module-config/manifest.json.md @@ -0,0 +1,71 @@ +--- +title: 'manifest.json Config' +sidebar_label: 'manifest.json file' +sidebar_position: 2 +description: Specification of the Companion manifest file +--- + +Starting with Companion 3.0, Companion looks at the `companion/manifest.json` file for module information (before 3.0 it looked at `package.json`) This provides a companion specific and programming language agnostic manifest about your module. In the future this will allow us to do more powerful things! + +You can see the auto-generated documentation for this file as a typescript interface [here](https://bitfocus.github.io/companion-module-base/interfaces/ModuleManifest.html). + +Tip: At any point you can validate your `manifest.json` by running `yarn companion-module-check`. + +### Format + +If you are comfortable reading or working with JSON Schema, you can find the formal definition [here](https://github.com/bitfocus/companion-module-base/blob/main/assets/manifest.schema.json) + +A full manifest definition is: + +```json +{ + "id": "fake-module", + "name": "fake module", + "shortname": "fake", + "description": "Fake Module", + "manufacturer": "Fake Module", + "products": ["Fake"], + "keywords": ["Fake"], + "version": "0.0.0", + "license": "MIT", + "repository": "git+https://github.com/bitfocus/companion-module-fake-module.git", + "bugs": "https://github.com/bitfocus/companion-module-fake-module/issues", + "maintainers": [ + { + "name": "Your name", + "email": "your.email@example.com" + } + ], + "legacyIds": [], + "runtime": { + "type": "node22", + "api": "nodejs-ipc", + "apiVersion": "0.0.0", + "entrypoint": "../dist/index.js" + } +} +``` + +### Properties + +- `id` unique id of your module. This has to match the repository name excluding the `companion-module-` +- `name` ??? +- `shortname` ??? +- `description` ??? +- `manufacturer` ??? +- `products` An array of strings with the names of the products supported by your module. Often there is only a single product or sometimes the name of a series of products is better known than the products itself, then it is also good to give the series name. Your module will be listed with all associated product names. +- `keywords` Keywords to allow users to more easily find your module by searching, please do not repeat the manufacturer or product names here. +- `version` Version of your module. This should be left as `0.0.0`, the build process populates with the value in your `package.json` +- `license` License of your module code. This needs to be something MIT compatible to be allowed to be distributed with the official builds +- `repository` Git URL to the repository +- `bugs` URL to the issue tracker. Users can follow this link to report bugs +- `maintainers` List of maintainers of this module. +- `legacyIds` (Optional) List of old module ids. If the module has been renamed, the old names should be listed here to allow for seamless upgrading of old configs +- `runtime` This defines the runtime requirements of your module. Described more below + +The runtime block is defined as: + +- `type` This can be either `node18` or `node22`, depending on the required version of Node.js. In the future this may allow for other languages to be used. +- `api` This must be `nodejs-ipc`. It defines the protocol used between your module and Companion. In the future more options will be possible, to allow for other languages. +- `apiVersion` This should be left as `0.0.0`, the build process populates with the correct value +- `entrypoint` The main javascript file for your module. This is what companion will execute to start your module. diff --git a/for-developers/module-development/module-config/typescript-config.md b/for-developers/module-development/module-config/typescript-config.md new file mode 100644 index 0000000..92cb578 --- /dev/null +++ b/for-developers/module-development/module-config/typescript-config.md @@ -0,0 +1,68 @@ +--- +title: 'TypeScript Config with tsconfig.json' +sidebar_label: 'TypeScript Config' +sidebar_position: 5 +description: Configuring the module to work with TypeScript. +--- + +:::important + +This is an advanced topic. If you use the [recommended templates](./file-structure.md), a default typescript +config file is included and you will generally not want to change it. + +::: + +The [recommended templates](./file-structure.md) provide typescript config presets in _tsconfig.json_ that we believe to be best practise, but they can be configured to be too strict for some, or may need to be modified if you change the name of the source or destination directories. + +A typical _tsconfig.json_ file looks like: + +```json +{ + "extends": "@companion-module/tools/tsconfig/node22/recommended", + "include": ["src/**/*.ts"], + "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"], + "compilerOptions": { + "outDir": "./dist", + "baseUrl": "./", + "paths": { + "*": ["./node_modules/*"] + } + } +} +``` + +Our TypeScript template splits it into two files: + +```json +//tsconfig.json +{ + "extends": "./tsconfig.build.json", + "include": ["src/**/*.ts"], + "exclude": ["node_modules/**"], + "compilerOptions": { + "types": ["node" ] + } +} +``` + +```json +// tsconfig.build.json +{ + "extends": "@companion-module/tools/tsconfig/node22/recommended", + "include": ["src/**/*.ts"], + "exclude": ["node_modules/**", "src/**/*spec.ts", "src/**/__tests__/*", "src/**/__mocks__/*"], + "compilerOptions": { + "outDir": "./dist", + "baseUrl": "./", + "paths": { + "*": ["./node_modules/*"] + }, + "module": "Node16", + "moduleResolution": "Node16" + } +} +``` + +You are free to override properties as you wish, this is only our recommendation after all. + +If you have any suggestions on changes to make to this base tsconfig, do [open an issue](https://github.com/bitfocus/companion-module-tools/issues) to let us know. We hope to collect some alteranate presets along with recommended. diff --git a/for-developers/module-development/module-config/unit-testing.md b/for-developers/module-development/module-config/unit-testing.md new file mode 100644 index 0000000..ecec7f4 --- /dev/null +++ b/for-developers/module-development/module-config/unit-testing.md @@ -0,0 +1,20 @@ +--- +title: 'Unit Testing Considerations' +sidebar_label: 'Unit Testing Config' +sidebar_position: 6 +description: Unit testing config and advice. +--- + +If you choose to include unit tests in your module repo, you can choose a framework such as Vitest, Mocha or Jest. [Vitest](https://vitest.dev/) may be especially useful for its native support of TypeScript. However, unit testing of modules is not very common: We find it quite hard to test most modules as to do so requires mocking-up the device connection. + +For TypeScript repos, you may need to add information to `compilerOptions": {"types": [] }`. + +If you have any suggestions on mocks, tooling or examples we should provide to make this easier, we are open to suggestions! + +Some modules which are known to have some unit tests: + +- [grassvalley-kaleido-solo](https://github.com/bitfocus/companion-module-grassvalley-kaleido-solo/blob/main/index.test.js) +- [etc-eos](https://github.com/bitfocus/companion-module-etc-eos/blob/master/main.test.js) +- [youtube-live](https://github.com/bitfocus/companion-module-youtube-live/tree/master/src/__tests__) + +Feel free to add yours here. diff --git a/for-developers/module-development/module-development-101.md b/for-developers/module-development/module-development-101.md index edcb431..2ce40a9 100644 --- a/for-developers/module-development/module-development-101.md +++ b/for-developers/module-development/module-development-101.md @@ -1,81 +1,44 @@ --- title: Module Development 101 sidebar_label: Module Development 101 -sidebar_position: 2 +sidebar_position: 5 description: The essential overview of Companion module development. --- -With your environment setup, it is time to start looking at modules. +With your [local environment setup](./home.md), it is time to start looking at modules in greater detail. By now you should be familiar with Git and GitHub workflows, but if not, start by reading our [GitHub Crash Course](../git-workflows/git-crashcourse.md). -Companion uses plug-ins to expand its capabilities, we call these plug-ins modules. For every device you can control with Companion there is a "module", which is both a Javascript and Companion term. +The general lifecycle of a module starts when you create a local repository as described in the [Getting Started](./home.md) page. Next you fill in the code to control your device. Finally you release it for others to use. Once you have completed one cycle, you continue by maintaining and updating your codebase. -## Changing existing modules +## What makes up a module? -When you want to make changes to an existing module, you need to fork the module's repository. This gives you somewhere to store your changes, so that you can open a request for them to be merged into the official version. -You can find the GitHub repository for each module by searching [here](https://github.com/bitfocus), there is a fork option in the top right of each module page. -You can then clone this version into your `companion-module-dev` folder. +There are a few files that make up every module. Please familiarize yourself with the basic structure described in our pages on [Module Configuration](module-config/file-structure). In particular, _package.json_, +[_companion/manifest.json_](./module-config/manifest.json.md) and _companion/HELP.md_ define the identity of the module. Once these are defined, you will spend most of your time crafting the module source code. -## Creating a new module +## The module source code -With over 700 published modules, often you'll want to use another module's code as your base. First you'll need to create a directory to develop your module. +While you can handle all your module's code in one big file, we strongly recommend splitting it across several files as illustrated [here](./module-config/file-structure#file-structure). -Before you start, make sure there isn't already a module for the same device [here](https://developer.bitfocus.io/modules/companion-connection/discover). Perhaps there is one that is for a similar device which uses the same protocol? +To understand what is needed in a module it helps to understand how the code is used. Your module is presented to Companion as a class that extends the module base class. A user can add one or more _instances_ of your module to their Companion site. When Companion starts up, it initializes each instance of the module by starting a new process and passing configuration information, as described next. -Make a directory `companion-module-mymanufacturer-myproduct` inside your `companion-module-dev`. We follow a `mymanufacturer-myproduct` naming scheme, try to think of what is most appropriate for your device. -Are there other similar device by the manufacturer that use the same protocol that the module could support later on? If so try and name it to more easily allow for that. - -You now need to decide if you want to start from scratch, or start with the code from another module. Github provides a `download as zip` option, so you can download either the [template](https://github.com/bitfocus/companion-module-template-js) or another module (the [generic](https://github.com/bitfocus?q=generic&type=source&language=&sort=) modules are often a good starting point). Extract the zip to the folder you created. - -Note: By using another module as a base, you could inherit some subtle misconfigurations or deviations they have made from our recommendations. The guide should still make sense, but may not be a perfect match. - -In a shell inside the new folder, you should run the following: - -1. `yarn` This will install any dependencies needed by the module. - -In your IDE of choice (we recommend [VS Code](https://code.visualstudio.com/)), you should start by editing the name of the module to match what yours is called. The search feature is really helpful for this! - -Now have a look around and see what you can figure our from the code. -These other pages will help explain some of the functionality that the module exposes to users: - -TO-DO: Update links - -- [Module Configuration](https://github.com/bitfocus/companion-module-base/wiki/Module-development-101) -- [Actions](https://github.com/bitfocus/companion-module-base/wiki/Actions) -- [Feedbacks](https://github.com/bitfocus/companion-module-base/wiki/Feedbacks) -- [Presets](https://github.com/bitfocus/companion-module-base/wiki/Presets) -- [Variables](https://github.com/bitfocus/companion-module-base/wiki/Variables) - -## What makes up a module - -There are a few files that make up every module. To get an overview of what these are, please see [File Structure](https://github.com/bitfocus/companion-module-base/wiki/File-Structure). - -## The Module source code - -While you can handle all your module's stuff in one big file, we strongly recommend splitting it across several files, but it's up to you. -When the user adds an instance in Companion, it instantiates the module. This will start your module in a new process, and do some basic setup with Companion. - -There are various tasks to do to get a working module, which can be divided in several categories: +### Base module implementation -1. Base module implementation -2. Providing functionality to the user -3. The code behind all that stuff +In the [typical module structure](./module-config/file-structure#file-structure), the entrypoint and module class are defined in _src/main.ts_. When your module is started, first the `constructor` of your module's class will be called, followed by your [upgrade scripts](https://github.com/bitfocus/companion-module-base/wiki/Upgrade-scripts) and then the `init` method. -### Base module implementation +Your constructor should only do some minimal class setup. It does not have access to the configuration information, so it should not be used to start doing things. Instead... -When your module is started, first the `constructor` will be called, followed by your [upgrade scripts](https://github.com/bitfocus/companion-module-base/wiki/Upgrade-scripts) and then the `init` method. -Your constructor should only do some minimal class setup. It does not have access to your instance config, so cannot be used to start doing things. +The `init` method is passed any previously-stored user-config information. (The structure of the user-config is defined by you and used as the type of the generic base class.) Inside the `init` method you should initiate the connection to your device (but dont await it! Instead use the The `updateStatus()` method to inform Companion asynchronously of the connection status). Here you also pass the module's actions, feedbacks, variables and presets to Companion for the first time. -Inside of the `init` method you should initiate the connection to your device (but dont await it!) and get everything working ready for actions and feedbacks to start being used. +Once the module is running the `configUpdated` (but not `init`) method will be called if the user changes the config on the Connections page. (You can programmatically change the user config by calling `saveConfig()`, defined in the base class.) You can also update the action, feedback, etc. definitions at any time. When the module gets deleted or disabled the `destroy` function is called. here you should clean-up whatever you don't need anymore. Make sure to not leave timers running, as that can cause performance problems in companion as the leaked timers start piling up! ### Providing functionality to the user -Most (so far all) modules do want to provide some interaction with the user. The possible items are stored in json objects. This splits up in several categories. +Your module provides interaction with the user by defining user-configurations, actions, feedbacks, and variables. In addition you can define "preset" buttons that predefine combinations of common actions and feedbacks for the user's convenience. These presets can be dragged onto the button grid for "instant button configuration". -TO-DO: Update links +TODO: Update links -- [Module Configuration](https://github.com/bitfocus/companion-module-base/wiki/Module-development-101) +- [Module Configuration](module-config/file-structure.md) - [Actions](https://github.com/bitfocus/companion-module-base/wiki/Actions) - [Feedbacks](https://github.com/bitfocus/companion-module-base/wiki/Feedbacks) - [Presets](https://github.com/bitfocus/companion-module-base/wiki/Presets) @@ -83,10 +46,9 @@ TO-DO: Update links ### Printing to log -While developing you might want to print some info or variables to the console or the in companion log. -The below commands will help you do just that: +You might want to print some info or variables to the console or the in companion log both to aid in development and to help users identify what may have gone wrong -For printing to the module debug log: +For printing to the module debug log use: - `console.log('your data/message');` @@ -94,7 +56,7 @@ And if you want it in the log in the web interface, see [the log method](https:/ ## Sharing your code -Once your module is tested and you are ready to release it publicly, please follow the guide for [releasing your module](https://github.com/bitfocus/companion-module-base/wiki/Releasing-your-module). +Once your module is tested and you are ready to release it publicly, you will use the [BitFocus Developer Portal](https://developer.bitfocus.io/modules/companion-connection/discover) to list it with Companion. Please follow the guide for [releasing your module](https://github.com/bitfocus/companion-module-base/wiki/Releasing-your-module). If your module it not intended for public release, or you want to share it locally for testing, you can also read the guide on [packaging your module](https://github.com/bitfocus/companion-module-base/wiki/Module-packaging). diff --git a/for-developers/module-development/module-lifecycle/_category_.json b/for-developers/module-development/module-lifecycle/_category_.json new file mode 100644 index 0000000..8a18914 --- /dev/null +++ b/for-developers/module-development/module-lifecycle/_category_.json @@ -0,0 +1,11 @@ +{ + "label": "Module LifeCycle", + "position": 30, + "link": { + "type": "generated-index", + "title": "Maintaining you Companion module over time" + }, + "customProps": { + "description": "The task necessary to maintain and upgrade a module repository over time." + } +} diff --git a/for-developers/module-development/module-lifecycle/companion-module-library.md b/for-developers/module-development/module-lifecycle/companion-module-library.md new file mode 100644 index 0000000..0770e34 --- /dev/null +++ b/for-developers/module-development/module-lifecycle/companion-module-library.md @@ -0,0 +1,50 @@ +--- +title: 'The Companion Module Libraries' +sidebar_label: '@companion-module Libraries' +sidebar_position: 7 +description: Explanation of the companion module library. +--- + +Since Companion version 3.0, we have used `@companion-module/base` and `@companion-module/tools` libraries as part of the module API. + +The libraries are available on [npm](https://www.npmjs.com/) and are installed automatically when you run `yarn install`. + +## What is the purpose of each? + +### @companion-module/base + +This library is added as a regular dependency of your module. It contains various helpers and utilities that we think are useful to modules, as well as the main base class that your module should extend. + +If you have any suggestions for new utilities or helpers that would benefit many modules, let us know and we shall consider including it. + +### @companion-module/tools + +This library should be added as a devDependency of your module. It contains various bits of tooling and commands needed during the development or distribution of modules. + +For example, it contains a script to bundle your module for distribution and some eslint and typescript config presets that can be used. + +We expect every module to use the build script provided from this, the rest is optional extras that you can choose to use if you like. + +## When should I update them? + +### @companion-module/base + +The version of this library determines the versions of Companion your module can be run with. See the [version compatibility table](https://github.com/bitfocus/companion-module-base#readme). + +For example, modules targeting Companion 3.0 should use @companion-module/base v1.4.3 (or any 1.x release back to v1.0.0). Modules targeting Companion 3.1 should use v1.5. A module written for 3.0 will run on 3.1, but a module written for 3.1 may not be compatible with 3.0. + +We recommend targeting the previous stable release of Companion. This will allow users who have not yet updated Companion to use the latest version of your module. But of course, you may wish to use newer versions if there are features that your module will benefit from. + +### @companion-module/tools + +This library is less picky about the version, and you will benefit by running an up to date version. + +## Why were they created? - A bit of history + +Prior to Companion version 3, modules would pull various bits of code from the Companion code. This was problematic as it meant that modules had to live in one of a few specific locations to be able to access the code. In release builds, they had to be built in before Companion was packaged. + +This was the main issue that was blocking our ability to support adding newer versions of modules into already released versions. + +Everything that modules need to access Companion is now located inside of `@companion-module/base`. Some things have not been made available, as they were determined to not be appropriate and alternatives will be recommended to the few module authors who utilised them. + +Another benefit of this separation is that it allows us to better isolate modules from being tied to specific versions of Companion. The `@companion-module/base` acts as a stable barrier between the two. It has intentionally been kept separate from the rest of the Companion code, so that changes made here get an extra level of scrutiny, as we want to guarantee backwards compatibility as much as possible. For example, in Companion version 2.x changes made to the module api occasionally required fixes to be applied to multiple modules. By having this barrier, we can avoid such problems for as long as possible, and can more easily create compatibility layers as needed. diff --git a/for-developers/module-development/module-lifecycle/updating-nodejs.md b/for-developers/module-development/module-lifecycle/updating-nodejs.md new file mode 100644 index 0000000..6b68fcf --- /dev/null +++ b/for-developers/module-development/module-lifecycle/updating-nodejs.md @@ -0,0 +1,32 @@ +--- +title: 'Updating the Node.JS Version in your Module' +sidebar_label: 'Updating Node.JS' +sidebar_position: 8 +description: Explanation of the companion module library. +--- + +Nodejs often makes new releases. In the major version jumps, these can include some breaking changes which could impact your module, so we don't want to do that without you being aware of it. + +Since Companion version 3.0, we've required modules to be compatible with Node.JS v18. Starting with @companion-module/base v1.11, Node.JS v22 has been supported. + +When developing your module, you need to have a similar version of nodejs installed, for when you run `yarn`. Companion v4 supports both v18 and v22 for modules, but may provide a different minor version than you have used. + +:::note + +Whenever Companion runs your module, it will do so with the version of nodejs that we distribute, not the one you have installed yourself. + +::: + +If you are using a node version manager, you can install an updated nodejs to your system, by doing something like `fnm install 22` and `fnm use 22`. See [our instructions here](../../setting-up-developer-environment.md). + +### Changing your module's version of nodejs + +Companion knows what version of nodejs your module is compatible with by looking at the `companion/manifest.json` in your module. Inside the `runtime` object, is a property `type` which should be set to something like `node18` or `node22`. Currently these are the only valid values, others will be allowed in the future. If this is an unknown value, your module will fail to run when adding an instance inside Companion. + +Make sure that your version of the `@companion-module/base` dependency is appropriate for the version of Companion too. A table of compatibility is listed in [the readme](https://github.com/bitfocus/companion-module-base#supported-versions-of-this-library) + +If you are using TypeScript, you should update your preset to have sensible `target` and `lib` values. The config presets in `@companion-module/tools`, have versions for each supported version of nodejs to make this easier for you. + +If your module has an `engines` field in the package.json, make sure that has a sensible value. + +Once you have done this, give it a test in Companion and ensure everything is working as expected. diff --git a/for-developers/setting-up-WSL.md b/for-developers/setting-up-WSL.md index 57c89d6..df5aa78 100644 --- a/for-developers/setting-up-WSL.md +++ b/for-developers/setting-up-WSL.md @@ -1,7 +1,7 @@ --- title: Setting Up WSL for the Developer Environment -sidebar_label: '- WSL Setup Notes' -sidebar_position: 0.2 +sidebar_label: '└─ WSL Setup Notes' +sidebar_position: 1.1 description: How to set up Windows System for Linux (WSL) for Companion development --- diff --git a/for-developers/setting-up-developer-environment.md b/for-developers/setting-up-developer-environment.md index 8e88182..5b45351 100644 --- a/for-developers/setting-up-developer-environment.md +++ b/for-developers/setting-up-developer-environment.md @@ -1,7 +1,7 @@ --- title: Setting up a Developer Environment sidebar_label: Setup the Developer Tools -sidebar_position: 0.1 +sidebar_position: 1 description: Setting up a Developer Environment ---