From 6e3303016eced3bf8897b15eb0961741d75602e2 Mon Sep 17 00:00:00 2001 From: 3v0k4 Date: Mon, 17 Feb 2025 17:59:40 +0100 Subject: [PATCH 1/6] chore: enable split by test examples by default --- docusaurus/docs/ruby/circleci.mdx | 5 ----- docusaurus/docs/ruby/reference.md | 8 +++++--- docusaurus/docs/ruby/split-by-test-examples.mdx | 6 +++--- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/docusaurus/docs/ruby/circleci.mdx b/docusaurus/docs/ruby/circleci.mdx index 19cd65bb..845b7d81 100644 --- a/docusaurus/docs/ruby/circleci.mdx +++ b/docusaurus/docs/ruby/circleci.mdx @@ -129,8 +129,6 @@ Remove [`KNAPSACK_PRO_FIXED_QUEUE_SPLIT`](reference.md#knapsack_pro_fixed_queue_ command: | mkdir -p /tmp/test-results - export KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true - # highlight-start export KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=/tmp/tests_to_run.txt # Retrieve the tests to run (all or just the failed ones), and let Knapsack Pro split them optimally. @@ -149,7 +147,6 @@ Remove [`KNAPSACK_PRO_FIXED_QUEUE_SPLIT`](reference.md#knapsack_pro_fixed_queue_ The snippet above: - [Collects metadata with the JUnit XML formatter](#collect-metadata-in-queue-mode) -- uses [`KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES`](split-by-test-examples.mdx) to split slow spec files by test examples - uses [`KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE`](reference.md#knapsack_pro_test_file_list_source_file) to specify what tests to run
@@ -162,8 +159,6 @@ The snippet above: export CIRCLE_TEST_REPORTS=/tmp/test-results mkdir -p $CIRCLE_TEST_REPORTS - export KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true - # highlight-start export KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE=/tmp/tests_to_run.txt # Retrieve the tests to run (all or just the failed ones), and let Knapsack Pro split them optimally. diff --git a/docusaurus/docs/ruby/reference.md b/docusaurus/docs/ruby/reference.md index b0822a01..9e9e6ef7 100644 --- a/docusaurus/docs/ruby/reference.md +++ b/docusaurus/docs/ruby/reference.md @@ -310,9 +310,11 @@ Parallelize test examples (instead of files) across CI nodes. ::: -```bash -KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true -``` +Default: `true` + +Available: +- `true`: distribute test examples across CI nodes (more balanced builds) +- `false`: distribute whole test files across CI nodes Make sure to read the details in [Split by test examples](split-by-test-examples.mdx). diff --git a/docusaurus/docs/ruby/split-by-test-examples.mdx b/docusaurus/docs/ruby/split-by-test-examples.mdx index f80adc5a..2de256a0 100644 --- a/docusaurus/docs/ruby/split-by-test-examples.mdx +++ b/docusaurus/docs/ruby/split-by-test-examples.mdx @@ -25,7 +25,7 @@ Only RSpec >= 3.3.0 is supported. [Let us know](https://knapsackpro.com/contact) ::: -You can set [`KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES`](reference.md#knapsack_pro_rspec_split_by_test_examples-rspec) to parallelize tests across CI nodes by example (`it`/`specify`). This is useful when you have slow test files but don't want to manually split test examples into smaller test files. +By default, Knapsack Pro parallelizes tests across CI nodes by example (`it`/`specify`). This is useful when you have slow test files but don't want to manually split test examples into smaller test files. You can disable this feature with [`KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=false`](reference.md#knapsack_pro_rspec_split_by_test_examples-rspec). As an example, imagine you have two test files in your suite: @@ -56,7 +56,7 @@ As an example, imagine you have two test files in your suite: On your [Knapsack Pro dashboard](../overview/index.mdx#dashboard), you can see the yellow highlight because of the bottleneck: if you run those tests on 2 parallel CI nodes, the total execution time would be 6.5 minutes instead of the optimal 4.5 minutes (2.5 minutes + 6.5 minutes divided by 2 CI nodes). -By enabling `KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES`, the bottleneck disappears because Knapsack Pro can distribute tests so that each CI node is balanced (e.g., 4.5 minutes + 4.5 minutes): +When splitting by test examples, the bottleneck disappears because Knapsack Pro distributes individual tests so that each CI node is balanced (e.g., 4.5 minutes + 4.5 minutes):
@@ -116,7 +116,7 @@ Does not support `--tag` ::: -Due to the [RSpec internals](https://github.com/rspec/rspec-core/issues/2522), `--tag` might be ignored when used together with `KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES`. But you can use the `KNAPSACK_PRO_TEST_FILE_*` [environment variables](reference.md) to filter the test files to run. +Due to the [RSpec internals](https://github.com/rspec/rspec-core/issues/2522), `--tag` might be ignored when used together with Split by test examples. But you can use the `KNAPSACK_PRO_TEST_FILE_*` [environment variables](reference.md) to filter the test files to run. :::caution From 1acf3b21540dc884e133f5c968c6b13e34893477 Mon Sep 17 00:00:00 2001 From: Artur Trzop Date: Tue, 25 Feb 2025 14:09:25 +0100 Subject: [PATCH 2/6] update(docs): RSpec tags are supported for the RSpec split by test examples feature. --- docusaurus/docs/ruby/reference.md | 3 +-- docusaurus/docs/ruby/split-by-test-examples.mdx | 9 --------- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/docusaurus/docs/ruby/reference.md b/docusaurus/docs/ruby/reference.md index 9e9e6ef7..12b43808 100644 --- a/docusaurus/docs/ruby/reference.md +++ b/docusaurus/docs/ruby/reference.md @@ -306,14 +306,13 @@ Parallelize test examples (instead of files) across CI nodes. - Requires RSpec >= 3.3.0 - Does not support `run_all_when_everything_filtered` -- Does not support `--tag` ::: Default: `true` Available: -- `true`: distribute test examples across CI nodes (more balanced builds) +- `true`: distribute test examples for slow test files across CI nodes (more balanced builds) - `false`: distribute whole test files across CI nodes Make sure to read the details in [Split by test examples](split-by-test-examples.mdx). diff --git a/docusaurus/docs/ruby/split-by-test-examples.mdx b/docusaurus/docs/ruby/split-by-test-examples.mdx index 2de256a0..fd0ddd6c 100644 --- a/docusaurus/docs/ruby/split-by-test-examples.mdx +++ b/docusaurus/docs/ruby/split-by-test-examples.mdx @@ -21,7 +21,6 @@ import { IconExternalLink } from '@site/src/components/IconExternalLink' Only RSpec >= 3.3.0 is supported. [Let us know](https://knapsackpro.com/contact) if you use a different test runner. As an alternative, consider: - spreading test examples into multiple files -- tagging test examples (e.g., RSpec's [`--tag`](rspec.mdx#run-a-subset-of-tests)) ::: @@ -112,14 +111,6 @@ We recommend running at least 2 CI builds after you enable this feature or chang :::caution -Does not support `--tag` - -::: - -Due to the [RSpec internals](https://github.com/rspec/rspec-core/issues/2522), `--tag` might be ignored when used together with Split by test examples. But you can use the `KNAPSACK_PRO_TEST_FILE_*` [environment variables](reference.md) to filter the test files to run. - -:::caution - Does not support [`run_all_when_everything_filtered`](rspec.mdx#some-of-my-test-files-are-not-executed) ::: From 32b7fca60b240d7db36232f3c28320dbb98f6141 Mon Sep 17 00:00:00 2001 From: Artur Trzop Date: Tue, 25 Feb 2025 16:13:49 +0100 Subject: [PATCH 3/6] Update split-by-test-examples.mdx --- docusaurus/docs/ruby/split-by-test-examples.mdx | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docusaurus/docs/ruby/split-by-test-examples.mdx b/docusaurus/docs/ruby/split-by-test-examples.mdx index fd0ddd6c..4088421f 100644 --- a/docusaurus/docs/ruby/split-by-test-examples.mdx +++ b/docusaurus/docs/ruby/split-by-test-examples.mdx @@ -18,9 +18,7 @@ import { IconExternalLink } from '@site/src/components/IconExternalLink' :::caution -Only RSpec >= 3.3.0 is supported. [Let us know](https://knapsackpro.com/contact) if you use a different test runner. As an alternative, consider: - -- spreading test examples into multiple files +Only RSpec >= 3.3.0 is supported. [Let us know](https://knapsackpro.com/contact) if you use a different test runner. As an alternative, consider spreading test examples into multiple files. ::: From a34234e1acf751285c25c582906a20081b3cf075 Mon Sep 17 00:00:00 2001 From: Artur Trzop Date: Tue, 25 Feb 2025 16:16:52 +0100 Subject: [PATCH 4/6] Update split-by-test-examples.mdx --- docusaurus/docs/ruby/split-by-test-examples.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docusaurus/docs/ruby/split-by-test-examples.mdx b/docusaurus/docs/ruby/split-by-test-examples.mdx index 4088421f..40c4e030 100644 --- a/docusaurus/docs/ruby/split-by-test-examples.mdx +++ b/docusaurus/docs/ruby/split-by-test-examples.mdx @@ -18,7 +18,9 @@ import { IconExternalLink } from '@site/src/components/IconExternalLink' :::caution -Only RSpec >= 3.3.0 is supported. [Let us know](https://knapsackpro.com/contact) if you use a different test runner. As an alternative, consider spreading test examples into multiple files. +Only RSpec is supported, [let us know](https://knapsackpro.com/contact) if you use a different test runner. + +As an alternative, consider spreading test examples into multiple files. ::: From 967553bb7f05d7ad75646a12750f750d79f31d76 Mon Sep 17 00:00:00 2001 From: Artur Trzop Date: Tue, 25 Feb 2025 16:20:25 +0100 Subject: [PATCH 5/6] Update rspec.mdx --- docusaurus/docs/ruby/rspec.mdx | 2 -- 1 file changed, 2 deletions(-) diff --git a/docusaurus/docs/ruby/rspec.mdx b/docusaurus/docs/ruby/rspec.mdx index eb078603..6708254b 100644 --- a/docusaurus/docs/ruby/rspec.mdx +++ b/docusaurus/docs/ruby/rspec.mdx @@ -285,8 +285,6 @@ RSpec.configure do |c| end ``` -If you are using RSpec in Queue Mode and [Split by test example](split-by-test-examples.mdx), `--tag` is not supported. - ### Some tests are failing in Queue Mode Since Knapsack Pro [ignores `.rspec`](rspec.mdx#rspec-is-ignored-in-queue-mode) and many projects use it to require `spec_helper.rb` or `rails_helper.rb`, some tests may be falling. Make sure you either require the correct helper at the top of each test file or pass it as an argument: From 9a3a74fc5b29aa142cff97820b74e4a0461c0121 Mon Sep 17 00:00:00 2001 From: Artur Trzop Date: Tue, 25 Feb 2025 16:47:58 +0100 Subject: [PATCH 6/6] Update reference.md --- docusaurus/docs/ruby/reference.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docusaurus/docs/ruby/reference.md b/docusaurus/docs/ruby/reference.md index 12b43808..d54da247 100644 --- a/docusaurus/docs/ruby/reference.md +++ b/docusaurus/docs/ruby/reference.md @@ -304,7 +304,6 @@ Parallelize test examples (instead of files) across CI nodes. :::caution -- Requires RSpec >= 3.3.0 - Does not support `run_all_when_everything_filtered` :::