Skip to content

Add flag name suggestions for misspelled flags#4723

Open
simonfaltum wants to merge 5 commits intomainfrom
simonfaltum/flag-suggestions
Open

Add flag name suggestions for misspelled flags#4723
simonfaltum wants to merge 5 commits intomainfrom
simonfaltum/flag-suggestions

Conversation

@simonfaltum
Copy link
Member

@simonfaltum simonfaltum commented Mar 12, 2026

Why

When users misspell a flag name (e.g., --outpu instead of --output), they get a generic "unknown flag" error with no help. Cobra already suggests corrections for misspelled command names, but not flags.

Changes

Before: Misspelled flags produce a generic "unknown flag" error with no guidance.
Now: The flagErrorFunc suggests the closest matching flag using Levenshtein distance (threshold of 2, matching Cobra's own suggestion threshold for commands).

Both long flags (--flagname) and shorthand flags (-x) are handled. Hidden and deprecated flags are excluded from suggestions. A small Levenshtein distance function is included inline (no new dependencies).

Test plan

  • Unit tests for suggestion matching (close match, no match, shorthand)
  • Unit tests for hidden flag exclusion
  • Unit tests for the Levenshtein distance function
  • Regression test for Cobra's error format parsing
  • make checks passes

When users misspell a flag (e.g., --outpu instead of --output), they now
get a "Did you mean" suggestion. This extends Cobra's existing command
suggestion behavior to flags using Levenshtein distance with a threshold
of 2. Hidden and deprecated flags are excluded from suggestions. Both
long flags (--flagname) and shorthand flags (-x) are handled.

Co-authored-by: Isaac
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 12, 2026

Commit: a0ef7d4

Run: 23056330116

Env ❌​FAIL 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🔄​ aws linux 3 7 7 266 787 13:54
💚​ aws windows 8 7 270 785 11:11
🔄​ aws-ucws linux 5 7 7 361 702 23:06
🔄​ aws-ucws windows 4 8 7 363 700 18:52
❌​ azure linux 2 1 2 9 268 785 13:57
❌​ azure windows 2 1 2 9 270 783 11:29
🔄​ azure-ucws linux 3 1 9 368 698 25:17
🔄​ azure-ucws windows 5 2 9 367 696 18:13
💚​ gcp linux 2 9 267 788 12:18
🔄​ gcp windows 2 2 9 267 786 11:05
31 interesting tests: 14 flaky, 7 SKIP, 6 RECOVERED, 4 FAIL
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🔄​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/model_serving_endpoints/basic 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p 🔄​f
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/resources/permissions/dashboards/create 🔄​f ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/permissions/dashboards/create/DATABRICKS_BUNDLE_ENGINE=terraform 🔄​f ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/ssh/connection 🔄​f 💚​R 🔄​f 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestGenerateFromExistingPipelineAndDeploy ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestSyncEnsureRemotePathIsUsableIfRepoExists ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestSyncFullFileSync ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestFetchRepositoryInfoAPI_FromRepo ✅​p ✅​p ✅​p ✅​p 🔄​f 🔄​f 🔄​f 🔄​f ✅​p ✅​p
❌​ TestFetchRepositoryInfoAPI_FromRepo//Workspace/Repos/222cace5-bdb4-4852-97e6-7cb1109cc0e9/integration-test-474dd112c4bb4e32bb0d89bba2806953 ❌​F
❌​ TestFetchRepositoryInfoAPI_FromRepo//Workspace/Repos/222cace5-bdb4-4852-97e6-7cb1109cc0e9/integration-test-474dd112c4bb4e32bb0d89bba2806953/knowledge_base/dashboard_nyc_taxi ❌​F
❌​ TestFetchRepositoryInfoAPI_FromRepo//Workspace/Repos/222cace5-bdb4-4852-97e6-7cb1109cc0e9/integration-test-90295d0d9f8746dabb4696290571526b ❌​F
❌​ TestFetchRepositoryInfoAPI_FromRepo//Workspace/Repos/222cace5-bdb4-4852-97e6-7cb1109cc0e9/integration-test-90295d0d9f8746dabb4696290571526b/knowledge_base/dashboard_nyc_taxi ❌​F
Top 50 slowest tests (at least 2 minutes):
duration env testname
7:47 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
5:54 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
5:22 azure-ucws linux TestAccept/ssh/connect-serverless-gpu
5:21 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct
4:46 azure-ucws windows TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct
4:44 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:38 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:32 azure-ucws windows TestAccept/bundle/resources/volumes/recreate/DATABRICKS_BUNDLE_ENGINE=direct
4:25 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:23 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:17 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:15 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:07 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:07 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:05 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:02 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:59 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:58 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:50 azure-ucws linux TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform
3:50 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:46 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:40 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:40 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:33 aws-ucws windows TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:32 azure-ucws windows TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
3:25 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:19 azure-ucws linux TestAccept/bundle/resources/jobs/check-metadata/DATABRICKS_BUNDLE_ENGINE=direct
3:17 aws-ucws linux TestAccept/bundle/resources/permissions/jobs/delete_one/cloud/DATABRICKS_BUNDLE_ENGINE=terraform
3:13 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:12 aws-ucws linux TestAccept/bundle/resources/dashboards/generate_inplace/DATABRICKS_BUNDLE_ENGINE=terraform
3:06 azure-ucws windows TestAccept/bundle/resources/volumes/recreate/DATABRICKS_BUNDLE_ENGINE=terraform
3:02 aws-ucws linux TestAccept/bundle/resources/jobs/check-metadata/DATABRICKS_BUNDLE_ENGINE=direct
3:02 aws-ucws linux TestAccept/bundle/resources/dashboards/generate_inplace/DATABRICKS_BUNDLE_ENGINE=direct
3:01 aws-ucws windows TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:57 aws-ucws windows TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_14.3_LTS
2:56 aws-ucws linux TestAccept/ssh/connect-serverless-gpu
2:49 aws-ucws windows TestAccept/bundle/generate/auto-bind
2:45 azure-ucws windows TestAccept/bundle/resources/permissions/jobs/delete_one/cloud/DATABRICKS_BUNDLE_ENGINE=terraform
2:41 aws-ucws windows TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=direct
2:41 aws-ucws linux TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform
2:41 azure-ucws linux TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct
2:37 azure-ucws linux TestAccept/bundle/resources/permissions/jobs/delete_one/cloud/DATABRICKS_BUNDLE_ENGINE=direct
2:35 aws-ucws linux TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform
2:32 aws-ucws linux TestAccept/bundle/resources/permissions/jobs/delete_one/cloud/DATABRICKS_BUNDLE_ENGINE=direct
2:29 azure-ucws windows TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_13.3_LTS
2:24 gcp windows TestAccept/bundle/deployment/bind/job/generate-and-bind/DATABRICKS_BUNDLE_ENGINE=terraform
2:23 aws-ucws windows TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform
2:20 azure linux TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 aws-ucws windows TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform

@simonfaltum simonfaltum marked this pull request as ready for review March 13, 2026 10:16
Copy link
Contributor

@shreyas-goenka shreyas-goenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This review was posted by Claude (AI assistant). Shreyas will do a separate, more thorough review pass.

Priority: HIGH — Should use typed errors instead of string parsing

MAJOR: String parsing error messages is fragile — use pflag.NotExistError typed errors

The PR parses error message strings with strings.HasPrefix(msg, "unknown flag: ") to extract flag names. However, pflag v1.0.10 (already a dependency) provides a structured pflag.NotExistError type with GetSpecifiedName() and GetSpecifiedShortnames() methods.

Should use: errors.As(err, &pflag.NotExistError{}) with the accessor methods instead of string parsing. This eliminates:

  • Dependency on exact error message wording
  • Brittle shorthand character extraction (rest[0] != '\'')
  • The prefix constant definitions

MEDIUM: ShorthandDeprecated filtering too aggressive

In findClosestFlag, the check f.ShorthandDeprecated != "" excludes the entire flag from long-flag suggestions. ShorthandDeprecated only means the shorthand is deprecated, not the long flag itself. A flag with a deprecated shorthand should still appear in long-flag suggestions.

What looks good

  • Clean separation in its own file, minimal integration point in root.go
  • Levenshtein threshold of 2 matches Cobra's own command suggestion threshold
  • Error wrapping with %w preserves error chain

Missing tests

  • No integration test through Cobra's actual flag parsing
  • No test for ShorthandDeprecated specifically
  • No test for tie-breaking behavior for equidistant flags

The typed error approach is the main blocker for this PR.

…ated filtering, add tests

Address PR review comments:
- Replace string parsing (HasPrefix on error messages) with errors.As
  matching on pflag.NotExistError, using GetSpecifiedName() and
  GetSpecifiedShortnames() to extract flag info.
- Fix findClosestFlag to no longer exclude flags with ShorthandDeprecated
  from long-flag suggestions. Only exclude deprecated shorthands from
  shorthand suggestions (in findClosestShorthand).
- Add tests: integration through flagErrorFunc, ShorthandDeprecated
  filtering for both long and short suggestions, tie-breaking for
  equidistant flags.
- Rewrite existing tests to use real Cobra flag parsing instead of
  hand-crafted error strings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants