Add comprehensive YAML configuration validation system for extractor#788
Open
MO2k4 wants to merge 1 commit intoAzure:mainfrom
Open
Add comprehensive YAML configuration validation system for extractor#788MO2k4 wants to merge 1 commit intoAzure:mainfrom
MO2k4 wants to merge 1 commit intoAzure:mainfrom
Conversation
- Implement ConfigurationValidator class for validating configuration.extractor.yaml files - Add automatic validation during configuration loading in extractor app - Create standalone CLI validation command: ./extractor validate-config <file> - Comprehensive validation rules: - Empty YAML section detection - Duplicate entry validation across all configuration sections - String type validation for API/Product/Group names - Naming convention enforcement (no whitespace/special chars in names) - Error handling with detailed messages and YAML line numbers - Updated documentation with validation guidelines and examples - Added example configuration files for testing and reference - Integration with existing LanguageExt Either<T,U> error handling patterns
Contributor
|
@MO2k4 - thanks for the PR. I unfortunately have to reject it as-is. We made a lot of configuration changes in our next version, which will conflict with your updates. You can take a look at the v7 code to see where we're going. The CHANGELOG is incomplete but shows some configuration changes we made. Happy to get feedback. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Problem Solved
Currently, the APIops extractor tool processes
configuration.extractor.yamlfiles without validation, leading to runtime errors that are difficult to diagnose when configuration files contain:Users often encounter cryptic error messages during extraction, making it challenging to identify and fix configuration issues quickly.
🛠️ Solution Implemented
This PR introduces a comprehensive YAML configuration validation system that:
Core Features:
Validation Rules:
Integration Points:
./extractor validate-config <file>for manual validation📁 Files Added/Modified
New Files:
tools/code/common/ConfigurationValidator.cs- Core validation logictools/code/extractor/ConfigurationValidationCommand.cs- CLI validation tooltools/code/extractor-config/configuration.extractor.example.yaml- Valid exampletools/code/extractor-config/configuration.extractor.invalid-example.yaml- Invalid example for testingModified Files:
tools/code/common/Configuration.cs- Integrated validation into config loadingtools/code/extractor/Configuration.cs- Added extractor-specific validationtools/code/extractor/App.cs- Added validation at application startuptools/code/extractor/Program.cs- Added validation command supportdocs/apiops/3-apimTools/apiops-2-1-tools-extractor.md- Updated documentationtools/README.md- Added validation guidelinesconfiguration.extractor.yaml- Added validation comments📋 Usage Examples
Automatic Validation (during extraction):
./extractor extract --configuration-file invalid-config.yaml # Output: Configuration validation failed: Duplicate entry 'my-api' found in both apis and products sections (line 15)Manual Validation:
Error Message Examples:
Empty section detected: 'apis' section is empty (line 10)Duplicate entry 'my-product' found in both products and groups sections (line 25)Invalid entry type: Expected string but found object for entry 'test-api' (line 15)Invalid naming convention: Entry 'my api' contains whitespace or special characters (line 20)🧪 Testing
📖 Documentation
🔄 Backward Compatibility
🎁 Benefits for Users