fix: use lazy imports to speed up module import time (#476)#501
Closed
manmohan-shaw-okta wants to merge 4 commits intomasterfrom
Closed
fix: use lazy imports to speed up module import time (#476)#501manmohan-shaw-okta wants to merge 4 commits intomasterfrom
manmohan-shaw-okta wants to merge 4 commits intomasterfrom
Conversation
Replace eager imports in okta/__init__.py, okta/api/__init__.py, and okta/models/__init__.py with lazy-loading using PEP 562 module __getattr__. Previously, 'import okta' eagerly loaded ~1900 classes (116 API classes + 1773 model classes), each pulling in pydantic, typing, etc. This caused import times of 2-5 seconds. With lazy loading, modules are only imported when their attributes are first accessed, reducing 'import okta' time from ~2s to ~0.007s (~280x faster).
…lliseconds This change addresses a critical performance issue where importing the okta-sdk-python module took nearly 2 seconds instead of milliseconds due to eager importing of all models and API classes. **Problem:** - Initial import of `okta` module was loading 1000+ model classes and 100+ API classes eagerly - Each model had complex Pydantic validation and inheritance chains - Import time was ~2 seconds, blocking application startup - Related to issue #476 **Solution:** Implemented intelligent lazy loading across three levels: 1. **Package Level (`okta/__init__.py`):** - Lazy load API classes and models using `__getattr__` hook - Only core SDK components (ApiClient, Configuration, etc.) are eagerly loaded - Models and API classes loaded on-demand when first accessed 2. **API Level (`okta/api/__init__.py`):** - Lazy load individual API classes using `__getattr__` pattern - Thread-safe imports with locking mechanism - Maintains backward compatibility 3. **Models Level (`okta/models/__init__.py`):** - Dynamic model grouping based on discriminator relationships - Preloads dependent models together to maintain class identity consistency - Critical for Pydantic validation with polymorphic models - Groups auto-generated from OpenAPI discriminator mappings - Manual field dependencies for complex model relationships **Key Technical Changes:** - **Template Updates:** - `__init__model.mustache`: Added lazy loading with discriminator-based grouping - `__init__package.mustache`: Implemented `__getattr__` for lazy loading - `api.mustache`: Fixed Content-Type header handling for non-consumes endpoints - `model_generic.mustache`: Import discriminator children from models package for class identity - **Model Grouping Strategy:** - Auto-groups models with parent-child discriminator relationships - Manual groups for field-level dependencies (e.g., DeviceAssurance → OSVersion) - Prevents Pydantic validation errors from duplicate class instances - Thread-safe lazy loading with locking - **Generated Code Changes:** - All API classes updated with lazy import support - All model classes updated to use centralized imports for discriminator handling - Models now import from `okta.models` instead of direct file paths **Testing:** - Updated GitHub Actions workflow to run pytest - Removed outdated commented test code in test_applications_it.py - All integration tests passing with new lazy loading implementation - Thread-safe implementation prevents race conditions in concurrent imports **Performance Impact:** - Import time: ~2000ms → <100ms (20x improvement) - No runtime performance impact after initial load - Maintains full backward compatibility - Models loaded on-demand only when used **Backward Compatibility:** - All existing import patterns continue to work - `from okta.models import ModelName` still functional - `from okta import ApiClass` still functional - No breaking changes to public API Fixes #476
…#504 contains the permanent fix for this issue. Once it's merged will sync it.
Contributor
|
Closing this PR as the issue has been addressed in the PR #505 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace eager imports in okta/init.py, okta/api/init.py, and okta/models/init.py with lazy-loading using PEP 562 module getattr.
Previously, 'import okta' eagerly loaded ~1900 classes (116 API classes + 1773 model classes), each pulling in pydantic, typing, etc. This caused import times of 2-5 seconds.
With lazy loading, modules are only imported when their attributes are first accessed, reducing 'import okta' time from ~2s to ~0.007s (~280x faster).