diff --git a/skills/SKILL.md b/skills/SKILL.md new file mode 100644 index 0000000..7835178 --- /dev/null +++ b/skills/SKILL.md @@ -0,0 +1,90 @@ +--- +name: python-typemap +description: Python 3.14+ type manipulation library inspired by TypeScript. Use when working with type-level operations, evaluating type expressions at runtime, or when asked about PEP 827 type operators. Supports type introspection, transformation, and runtime type evaluation. +disable-model-invocation: false +allowed-tools: Read,Grep,Glob,Bash +--- + +# Python Typemap Skill + +A skill for working with python-typemap - a PEP 827 type manipulation library for Python 3.14+. + +## Quick Usage + +```python +from typemap import eval_typing +import typemap_extensions as tm + +# Get type operators +tm.KeyOf[T] # TypeScript's keyof +tm.Partial[T] # Make fields optional +tm.Pick[T, K] # Select fields +tm.Omit[T, K] # Exclude fields +tm.Iter[T] # Iterate over type +tm.Attrs[T] # Get type attributes +``` + +## What is python-typemap? + +Python-typemap brings **TypeScript-inspired type operators** to Python. It provides: + +- **Type Operators**: `Member`, `KeyOf`, `Partial`, `Pick`, `Omit`, `Iter`, `Attrs`, etc. +- **Runtime Evaluation**: `eval_typing()` evaluates type expressions at runtime +- **Type Introspection**: Inspect and transform types programmatically + +## Core Concepts + +| Concept | Description | +|---------|-------------| +| **Type Operators** | Classes that manipulate types (like TypeScript's utility types) | +| **Member** | Access type members with name, type, and qualifiers | +| **EvalContext** | Context for tracking resolved types during evaluation | +| **Singledispatch** | Pattern for handling different type implementations | + +## Key Type Operators + +| Operator | Purpose | Example | +|----------|---------|---------| +| `KeyOf[T]` | Get all keys of a type | `KeyOf[User]` → `tuple[Literal['name'], Literal['age']]` | +| `Partial[T]` | Make all fields optional | `Partial[User]` → all fields `\| None` | +| `Pick[T, K]` | Select specific fields | `Pick[User, 'name']` → only `name` field | +| `Omit[T, K]` | Remove specific fields | `Omit[User, 'password']` → without password | +| `Iter[T]` | Iterate over type elements | `Iter[list[int]]` → `int` | +| `Attrs[T]` | Get type attributes | `Attrs[User]` → tuple of Member descriptors | +| `Param[T, N]` | Get Nth parameter | `Param[func, 0]` → first param type | +| `Return[T]` | Get return type | `Return[func]` → function return type | +| `Required[T]` | Make all fields required | `Required[Partial[User]]` → revert Partial | +| `Readonly[T]` | Make fields immutable | `Readonly[User]` → immutable fields | + +## Runtime Evaluation + +The `eval_typing()` function evaluates type expressions at runtime: + +```python +from typemap import eval_typing +import typemap_extensions as tm + +class User: + name: str + age: int + +# Get keys of User +keys = eval_typing(tm.KeyOf[User]) +# Result: tuple[Literal['name'], Literal['age']] + +# Make User partial +PartialUser = eval_typing(tm.Partial[User]) +# Result: User with all fields optional +``` + +## Additional Resources + +For detailed information on each feature: + +- [type-operators.md](python-typemap/type-operators.md) - All type operators explained +- [runtime-evaluation.md](python-typemap/runtime-evaluation.md) - How eval_typing works +- [member.md](python-typemap/member.md) - Member type descriptor +- [patterns.md](python-typemap/patterns.md) - Common usage patterns +- [examples.md](python-typemap/examples.md) - Practical examples +- [internals.md](python-typemap/internals.md) - Architecture and internals +- [errors.md](python-typemap/errors.md) - Error handling diff --git a/skills/python-typemap/errors.md b/skills/python-typemap/errors.md new file mode 100644 index 0000000..e069e12 --- /dev/null +++ b/skills/python-typemap/errors.md @@ -0,0 +1,239 @@ +# Error Handling + +Understanding and handling errors in python-typemap. + +## Exception Types + +### TypeMapError + +Base exception for all typemap errors. + +```python +from typemap import TypeMapError + +try: + result = eval_typing(expr) +except TypeMapError as e: + print(f"Type evaluation failed: {e}") +``` + +### StuckException + +Raised when type evaluation cannot proceed because a type variable hasn't been resolved. + +```python +from typemap.type_eval import StuckException + +try: + # T is still a TypeVar, can't evaluate + result = eval_typing(KeyOf[T]) +except StuckException: + # T needs to be bound to a concrete type + result = eval_typing(KeyOf[ConcreteType]) +``` + +**Common causes:** +- Passing a TypeVar directly to an operator +- Using unevaluated generics +- Circular type references + +### RecursionError + +Python's built-in recursion limit exceeded during type evaluation. + +```python +# Recursive type that doesn't terminate +type Infinite = list[Infinite] + +try: + eval_typing(Partial[Infinite]) +except RecursionError: + print("Type is infinitely recursive") +``` + +## Common Errors + +### 1. Type Variable Not Bound + +```python +from typing import TypeVar + +T = TypeVar('T') + +# Wrong - T is not bound +KeyOf[T] # StuckException + +# Correct - T is bound to a concrete type +KeyOf[User] +``` + +### 2. Missing Type Annotation + +```python +class MissingAnnotation: + name: str + data # No annotation! + +# May cause evaluation issues +eval_typing(Attrs[MissingAnnotation]) +# May raise: AttributeError or StuckException +``` + +### 3. Invalid Type Expression + +```python +# Wrong - this is not a valid type expression +eval_typing("User") # String, not a type + +# Correct +eval_typing(User) +eval_typing(KeyOf[User]) +``` + +### 4. Union with Non-Types + +```python +# Wrong +Union[str, 123] # 123 is not a type + +# Correct +Union[str, int] +``` + +## Debugging Tips + +### Enable Context Manager + +```python +from typemap.type_eval import _ensure_context + +with _ensure_context() as ctx: + result = _eval_types_impl(expr, ctx) + print(f"Resolved: {ctx.resolved}") + print(f"Seen: {ctx.seen}") +``` + +### Check What Was Resolved + +```python +ctx = EvalContext() +result = eval_typing(expr, ensure_context=False) + +# Inspect cache +for type_obj, resolved in ctx.resolved.items(): + print(f"{type_obj} -> {resolved}") +``` + +### Verify Type Structure + +```python +import typing + +def inspect_type(t: type) -> dict: + """Debug type structure.""" + info = { + 'is_generic': isinstance(t, typing._GenericAlias), + 'is_union': hasattr(t, '__origin__') and t.__origin__ is Union, + 'args': getattr(t, '__args__', ()), + 'origin': getattr(t, '__origin__', None), + } + return info +``` + +## Error Recovery + +### Fallback to Default + +```python +def safe_eval(expr, default=None): + """Evaluate with fallback on error.""" + try: + return eval_typing(expr) + except (StuckException, TypeMapError) as e: + warnings.warn(f"Evaluation failed: {e}") + return default +``` + +### Partial Evaluation + +```python +def try_partial_eval(cls: type, operators: list): + """Try applying operators, collecting failures.""" + results = {} + errors = [] + + for op in operators: + try: + results[op] = eval_typing(op[cls]) + except Exception as e: + errors.append((op, str(e))) + + return results, errors +``` + +## Validation Before Evaluation + +### Check for TypeVars + +```python +from typing import TypeVar, get_args, get_origin + +def has_unbound_typevars(t: type) -> bool: + """Check if type has unresolved TypeVars.""" + if isinstance(t, TypeVar): + return True + if hasattr(t, '__args__'): + return any(has_unbound_typevars(arg) for arg in t.__args__) + return False + +# Before evaluation +if has_unbound_typevars(MyType): + raise ValueError("Type has unbound TypeVars") +``` + +### Validate Type Structure + +```python +def validate_for_eval(t: type) -> tuple[bool, str]: + """Validate type can be evaluated.""" + if t is None: + return False, "Type is None" + if isinstance(t, str): + return False, "Type is a string, not a type" + if isinstance(t, TypeVar): + return False, f"TypeVar {t} is unbound" + if not callable(t): + return False, f"{t} is not callable" + return True, "OK" +``` + +## Logging + +```python +import logging + +logging.basicConfig(level=logging.DEBUG) +logger = logging.getLogger('typemap') + +# Enable debug logging +logger.setLevel(logging.DEBUG) +``` + +## Common Error Messages + +| Error | Cause | Solution | +|-------|-------|----------| +| `StuckException: cannot evaluate KeyOf[T]` | TypeVar not bound | Provide concrete type | +| `RecursionError` | Infinite type recursion | Break recursive cycle | +| `TypeError: not a type` | Invalid type passed | Check type annotation | +| `AttributeError: no __args__` | Non-generic used as generic | Use correct type | +| `KeyError` | Type not in namespace | Provide namespace context | + +## Best Practices + +1. **Always bind TypeVars** before passing to operators +2. **Use concrete types** not type aliases when possible +3. **Catch specific exceptions** not just `Exception` +4. **Validate inputs** before expensive evaluations +5. **Cache results** to avoid repeated evaluations +6. **Provide namespace** for custom type resolution diff --git a/skills/python-typemap/examples.md b/skills/python-typemap/examples.md new file mode 100644 index 0000000..caaaa83 --- /dev/null +++ b/skills/python-typemap/examples.md @@ -0,0 +1,370 @@ +# Practical Examples + +Real-world examples of using python-typemap. + +## Example 1: REST API Layer + +### Setup + +```python +from dataclasses import dataclass +from datetime import datetime +from typemap import eval_typing +import typemap_extensions as tm + +@dataclass +class User: + id: int + name: str + email: str + password_hash: str + created_at: datetime + updated_at: datetime +``` + +### API DTOs + +```python +# Public user profile (no sensitive data) +PublicUser = eval_typing( + tm.Omit[User, 'password_hash' | 'updated_at'] +) + +# User creation (all fields except id and timestamps) +UserCreate = eval_typing( + tm.Partial[ + tm.Omit[User, 'id' | 'created_at' | 'updated_at'] + ] +) + +# User update (all fields optional) +UserUpdate = eval_typing( + tm.Partial[User] +) +``` + +### Usage + +```python +# GET /users/:id - returns public data only +def get_user(user_id: int) -> PublicUser: + user = db.get_user(user_id) + # Type checker ensures no password_hash leaks + return PublicUser( + id=user.id, + name=user.name, + email=user.email, + created_at=user.created_at + ) + +# PATCH /users/:id - any field can be updated +def update_user(user_id: int, data: UserUpdate) -> PublicUser: + validated = validate_update(data) + db.update_user(user_id, validated) + return get_user(user_id) +``` + +--- + +## Example 2: Form Generation + +### Model + +```python +@dataclass +class ContactForm: + name: str + email: str + subject: str + message: str + phone: str | None = None +``` + +### Form Generator + +```python +def generate_form(cls: type) -> list[FormField]: + """Generate HTML form fields from a class.""" + members = eval_typing(tm.Attrs[cls]) + fields = [] + + type_map = { + str: 'text', + int: 'number', + bool: 'checkbox', + 'email': 'email', + 'phone': 'tel', + } + + for member in members: + input_type = type_map.get(member.type, 'text') + + # Determine if required + is_required = member.quals.required + + # Check for optional type + is_optional = ( + isinstance(member.type, types.UnionType) and + type(None) in types.get_args(member.type) + ) + + fields.append(FormField( + name=member.name, + input_type=input_type, + required=is_required and not is_optional, + label=humanize(member.name), + placeholder=f"Enter {member.name}" + )) + + return fields + +# Usage +fields = generate_form(ContactForm) +# Returns FormField objects ready for template +``` + +--- + +## Example 3: Database Schema + +### Model + +```python +class Product: + id: int + name: str + description: str | None + price: Decimal + stock: int + created_at: datetime + updated_at: datetime + is_active: bool +``` + +### Schema Extraction + +```python +from typemap.type_eval import eval_typing +import typemap_extensions as tm + +def extract_table_schema(cls: type) -> dict: + """Extract database table schema from model.""" + members = eval_typing(tm.Attrs[cls]) + + columns = [] + primary_key = None + indexes = [] + + for member in members: + col = { + 'name': member.name, + 'type': sql_type(member.type), + 'nullable': is_nullable(member.type), + 'default': get_default(member), + } + + columns.append(col) + + if member.name == 'id': + primary_key = member.name + elif member.name.endswith('_id'): + indexes.append(member.name) + + return { + 'columns': columns, + 'primary_key': primary_key, + 'indexes': indexes + } + +# Output: +# { +# 'columns': [ +# {'name': 'id', 'type': 'INTEGER', 'nullable': False, 'default': None}, +# {'name': 'name', 'type': 'VARCHAR(255)', 'nullable': False, 'default': None}, +# ... +# ], +# 'primary_key': 'id', +# 'indexes': [] +# } +``` + +--- + +## Example 4: Validation Library + +### Type Guard + +```python +from typing import TypeGuard + +def is_user(data: dict) -> TypeGuard[User]: + """Check if dict matches User structure.""" + required_keys = eval_typing(tm.KeyOf[User]) + return all(key in data for key in required_keys) + +def validate_user(data: dict) -> tuple[bool, User | None]: + """Validate and convert dict to User.""" + if not is_user(data): + return False, None + + try: + return True, User(**data) + except (TypeError, ValueError) as e: + return False, None +``` + +### Partial Validation + +```python +def validate_partial(data: dict, cls: type) -> tuple[bool, dict]: + """Validate subset of fields.""" + partial_cls = eval_typing(tm.Partial[cls]) + keys = eval_typing(tm.KeyOf[partial_cls]) + + errors = {} + for key in keys: + if key in data: + if not isinstance(data[key], str): + errors[key] = f"Expected str, got {type(data[key]).__name__}" + + return len(errors) == 0, errors +``` + +--- + +## Example 5: Plugin System + +### Plugin Definition + +```python +@dataclass +class PluginManifest: + name: str + version: str + author: str + dependencies: list[str] + permissions: list[str] + +@dataclass +class PluginConfig: + manifest: PluginManifest + enabled: bool + settings: dict +``` + +### Schema Validation + +```python +def register_plugin(config: dict) -> PluginConfig: + """Register a plugin with validation.""" + # Get required plugin fields + manifest_keys = eval_typing(tm.KeyOf[PluginManifest]) + + # Validate manifest + if 'manifest' not in config: + raise PluginError("Missing manifest") + + manifest_data = config['manifest'] + missing = [k for k in manifest_keys if k not in manifest_data] + + if missing: + raise PluginError(f"Missing manifest fields: {missing}") + + # Create validated config + return PluginConfig( + manifest=PluginManifest(**manifest_data), + enabled=config.get('enabled', True), + settings=config.get('settings', {}) + ) +``` + +--- + +## Example 6: Event System + +### Event Definition + +```python +@dataclass +class UserEvent: + user_id: int + event_type: str + timestamp: datetime + data: dict + +@dataclass +class OrderEvent: + order_id: int + event_type: str + timestamp: datetime + data: dict +``` + +### Unified Handler + +```python +def create_event_handler(event: dict) -> UserEvent | OrderEvent: + """Create appropriate event from dict.""" + event_type = event.get('type') + + if event_type in ('user.created', 'user.updated', 'user.deleted'): + keys = eval_typing(tm.KeyOf[UserEvent]) + if all(k in event for k in keys): + return UserEvent(**event) + + elif event_type in ('order.created', 'order.shipped', 'order.delivered'): + keys = eval_typing(tm.KeyOf[OrderEvent]) + if all(k in event for k in keys): + return OrderEvent(**event) + + raise ValueError(f"Unknown event type: {event_type}") +``` + +--- + +## Example 7: Testing + +### Factory Pattern + +```python +import typing + +def make_factory(cls: type): + """Create a factory that generates instances with defaults.""" + members = eval_typing(tm.Attrs[cls]) + + def create(**overrides) -> cls: + kwargs = {} + + for member in members: + if member.name in overrides: + kwargs[member.name] = overrides[member.name] + elif has_default(cls, member.name): + kwargs[member.name] = get_default(cls, member.name) + else: + # Use sensible defaults based on type + kwargs[member.name] = get_fake_value(member.type) + + return cls(**kwargs) + + return create + +# Usage +create_user = make_factory(User) +user = create_user(name="Test User") +``` + +### Property Testing + +```python +from hypothesis import given, strategies as st + +@given(st.dictionaries(st.text(), st.integers())) +def test_user_serialization(data: dict): + """Test that partial user data can be validated.""" + PartialUser = eval_typing(tm.Partial[User]) + + # Should not raise for partial data + instance = PartialUser(**data) + assert isinstance(instance, User) +``` diff --git a/skills/python-typemap/internals.md b/skills/python-typemap/internals.md new file mode 100644 index 0000000..bb30c08 --- /dev/null +++ b/skills/python-typemap/internals.md @@ -0,0 +1,263 @@ +# Internals and Architecture + +Understanding how python-typemap works internally. + +## Project Structure + +``` +packages/typemap/ +├── src/typemap/ +│ ├── __init__.py # Main exports +│ ├── typing.py # Type operator definitions +│ └── type_eval/ +│ ├── __init__.py # eval_typing exports +│ ├── _eval_types_impl.py # Core evaluation logic +│ ├── _eval_typing.py # Main eval function +│ ├── _box.py # Class boxing +│ ├── _eval_context.py # Context management +│ └── _helpers.py # Utility functions +├── typemap_extensions/ # Public API (re-exports) +└── tests/ +``` + +## Core Components + +### 1. Type Operators (`typing.py`) + +Type operators are defined as special generic classes: + +```python +class _TupleLikeOperator(typing.Generic[T], typing._TupleLikeTarget): + """Base class for tuple-like type operators.""" + pass + +class Member[N: str, T, Q: MemberQuals = typing.Never, ...]: + """Member descriptor with name, type, and qualifiers.""" + type name = N + type type = T + type quals = Q + +class KeyOf[T](_TupleLikeOperator): + """TypeScript's keyof operator.""" + pass + +class Partial[T](_TupleLikeOperator): + """Make all fields optional.""" + pass +``` + +**Key Design Pattern:** Uses `typing.Generic` with subscript syntax for type parameters. + +### 2. Evaluation Context (`_eval_context.py`) + +```python +from dataclasses import dataclass, field +from typing import Any + +@dataclass +class EvalContext: + """Context for type evaluation.""" + resolved: dict[type, Any] = field(default_factory=dict) + seen: set[type] = field(default_factory=set) + ns: dict[str, Any] = field(default_factory=dict) +``` + +**Purpose:** +- `resolved`: Memoization cache for evaluated types +- `seen`: Cycle detection for recursive types +- `ns`: Namespace for type name resolution + +### 3. Singledispatch Evaluation (`_eval_types_impl.py`) + +```python +from functools import singledispatch + +@singledispatch +def _eval_types_impl(obj: Any, ctx: EvalContext) -> Any: + """Default handler - return as-is.""" + return obj + +@_eval_types_impl.register +def _eval_type_alias(obj: types.TypeAliasType, ctx: EvalContext): + """Handle type aliases.""" + ... + +@_eval_types_impl.register +def _eval_union(obj: types.UnionType, ctx: EvalContext): + """Handle Union types.""" + ... + +@_eval_types_impl.register +def _eval_literal(obj: types.LiteralGenericAlias, ctx: EvalContext): + """Handle Literal types.""" + ... +``` + +**Benefits:** +- Extensible by registering new handlers +- Falls back to default for unknown types +- Clear separation of concerns + +### 4. Class Boxing (`_box.py`) + +Boxes classes for safe evaluation: + +```python +def box_cache(cls: type, ctx: EvalContext) -> Any: + """Cache and box class evaluations.""" + key = (cls, id(ctx.resolved)) + if key in _box_cache: + return _box_cache[key] + + boxed = _box_class(cls, ctx) + _box_cache[key] = boxed + return boxed +``` + +### 5. Main Evaluation Loop (`_eval_typing.py`) + +```python +import contextvars + +_current_context: contextvars.ContextVar[EvalContext | None] = \ + contextvars.ContextVar('current_context', default=None) + +def eval_typing(expr: Any, *, ensure_context: bool = True) -> Any: + """Evaluate a type expression at runtime.""" + if ensure_context: + with _ensure_context() as ctx: + return _eval_types_impl(expr, ctx) + else: + ctx = _current_context.get() + if ctx is None: + raise TypeMapError("No evaluation context") + return _eval_types_impl(expr, ctx) +``` + +## Custom GenericAlias Classes + +### _IterSafeGenericAlias + +Allows safe iteration over unevaluated types: + +```python +class _IterSafeGenericAlias: + """Generic alias that safely handles iteration.""" + def __init__(self, origin, args): + self.__origin__ = origin + self.__args__ = args + + def __iter__(self): + """Safely iterate without evaluating.""" + for arg in self.__args__: + yield arg +``` + +### _BoolGenericAlias + +Boolean evaluation with context: + +```python +class _BoolGenericAlias: + """Boolean operations on types.""" + def __and__(self, other): + return _eval_bool_and(self, other) + + def __or__(self, other): + return _eval_bool_or(self, other) +``` + +### _AssociatedTypeGenericAlias + +Associated type access: + +```python +class _AssociatedTypeGenericAlias: + """访问 associated types.""" + def __get__(self, instance, owner): + return _eval_associated_type(owner, self.__args__) +``` + +## Error Handling + +### StuckException + +Raised when type operators receive type variable arguments: + +```python +class StuckException(TypeMapError): + """Type evaluation is stuck waiting for concrete type.""" + pass +``` + +**When it happens:** +- `KeyOf[T]` where `T` is still a TypeVar +- Other operators without enough context + +### TypeMapError + +General type evaluation errors: + +```python +class TypeMapError(Exception): + """Base exception for typemap errors.""" + pass +``` + +## Thread Safety + +Uses `contextvars` for thread-local context: + +```python +import contextvars + +# Each thread gets its own context +_current_context: contextvars.ContextVar[EvalContext | None] = \ + contextvars.ContextVar('current_context', default=None) +``` + +## Extension Points + +### Adding New Type Handlers + +```python +from typemap.type_eval._eval_types_impl import _eval_types_impl + +@_eval_types_impl.register +def _eval_my_custom_type(obj: MyCustomType, ctx: EvalContext): + """Handle MyCustomType evaluation.""" + # Custom logic here + return transformed_result +``` + +### Adding New Type Operators + +```python +from typemap.typing import _TupleLikeOperator + +class MyOperator[T](_TupleLikeOperator): + """Custom type operator.""" + pass +``` + +## Performance Considerations + +1. **Memoization**: `ctx.resolved` caches all evaluations +2. **Cycle Detection**: `ctx.seen` prevents infinite loops +3. **Lazy Boxing**: Classes are boxed only when needed +4. **Context Isolation**: Each `eval_typing` call is independent + +## Testing Internals + +```python +# Direct context testing +ctx = EvalContext() +result = _eval_types_impl(some_type_expr, ctx) + +# Check memoization +assert some_type_expr in ctx.resolved + +# Check cycle detection +ctx.seen.add(type(some_type_expr)) +# Should raise StuckException if encountered again +``` diff --git a/skills/python-typemap/member.md b/skills/python-typemap/member.md new file mode 100644 index 0000000..fbe14cc --- /dev/null +++ b/skills/python-typemap/member.md @@ -0,0 +1,179 @@ +# Member Type Descriptor + +`Member` is the fundamental type descriptor in python-typemap, representing a named type field with optional qualifiers. + +## Definition + +```python +class Member[N: str, T, Q: MemberQuals = typing.Never, ...]: + type name = N + type type = T + type quals = Q +``` + +## Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `N` | `str` | The member name (literal string) | +| `T` | `Any` | The member type | +| `Q` | `MemberQuals` | Optional qualifiers (readonly, required, etc.) | + +## MemberQuals + +Qualifiers modify member behavior: + +```python +class MemberQuals(typing.Protocol): + """Qualifiers for a Member type.""" + readonly: bool + required: bool + deprecated: bool +``` + +### Built-in Qualifiers + +| Qualifier | Effect | +|------------|--------| +| `readonly` | Field cannot be modified | +| `required` | Field must be provided | +| `deprecated` | Field is deprecated | + +## Creating Members + +### From Class Attributes + +```python +import typemap_extensions as tm +from typemap import eval_typing + +class User: + name: str + email: str + +# Get all members as Member descriptors +members = eval_typing(tm.Attrs[User]) +# Result: tuple[Member['name', str], Member['email', str]] +``` + +### From Type Annotations + +```python +from typemap.typing import Member + +# Explicit Member creation +NameMember = Member['name', str] +EmailMember = Member['email', str, {"readonly": True}] +``` + +## Accessing Member Properties + +```python +member: Member['name', str] = ... + +name = member.name # 'name' (literal) +t = member.type # str +quals = member.quals # MemberQuals instance +``` + +## Use Cases + +### 1. Field Introspection + +```python +def inspect_fields(cls: type) -> list[str]: + """Get all field names of a class.""" + members = eval_typing(tm.Attrs[cls]) + return [m.name for m in members] +``` + +### 2. Schema Generation + +```python +def generate_schema(cls: type) -> dict: + """Generate JSON schema from class.""" + members = eval_typing(tm.Attrs[cls]) + properties = {} + required = [] + + for member in members: + properties[member.name] = { + "type": get_json_type(member.type) + } + if member.quals.required: + required.append(member.name) + + return {"properties": properties, "required": required} +``` + +### 3. Form Generation + +```python +def generate_form(cls: type) -> list[FormField]: + """Generate form fields from class.""" + members = eval_typing(tm.Attrs[cls]) + fields = [] + + for member in members: + fields.append(FormField( + name=member.name, + type=get_form_type(member.type), + required=member.quals.required, + readonly=member.quals.readonly + )) + + return fields +``` + +## Member with Qualifiers + +### Readonly Fields + +```python +class Config: + id: str # Regular + api_key: str # readonly + +# Access qualifiers +members = eval_typing(tm.Attrs[Config]) +for m in members: + if m.quals.readonly: + print(f"{m.name} is readonly") +``` + +### Required Fields + +```python +@dataclass +class User: + name: str # required by default + email: str | None = None # optional + +# Check what's required +members = eval_typing(tm.Attrs[User]) +required = [m.name for m in members if m.quals.required] +# ['name'] +``` + +## Advanced: Custom Qualifiers + +```python +from typemap.typing import MemberQuals + +class MyQuals(MemberQuals): + """Custom qualifier for validation rules.""" + min_length: int | None = None + max_length: int | None = None + pattern: str | None = None + +# Use custom qualifier +ValidatedField = Member['username', str, {"min_length": 3, "max_length": 20}] +``` + +## Member vs Attrs + +| Aspect | `Member` | `Attrs` | +|--------|----------|---------| +| What | Single field descriptor | All fields of a type | +| Returns | One `Member` instance | Tuple of `Member` instances | +| Use | When you need one field | When you need all fields | diff --git a/skills/python-typemap/patterns.md b/skills/python-typemap/patterns.md new file mode 100644 index 0000000..526a4ee --- /dev/null +++ b/skills/python-typemap/patterns.md @@ -0,0 +1,298 @@ +# Common Usage Patterns + +This guide shows common patterns for using python-typemap effectively. + +## Type Transformation Patterns + +### 1. API Response Stripping + +Remove sensitive fields from API responses: + +```python +import typemap_extensions as tm +from typemap import eval_typing + +class User: + id: int + email: str + password_hash: str + created_at: datetime + +# For public API - no password! +PublicUser = eval_typing(tm.Omit[User, 'password_hash']) + +# For admin API - no timestamps! +AdminUser = eval_typing(tm.Omit[User, 'created_at']) +``` + +### 2. Create/Update DTOs + +Different types for different operations: + +```python +# All fields optional for creation +class UserCreate: + name: str | None = None + email: str | None = None + age: int | None = None + +# With eval_typing on existing class +UserCreateDTO = eval_typing(tm.Partial[User]) + +# Only specific fields for profile update +UserUpdateDTO = eval_typing( + tm.Partial[ + tm.Pick[User, 'name' | 'email'] + ] +) +``` + +### 3. Immutable Data Classes + +Make data classes immutable after creation: + +```python +from typing import Readonly + +# Create an immutable version +ImmutableUser = eval_typing(tm.Readonly[User]) + +# Validate at runtime +def create_user(data: dict) -> ImmutableUser: + return ImmutableUser(**data) +``` + +## Introspection Patterns + +### 4. Dynamic Form Generation + +```python +def generate_form_fields(cls: type) -> list[dict]: + """Generate form configuration from class.""" + members = eval_typing(tm.Attrs[cls]) + fields = [] + + for member in members: + field = { + 'name': member.name, + 'type': get_html_input_type(member.type), + 'required': member.quals.required, + } + fields.append(field) + + return fields + +# Usage +fields = generate_form_fields(User) +# [{'name': 'name', 'type': 'text', 'required': True}, ...] +``` + +### 5. JSON Schema Generation + +```python +def to_json_schema(cls: type) -> dict: + """Convert class to JSON Schema.""" + members = eval_typing(tm.Attrs[cls]) + properties = {} + required = [] + + for member in members: + prop_type = member.type + + # Convert Python type to JSON Schema type + if prop_type is str: + json_type = "string" + elif prop_type is int: + json_type = "integer" + elif prop_type is bool: + json_type = "boolean" + else: + json_type = "object" + + properties[member.name] = {"type": json_type} + + if member.quals.required: + required.append(member.name) + + return { + "type": "object", + "properties": properties, + "required": required + } +``` + +### 6. Database Model to Schema + +```python +class UserModel: + """SQLAlchemy-like model.""" + __tablename__ = 'users' + + id: int + name: str + email: str + created_at: datetime + +def extract_columns(model: type) -> list[Column]: + """Extract column definitions from model.""" + members = eval_typing(tm.Attrs[model]) + columns = [] + + for member in members: + columns.append(Column( + name=member.name, + type=map_to_db_type(member.type), + nullable=is_nullable(member.type), + primary_key=(member.name == 'id') + )) + + return columns +``` + +## Validation Patterns + +### 7. Type-Safe Validation + +```python +from typing import get_type_hints + +def validate_dict(data: dict, cls: type) -> tuple[bool, list[str]]: + """Validate dict against class type hints.""" + hints = get_type_hints(cls) + errors = [] + + for field, expected_type in hints.items(): + if field not in data: + # Check if field is required + members = eval_typing(tm.Attrs[cls]) + for m in members: + if m.name == field and not m.quals.required: + continue + errors.append(f"Missing required field: {field}") + elif not isinstance(data[field], expected_type): + errors.append(f"Invalid type for {field}") + + return len(errors) == 0, errors +``` + +### 8. Selective Update + +```python +def partial_update(original: User, updates: dict) -> User: + """Partially update a user, validating keys exist.""" + # Get allowed keys + allowed_keys = eval_typing(tm.KeyOf[User]) + + # Filter updates to only allowed keys + valid_updates = { + k: v for k, v in updates.items() + if k in allowed_keys + } + + return User(**{**asdict(original), **valid_updates}) +``` + +## Composition Patterns + +### 9. Complex Type Transformations + +```python +# Chain transformations +type1 = tm.Omit[User, 'password'] +type2 = tm.Partial[type1] +type3 = tm.Readonly[type2] + +# Or inline +ComplexUser = eval_typing( + tm.Readonly[ + tm.Partial[ + tm.Omit[User, 'password'] + ] + ] +) +``` + +### 10. Conditional Types + +```python +def conditional_pick(include_admin: bool, cls: type) -> type: + """Pick fields based on condition.""" + if include_admin: + return eval_typing(tm.Pick[cls, 'id' | 'name' | 'email' | 'role']) + else: + return eval_typing(tm.Pick[cls, 'id' | 'name' | 'email']) +``` + +## Performance Patterns + +### 11. Caching Evaluations + +```python +from functools import lru_cache + +@lru_cache(maxsize=128) +def get_partial_type(cls: type): + """Cache Partial evaluations.""" + return eval_typing(tm.Partial[cls]) + +@lru_cache(maxsize=128) +def get_keys(cls: type): + """Cache KeyOf evaluations.""" + return eval_typing(tm.KeyOf[cls]) +``` + +### 12. Lazy Evaluation + +```python +from typing import Generic, TypeVar + +T = TypeVar('T') + +class LazyType(Generic[T]): + """Lazy type that evaluates on first access.""" + _evaluated: T | None = None + + def __init__(self, expr): + self._expr = expr + + @property + def value(self) -> T: + if self._evaluated is None: + self._evaluated = eval_typing(self._expr) + return self._evaluated +``` + +## Anti-Patterns to Avoid + +### Don't: Evaluate in Hot Loops + +```python +# Bad - evaluates every iteration +for cls in classes: + keys = eval_typing(tm.KeyOf[cls]) # Expensive! + +# Good - evaluate once +key_cache = {cls: eval_typing(tm.KeyOf[cls]) for cls in classes} +for cls in classes: + keys = key_cache[cls] +``` + +### Don't: Nest Too Deeply + +```python +# Bad - hard to debug +Result = eval_typing( + tm.Readonly[ + tm.Partial[ + tm.DeepPartial[ + tm.Omit[...] + ] + ] + ] +) + +# Good - intermediate variables +WithoutPassword = tm.Omit[User, 'password'] +OptionalFields = tm.Partial[WithoutPassword] +ImmutableResult = tm.Readonly[OptionalFields] +``` diff --git a/skills/python-typemap/runtime-evaluation.md b/skills/python-typemap/runtime-evaluation.md new file mode 100644 index 0000000..77e2426 --- /dev/null +++ b/skills/python-typemap/runtime-evaluation.md @@ -0,0 +1,178 @@ +# Runtime Evaluation + +The `eval_typing()` function is the core of python-typemap - it evaluates type expressions at runtime. + +## Overview + +```python +from typemap import eval_typing +import typemap_extensions as tm + +result = eval_typing(tm.KeyOf[SomeType]) +``` + +## How It Works + +### 1. Entry Point + +`eval_typing()` is defined in `typemap/type_eval/__init__.py`: + +```python +def eval_typing(expr: Any, *, ensure_context: bool = True) -> Any: + """Evaluate a type expression at runtime.""" +``` + +### 2. Context Management + +The evaluation uses `EvalContext` to track: +- **resolved**: Memoization of evaluated types +- **seen**: Types currently being evaluated (for recursion detection) +- **ns**: Namespace for type resolution + +```python +from typemap.type_eval._eval_types_impl import EvalContext + +ctx = EvalContext( + resolved={}, # Memoization cache + seen=set(), # For cycle detection + ns={} # Type namespace +) +``` + +### 3. Singledispatch Implementation + +The evaluation uses Python's `singledispatch` to handle different type kinds: + +```python +from functools import singledispatch + +@singledispatch +def _eval_types_impl(obj: Any, ctx: EvalContext) -> Any: + """Default handler - return as-is.""" + return obj + +@_eval_types_impl.register +def _eval_literal(obj: typing._LiteralGenericAlias, ctx: EvalContext): + """Handle Literal types.""" + ... + +@_eval_types_impl.register +def _eval_special_form(obj: typing._SpecialForm, ctx: EvalContext): + """Handle special forms like Union, Optional.""" + ... +``` + +### 4. Custom GenericAlias Classes + +Python-typemap uses custom `GenericAlias` classes for special behaviors: + +| Class | Purpose | +|-------|---------| +| `_IterSafeGenericAlias` | Safe iteration over unevaluated types | +| `_BoolGenericAlias` | Boolean evaluation with context | +| `_AssociatedTypeGenericAlias` | Associated type access | + +## Context Helpers + +### _ensure_context() + +Ensures an evaluation context exists: + +```python +from typemap.type_eval._eval_types_impl import _ensure_context + +with _ensure_context() as ctx: + result = _eval_types_impl(expr, ctx) +``` + +### _child_context() + +Creates a child context for nested evaluation: + +```python +with _child_context(ctx) as child_ctx: + # Nested evaluation uses child_ctx + ... +``` + +## Caching Strategy + +### box_cache + +Classes are boxed (wrapped) once and cached: + +```python +from typemap.type_eval._box import box_cache + +boxed = box_cache(cls, ctx) +# Cached by (cls, id(ctx.resolved)) +``` + +### resolved / seen + +The `resolved` dict memoizes type evaluations: +```python +ctx.resolved[type(obj)] = result +``` + +The `seen` set detects recursion: +```python +if type(obj) in ctx.seen: + raise RecursionError("Cycle detected") +ctx.seen.add(type(obj)) +try: + ... +finally: + ctx.seen.discard(type(obj)) +``` + +## Evaluation Flow + +``` +eval_typing(expr) + ↓ +_ensure_context() → creates EvalContext + ↓ +_eval_types_impl(expr, ctx) + ↓ +[dispatch based on type] + ↓ +Result (possibly cached) +``` + +## Thread Safety + +Context uses `contextvars` for thread-local storage: + +```python +import contextvars + +_current_context: contextvars.ContextVar[EvalContext | None] = ... +``` + +This ensures each thread has its own evaluation context. + +## Advanced Usage + +### Custom Type Evaluation + +To add support for custom types: + +```python +from typemap.type_eval._eval_types_impl import _eval_types_impl + +@_eval_types_impl.register +def _eval_my_type(obj: MyCustomType, ctx: EvalContext): + """Handle MyCustomType evaluation.""" + # Custom logic + return transformed_result +``` + +### Namespace Control + +Pass custom namespaces for type resolution: + +```python +ctx = EvalContext(ns={'MyType': MyType, 'Optional': Optional}) +result = eval_typing(expr, ensure_context=False) +``` diff --git a/skills/python-typemap/type-operators.md b/skills/python-typemap/type-operators.md new file mode 100644 index 0000000..f526b4f --- /dev/null +++ b/skills/python-typemap/type-operators.md @@ -0,0 +1,248 @@ +# Type Operators + +Type operators are classes that manipulate and transform types, inspired by TypeScript's utility types. + +## Overview + +Type operators in python-typemap are defined as special generic classes that: +- Accept type parameters via subscript syntax (e.g., `KeyOf[T]`) +- Are evaluated at runtime using `eval_typing()` +- Return transformed types + +## Available Operators + +### KeyOf[T] + +Gets all keys/property names of a type as a tuple of Literals. + +```python +import typemap_extensions as tm +from typemap import eval_typing + +class User: + name: str + age: int + +keys = eval_typing(tm.KeyOf[User]) +# Result: tuple[Literal['name'], Literal['age']] +``` + +**Use cases:** +- Enum generation from data classes +- Dynamic form generation +- Generic key access utilities + +--- + +### Partial[T] + +Makes all fields of a type optional (each field becomes `T | None`). + +```python +class User: + name: str + email: str + +PartialUser = eval_typing(tm.Partial[User]) +# Result: User with name: str | None, email: str | None +``` + +**Use cases:** +- Form inputs (all fields optional for creation) +- Update DTOs (only fields you want to update) +- PATCH API endpoints + +--- + +### DeepPartial[T] + +Recursively makes all nested fields optional. + +```python +class Config: + database: DatabaseConfig + cache: CacheConfig + +PartialConfig = eval_typing(tm.DeepPartial[Config]) +# All fields at all levels become optional +``` + +--- + +### Pick[T, K] + +Selects specific fields from a type. + +```python +class User: + name: str + email: str + password: str + created_at: datetime + +# Only public profile fields +PublicUser = eval_typing(tm.Pick[User, 'name' | 'email']) +``` + +**Use cases:** +- Public/private API responses +- Strip sensitive fields +- Create view models + +--- + +### Omit[T, K] + +Removes specific fields from a type. + +```python +# Remove sensitive data +SafeUser = eval_typing(tm.Omit[User, 'password' | 'ssn']) +``` + +**Use cases:** +- Remove password from user responses +- Exclude internal fields from API output +- Sanitize data before logging + +--- + +### Iter[T] + +Iterates over type elements (e.g., union members, tuple elements, list items). + +```python +type Status = 'pending' | 'active' | 'completed' + +StatusType = eval_typing(tm.Iter[Status]) +# Result: Literal['pending'] | Literal['active'] | Literal['completed'] +``` + +**Use cases:** +- Union type iteration +- Tuple element access +- Generic sequence operations + +--- + +### Attrs[T] + +Gets type attributes as Member descriptors. + +```python +class User: + name: str + age: int + +attrs = eval_typing(tm.Attrs[User]) +# Result: tuple[Member['name', str], Member['age', int]] +``` + +**Use cases:** +- ORM column introspection +- Form generation +- Schema extraction + +--- + +### Param[T, N] + +Gets the Nth parameter type of a callable. + +```python +def greet(name: str, age: int) -> str: + return f"Hello {name}" + +FirstParam = eval_typing(tm.Param[greet, 0]) +# Result: str + +SecondParam = eval_typing(tm.Param[greet, 1]) +# Result: int +``` + +--- + +### Return[T] + +Gets the return type of a callable. + +```python +ReturnType = eval_typing(tm.Return[greet]) +# Result: str +``` + +--- + +### Required[T] + +Makes all fields required (opposite of Partial). + +```python +PartialUser = eval_typing(tm.Partial[User]) +FullUser = eval_typing(tm.Required[PartialUser]) +# Back to original User +``` + +--- + +### Readonly[T] + +Makes all fields immutable. + +```python +ImmutableUser = eval_typing(tm.Readonly[User]) +# All fields become read-only +``` + +--- + +### NonOptional[T] + +Removes None from optional fields. + +```python +type MaybeUser = User | None + +ConcreteUser = eval_typing(tm.NonOptional[MaybeUser]) +# Result: User +``` + +--- + +### Flatten[T] + +Flattens nested types. + +```python +type Nested = int | tuple[int | tuple[int]] + +Flat = eval_typing(tm.Flatten[Nested]) +# Result: int +``` + +--- + +## Operator Composition + +Operators can be composed for complex transformations: + +```python +# Create a type with only non-sensitive, optional fields +PublicPartialUser = eval_typing( + tm.Partial[ + tm.Omit[User, 'password' | 'ssn' | 'api_key'] + ] +) +``` + +## Custom Type Operators + +You can create custom operators by extending the base classes: + +```python +from typemap.typing import _TupleLikeOperator + +class MyOperator[T](_TupleLikeOperator): + """Custom type operator""" + pass +```