Phase 1: Implement Eq. 21-22 delta-rule momentum variant

[georgepullen]

Purpose

Implement Eq. 21-22 delta-rule momentum path in inner optimizer with testable math-level fidelity.

Mandatory Reading (blocking)

First comment must summarize:

• reports/NL_IMPLEMENTATION_ORACLE.md section 6.1.2
• reports/paper/NL-print.extracted.clean.txt Eq. (21), Eq. (22)
• src/nested_learning/optim/deep.py current l2_objective branch

Required Code Anchors

• src/nested_learning/optim/deep.py
• src/nested_learning/optim/factory.py
• src/nested_learning/optim/manager.py
• tests in tests/

Scope

• Add new variant name: delta_rule.
• Implement update equivalent to:
  • m_{i+1} = (alpha*I - g^T g)m_i - eta*P*g
• Support P=I and diagonal preconditioned mode.
• Keep legacy variants unchanged for reproducibility.

Test Requirements

• Analytic toy tensor test against closed-form expected update.
• State-shape and determinism tests.
• No regressions in existing optimizer suite.

Deliverables

• New variant + config support.
• Dedicated test file and ablation config.

Acceptance Criteria

• Tests prove formula-level correctness for toy case.
• 5k pilot run with delta_rule is finite and emits expected telemetry.
• First issue comment contains mandatory reading summary.

[georgepullen]

Read summary (oracle + paper + current code) before implementation:

• Oracle §6.1.2 identifies Eq. 21-22 as a primary gap: current l2_objective in DeepMomentum is heuristic (grad + 0.1*mean(grad)), not the paper delta-rule recurrence.
• Paper Eq. (21)-(22) requires momentum update of the form m_{i+1} = (alpha*I - g^T g)m_i - eta*P*g, i.e. retain/forget term plus gradient write term.
• src/nested_learning/optim/deep.py is the correct insertion point because all level optimizers route through this module via optim/factory.py and optim/manager.py.
• Existing variants (preconditioned, dmgd, muon, nl_l2_precond) are used for reproducibility and should remain behaviorally unchanged.
• Fidelity requirement for this ticket is equation-structure correctness on toy tensors + deterministic state updates, not full benchmark parity.

Code anchors used:

• src/nested_learning/optim/deep.py
• src/nested_learning/optim/factory.py
• src/nested_learning/optim/manager.py
• tests/test_optim.py
• tests/test_paper_faithful_configs.py

Implementation update landed:

• Added new DeepMomentum variant: delta_rule with recurrence matching Eq. 21-22 structure.
• Supports P=I (delta_preconditioner=identity) and diagonal preconditioned mode (delta_preconditioner=adam_diag).
• Kept legacy variants unchanged.
• Added analytic toy tests for recurrence and preconditioned mode.
• Added ablation config: configs/ablations/selfmod_delta_rule.yaml.