Python: [Bug]: ENABLE_SENSITIVE_DATA env var ignored when load_dotenv() runs after module import

[droideronline]

Description

What happened?
When setting ENABLE_SENSITIVE_DATA=true in a .env file and using load_dotenv() before calling configure_otel_providers() or enable_instrumentation(), sensitive data logging remains disabled (enable_sensitive_data stays False).

What did you expect to happen?
ENABLE_SENSITIVE_DATA=true should be respected regardless of whether it was set before or after the module was imported, since load_dotenv() populates os.environ before the user calls configure_otel_providers().

Steps to reproduce:

1. Create a .env file with ENABLE_SENSITIVE_DATA=true
2. In your script: import agent_framework, then call load_dotenv(), then call configure_otel_providers()
3. Check OBSERVABILITY_SETTINGS.enable_sensitive_data — it is False

Code Sample

from dotenv import load_dotenv
load_dotenv()  # populates os.environ with ENABLE_SENSITIVE_DATA=true

from agent_framework.observability import configure_otel_providers, OBSERVABILITY_SETTINGS

configure_otel_providers()
print(OBSERVABILITY_SETTINGS.enable_sensitive_data)  # False — should be True

Root Cause

OBSERVABILITY_SETTINGS = ObservabilitySettings() is constructed at module import time (line 884 of observability.py). At that point, os.environ does not yet contain the .env values because load_dotenv() hasn't been called. The cached enable_sensitive_data=False is never refreshed.

configure_otel_providers() and enable_instrumentation() force enable_instrumentation=True but do not re-read ENABLE_SENSITIVE_DATA from os.environ when the parameter is None.

The same timing issue affects VS_CODE_EXTENSION_PORT in configure_otel_providers().

Package Versions

agent-framework-core: latest (main branch)

Python Version

Python 3.13

Additional Context

This is the standard usage pattern shown in samples — load_dotenv() is called after imports but before configure_otel_providers(). The env_file_path branch of configure_otel_providers() works correctly because it constructs a fresh ObservabilitySettings(**settings_kwargs), but the common else branch (no env_file_path) does not.

[eavanvalkenburg]

Please check with rc1, I merged a change in how we handle env values

[droideronline]

Verified this from the latest main (0086d38f, tagged python-1.0.0rc1), which includes your .env handling changes from #4032. The issue still reproduces because #4032 fixed load_settings() precedence at construction time, but the module-level OBSERVABILITY_SETTINGS singleton is already constructed at import time before load_dotenv() populates os.environ. The stale cached value is never refreshed by configure_otel_providers() or enable_instrumentation(). Took some time to trace through the root cause. Would appreciate if you could review the PR (#4120) and its test cases to understand the fix in detail.