Change default sampler to Rate Limited Sampler

sdk/monitor/azure-monitor-opentelemetry/CHANGELOG.md

@@ -8,6 +8,8 @@
   ([#44902](https://github.com/Azure/azure-sdk-for-python/pull/44902))
 
 ### Breaking Changes
+- The default sampling behavior has been changed from ApplicationInsightsSampler with 100% sampling (all traces sampled) to RateLimitedSampler with 5.0 traces per second. This change significantly reduces telemetry volume for high-traffic applications and provides better cost optimization out of the box. Impact: Applications with more than 5 requests per second will see fewer traces exported by default.
+  ([#44925](https://github.com/Azure/azure-sdk-for-python/pull/44925))
 
 ### Bugs Fixed
 

sdk/monitor/azure-monitor-opentelemetry/README.md

@@ -70,7 +70,8 @@ You can use `configure_azure_monitor` to set up instrumentation for your app to
 | `views` | A list of [views][ot_view] that will be used to customize metrics exported by the SDK. | `N/A` |
 | `log_record_processors` | A list of [log record processors][ot_log_record_processor] that will process log records before they are exported. | `N/A` |
 | `metric_readers` | A list of [metric reader][ot_metric_reader] that will process metric readers before they are exported | `N/A` |
-| `traces_per_second` | Configures the Rate Limited sampler by specifying the maximum number of traces to sample per second. When set, this automatically enables the rate-limited sampler. Alternatively, you can configure sampling using the `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG` environment variables as described in the table below. Please note that the sampling configuration via environment variables will have precedence over the sampling exporter/distro options. | `N/A`
+| `sampling_ratio` | Configures the Application Insights Sampler specifying the percentage of traces to be sampled. Has to be a value between 0 and 1. For example, 0.1 sampling ratio would mean sampling 10% of all the traces. Defaults to 1.0 i.e 100% sampling rate. **Please note that the sampling configuration via environment variables will have precedence over the sampling exporter/distro options.** | `N/A`
+| `traces_per_second` | Configures the Rate Limited Sampler by specifying the maximum number of traces to sample per second. When set, this automatically enables the rate-limited sampler. Alternatively, you can configure sampling using the `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG` environment variables as described in the table below. To switch back to fixed percentage sampler, set the sampling_ratio to the desired value, ex. 1.0. Please refer to [sampling_configurations] sample file for more clarity on sampler configuration. **Please note that the sampling configuration via environment variables will have precedence over the sampling exporter/distro options.** | `N/A`
 
 You can configure further with [OpenTelemetry environment variables][ot_env_vars].
 
@@ -82,8 +83,8 @@ You can configure further with [OpenTelemetry environment variables][ot_env_vars
 | `OTEL_TRACES_EXPORTER` | If set to `None`, disables collection and export of distributed tracing telemetry. |
 | `OTEL_BLRP_SCHEDULE_DELAY` | Specifies the logging export interval in milliseconds. Defaults to 5000. |
 | `OTEL_BSP_SCHEDULE_DELAY` | Specifies the distributed tracing export interval in milliseconds. Defaults to 5000. |
-| `OTEL_TRACES_SAMPLER` | Specifies the sampler to be used for traces. Supports `always_on`, `always_off`, `trace_id_ratio`, `parentbased_always_on`, `parentbased_always_off`, `parentbased_trace_id_ratio`, [application_insights_sampling] and [rate_limited_sampling]. Use `microsoft.fixed_percentage` for the Application Insights sampler or `microsoft.rate_limited` for the Rate Limited sampler. |
-| `OTEL_TRACES_SAMPLER_ARG` | Specifies the sampling parameter for the configured sampler. For the standard OpenTelemetry samplers `trace_id_ratio` and `parentbased_trace_id_ratio`, this is the sampling ratio in the range [0.0, 1.0]. Not needed to be specified for `always_on`, `always_off`, `parentbased_always_on`, or `parentbased_always_off` samplers. For the Application Insights sampler, this sets the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling] with accepted values in the range [0,1]. Defaults to 1.0 (no sampling). For the Rate Limited sampler, this sets the maximum traces per second to be [sampled][rate_limited_sampler]. For example, 0.5 means one trace every two seconds, while 5.0 means five traces per second. |
+| `OTEL_TRACES_SAMPLER` | Specifies the sampler to be used for traces. Supports `always_on`, `always_off`, `trace_id_ratio`, `parentbased_always_on`, `parentbased_always_off`, `parentbased_trace_id_ratio`, [application_insights_sampling] and [rate_limited_sampling]. Use `microsoft.fixed_percentage` for the Application Insights sampler or `microsoft.rate_limited` for the Rate Limited sampler. **The default sampler is `Rate Limited Sampler` with the default value of 5.0 traces per second**|
+| `OTEL_TRACES_SAMPLER_ARG` | Specifies the sampling parameter for the configured sampler. For the standard OpenTelemetry samplers `trace_id_ratio` and `parentbased_trace_id_ratio`, this is the sampling ratio in the range [0.0, 1.0]. Not needed to be specified for `always_on`, `always_off`, `parentbased_always_on`, or `parentbased_always_off` samplers. For the Application Insights sampler, this sets the ratio of distributed tracing telemetry to be [sampled][application_insights_sampling] with accepted values in the range [0,1]. For the Rate Limited sampler, this sets the maximum traces per second to be [sampled][rate_limited_sampler]. Defaults to 5.0 traces per second. For example, 0.5 means one trace every two seconds, while 5.0 means five traces per second. |
 | `OTEL_PYTHON_DISABLED_INSTRUMENTATIONS` | Specifies which of the supported instrumentations to disable. Disabled instrumentations will not be instrumented as part of `configure_azure_monitor`. However, they can still be manually instrumented with `instrument()` directly. Accepts a comma-separated list of lowercase [Library Names](#officially-supported-instrumentations). For example, set to `"psycopg2,fastapi"` to disable the Psycopg2 and FastAPI instrumentations. Defaults to an empty list, enabling all supported instrumentations. |
 | `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` | An experimental OpenTelemetry environment variable used to specify Resource Detectors to be used to generate Resource Attributes. This is an experimental feature and the name of this variable and its behavior can change in a non-backwards compatible way. Defaults to "azure_app_service,azure_vm" to enable the [Azure Resource Detectors][ot_resource_detector_azure] for Azure App Service and Azure VM. To add or remove specific resource detectors, set the environment variable accordingly. See the [OpenTelemetry Python Resource Detector Documentation][ot_python_resource_detectors] for more. |
 
@@ -268,4 +269,5 @@ contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additio
 [python_logging_formatter]: https://docs.python.org/3/library/logging.html#formatter-objects
 [python_logging_level]: https://docs.python.org/3/library/logging.html#levels
 [samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-opentelemetry/samples
+[sampling_configurations]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-opentelemetry/samples/tracing/sampling_configurations.py
 [samples_manual]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-opentelemetry/samples/tracing/manually_instrumented.py

sdk/monitor/azure-monitor-opentelemetry/azure/monitor/opentelemetry/_configure.py

@@ -166,15 +166,15 @@ def _setup_tracing(configurations: Dict[str, ConfigurationValue]):
         sampler_type = configurations[SAMPLER_TYPE]
         sampler = _get_sampler_from_name(sampler_type, sampler_arg)
         tracer_provider = TracerProvider(sampler=sampler, resource=resource)
-    elif SAMPLING_TRACES_PER_SECOND_ARG in configurations:
-        traces_per_second = configurations[SAMPLING_TRACES_PER_SECOND_ARG]
+    elif SAMPLING_RATIO_ARG in configurations:
+        sampling_ratio = configurations[SAMPLING_RATIO_ARG]
         tracer_provider = TracerProvider(
-            sampler=RateLimitedSampler(target_spans_per_second_limit=cast(float, traces_per_second)), resource=resource
+            sampler=ApplicationInsightsSampler(sampling_ratio=cast(float, sampling_ratio)), resource=resource
         )
     else:
-        sampling_ratio = configurations[SAMPLING_RATIO_ARG]
+        traces_per_second = configurations[SAMPLING_TRACES_PER_SECOND_ARG]
         tracer_provider = TracerProvider(
-            sampler=ApplicationInsightsSampler(sampling_ratio=cast(float, sampling_ratio)), resource=resource
+            sampler=RateLimitedSampler(target_spans_per_second_limit=cast(float, traces_per_second)), resource=resource
         )
 
     for span_processor in configurations[SPAN_PROCESSORS_ARG]:  # type: ignore

sdk/monitor/azure-monitor-opentelemetry/azure/monitor/opentelemetry/_utils/configurations.py

@@ -168,27 +168,28 @@ def _default_resource(configurations):
 # pylint: disable=too-many-statements,too-many-branches
 def _default_sampling_ratio(configurations):
     default_value = 1.0
+    default_value_for_rate_limited_sampler = 5.0
     sampler_type = environ.get(OTEL_TRACES_SAMPLER)
     sampler_arg = environ.get(OTEL_TRACES_SAMPLER_ARG)
 
     # Handle rate-limited sampler
     if sampler_type == RATE_LIMITED_SAMPLER:
         try:
-            sampler_value = float(sampler_arg) if sampler_arg is not None else default_value
+            sampler_value = float(sampler_arg) if sampler_arg is not None else default_value_for_rate_limited_sampler
             if sampler_value < 0.0:
                 _logger.error("Invalid value for OTEL_TRACES_SAMPLER_ARG. It should be a non-negative number.")
-                sampler_value = default_value
+                sampler_value = default_value_for_rate_limited_sampler
             else:
                 _logger.info("Using rate limited sampler: %s traces per second", sampler_value)
             configurations[SAMPLING_TRACES_PER_SECOND_ARG] = sampler_value
         except ValueError as e:
             _logger.error(  # pylint: disable=C
                 _INVALID_TRACES_PER_SECOND_MESSAGE,
                 OTEL_TRACES_SAMPLER_ARG,
-                default_value,
+                default_value_for_rate_limited_sampler,
                 e,
             )
-            configurations[SAMPLING_TRACES_PER_SECOND_ARG] = default_value
+            configurations[SAMPLING_TRACES_PER_SECOND_ARG] = default_value_for_rate_limited_sampler
 
     # Handle fixed percentage sampler
     elif sampler_type in (FIXED_PERCENTAGE_SAMPLER, "microsoft.fixed.percentage"):  # to support older string
@@ -271,15 +272,15 @@ def _default_sampling_ratio(configurations):
 
     # Handle all other cases (no sampler type specified or unsupported sampler type)
     else:
-        if configurations.get(SAMPLING_RATIO_ARG) is None:
-            configurations[SAMPLING_RATIO_ARG] = default_value
+        if configurations.get(SAMPLING_TRACES_PER_SECOND_ARG) is None:
+            configurations[SAMPLING_TRACES_PER_SECOND_ARG] = default_value_for_rate_limited_sampler
         if sampler_type is not None:
             _logger.error(  # pylint: disable=C
                 "Invalid argument for the sampler to be used for tracing. "
                 "Supported values are %s. Defaulting to %s: %s",
                 SUPPORTED_OTEL_SAMPLERS,
-                FIXED_PERCENTAGE_SAMPLER,
-                configurations[SAMPLING_RATIO_ARG],
+                RATE_LIMITED_SAMPLER,
+                configurations[SAMPLING_TRACES_PER_SECOND_ARG],
             )
 
 

sdk/monitor/azure-monitor-opentelemetry/samples/tracing/sampling_configurations.py

@@ -17,8 +17,8 @@
 
 # Using trace_id_ratio sampler
 # Set the OTEL_TRACES_SAMPLER environment variable to "trace_id_ratio"
-# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.1, it has to be a number between 0 and 1, else it will throw an error and default to 1.0
-# The sampling rate is 0.1 means approximately 10% of your traces are sent
+# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.1; it must be a number between 0 and 1 or it defaults to 1.0.
+# A sampling rate of 0.1 means approximately 10% of your traces are sent.
 
 # Using parentbased_always_on sampler
 # Set the OTEL_TRACES_SAMPLER environment variable to "parentbased_always_on"
@@ -30,26 +30,40 @@
 
 # Using parentbased_trace_id_ratio sampler
 # Set the OTEL_TRACES_SAMPLER environment variable to "parentbased_trace_id_ratio"
-# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.45, it has to be a number between 0 and 1, else it will throw an error and default to 1.0
-# The sampling rate is 0.45 means approximately 45% of your traces are sent
+# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.45; it must be a number between 0 and 1 or it defaults to 1.0.
+# A sampling rate of 0.45 means approximately 45% of your traces are sent.
 
-# Using rate limited sampler
+# Using rate limited sampler (this is the default sampler)
 # Set the OTEL_TRACES_SAMPLER environment variable to "microsoft.rate_limited"
 # Set the OTEL_TRACES_SAMPLER_ARG environment variable to the desired rate limit (e.g., 0.5 means one trace every two seconds, while 5.0 means five traces per second)
+# You can also configure the rate limited sampler by passing the `traces_per_second` argument to `configure_azure_monitor`.
+
+"""
+    configure_azure_monitor (
+        traces_per_second: 0.5,
+    ) 
+"""
 
 # Using fixed percentage sampler
 # Set the OTEL_TRACES_SAMPLER environment variable to "microsoft.fixed_percentage"
-# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.2, it has to be a number between 0 and 1, else it will throw an error and default to 1.0
+# Set the OTEL_TRACES_SAMPLER_ARG environment variable to 0.2; it must be a number between 0 and 1 or it defaults to 1.0.
+# When configuring sampling via `configure_azure_monitor`, the default sampler is rate limited. To use the classic Application Insights sampler instead, set `sampling_ratio` to 1.0. # pylint: disable=line-too-long
+
+"""
+    configure_azure_monitor (
+        sampling_ratio: 1.0,
+    ) 
+"""
 
 # Using trace_based_sampling configuration # cspell: ignore unsampled
 # Determines whether the logger should drop log records associated with unsampled traces.
-# Passing the enable_trace_based_sampling_for_logs=True argument to configure_azure_monitor ensure that log records associated with unsampled traces are dropped by the `Logger`.
+# Passing the enable_trace_based_sampling_for_logs=True argument to configure_azure_monitor ensures that log records associated with unsampled traces are dropped by the logger.
 # A log record is considered associated with an unsampled trace if it has a valid `SpanId` and its `TraceFlags` indicate that the trace is unsampled.
-# The value of this config is False by default
+# The value of this config is False by default.
 
 """
     configure_azure_monitor (
-        "enable_trace_based_sampling_for_logs": True,
+        enable_trace_based_sampling_for_logs: True,
     ) 
 """
 

sdk/monitor/azure-monitor-opentelemetry/tests/utils/test_configurations.py

@@ -151,7 +151,7 @@ def test_get_configurations_defaults(self, resource_create_mock):
         self.assertEqual(configurations["resource"].attributes, TEST_DEFAULT_RESOURCE.attributes)
         self.assertEqual(environ[OTEL_EXPERIMENTAL_RESOURCE_DETECTORS], "azure_app_service,azure_vm")
         resource_create_mock.assert_called_once_with()
-        self.assertEqual(configurations["sampling_ratio"], 1.0)
+        self.assertEqual(configurations["traces_per_second"], 5.0)
         self.assertTrue("credential" not in configurations)
         self.assertTrue("storage_directory" not in configurations)
         self.assertEqual(configurations["enable_live_metrics"], True)
@@ -199,7 +199,7 @@ def test_get_configurations_env_vars(self, resource_create_mock):
         self.assertEqual(configurations["resource"].attributes, TEST_DEFAULT_RESOURCE.attributes)
         self.assertEqual(environ[OTEL_EXPERIMENTAL_RESOURCE_DETECTORS], "custom_resource_detector")
         resource_create_mock.assert_called_once_with()
-        self.assertEqual(configurations["sampling_ratio"], 1.0)
+        self.assertEqual(configurations["traces_per_second"], 5.0)
 
     @patch.dict(
         "os.environ",
@@ -527,7 +527,7 @@ def test_get_configurations_env_vars_no_preference(self, resource_create_mock):
         self.assertEqual(configurations["resource"].attributes, TEST_DEFAULT_RESOURCE.attributes)
         self.assertEqual(environ[OTEL_EXPERIMENTAL_RESOURCE_DETECTORS], "custom_resource_detector")
         resource_create_mock.assert_called_once_with()
-        self.assertEqual(configurations["sampling_ratio"], 1.0)
+        self.assertEqual(configurations["traces_per_second"], 5.0)
 
     @patch.dict(
         "os.environ",
@@ -565,7 +565,7 @@ def test_get_configurations_env_vars_check_default(self, resource_create_mock):
         self.assertEqual(configurations["resource"].attributes, TEST_DEFAULT_RESOURCE.attributes)
         self.assertEqual(environ[OTEL_EXPERIMENTAL_RESOURCE_DETECTORS], "custom_resource_detector")
         resource_create_mock.assert_called_once_with()
-        self.assertEqual(configurations["sampling_ratio"], 1.0)
+        self.assertEqual(configurations["traces_per_second"], 5.0)
 
     @patch.dict(
         "os.environ",
@@ -854,7 +854,7 @@ def test_get_configurations_env_vars_no_sampling_env_set(self, resource_create_m
 
         self.assertTrue("connection_string" not in configurations)
         self.assertEqual(configurations["resource"].attributes, TEST_DEFAULT_RESOURCE.attributes)
-        self.assertEqual(configurations["sampling_ratio"], 1.0)
+        self.assertEqual(configurations["traces_per_second"], 5.0)
 
     # Tests for the _get_sampler_from_name function
     def test_get_sampler_from_name_always_on_off(self):