Expose evaluations and metrics

generated/.openapi-generator/FILES

@@ -29,6 +29,11 @@ docs/ImageQueriesApi.md
 docs/ImageQuery.md
 docs/ImageQueryTypeEnum.md
 docs/InlineResponse200.md
+docs/InlineResponse2001.md
+docs/InlineResponse2001EvaluationResults.md
+docs/InlineResponse2002.md
+docs/InlineResponse200Summary.md
+docs/InlineResponse200SummaryClassCounts.md
 docs/LabelValue.md
 docs/LabelValueRequest.md
 docs/LabelsApi.md
@@ -42,6 +47,8 @@ docs/PaginatedDetectorList.md
 docs/PaginatedImageQueryList.md
 docs/PaginatedRuleList.md
 docs/PatchedDetectorRequest.md
+docs/PayloadTemplate.md
+docs/PayloadTemplateRequest.md
 docs/ROI.md
 docs/ROIRequest.md
 docs/ResultTypeEnum.md
@@ -96,6 +103,11 @@ groundlight_openapi_client/model/escalation_type_enum.py
 groundlight_openapi_client/model/image_query.py
 groundlight_openapi_client/model/image_query_type_enum.py
 groundlight_openapi_client/model/inline_response200.py
+groundlight_openapi_client/model/inline_response2001.py
+groundlight_openapi_client/model/inline_response2001_evaluation_results.py
+groundlight_openapi_client/model/inline_response2002.py
+groundlight_openapi_client/model/inline_response200_summary.py
+groundlight_openapi_client/model/inline_response200_summary_class_counts.py
 groundlight_openapi_client/model/label_value.py
 groundlight_openapi_client/model/label_value_request.py
 groundlight_openapi_client/model/mode_enum.py
@@ -107,6 +119,8 @@ groundlight_openapi_client/model/paginated_detector_list.py
 groundlight_openapi_client/model/paginated_image_query_list.py
 groundlight_openapi_client/model/paginated_rule_list.py
 groundlight_openapi_client/model/patched_detector_request.py
+groundlight_openapi_client/model/payload_template.py
+groundlight_openapi_client/model/payload_template_request.py
 groundlight_openapi_client/model/result_type_enum.py
 groundlight_openapi_client/model/roi.py
 groundlight_openapi_client/model/roi_request.py
@@ -128,4 +142,6 @@ setup.cfg
 setup.py
 test-requirements.txt
 test/__init__.py
+test/test_payload_template.py
+test/test_payload_template_request.py
 tox.ini

generated/README.md

@@ -94,6 +94,7 @@ rule_request = RuleRequest(
             WebhookActionRequest(
                 url="url_example",
                 include_image=True,
+                payload_template=None,
             ),
         ],
     ) # RuleRequest | 
@@ -122,6 +123,8 @@ Class | Method | HTTP request | Description
 *DetectorsApi* | [**create_detector**](docs/DetectorsApi.md#create_detector) | **POST** /v1/detectors | 
 *DetectorsApi* | [**delete_detector**](docs/DetectorsApi.md#delete_detector) | **DELETE** /v1/detectors/{id} | 
 *DetectorsApi* | [**get_detector**](docs/DetectorsApi.md#get_detector) | **GET** /v1/detectors/{id} | 
+*DetectorsApi* | [**get_detector_evaluation**](docs/DetectorsApi.md#get_detector_evaluation) | **GET** /v1/detectors/{id}/evaluation | 
+*DetectorsApi* | [**get_detector_metrics**](docs/DetectorsApi.md#get_detector_metrics) | **GET** /v1/detectors/{detector_id}/metrics | 
 *DetectorsApi* | [**list_detectors**](docs/DetectorsApi.md#list_detectors) | **GET** /v1/detectors | 
 *DetectorsApi* | [**update_detector**](docs/DetectorsApi.md#update_detector) | **PATCH** /v1/detectors/{id} | 
 *EdgeApi* | [**get_model_urls**](docs/EdgeApi.md#get_model_urls) | **GET** /v1/edge/fetch-model-urls/{detector_id}/ | 
@@ -160,6 +163,11 @@ Class | Method | HTTP request | Description
  - [ImageQuery](docs/ImageQuery.md)
  - [ImageQueryTypeEnum](docs/ImageQueryTypeEnum.md)
  - [InlineResponse200](docs/InlineResponse200.md)
+ - [InlineResponse2001](docs/InlineResponse2001.md)
+ - [InlineResponse2001EvaluationResults](docs/InlineResponse2001EvaluationResults.md)
+ - [InlineResponse2002](docs/InlineResponse2002.md)
+ - [InlineResponse200Summary](docs/InlineResponse200Summary.md)
+ - [InlineResponse200SummaryClassCounts](docs/InlineResponse200SummaryClassCounts.md)
  - [LabelValue](docs/LabelValue.md)
  - [LabelValueRequest](docs/LabelValueRequest.md)
  - [ModeEnum](docs/ModeEnum.md)
@@ -171,6 +179,8 @@ Class | Method | HTTP request | Description
  - [PaginatedImageQueryList](docs/PaginatedImageQueryList.md)
  - [PaginatedRuleList](docs/PaginatedRuleList.md)
  - [PatchedDetectorRequest](docs/PatchedDetectorRequest.md)
+ - [PayloadTemplate](docs/PayloadTemplate.md)
+ - [PayloadTemplateRequest](docs/PayloadTemplateRequest.md)
  - [ROI](docs/ROI.md)
  - [ROIRequest](docs/ROIRequest.md)
  - [ResultTypeEnum](docs/ResultTypeEnum.md)

generated/docs/ActionsApi.md

@@ -69,6 +69,7 @@ with groundlight_openapi_client.ApiClient(configuration) as api_client:
             WebhookActionRequest(
                 url="url_example",
                 include_image=True,
+                payload_template=None,
             ),
         ],
     ) # RuleRequest | 

generated/docs/DetectorsApi.md

@@ -7,6 +7,8 @@ Method | HTTP request | Description
 [**create_detector**](DetectorsApi.md#create_detector) | **POST** /v1/detectors | 
 [**delete_detector**](DetectorsApi.md#delete_detector) | **DELETE** /v1/detectors/{id} | 
 [**get_detector**](DetectorsApi.md#get_detector) | **GET** /v1/detectors/{id} | 
+[**get_detector_evaluation**](DetectorsApi.md#get_detector_evaluation) | **GET** /v1/detectors/{id}/evaluation | 
+[**get_detector_metrics**](DetectorsApi.md#get_detector_metrics) | **GET** /v1/detectors/{detector_id}/metrics | 
 [**list_detectors**](DetectorsApi.md#list_detectors) | **GET** /v1/detectors | 
 [**update_detector**](DetectorsApi.md#update_detector) | **PATCH** /v1/detectors/{id} | 
 
@@ -243,6 +245,160 @@ Name | Type | Description  | Notes
  - **Accept**: application/json
 
 
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** |  |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+
+# **get_detector_evaluation**
+> InlineResponse2001 get_detector_evaluation(id)
+
+
+
+Get Detector evaluation results. The result is null if there isn't enough ground truth data to evaluate the detector. Returns the time of the evaulation, total ground truth labels, the ml based kfold accuracies, and the system accuracies at different confidence thresholds
+
+### Example
+
+* Api Key Authentication (ApiToken):
+
+```python
+import time
+import groundlight_openapi_client
+from groundlight_openapi_client.api import detectors_api
+from groundlight_openapi_client.model.inline_response2001 import InlineResponse2001
+from pprint import pprint
+# Defining the host is optional and defaults to https://api.groundlight.ai/device-api
+# See configuration.py for a list of all supported configuration parameters.
+configuration = groundlight_openapi_client.Configuration(
+    host = "https://api.groundlight.ai/device-api"
+)
+
+# The client must configure the authentication and authorization parameters
+# in accordance with the API server security policy.
+# Examples for each auth method are provided below, use the example that
+# satisfies your auth use case.
+
+# Configure API key authorization: ApiToken
+configuration.api_key['ApiToken'] = 'YOUR_API_KEY'
+
+# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
+# configuration.api_key_prefix['ApiToken'] = 'Bearer'
+
+# Enter a context with an instance of the API client
+with groundlight_openapi_client.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = detectors_api.DetectorsApi(api_client)
+    id = "id_example" # str | 
+
+    # example passing only required values which don't have defaults set
+    try:
+        api_response = api_instance.get_detector_evaluation(id)
+        pprint(api_response)
+    except groundlight_openapi_client.ApiException as e:
+        print("Exception when calling DetectorsApi->get_detector_evaluation: %s\n" % e)
+```
+
+
+### Parameters
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **id** | **str**|  |
+
+### Return type
+
+[**InlineResponse2001**](InlineResponse2001.md)
+
+### Authorization
+
+[ApiToken](../README.md#ApiToken)
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: application/json
+
+
+### HTTP response details
+
+| Status code | Description | Response headers |
+|-------------|-------------|------------------|
+**200** |  |  -  |
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+
+# **get_detector_metrics**
+> InlineResponse200 get_detector_metrics(detector_id)
+
+
+
+Get Detector metrics, primarily the counts of different types of labels
+
+### Example
+
+* Api Key Authentication (ApiToken):
+
+```python
+import time
+import groundlight_openapi_client
+from groundlight_openapi_client.api import detectors_api
+from groundlight_openapi_client.model.inline_response200 import InlineResponse200
+from pprint import pprint
+# Defining the host is optional and defaults to https://api.groundlight.ai/device-api
+# See configuration.py for a list of all supported configuration parameters.
+configuration = groundlight_openapi_client.Configuration(
+    host = "https://api.groundlight.ai/device-api"
+)
+
+# The client must configure the authentication and authorization parameters
+# in accordance with the API server security policy.
+# Examples for each auth method are provided below, use the example that
+# satisfies your auth use case.
+
+# Configure API key authorization: ApiToken
+configuration.api_key['ApiToken'] = 'YOUR_API_KEY'
+
+# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
+# configuration.api_key_prefix['ApiToken'] = 'Bearer'
+
+# Enter a context with an instance of the API client
+with groundlight_openapi_client.ApiClient(configuration) as api_client:
+    # Create an instance of the API class
+    api_instance = detectors_api.DetectorsApi(api_client)
+    detector_id = "detector_id_example" # str | 
+
+    # example passing only required values which don't have defaults set
+    try:
+        api_response = api_instance.get_detector_metrics(detector_id)
+        pprint(api_response)
+    except groundlight_openapi_client.ApiException as e:
+        print("Exception when calling DetectorsApi->get_detector_metrics: %s\n" % e)
+```
+
+
+### Parameters
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **detector_id** | **str**|  |
+
+### Return type
+
+[**InlineResponse200**](InlineResponse200.md)
+
+### Authorization
+
+[ApiToken](../README.md#ApiToken)
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: application/json
+
+
 ### HTTP response details
 
 | Status code | Description | Response headers |

generated/docs/InlineResponse200.md

@@ -4,7 +4,7 @@
 ## Properties
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
-**username** | **str** | The user's username | [optional] 
+**summary** | [**InlineResponse200Summary**](InlineResponse200Summary.md) |  | [optional] 
 **any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

generated/docs/InlineResponse2001.md

@@ -0,0 +1,12 @@
+# InlineResponse2001
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**evaluation_results** | [**InlineResponse2001EvaluationResults**](InlineResponse2001EvaluationResults.md) |  | [optional] 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/InlineResponse2001EvaluationResults.md

@@ -0,0 +1,19 @@
+# InlineResponse2001EvaluationResults
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**eval_timestamp** | **datetime** |  | [optional] 
+**total_ground_truth_examples** | **int, none_type** |  | [optional] 
+**kfold_pooled__balanced_accuracy** | **float** |  | [optional] 
+**kfold_pooled__positive_accuracy** | **float** |  | [optional] 
+**kfold_pooled__negative_accuracy** | **float** |  | [optional] 
+**balanced_system_accuracies** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**positive_system_accuracies** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**negative_system_accuracies** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/InlineResponse2002.md

@@ -0,0 +1,12 @@
+# InlineResponse2002
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**username** | **str** | The user's username | [optional] 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/InlineResponse200Summary.md

@@ -0,0 +1,16 @@
+# InlineResponse200Summary
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**num_ground_truth** | **int** |  | [optional] 
+**num_current_source_human** | **int** |  | [optional] 
+**class_counts** | [**InlineResponse200SummaryClassCounts**](InlineResponse200SummaryClassCounts.md) |  | [optional] 
+**unconfident_counts** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**total_iqs** | **int** |  | [optional] 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/InlineResponse200SummaryClassCounts.md

@@ -0,0 +1,16 @@
+# InlineResponse200SummaryClassCounts
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**source_ml** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**source_human** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**cloud_labeler** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**cloud** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**total** | **{str: (bool, date, datetime, dict, float, int, list, str, none_type)}** |  | [optional] 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/PayloadTemplate.md

@@ -0,0 +1,12 @@
+# PayloadTemplate
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**template** | **str** |  | 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/PayloadTemplateRequest.md

@@ -0,0 +1,12 @@
+# PayloadTemplateRequest
+
+
+## Properties
+Name | Type | Description | Notes
+------------ | ------------- | ------------- | -------------
+**template** | **str** |  | 
+**any string name** | **bool, date, datetime, dict, float, int, list, str, none_type** | any string name can be used but the value must be the correct type | [optional]
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+
+

generated/docs/TextRecognitionResult.md

@@ -4,7 +4,7 @@
 ## Properties
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
-**text** | **str, none_type** |  | 
+**text** | **str** |  | 
 **truncated** | **bool** |  | 
 **confidence** | **float, none_type** |  | [optional] 
 **source** | **str** |  | [optional]