What would you like to happen?
What is the problem?
When using BigQuery Storage Write API (STORAGE_WRITE_API method) and write operations fail, the BigQueryStorageApiInsertError object does not contain information about which
table the error occurred on. This makes it difficult for users to identify and troubleshoot errors, especially when writing to multiple tables.
Current behavior:
BigQueryStorageApiInsertError only contains the row data and error message
- Users cannot determine which table caused the error without additional logging
Expected behavior:
BigQueryStorageApiInsertError should include table identification (project, dataset, table)
- API should be consistent with
BigQueryInsertError (used by STREAMING_INSERTS method), which provides table information via TableReference
Proposed solution:
Add the following to BigQueryStorageApiInsertError:
tableUrn field (format: projects/{project}/datasets/{dataset}/tables/{table})
- Convenience methods:
getProjectId(), getDatasetId(), getTableId()
Use case:
This is particularly useful for:
- Error monitoring and alerting systems
- Debugging write failures in multi-table pipelines
- Logging and auditing
Related:
- Consistent with existing
BigQueryInsertError API
- Uses standard
TableDestination.getTableUrn() format
Additional context:
I'm willing to contribute a PR for this enhancement.
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
What would you like to happen?
What is the problem?
When using BigQuery Storage Write API (
STORAGE_WRITE_APImethod) and write operations fail, theBigQueryStorageApiInsertErrorobject does not contain information about whichtable the error occurred on. This makes it difficult for users to identify and troubleshoot errors, especially when writing to multiple tables.
Current behavior:
BigQueryStorageApiInsertErroronly contains the row data and error messageExpected behavior:
BigQueryStorageApiInsertErrorshould include table identification (project, dataset, table)BigQueryInsertError(used bySTREAMING_INSERTSmethod), which provides table information viaTableReferenceProposed solution:
Add the following to
BigQueryStorageApiInsertError:tableUrnfield (format:projects/{project}/datasets/{dataset}/tables/{table})getProjectId(),getDatasetId(),getTableId()Use case:
This is particularly useful for:
Related:
BigQueryInsertErrorAPITableDestination.getTableUrn()formatAdditional context:
I'm willing to contribute a PR for this enhancement.
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components