bigquery: add policyTags support to ignore_schema_changes on google_b…

[GraceAtwood]

Description

Add support for policyTags as a value in the ignore_schema_changes virtual field on google_bigquery_table. This mirrors the existing dataPolicies implementation (PR #23495, PR #25721) and allows users to ignore externally-applied policy tag changes without having to ignore the entire schema.

Problem

BigQuery policy tags (policyTags) are often managed outside Terraform by automated tools (e.g., PII auto-taggers that scan tables daily). Since the schema attribute is a single JSON blob, ignore_changes cannot target individual fields within it — users must either ignore the entire schema or manually track every tag change in their Terraform config.

The ignore_schema_changes field already supports dataPolicies (data masking policies). This PR extends it to also support policyTags (column-level access control tags), which are a different schema field (TableFieldSchema.policyTags vs TableFieldSchema.dataPolicies).

Related Issues

• hashicorp/terraform-provider-google#8724 — Investigate reworking bigquery table schema field (open since 2021)
• hashicorp/terraform-provider-google#19893 — ignore_changes to specific properties within JSON-encoded attribute
• Google Dev Forum: Terraform ignore_changes on PolicyTags in BigQuery

Behavior

When ignore_schema_changes = ["policyTags"] is set:

• Policy tags applied outside Terraform (e.g., by an auto-tagger) are preserved during updates
• Policy tags explicitly set in the Terraform config still override live state
• Diff suppression prevents plan noise from external tag changes
• Recursive handling supports nested RECORD/STRUCT types

Changes

• resource_bigquery_table.go.tmpl: Added mergePolicyTags() and mergePolicyTagsIntoMap() functions, updated diff suppression, CustomizeDiff, and Update logic
• resource_bigquery_table_internal_test.go.tmpl: Unit tests for diff suppress, merge struct, merge map, isChangeable
• resource_bigquery_table_test.go: 3-stage acceptance test (TestAccBigQueryTable_PolicyTagsMerge)
• bigquery_table.html.markdown: Updated docs to mention policyTags alongside dataPolicies

bigquery: added `policyTags` support to `ignore_schema_changes` in `google_bigquery_table` resource

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@slevenick, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[github-actions]

@slevenick This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 609 insertions(+), 13 deletions(-))
google-beta provider: Diff ( 4 files changed, 609 insertions(+), 13 deletions(-))

[modular-magician]

Tests analytics

Total tests: 150
Passed tests: 139
Skipped tests: 10
Affected tests: 1

Click here to see the affected service packages

• bigquery

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccBigQueryTable_PolicyTagsMerge

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccBigQueryTable_PolicyTagsMerge [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

---

🟢 All tests passed!

View the build log or the debug log for each test

[slevenick]

@wj-chen please take a look and see if this fits this resource

[github-actions]

@GoogleCloudPlatform/terraform-team @slevenick This PR has been waiting for review for 1 week. Please take a look! Use the label disable-review-reminders to disable these notifications.

[slevenick]

waiting on review from @wj-chen

[sachinpro]

Can we merge mergePolicyTagsIntoMap and mergeDataPoliciesIntoMap functions into a single one like:

// mergeSchemaMapProperties merges live state properties into the TF config map representation.
// propertyKey expects either "dataPolicies" or "policyTags".
func mergeSchemaMapProperties(old, new []interface{}, propertyKey string) {
	oldMap := make(map[string]map[string]interface{})
	for _, v := range old {
		if m, ok := v.(map[string]interface{}); ok {
			if name, ok := m["name"].(string); ok {
				oldMap[name] = m
			}
		}
	}

	for _, v := range new {
		newField, ok := v.(map[string]interface{})
		if !ok {
			continue
		}
		name := newField["name"].(string)

		if oldField, exists := oldMap[name]; exists {
			// If config doesn't have the property, but backend does, copy it over
			if _, specified := newField[propertyKey]; !specified {
				if val, hasVal := oldField[propertyKey]; hasVal {
					newField[propertyKey] = val
					log.Printf("[DEBUG] Added live %s to schema: %v", propertyKey, val)
				}
			}

			// Recursively handle nested fields (RECORD types)
			if oldNested, ok1 := oldField["fields"].([]interface{}); ok1 {
				if newNested, ok2 := newField["fields"].([]interface{}); ok2 {
					// Recursive call with the same propertyKey
					mergeSchemaMapProperties(oldNested, newNested, propertyKey)
				}
			}
		}
	}
}

[sachinpro]

Similar to the other comment, we can also have a common function for this:

// mergeSchemaFieldProperties merges live state properties (like dataPolicies or policyTags)
// into the TF config struct if they are not defined in the raw config.
// propertyKey expects either "dataPolicies" or "policyTags".
func mergeSchemaFieldProperties(configFields []*bigquery.TableFieldSchema, liveFields []*bigquery.TableFieldSchema, rawSchema []interface{}, propertyKey string) {
	liveMap := make(map[string]*bigquery.TableFieldSchema)
	if liveFields != nil {
		for _, f := range liveFields {
			liveMap[f.Name] = f
		}
	}

	rawMap := make(map[string]map[string]interface{})
	for _, item := range rawSchema {
		if m, ok := item.(map[string]interface{}); ok {
			if name, ok := m["name"].(string); ok {
				rawMap[name] = m
			}
		}
	}

	for _, configField := range configFields {
		rawField, rawExists := rawMap[configField.Name]
		liveField := liveMap[configField.Name]

		// Recursively handle nested fields (RECORD/STRUCT types)
		if len(configField.Fields) > 0 {
			var liveNested []*bigquery.TableFieldSchema
			if liveField != nil {
				liveNested = liveField.Fields
			}
			var rawNested []interface{}
			if rawExists {
				if fields, ok := rawField["fields"].([]interface{}); ok {
					rawNested = fields
				}
			}
			// Recursive call with the same propertyKey
			mergeSchemaFieldProperties(configField.Fields, liveNested, rawNested, propertyKey)
		}

		// Handle specific property logic
		isSpecified := false
		if rawExists {
			// Check if the property key explicitly exists in the raw config JSON
			_, isSpecified = rawField[propertyKey]
		}

		// If the user did NOT specify the property in the config,
		// and it exists on the server, copy it to the config struct.
		if !isSpecified && liveField != nil {
			switch propertyKey {
			case "dataPolicies":
				if len(liveField.DataPolicies) > 0 {
					configField.DataPolicies = liveField.DataPolicies
				}
			case "policyTags":
				if liveField.PolicyTags != nil && len(liveField.PolicyTags.Names) > 0 {
					configField.PolicyTags = liveField.PolicyTags
				}
			}
		}
	}
}

This makes more sense if there can be other ignoreSchemaChanges properties which need similar merging logic.

If we don't anticipate more properties like these, I am fine with duplication.

[sachinpro]

@wj-chen Any inputs on this?

[wj-chen]

Yes, prefer consolidating the merging logic into a common helper if it doesn't differ too much between the two. This would reduce duplication and the need for conditional blocks.

[github-actions]

@GraceAtwood, this PR is waiting for action from you. If no action is taken, this PR will be closed in 28 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

[github-actions]

@GraceAtwood, this PR is waiting for action from you. If no action is taken, this PR will be closed in 14 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.

[github-actions]

@GraceAtwood, this PR is waiting for action from you. If no action is taken, this PR will be closed in 14 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

This notification can be disabled with the disable-automatic-closure label.