HIT: add data quality flag and uncertainty variable

[tech3371]

Summary

This became need and requirement because of HIT delivery custom L0 file to SDC due to FSW issue.

These custom L0 file has descriptor pha-telemetry-corrected.

Requirements

The more specific the better.

• Add new CDF variable in L1A for quality flag, first bit specifying a custom data from HIT so that user knows to use that with caution. Do something similar to SWE flags. If l0 file descriptor has pha-telemetry-corrected, we need to set that flag to true.

Related tickets

No response

[vmartinez-cu]

A systematic uncertainty variable already exists for L2 data. Sounds like an update to how this is calculated may be needed

imap_processing/imap_processing/hit/l2/hit_l2.py

Lines 426 to 459 in ad23c79

	def add_systematic_uncertainties(dataset: xr.Dataset, particle: str) -> xr.Dataset:
	"""
	Add systematic uncertainties to the dataset.
	Add systematic uncertainties to the dataset. Just zeros for now.
	To change if/when HIT determines there are systematic uncertainties.
	Parameters
	----------
	dataset : xr.Dataset
	The dataset to add the systematic uncertainties to
	which contain the particle data variables.
	particle : str
	The particle name.
	Returns
	-------
	updated_ds : xr.Dataset
	The dataset with the systematic uncertainties added.
	"""
	updated_ds = dataset.copy()
	updated_ds[f"{particle}_sys_err_minus"] = xr.DataArray(
	data=np.zeros(updated_ds[particle].shape, dtype=np.float32),
	dims=updated_ds[particle].dims,
	name=f"{particle}_sys_err_minus",
	)
	updated_ds[f"{particle}_sys_err_plus"] = xr.DataArray(
	data=np.zeros(updated_ds[particle].shape, dtype=np.float32),
	dims=updated_ds[particle].dims,
	name=f"{particle}_sys_err_plus",
	)
	return updated_ds

[vmartinez-cu]

It sounds like the data quality flag may need to be carried through to L2. HIT doesn't currently have a data quality flag variable, though it looks like a flag exists in quality_flags.py that can be imported

imap_processing/imap_processing/hit/l2/hit_l2.py

Lines 426 to 459 in ad23c79

	def add_systematic_uncertainties(dataset: xr.Dataset, particle: str) -> xr.Dataset:
	"""
	Add systematic uncertainties to the dataset.
	Add systematic uncertainties to the dataset. Just zeros for now.
	To change if/when HIT determines there are systematic uncertainties.
	Parameters
	----------
	dataset : xr.Dataset
	The dataset to add the systematic uncertainties to
	which contain the particle data variables.
	particle : str
	The particle name.
	Returns
	-------
	updated_ds : xr.Dataset
	The dataset with the systematic uncertainties added.
	"""
	updated_ds = dataset.copy()
	updated_ds[f"{particle}_sys_err_minus"] = xr.DataArray(
	data=np.zeros(updated_ds[particle].shape, dtype=np.float32),
	dims=updated_ds[particle].dims,
	name=f"{particle}_sys_err_minus",
	)
	updated_ds[f"{particle}_sys_err_plus"] = xr.DataArray(
	data=np.zeros(updated_ds[particle].shape, dtype=np.float32),
	dims=updated_ds[particle].dims,
	name=f"{particle}_sys_err_plus",
	)
	return updated_ds

[tech3371]

good to know @vmartinez-cu , then I will remove adding statistic uncertainty variable to l1a CDF from ticket checklist.

[jtniehof]

Release 1 handling plan is to put a note in the release notes that all the HIT data will be affected by this. Flag can be added later and we can note in those release notes to look for the flag.