Ingest Zhang18 spectra from FITS file

[ying2212]

Short description: Include what type of data being ingested and appropriate references.

Link to relevant issue: #529

For data ingests:

• includes script used for ingest
• includes modified JSON files
• Add new tests
• Update the Versions table

[kelle]

Great start with converting the spectra! My one suggestion at this point is to do the various instruments/modes separately. Right now, it looks like the wavelength units for "Xshooter_VIS" isn't quite right but "OSIRIS" looks good.

[kelle]

The spectra are looking great! But it looks like there are still some spectra which are failing or being skipped. We need to figure out what's going on with those.

[ying2212]

I’m running into errors with ingesting access_url in the Spectra table, seems like it only accepts URL format, but I’m working with local FITS files. I asked Carolina, and she said SIMPLE bdnyc has cloud storage for the files. Should I pull them from there instead?

[ying2212]

The spectra are looking great! But it looks like there are still some spectra which are failing or being skipped. We need to figure out what's going on with those.

I believe the failed file is a README file that is storing some reference info. For the skipped files are Xshooter instrument as I’m doing Xshooter and OSIRIS separately, and those skipped files have already been plotted up above!

[kelle]

For the ORSIRS data, let's trim their wavelength ranges like so:

spectrum = Spectrum.read(file_path) 
if filename == "ULAS_J230443.30+093423.9_sdL0_OSIRIS_GTC63-13A_Primeval-IV.fits":
        spectrum = extract_region(spectrum, SpectralRegion(0.625*u.um, 0.98*u.um))
if filename == "ULAS_J141832.35+025323.0_sdL0_comb_OSIRIS_scombine_Primeval-IV.fits":
        spectrum = extract_region(spectrum, SpectralRegion(0.62*u.um, 0.975*u.um))
else:
        spectrum = extract_region(spectrum, SpectralRegion(0.54*u.um, 1.02*u.um))

[kelle]

This PR is stuck on trying to figure out what to do with the spectra that need to be binned down in order to see the spectra well.

[kelle]

We decided to just make two spectra, one binned and unbinned. New question: how to deal with duplicate metadata and primary keys?