Skip to content

Conversation

@kaseyLee123
Copy link
Contributor

@kaseyLee123 kaseyLee123 commented Jul 29, 2025

Short description: Include what type of data being ingested and appropriate references.

Link to relevant issue: Closes #518 and Closes #484 .

For data ingests:
[ ] includes script used for ingest
[ ] includes modified JSON files
[ ] Add new tests
[ ] Update the Versions table

@kaseyLee123 kaseyLee123 changed the title basic change Ingest modeled parameters sanghi23 Jul 29, 2025
@kaseyLee123 kaseyLee123 self-assigned this Jul 30, 2025
@kelle
Copy link
Collaborator

kelle commented Jul 30, 2025

This looks like great progress! Before adding new parameters, please add a test for the modeled parameters that counts the number of parameters with model = Null.

@kelle
Copy link
Collaborator

kelle commented Jul 30, 2025

Also update with main. It shouldn't impact this PR since it's all photometry and proper motions.

@kaseyLee123
Copy link
Contributor Author

I just edited pytest cases where I added one for checking which had a null value for model and also how many of each parameter was ingested (added one for L bol). I added counters in my script and have the number in the comments next to it and the pytest case numbers make sense when I do the math

@kaseyLee123
Copy link
Contributor Author

Just pushed the new pytest file. I have the JSON files (a little over 1000) ready to push whenever

Comment on lines 11 to 13
if len(t) > 0:
print("\nEntries found without a model")
print(t)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if statement is not needed since we expect some parameters to not have models.

Suggested change
if len(t) > 0:
print("\nEntries found without a model")
print(t)

@kelle
Copy link
Collaborator

kelle commented Jul 30, 2025

Awesome progress!

  • could you please try to push just 10-20 JSON files. If you're still using Github Desktop, use the checkboxes to Deselect All and then Select 10-20 by hand to add, commit, and push.
  • add tests which test the references for the modeled paramaters. How many Lbols from Fili15?

@kaseyLee123
Copy link
Contributor Author

Awesome progress!

  • could you please try to push just 10-20 JSON files. If you're still using Github Desktop, use the checkboxes to Deselect All and then Select 10-20 by hand to add, commit, and push.
  • add tests which test the references for the modeled paramaters. How many Lbols from Fili15?

Did you want the pytest case to check references for each parameter or just in general?

@kelle
Copy link
Collaborator

kelle commented Jul 30, 2025

For each parameter. How many Lbols from Fili15? How many Lbols from Sang24?

# delimiter: ','
# schema: astropy-2.0
Skipped,Reason
veryred-1527,"no source found, atmo model"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there really a source name called "veryred-1527"? @cosmicoder, any insight on what source this is?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Kelle, that is an old internal name that accidentally propagated through. The source is PSO J243.9421+67.2075. Let me know if there are any others. They should be updated in the latest version of the UltracoolSheet

@kelle
Copy link
Collaborator

kelle commented Jul 30, 2025

  • Please get the tests to pass locally with the new JSON files and push the test.py files, but not all of the JSONs yet.
  • Go ahead and also update the Versions.md. The new version will be 4.3.2025.10. You can guess a date. maybe Friday. :)

@kaseyLee123
Copy link
Contributor Author

just did! Not sure if the pytest case is the most efficient way because it seems a little long but. Also, the csv files did not have a ref column so I just put sang23 for all of them.

@kelle
Copy link
Collaborator

kelle commented Jul 31, 2025

Yep, this looks great. Once you ingest the one skipped source (PSO J243.9421+67.2075), go ahead and push the JSONs!

@kaseyLee123
Copy link
Contributor Author

just pushed

Copy link
Collaborator

@kelle kelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is done. Great work!!!

@kelle
Copy link
Collaborator

kelle commented Jul 31, 2025

Ohhhh, adding the adopted column! Let's go ahead and do that now since you're already modifying the schema.

  • Add the adopted column - set them all to Null or 0.
  • We don't have to actually adopt any measurements at the moment -- that will be hard to decide!
  • Add a test which counts the number of adopted measurements. Should be zero for this PR.

Copy link
Collaborator

@kelle kelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's tweak the schema a bit more and add the adopted column.

@kelle
Copy link
Collaborator

kelle commented Jul 31, 2025

Also, take a look at the ModeledParameters ingest template spreadsheet. Let's make sure what you're doing here is reflected in that template.

@kelle
Copy link
Collaborator

kelle commented Jul 31, 2025

Looking at the ingest template, let's also add the other_references column.

@kaseyLee123
Copy link
Contributor Author

Just did everything looks good I think

Copy link
Collaborator

@kelle kelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing fast work! You're knocking it out of the park!

@kelle kelle merged commit 369473d into SIMPLE-AstroDB:main Jul 31, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingest modeled parameters from Sanghi et al. 2023 Tweak Modeled Parameters table

3 participants