Skip to content

Merge discvr-25.3 to discvr-25.7 #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bbimber merged 16 commits into from
Aug 20, 2025
Merged

Merge discvr-25.3 to discvr-25.7 #195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
db6458f
Use lsid instead of objectId for ETL
bbimber Aug 5, 2025
9f86bac
Use Audit table for PMR ETL deletes
bbimber Aug 6, 2025
d79c6a7
Add celltype column to tcr stats table
bbimber Aug 6, 2025
22a4369
Add tcrdb column
bbimber Aug 7, 2025
c04c297
Add study validation notification
bbimber Aug 8, 2025
22c2039
Correct column name in ETL
bbimber Aug 8, 2025
0e99631
Add ETL task to verify source/dest row counts
bbimber Aug 8, 2025
a93767f
Use maxRows in SelectRowsCommand
bbimber Aug 10, 2025
5c19329
Update ETL batchSize
bbimber Aug 11, 2025
61ee678
Simplify mGAP ETL
bbimber Aug 12, 2025
48bd615
Many improvements to EHR-related ETLs
bbimber Aug 12, 2025
ad75704
Improve mGAP QC
bbimber Aug 13, 2025
ccef459
Add MCC column showing numAnimalsRequested
bbimber Aug 19, 2025
bfaffc2
Sql fix
bbimber Aug 19, 2025
550b30f
Add url to column
bbimber Aug 20, 2025
d17c281
Merge discvr-25.3 to discvr-25.7
bbimber Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions PMR/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ dependencies {
BuildUtils.addLabKeyDependency(project: project, config: "modules", depProjectPath: ":server:modules:premiumModules:dataintegration", depProjectConfig: "published", depExtension: "module")
BuildUtils.addLabKeyDependency(project: project, config: "modules", depProjectPath: ":server:modules:DiscvrLabKeyModules:discvrcore", depProjectConfig: "published", depExtension: "module")
BuildUtils.addLabKeyDependency(project: project, config: "modules", depProjectPath: ":server:modules:onprcEHRModules:GeneticsCore", depProjectConfig: "published", depExtension: "module")
BuildUtils.addLabKeyDependency(project: project, config: "modules", depProjectPath: ":server:modules:BimberLabKeyModules:primeseq", depProjectConfig: "published", depExtension: "module")

BuildUtils.addLabKeyDependency(project: project, config: "implementation", depProjectPath: ":server:modules:DiscvrLabKeyModules:Studies", depProjectConfig: "apiJarFile")
BuildUtils.addLabKeyDependency(project: project, config: "implementation", depProjectPath: ":server:modules:premiumModules:dataintegration", depProjectConfig: "apiJarFile")
Expand Down
198 changes: 0 additions & 198 deletions PMR/resources/etls/pmr-datasets.xml
View file
Open in desktop

This file was deleted.

43 changes: 0 additions & 43 deletions PMR/resources/etls/pmr-demographics.xml
View file
Open in desktop

This file was deleted.

79 changes: 76 additions & 3 deletions PMR/resources/etls/pmrClinicalData.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,91 @@
<transform id="demographics" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/pmr-demographics"/>
<setting name="transformId" value="{PMR}/prime-demographics"/>
</settings>
</taskref>
</transform>

<transform id="datasets" type="TaskrefTransformStep">
<transform id="birth" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/pmr-datasets"/>
<setting name="transformId" value="{PMR}/prime-birth"/>
</settings>
</taskref>
</transform>

<transform id="blooddraws" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-blooddraws"/>
</settings>
</taskref>
</transform>

<transform id="chemistryResults" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-chemistryResults"/>
</settings>
</taskref>
</transform>

<transform id="clinpathRuns" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-clinpathRuns"/>
</settings>
</taskref>
</transform>

<transform id="hematologyResults" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-hematologyResults"/>
</settings>
</taskref>
</transform>

<transform id="histology" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-histology"/>
</settings>
</taskref>
</transform>

<transform id="microbiology" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-microbiology"/>
</settings>
</taskref>
</transform>

<transform id="parentage" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-parentage"/>
</settings>
</taskref>
</transform>

<transform id="pathologyDiagnoses" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-pathologyDiagnoses"/>
</settings>
</taskref>
</transform>

<transform id="weight" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<settings>
<setting name="transformId" value="{PMR}/prime-weight"/>
</settings>
</taskref>
</transform>

</transforms>
<schedule>
<cron expression="0 30 1 * * ?"/>
Expand Down
Loading