UCL Scotch support for SWIFT by themkots · Pull Request #42 · SWIFTSIM/SWIFT

themkots · 2023-11-17T11:00:03Z

No description provided.

…to ucl-scotch-build

…ilation.

…to ucl-scotch-build

docs: New documentation on the installation and running with SWIFT

Update README-Scotch.md

fix: Formatting according to format.sh (clang-format)

fix: Clean up of excess files

fix: Formatting scripts

fix: Revert whitespace

fix: Adding defs of idx_t and IDX_MAX

* Fix for infinite loop in Brent's method * Fixed issues with missing unlocks/dependencies/debug checks for top level cells which aren't also super cells * GEAR-RT: theory * Deadlock detector * Update spin/jet AGN model * corrected number of items to write in tracers_io.h * Fix new time-step for spin-jet scheme * Grackle Cooling updates * Make the EAGLE_ICs example load the PS2020 cooling tables and not the Wiersma+09 ones * Update EAGLE_ICs README to mention how to load the Wiersma tables if needed * Add a new potential MWPotential2014 * Fix jet initial velocities * Fix EAGLE tracers' list of outputs * Add forcing terms to the hydro equations * Add barotropic EoS * Add a function so that we can mark memory as deliberately leaked at the end of the program. * Update README-Scotch.md * Update README-Scotch.md Co-authored-by: Ilektra Christidi <ilektra.christidi@ucl.ac.uk> * Update README-Scotch.md Co-authored-by: Ilektra Christidi <ilektra.christidi@ucl.ac.uk> * Update README-Scotch.md Co-authored-by: Ilektra Christidi <ilektra.christidi@ucl.ac.uk> * Update README-Scotch.md Co-authored-by: Ilektra Christidi <ilektra.christidi@ucl.ac.uk> --------- Co-authored-by: Yolan Uyttenhove <yolan.uyttenhove@ugent.be> Co-authored-by: Matthieu Schaller <schaller@strw.leidenuniv.nl> Co-authored-by: Mladen Ivkovic <mladen.ivkovic@hotmail.com> Co-authored-by: Filip Husko <filip.husko@durham.ac.uk> Co-authored-by: Yves Revaz <yves.revaz@epfl.ch> Co-authored-by: Darwin <darwin.roduit@epfl.ch> Co-authored-by: Ilektra Christidi <ilektra.christidi@ucl.ac.uk>

* fix: Passing architecture file as flag * Fixed prevailing SCOTCH invocation with configure script; made ArchData globally visible. * Fixed picking scotch when also have metis/parmetis * Fixed picking scotch when also have metis/parmetis * Fixed populating partition struct with scotch target arch file before calling initial partitioning. * Prevent destruction and empty regeneration of archdat structure. * Edited swift_fof.c to match target file updates of swift.c * Minor edits: replaced a HAVE_PARMETIS with a HAVE_METIS, edit a message to show either SCOTCH or METIS used, changed strategy flag from SCOTCH_STRATQUALITY to SCOTCH_STRATBALANCE * Added a mesage to notify when a successful SCOTCH repartition takes place * Dropping reference to METIS in pick_scotch * Changed format specifiers for SCOTCH weights reporting in debug check - now acceptable for both 32 and 64 bit versions. * Merged two separate ifdef chunks of HAVE_SCOTCH for better visibility * Dressed up naked mallocs; averaged [div by 2.0] the edge loads; removed commented out and unused code. * Updated string for averaged and not summed weights --------- Co-authored-by: Dan Giles <dangiles@Dans-MacBook-Pro.local> Co-authored-by: Themis Kotsialos <e.kotsialos@ucl.ac.uk>

* Initialised return_edge variable * Removed a typedef for idx_t and a define for SCOTCH_Idx; * Removed typedef from metis/parmetis - only code chunk.

…83647); makes scotch64 and scotch-local balance slightly better.

…33) * Update partition.c * Update configure.ac * Removal of refine in Scotch

* Update debug.c - minor edits * Update engine.c * Update swift.c * Update swift_fof.c * Update partition.c * Update partition.c

* Update README-Scotch.md * Update README-Scotch.md - Warning on compiler inclusion preferences * Update README-Scotch.md - prod runs table updates 01 * Update README-Scotch.md table runs 02 * Update README-Scotch.md remove old table * Update README-Scotch.md added note for Rome and Milan type of Cosma 8 nodes * Update README-Scotch.md * Update README-Scotch.md - prod runs table 03 * Update README-Scotch.md - large simulations dump minimisation --------- Co-authored-by: themkots <70752441+themkots@users.noreply.github.com>

MatthieuSchaller · 2024-07-03T09:09:00Z

@pwdraper, for the records, could we list here the issues we had not resolved?
I think the main one was the lack of 64-bits support and the inability for scotch to evolve a previous partition?

themkots · 2024-07-03T11:10:42Z

@MatthieuSchaller , @pwdraper , @DanGiles
There are the things that @MatthieuSchaller mentions just above
(32-bit linking where we expected 64-bit one, no refinement -- for the scotch-version, not the ptscotch one --
we had no time investigating ptsccotch, as per this issue messages, and I find hard to believe that ptscotch does not do
an incremental refinement -- otherwise how can it do parallelisation/mpi stuff ?? --)
There is also the hard-wired graph-connectivity issue that metis/parmetis seem not to cross over, but scotch does, i.e. when it finds more that <hard wired number? cannot remember right now - is in src - 64? something else?> nodes able to communicate with one another, it tries to leverage this large connectivity for load/compute/comms balance, but this results in runtime error from code when discovering so in graph partitioning call.

Wish we had more time to delve into those issues, and a) have 64-bit clean builds, b) explore the obvious strategy of incremental refinement, c) explore large connectivity for better / lower than say 5% gaps.

Ta.

MatthieuSchaller · 2024-07-03T14:00:24Z

Thanks Themys!

For the 64-bits build, do you recall whether it's a problem of creating a clean build or what is a more fundamental lack of support? My memory says it's the latter.

themkots · 2024-07-03T15:19:22Z

Yes, it is their source that links the 32 bit one --- could attempt a direct edit, or send a PR, if in github somehow, but not right now...

Dan Giles and others added 30 commits March 15, 2023 10:41

feat: Read in the metis dump files and generate a scotch graph

856b033

outputting the graph structure

386304b

Merge branch 'master' of https://github.com/UCL/swiftsim

9aadb06

feat: Building with Scotch

83533a0

Updating Makefile and adding pick-scotch function

4783ed5

Merge branch 'ucl-scotch-build' of https://github.com/UCL/swiftsim in…

a361b6a

…to ucl-scotch-build

Merge branch 'SWIFTSIM:master' into ucl-scotch-build

e157cca

fix: Builds on Myriad

4486162

Merge branch 'ucl-scotch-build' of https://github.com/UCL/swiftsim in…

efc7998

…to ucl-scotch-build

Update partition.c

b0302b4

Correction to scotch functions

ae3533a

Merge branch 'ucl-scotch-build' of https://github.com/UCL/swiftsim in…

99582fd

…to ucl-scotch-build

Adding test tgt file

89eb834

Addition of the .tgt which represents a Myriad node

dbd3f95

fix: comm matrix passed to scotch

1ee20ec

Merge branch 'SWIFTSIM:master' into ucl-scotch-build

6959b86

fix: Correction to partition_repatition

7654c83

Merge branch 'ucl-scotch-build' of https://github.com/UCL/swiftsim in…

c5a399d

…to ucl-scotch-build

fix: filling up the edge and vertex weights

c67ff38

updating the target arch file for one cosma node (8 NUMA regions)

6921ded

fix: Updating to read in target file

7e5898d

Adding a template multi cosma node job script

0bdf8f3

Adding template target files

cc90448

Merge branch 'SWIFTSIM:master' into ucl-scotch-build

251c8f3

Added scripts to create target file.

0144f84

Incremental edits.

dd440d0

Defining the mapping strategy

bb9e65d

Added SCOTCH_LIBS, SCOTCH_INCS to src/Makefile.am for Scotch MPI comp…

151a7e9

…ilation.

updated assignment of edges

b9e23d7

Merge branch 'ucl-scotch-build' of https://github.com/UCL/swiftsim in…

2c704dc

…to ucl-scotch-build

DanGiles and others added 26 commits August 21, 2023 13:09

Merge pull request #17 from UCL/dg-cleanup-docs

0f5c901

docs: New documentation on the installation and running with SWIFT

Update README-Scotch.md

c33a912

Merge pull request #19 from UCL/DanGiles-docs-update

5053cb6

Update README-Scotch.md

Update README-Scotch.md

e7c7479

Merge pull request #20 from UCL/DanGiles-patch-1

016ec08

Update README-Scotch.md

fix: Formatting according to clang-format.sh

ca0f018

Merge pull request #21 from UCL/dg-formatting

9c16bb6

fix: Formatting according to format.sh (clang-format)

fix: Clean up of excess files

0a2e732

Merge pull request #22 from UCL/dg-cleanup

5cb01a5

fix: Clean up of excess files

fix: Formatting scripts

3ab621c

Merge pull request #23 from UCL/dg-format-script

1b9439b

fix: Formatting scripts

fix: Revert whitespace

ca9deed

Merge pull request #24 from UCL/dg-revert-whitespace

9175f7d

fix: Revert whitespace

fix: Adding defs of idx_t

7c62a50

fix: Check for METIS and PARMETIS

6621820

adding scotch repartition_name[]

54c5ac4

Merge pull request #25 from UCL/dg-type-def

cb06b0e

fix: Adding defs of idx_t and IDX_MAX

Merge branch 'SWIFTSIM:master' into master

9766d6d

Initialised return_edge variable (#31)

5d78376

* Initialised return_edge variable * Removed a typedef for idx_t and a define for SCOTCH_Idx; * Removed typedef from metis/parmetis - only code chunk.

Downsized IDX_MAX to INT_MAX, from SCOTCH_NUMMAX.

f559337

Defined IDX_MAX explicitly, using 32-bit value, define IDX_MAX (21474…

0463253

…83647); makes scotch64 and scotch-local balance slightly better.

Addressing comments on configure.ac and removal on unused refine (#…

f656d4d

…33) * Update partition.c * Update configure.ac * Removal of refine in Scotch

Update files - minor edits and SCOTCH mentions in comments etc. (#36)

7fed283

* Update debug.c - minor edits * Update engine.c * Update swift.c * Update swift_fof.c * Update partition.c * Update partition.c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UCL Scotch support for SWIFT#42

UCL Scotch support for SWIFT#42
themkots wants to merge 69 commits intoSWIFTSIM:masterfrom
UCL:master

themkots commented Nov 17, 2023

Uh oh!

MatthieuSchaller commented Jul 3, 2024

Uh oh!

themkots commented Jul 3, 2024

Uh oh!

MatthieuSchaller commented Jul 3, 2024

Uh oh!

themkots commented Jul 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

themkots commented Nov 17, 2023

Uh oh!

MatthieuSchaller commented Jul 3, 2024

Uh oh!

themkots commented Jul 3, 2024

Uh oh!

MatthieuSchaller commented Jul 3, 2024

Uh oh!

themkots commented Jul 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants