|
136 | 136 | <div class="contents"> |
137 | 137 | <div class="textblock"><p><a class="anchor" id="autotoc_md29"></a></p> |
138 | 138 | <h1><a class="anchor" id="autotoc_md30"></a> |
| 139 | +Strong- & Weak-scaling</h1> |
| 140 | +<p>The <a href="case.py"><b>Scaling</b></a> case can exercise both weak- and strong-scaling. It adjusts itself depending on the number of requested ranks.</p> |
| 141 | +<p>This directory also contains a collection of scripts used to test strong-scaling on OLCF Frontier. They required modifying MFC to collect some metrics but are meant to serve as a reference to users wishing to run similar experiments.</p> |
| 142 | +<h2><a class="anchor" id="autotoc_md31"></a> |
| 143 | +Weak Scaling</h2> |
| 144 | +<p>Pass <code>--scaling weak</code>. The <code>--memory</code> option controls (approximately) how much memory each rank should use, in Gigabytes. The number of cells in each dimension is then adjusted according to the number of requested ranks and an approximation for the relation between cell count and memory usage. The problem size increases linearly with the number of ranks.</p> |
| 145 | +<h2><a class="anchor" id="autotoc_md32"></a> |
| 146 | +Strong Scaling</h2> |
| 147 | +<p>Pass <code>--scaling strong</code>. The <code>--memory</code> option controls (approximately) how much memory should be used in total during simulation, across all ranks, in Gigabytes. The problem size remains constant as the number of ranks increases.</p> |
| 148 | +<h2><a class="anchor" id="autotoc_md33"></a> |
| 149 | +Example</h2> |
| 150 | +<p>For example, to run a weak-scaling test that uses ~4GB of GPU memory per rank on 8 2-rank nodes with case optimization, one could:</p> |
| 151 | +<div class="fragment"><div class="line">./mfc.sh run examples/scaling/case.py -t pre_process simulation \</div> |
| 152 | +<div class="line"> -e batch -p mypartition -N 8 -n 2 -w "01:00:00" -# "MFC Weak Scaling" \</div> |
| 153 | +<div class="line"> --case-optimization -j 32 -- --scaling weak --memory 4</div> |
| 154 | +</div><!-- fragment --><h1><a class="anchor" id="autotoc_md34"></a> |
| 155 | +Lid-Driven Cavity Problem (2D)</h1> |
| 156 | +<p>Reference: Bezgin, D. A., & Buhendwa A. B., & Adams N. A. (2022). JAX-FLUIDS: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows. arXiv:2203.13760</p> |
| 157 | +<p>Reference: Ghia, U., & Ghia, K. N., & Shin, C. T. (1982). High-re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method. Journal of Computational Physics, 48, 387-411</p> |
| 158 | +<p>Video: <a href="https://youtube.com/shorts/JEP28scZrBM?feature=share">https://youtube.com/shorts/JEP28scZrBM?feature=share</a></p> |
| 159 | +<h2><a class="anchor" id="autotoc_md35"></a> |
| 160 | +Final Condition</h2> |
| 161 | +<div class="image"> |
| 162 | +<img src="final_condition-2D_lid_driven_cavity-example.png" alt=""/> |
| 163 | +<div class="caption"> |
| 164 | +Final Condition</div></div> |
| 165 | + <h2><a class="anchor" id="autotoc_md36"></a> |
| 166 | +Centerline Velocities</h2> |
| 167 | +<div class="image"> |
| 168 | +<img src="centerline_velocities-2D_lid_driven_cavity-example.png" alt=""/> |
| 169 | +<div class="caption"> |
| 170 | +Centerline Velocities</div></div> |
| 171 | + <h1><a class="anchor" id="autotoc_md37"></a> |
| 172 | +Lax shock tube problem (1D)</h1> |
| 173 | +<p>Reference: P. D. Lax, Weak solutions of nonlinear hyperbolic equations and their numerical computation, Communications on pure and applied mathematics 7 (1) (1954) 159–193.</p> |
| 174 | +<h2><a class="anchor" id="autotoc_md38"></a> |
| 175 | +Initial Condition</h2> |
| 176 | +<div class="image"> |
| 177 | +<img src="initial-1D_laxshocktube-example.png" alt=""/> |
| 178 | +<div class="caption"> |
| 179 | +Initial Condition</div></div> |
| 180 | + <h2><a class="anchor" id="autotoc_md39"></a> |
| 181 | +Result</h2> |
| 182 | +<div class="image"> |
| 183 | +<img src="result-1D_laxshocktube-example.png" alt=""/> |
| 184 | +<div class="caption"> |
| 185 | +Result</div></div> |
| 186 | + <h1><a class="anchor" id="autotoc_md40"></a> |
139 | 187 | Shu-Osher problem (1D)</h1> |
140 | 188 | <p>Reference: C. W. Shu, S. Osher, Efficient implementation of essentially non-oscillatory shock-capturing schemes, Journal of Computational Physics 77 (2) (1988) 439–471. doi:10.1016/0021-9991(88)90177-5.</p> |
141 | | -<h2><a class="anchor" id="autotoc_md31"></a> |
| 189 | +<h2><a class="anchor" id="autotoc_md41"></a> |
142 | 190 | Initial Condition</h2> |
143 | 191 | <div class="image"> |
144 | 192 | <img src="initial-1D_shuosher_old-example.png" alt=""/> |
145 | 193 | <div class="caption"> |
146 | 194 | Initial Condition</div></div> |
147 | | - <h2><a class="anchor" id="autotoc_md32"></a> |
| 195 | + <h2><a class="anchor" id="autotoc_md42"></a> |
148 | 196 | Result</h2> |
149 | 197 | <div class="image"> |
150 | 198 | <img src="result-1D_shuosher_old-example.png" alt=""/> |
151 | 199 | <div class="caption"> |
152 | 200 | Result</div></div> |
153 | | - <h1><a class="anchor" id="autotoc_md33"></a> |
154 | | -2D Riemann Test (2D)</h1> |
155 | | -<p>Reference: Chamarthi, A., & Hoffmann, N., & Nishikawa, H., & Frankel S. (2023). Implicit gradients based conservative numerical scheme for compressible flows. arXiv:2110.05461</p> |
156 | | -<h2><a class="anchor" id="autotoc_md34"></a> |
157 | | -Density Initial Condition</h2> |
158 | | -<div class="image"> |
159 | | -<img src="alpha_rho1_initial-2D_riemann_test-example.png" alt=""/> |
160 | | -<div class="caption"> |
161 | | -Density</div></div> |
162 | | - <h2><a class="anchor" id="autotoc_md35"></a> |
163 | | -Density Final Condition</h2> |
| 201 | + <h1><a class="anchor" id="autotoc_md43"></a> |
| 202 | +Shock Droplet (2D)</h1> |
| 203 | +<p>Reference: Panchal et. al., A Seven-Equation Diffused Interface Method for Resolved Multiphase Flows, JCP, 475 (2023)</p> |
| 204 | +<h2><a class="anchor" id="autotoc_md44"></a> |
| 205 | +Initial Condition</h2> |
164 | 206 | <div class="image"> |
165 | | -<img src="alpha_rho1_final-2D_riemann_test-example.png" alt=""/> |
| 207 | +<img src="initial-2D_shockdroplet-example.png" alt=""/> |
166 | 208 | <div class="caption"> |
167 | | -Density Norms</div></div> |
168 | | - <h1><a class="anchor" id="autotoc_md36"></a> |
| 209 | +Initial Condition</div></div> |
| 210 | + <h2><a class="anchor" id="autotoc_md45"></a> |
| 211 | +Result</h2> |
| 212 | +<p><img src="result-2D_shockdroplet-example.png" alt="" class="inline" title="Result"/> </p> |
| 213 | +<h1><a class="anchor" id="autotoc_md46"></a> |
169 | 214 | Titarev-Toro problem (1D)</h1> |
170 | 215 | <p>Reference: V. A. Titarev, E. F. Toro, Finite-volume WENO schemes for three-dimensional conservation laws, Journal of Computational Physics 201 (1) (2004) 238–260.</p> |
171 | | -<h2><a class="anchor" id="autotoc_md37"></a> |
| 216 | +<h2><a class="anchor" id="autotoc_md47"></a> |
172 | 217 | Initial Condition</h2> |
173 | 218 | <div class="image"> |
174 | 219 | <img src="initial-1D_titarevtorro-example.png" alt=""/> |
175 | 220 | <div class="caption"> |
176 | 221 | Initial Condition</div></div> |
177 | | - <h2><a class="anchor" id="autotoc_md38"></a> |
| 222 | + <h2><a class="anchor" id="autotoc_md48"></a> |
178 | 223 | Result</h2> |
179 | 224 | <div class="image"> |
180 | 225 | <img src="result-1D_titarevtorro-example.png" alt=""/> |
181 | 226 | <div class="caption"> |
182 | 227 | Result</div></div> |
183 | | - <h1><a class="anchor" id="autotoc_md39"></a> |
184 | | -Rayleigh-Taylor Instability (2D)</h1> |
185 | | -<h2><a class="anchor" id="autotoc_md40"></a> |
| 228 | + <h1><a class="anchor" id="autotoc_md49"></a> |
| 229 | +Rayleigh-Taylor Instability (3D)</h1> |
| 230 | +<h2><a class="anchor" id="autotoc_md50"></a> |
186 | 231 | Final Condition</h2> |
187 | 232 | <div class="image"> |
188 | | -<img src="final_condition-2D_rayleigh_taylor-example.png" alt=""/> |
| 233 | +<img src="final_condition-3D_rayleigh_taylor-example.png" alt=""/> |
189 | 234 | <div class="caption"> |
190 | 235 | Final Condition</div></div> |
191 | | - <h2><a class="anchor" id="autotoc_md41"></a> |
| 236 | + <h2><a class="anchor" id="autotoc_md51"></a> |
192 | 237 | Centerline Velocities</h2> |
193 | 238 | <p><img src="linear_theory.jpg" alt="Linear Theory Comparison" class="inline"/></p> |
194 | | -<h1><a class="anchor" id="autotoc_md42"></a> |
195 | | -Isentropic vortex problem (2D)</h1> |
196 | | -<p>Reference: Coralic, V., & Colonius, T. (2014). Finite-volume Weno scheme for viscous compressible multicomponent flows. Journal of Computational Physics, 274, 95–121. <a href="https://doi.org/10.1016/j.jcp.2014.06.003">https://doi.org/10.1016/j.jcp.2014.06.003</a></p> |
197 | | -<h2><a class="anchor" id="autotoc_md43"></a> |
198 | | -Density</h2> |
199 | | -<div class="image"> |
200 | | -<img src="alpha_rho1-2D_isentropicvortex-example.png" alt=""/> |
201 | | -<div class="caption"> |
202 | | -Density</div></div> |
203 | | - <h2><a class="anchor" id="autotoc_md44"></a> |
204 | | -Density Norms</h2> |
205 | | -<div class="image"> |
206 | | -<img src="density_norms-2D_isentropicvortex-example.png" alt=""/> |
207 | | -<div class="caption"> |
208 | | -Density Norms</div></div> |
209 | | - <h1><a class="anchor" id="autotoc_md45"></a> |
210 | | -2D Hardcodied IC Example</h1> |
211 | | -<h2><a class="anchor" id="autotoc_md46"></a> |
212 | | -Initial Condition</h2> |
213 | | -<div class="image"> |
214 | | -<img src="initial-2D_hardcodied_ic-example.png" alt=""/> |
215 | | -<div class="caption"> |
216 | | -Initial Condition</div></div> |
217 | | - <h2><a class="anchor" id="autotoc_md47"></a> |
218 | | -Result</h2> |
219 | | -<p><img src="result-2D_hardcodied_ic-example.png" alt="" class="inline" title="Result"/> </p> |
220 | | -<h1><a class="anchor" id="autotoc_md48"></a> |
221 | | -Rayleigh-Taylor Instability (3D)</h1> |
222 | | -<h2><a class="anchor" id="autotoc_md49"></a> |
| 239 | +<h1><a class="anchor" id="autotoc_md52"></a> |
| 240 | +Rayleigh-Taylor Instability (2D)</h1> |
| 241 | +<h2><a class="anchor" id="autotoc_md53"></a> |
223 | 242 | Final Condition</h2> |
224 | 243 | <div class="image"> |
225 | | -<img src="final_condition-3D_rayleigh_taylor-example.png" alt=""/> |
| 244 | +<img src="final_condition-2D_rayleigh_taylor-example.png" alt=""/> |
226 | 245 | <div class="caption"> |
227 | 246 | Final Condition</div></div> |
228 | | - <h2><a class="anchor" id="autotoc_md50"></a> |
| 247 | + <h2><a class="anchor" id="autotoc_md54"></a> |
229 | 248 | Centerline Velocities</h2> |
230 | 249 | <p><img src="linear_theory.jpg" alt="Linear Theory Comparison" class="inline"/></p> |
231 | | -<h1><a class="anchor" id="autotoc_md51"></a> |
232 | | -Strong- & Weak-scaling</h1> |
233 | | -<p>The <a href="case.py"><b>Scaling</b></a> case can exercise both weak- and strong-scaling. It adjusts itself depending on the number of requested ranks.</p> |
234 | | -<p>This directory also contains a collection of scripts used to test strong-scaling on OLCF Frontier. They required modifying MFC to collect some metrics but are meant to serve as a reference to users wishing to run similar experiments.</p> |
235 | | -<h2><a class="anchor" id="autotoc_md52"></a> |
236 | | -Weak Scaling</h2> |
237 | | -<p>Pass <code>--scaling weak</code>. The <code>--memory</code> option controls (approximately) how much memory each rank should use, in Gigabytes. The number of cells in each dimension is then adjusted according to the number of requested ranks and an approximation for the relation between cell count and memory usage. The problem size increases linearly with the number of ranks.</p> |
238 | | -<h2><a class="anchor" id="autotoc_md53"></a> |
239 | | -Strong Scaling</h2> |
240 | | -<p>Pass <code>--scaling strong</code>. The <code>--memory</code> option controls (approximately) how much memory should be used in total during simulation, across all ranks, in Gigabytes. The problem size remains constant as the number of ranks increases.</p> |
241 | | -<h2><a class="anchor" id="autotoc_md54"></a> |
242 | | -Example</h2> |
243 | | -<p>For example, to run a weak-scaling test that uses ~4GB of GPU memory per rank on 8 2-rank nodes with case optimization, one could:</p> |
244 | | -<div class="fragment"><div class="line">./mfc.sh run examples/scaling/case.py -t pre_process simulation \</div> |
245 | | -<div class="line"> -e batch -p mypartition -N 8 -n 2 -w "01:00:00" -# "MFC Weak Scaling" \</div> |
246 | | -<div class="line"> --case-optimization -j 32 -- --scaling weak --memory 4</div> |
247 | | -</div><!-- fragment --><h1><a class="anchor" id="autotoc_md55"></a> |
248 | | -Lid-Driven Cavity Problem (2D)</h1> |
249 | | -<p>Reference: Bezgin, D. A., & Buhendwa A. B., & Adams N. A. (2022). JAX-FLUIDS: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows. arXiv:2203.13760</p> |
250 | | -<p>Reference: Ghia, U., & Ghia, K. N., & Shin, C. T. (1982). High-re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method. Journal of Computational Physics, 48, 387-411</p> |
251 | | -<p>Video: <a href="https://youtube.com/shorts/JEP28scZrBM?feature=share">https://youtube.com/shorts/JEP28scZrBM?feature=share</a></p> |
| 250 | +<h1><a class="anchor" id="autotoc_md55"></a> |
| 251 | +2D Riemann Test (2D)</h1> |
| 252 | +<p>Reference: Chamarthi, A., & Hoffmann, N., & Nishikawa, H., & Frankel S. (2023). Implicit gradients based conservative numerical scheme for compressible flows. arXiv:2110.05461</p> |
252 | 253 | <h2><a class="anchor" id="autotoc_md56"></a> |
253 | | -Final Condition</h2> |
| 254 | +Density Initial Condition</h2> |
254 | 255 | <div class="image"> |
255 | | -<img src="final_condition-2D_lid_driven_cavity-example.png" alt=""/> |
| 256 | +<img src="alpha_rho1_initial-2D_riemann_test-example.png" alt=""/> |
256 | 257 | <div class="caption"> |
257 | | -Final Condition</div></div> |
| 258 | +Density</div></div> |
258 | 259 | <h2><a class="anchor" id="autotoc_md57"></a> |
259 | | -Centerline Velocities</h2> |
| 260 | +Density Final Condition</h2> |
260 | 261 | <div class="image"> |
261 | | -<img src="centerline_velocities-2D_lid_driven_cavity-example.png" alt=""/> |
| 262 | +<img src="alpha_rho1_final-2D_riemann_test-example.png" alt=""/> |
262 | 263 | <div class="caption"> |
263 | | -Centerline Velocities</div></div> |
| 264 | +Density Norms</div></div> |
264 | 265 | <h1><a class="anchor" id="autotoc_md58"></a> |
265 | | -Lax shock tube problem (1D)</h1> |
266 | | -<p>Reference: P. D. Lax, Weak solutions of nonlinear hyperbolic equations and their numerical computation, Communications on pure and applied mathematics 7 (1) (1954) 159–193.</p> |
| 266 | +2D Hardcodied IC Example</h1> |
267 | 267 | <h2><a class="anchor" id="autotoc_md59"></a> |
268 | 268 | Initial Condition</h2> |
269 | 269 | <div class="image"> |
270 | | -<img src="initial-1D_laxshocktube-example.png" alt=""/> |
| 270 | +<img src="initial-2D_hardcodied_ic-example.png" alt=""/> |
271 | 271 | <div class="caption"> |
272 | 272 | Initial Condition</div></div> |
273 | 273 | <h2><a class="anchor" id="autotoc_md60"></a> |
274 | 274 | Result</h2> |
275 | | -<div class="image"> |
276 | | -<img src="result-1D_laxshocktube-example.png" alt=""/> |
277 | | -<div class="caption"> |
278 | | -Result</div></div> |
279 | | - <h1><a class="anchor" id="autotoc_md61"></a> |
280 | | -Shock Droplet (2D)</h1> |
281 | | -<p>Reference: Panchal et. al., A Seven-Equation Diffused Interface Method for Resolved Multiphase Flows, JCP, 475 (2023)</p> |
| 275 | +<p><img src="result-2D_hardcodied_ic-example.png" alt="" class="inline" title="Result"/> </p> |
| 276 | +<h1><a class="anchor" id="autotoc_md61"></a> |
| 277 | +Isentropic vortex problem (2D)</h1> |
| 278 | +<p>Reference: Coralic, V., & Colonius, T. (2014). Finite-volume Weno scheme for viscous compressible multicomponent flows. Journal of Computational Physics, 274, 95–121. <a href="https://doi.org/10.1016/j.jcp.2014.06.003">https://doi.org/10.1016/j.jcp.2014.06.003</a></p> |
282 | 279 | <h2><a class="anchor" id="autotoc_md62"></a> |
283 | | -Initial Condition</h2> |
| 280 | +Density</h2> |
284 | 281 | <div class="image"> |
285 | | -<img src="initial-2D_shockdroplet-example.png" alt=""/> |
| 282 | +<img src="alpha_rho1-2D_isentropicvortex-example.png" alt=""/> |
286 | 283 | <div class="caption"> |
287 | | -Initial Condition</div></div> |
| 284 | +Density</div></div> |
288 | 285 | <h2><a class="anchor" id="autotoc_md63"></a> |
289 | | -Result</h2> |
| 286 | +Density Norms</h2> |
290 | 287 | <div class="image"> |
291 | | -<img src="result-2D_shockdroplet-example.png" alt=""/> |
| 288 | +<img src="density_norms-2D_isentropicvortex-example.png" alt=""/> |
292 | 289 | <div class="caption"> |
293 | | -Result</div></div> |
| 290 | +Density Norms</div></div> |
294 | 291 | </div></div><!-- contents --> |
295 | 292 | </div><!-- PageDoc --> |
296 | 293 | </div><!-- doc-content --> |
|
0 commit comments