[HPGMG Forum] HPGMG-FV v0.2 and v0.3

Sam Williams swwilliams at lbl.gov
Mon Aug 17 16:06:31 UTC 2015

As some of you may have noticed, there were two HPGMG-FV releases in quick succession.  The first, v0.2, represents a final version of the *2nd order* finite-volume operator, while the second v0.3, represents the move to a *4th order* finite-volume scheme in order to more fully utilize today's computing platforms.  It also includes some measurement of dynamic range (i.e. the ability to deliver constant performance over a range of per-node problem sizes).

Structurally, the 4th order FV version is very similar to the 2nd order version as both are based on Full Multigrid (FMG).  However, mathematically, 4th order moves from a cell-centered to cell-averaged discretization, uses a 4th order discretization of the Laplacian with 4th order volume averaged Dirichlet boundary conditions, implements three out-of-place GSRB's as a smoother, and uses volume-averaged 2nd order (V-cycle) and 4th order (F-Cycle) interpolation.  In practice, the 4th order method is both more compute intensive (more flops for the same DRAM data movement) and more network intensive (more messages and larger messages). 

Those wishing to submit system benchmark results for SC15 *must* use v0.3 of the benchmark.  For guidelines, please see http://crd.lbl.gov/departments/computer-science/performance-and-algorithms-research/research/hpgmg/submission/

Note, as the move from a degree-4 Chebyshev smoother to 3 GSRB's increases smoother time by roughly 50%, 4th order performance (DOF/s) will be load than 2nd order.  Nevertheless, the improvements in residual, error, and processor utilization more than justify this concession.  

Those not wishing to move to 4th order at this time and would rather to continue their research efforts using the older 2nd order should simply pull v0.2 of the repo.  This will provide a solid baseline for performance comparisons.  Alternately, one may pull the latest version and modify their build system to run the 2nd order operator:  
 - modify ./finite-volume/source/local.mk to use operators.7pt.c rather than operators.fv4.c
 - change the default smoother to Chebyshev (./configure --fv-smoother=cheby)

More information about the HPGMG-Forum mailing list