[HPGMG Forum] latest HPGMG-FV release transposes the data structure to facilitate OpenMP4/OpenACC/CUDA implementations

Samuel Williams swwilliams at lbl.gov
Thu Jan 22 06:03:33 UTC 2015

I didn't notice any substantive difference in performance (plus or minus) at moderate scale on Edison (Ivy bridge Xeon), hopper (mangy cours opteron), or Babbage (Knights corner).  I did include a few macros (detailed in the README) which can be tuned to avoid conflict misses/facilitate simd/avoid cache line splits.  Additionally, there was an alternate allocation approach in level.c (currently disabled) in which rather than performing one large malloc for all boxes and vectors, vectors were individually malloc'd.  The bulk malloc was consistently faster and is thus the default.

- Sam

> On Jan 21, 2015, at 9:43 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
> Does this change the performance on systems that aren't designed to
> maximize programmer pain?  It would be a shame if this interfered with
> performance on systems where the big compute and the big memory are
> co-located.
> Jeff
>> On Wed, Jan 21, 2015 at 10:59 PM, Sam Williams <swwilliams at lbl.gov> wrote:
>> FYI, the latest HPGMG-FV release transposes the internal data layout to facilitate any potential OpenMP4/OpenACC/CUDA implementations.  This should be completely transparent and does not affect any existing operators as the code previously did not assume data was contiguous.  It does however, create an additional set of pointers (one per vector) in level_type that point to the union of FP data across all boxes on that level for a specific vector.  Hopefully, this will facilitate any future OpenMP4/OpenACC/CUDA implementations as it allows one to perform one bulk copy (single omp target update/acc copyin/cudamemcpy statement) for *all* boxes on a level instead of having one update/copyin/cudamemcpy for *each* of a variable number of boxes.
>> At a high level, the figure below highlights the change.
>> _______________________________________________
>> HPGMG-Forum mailing list
>> HPGMG-Forum at hpgmg.org
>> https://hpgmg.org/lists/listinfo/hpgmg-forum
> -- 
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/

More information about the HPGMG-Forum mailing list