[HPGMG Forum] [EXTERNAL] Re: Acceptable rounding errors

Brian Van Straalen bvstraalen at lbl.gov
Mon Aug 3 17:51:50 UTC 2015

I think we are that stage that Sam finished with: We are currently interested/concerned about round-off and imprecise floating-point and the final accuracy.

I suspect that Jed is right and that mixed precision will have a hard time achieving proper truncation order, but I’m interested to see someone try.  We just need to be prepared for such submissions.

As Sam says, odd sized problems might fit the platform better and give the best ranking output but will not have a single cell bottom solver, so you can get dot product variations.

Some finite volume implementations (like Chombo) compute face flux values just once and re-use them, but FV schemes are typically defined in terms of exact arithmetic and most codes rely on recomputing.  I suspect future code transformations (scalarization, fusion, etc.) will increasing rely on commutative and associative transformations for performance.   Someone might want to try a segmental refinement technique at large levels of concurrency in the face of more pessimistic networks.

It might be a good idea to present a few slides at the SC15 BOF.  I won’t be there though (I did not have anything submitted to SC15 and so the deadline to request to attend has passed for LBL).


> On Aug 3, 2015, at 9:58 AM, Sam Williams <swwilliams at lbl.gov> wrote:
> I think scalable implementations *should* reduce the bottom problem size to a single process.  Nevertheless, I don't think we specify that the bottom solve *must* be run on a single process.  I think we only specify the restrictions on the dimensions of the bottom solve.  This flexibility could be beneficial for odd problem sizes like (11*2^k)^3 whose bottom solve may be 11^3.  Even if running on a single process, there can still be multiple threads, cores, or processors (e.g. single process spans a multisocket node) performing the bottom solve.  How one chooses to thread such an operations is an implementation choice.
> For the FV method, you are also required to sum the flux surfaces for each cell.  Obviously this is 6 terms that must be summed and not 1000, but I don't think we should mandate the order of that summation.  There is a deeper question as to whether adjacent cells must calculate common flux surfaces with the same order of operations.
> At this point, I'm not inclined to mandate reproducibility, but am curious to see quantitative/numerical data where it made a difference in HPGMG-FV.
> On Aug 3, 2015, at 9:20 AM, Jed Brown <jed at jedbrown.org> wrote:
>> Brian Van Straalen <bvstraalen at lbl.gov> writes:
>>> Once you reach the bottom solver for full multigrid you should not be
>>> in the network at all anymore.  You would be computing a dot product
>>> over a few hundred doubles in L1 memory.
>> And the order of those doubles doesn't depend on the
>> network/fine-grid/number of processors (at least not in FE), so it's
>> reproducible in any precision and thus I don't know why we're having
>> this particular discussion about dot products that occur only on the
>> coarsest (serial) grid.
>> The summation order during restriction might not be reproducible
>> (depends on implementation details including whether you use
>> MPI_Waitsome versus MPI_Waitall), but can be made reproducible and
>> parallel-invariant if desired.
>> _______________________________________________
>> HPGMG-Forum mailing list
>> HPGMG-Forum at hpgmg.org
>> https://hpgmg.org/lists/listinfo/hpgmg-forum

Brian Van Straalen         Lawrence Berkeley Lab
BVStraalen at lbl.gov         Computational Research
(510) 486-4976             Division (crd.lbl.gov)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://hpgmg.org/lists/archives/hpgmg-forum/attachments/20150803/4730ac24/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://hpgmg.org/lists/archives/hpgmg-forum/attachments/20150803/4730ac24/attachment-0001.bin>

More information about the HPGMG-Forum mailing list