[HPGMG Forum] what is a "good" or a "better" TOP500 ranking?

Mark Adams mfadams at lbl.gov
Sun Nov 30 21:20:59 UTC 2014


We have also been flabbergasted by the lack of rational design or analysis
of HPCG and HPL for that matter.  We have had many conversation to this
effect; many not appropriate for a public forum.   HPCG goes as far as
saying that they have a mix of small and large messages.  Sato from Riken
presented some very nice data (2nd attachment) in our BoF, that Sam brought
to my attention, of network occupancy rate on K for HPCG and HPGMG that
shows the limits of this long and short theory - the network is just not
being stressed once a subdomains are "saturated" with HPCG but it is still
rising at the end of this data.

But let me say that we have had a plan to collect data on some pool of
(mini) apps, design metrics, and correlate HPGMG via these metrics with the
application pool, for analysis and design.  This issues came up in the BoF;
I mentioned that this is the weakest part of the HPGMG project, not for
lack of interest but for lack of resources.  We are an all volunteer effort
(until very recently) and no one has volunteered to do this.  We have plans
in the works to address this but they are not ready for public consumption.

We have had some contributed data along these lines.  The LLNL folks, Bert
Still and Ian Karlan, collected data from a variety of (CORAL?) apps (with
high intensity apps like QBOX removed), and HPCG, HPL, STREAMS, and HPGMG
(FV & FE).  I presented this at ISC14 (attached) and it was even tweeted.
This shows the very nice correlation of STREAMS and HPCG, and better
correlation of HPGMG to this app data in arithmetic intensity like
metrics.  As I said our effort in this area has floundered but we (Jed)
have concrete plans to address it.


[image: Inline image 1]
[image: Inline image 1]

On Sun, Nov 30, 2014 at 1:34 PM, Horst Simon <hdsimon at lbl.gov> wrote:

> Like many of you I returned from SC14 with many questions to think about,
> among others "what is a 'better' benchmark?"  Here is a summary of at least
> three conversations that I had at SC14 between "M" (that is me), and "C" a
> colleague (a synthesis of several conversations.
> C: Well, I am really glad that HPCG (or HPGMG) is being developed that
> will make the TOP500 more realistic.
> M: What do you mean by more realistic?
> C: It is well known that HPL is not a good benchmark to measure the
> performance of real systems. If we replace HPL with HPCG (or HPGMG) then we
> would not get such a distortion of the performance, for example all those
> GPU based system would not be ranked as high.
> M: But why do you think that HPCG (or HPGMG) is better?
> C: ... long technical argument involving bisection bandwidth, mixture of
> long and short messages, real applications that don't solve dense linear
> systems, streams benchmark etc.
> M: But why do think this is "better"? What do mean by "better"? I think
> that "better" should imply that any new benchmark would in some sense be a
> better approximation to the application workload. Can you prove this?
> ..... etc.
> What was really striking about these type of conversations was the fact
> how little our community is thinking scientifically. If you want to do
> something better, then  you first have to define what you are actually
> measuring. So how do we really measure the applications performance of a
> petascale platform? I can think of many applications where HPCG (and HPGM)
> as irrelevant to the application as is HPL.
> Horst
> _______________________________________________
> HPGMG-Forum mailing list
> HPGMG-Forum at hpgmg.org
> https://hpgmg.org/lists/listinfo/hpgmg-forum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://hpgmg.org/lists/archives/hpgmg-forum/attachments/20141130/e47ce586/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 78918 bytes
Desc: not available
URL: <https://hpgmg.org/lists/archives/hpgmg-forum/attachments/20141130/e47ce586/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 17270 bytes
Desc: not available
URL: <https://hpgmg.org/lists/archives/hpgmg-forum/attachments/20141130/e47ce586/attachment-0003.png>

More information about the HPGMG-Forum mailing list