[HPGMG Forum] what is a "good" or a "better" TOP500 ranking?

Horst Simon hdsimon at lbl.gov
Sun Nov 30 18:34:37 UTC 2014

Like many of you I returned from SC14 with many questions to think 
about, among others "what is a 'better' benchmark?"  Here is a summary 
of at least three conversations that I had at SC14 between "M" (that is 
me), and "C" a colleague (a synthesis of several conversations.

C: Well, I am really glad that HPCG (or HPGMG) is being developed that 
will make the TOP500 more realistic.
M: What do you mean by more realistic?
C: It is well known that HPL is not a good benchmark to measure the 
performance of real systems. If we replace HPL with HPCG (or HPGMG) then 
we would not get such a distortion of the performance, for example all 
those GPU based system would not be ranked as high.
M: But why do you think that HPCG (or HPGMG) is better?
C: ... long technical argument involving bisection bandwidth, mixture of 
long and short messages, real applications that don't solve dense linear 
systems, streams benchmark etc.
M: But why do think this is "better"? What do mean by "better"? I think 
that "better" should imply that any new benchmark would in some sense be 
a better approximation to the application workload. Can you prove this?
..... etc.

What was really striking about these type of conversations was the fact 
how little our community is thinking scientifically. If you want to do 
something better, then  you first have to define what you are actually 
measuring. So how do we really measure the applications performance of a 
petascale platform? I can think of many applications where HPCG (and 
HPGM) as irrelevant to the application as is HPL.


More information about the HPGMG-Forum mailing list