[HPGMG Forum] what is a "good" or a "better" TOP500 ranking?
hdsimon at lbl.gov
Sun Nov 30 18:34:37 UTC 2014
Like many of you I returned from SC14 with many questions to think
about, among others "what is a 'better' benchmark?" Here is a summary
of at least three conversations that I had at SC14 between "M" (that is
me), and "C" a colleague (a synthesis of several conversations.
C: Well, I am really glad that HPCG (or HPGMG) is being developed that
will make the TOP500 more realistic.
M: What do you mean by more realistic?
C: It is well known that HPL is not a good benchmark to measure the
performance of real systems. If we replace HPL with HPCG (or HPGMG) then
we would not get such a distortion of the performance, for example all
those GPU based system would not be ranked as high.
M: But why do you think that HPCG (or HPGMG) is better?
C: ... long technical argument involving bisection bandwidth, mixture of
long and short messages, real applications that don't solve dense linear
systems, streams benchmark etc.
M: But why do think this is "better"? What do mean by "better"? I think
that "better" should imply that any new benchmark would in some sense be
a better approximation to the application workload. Can you prove this?
What was really striking about these type of conversations was the fact
how little our community is thinking scientifically. If you want to do
something better, then you first have to define what you are actually
measuring. So how do we really measure the applications performance of a
petascale platform? I can think of many applications where HPCG (and
HPGM) as irrelevant to the application as is HPL.
More information about the HPGMG-Forum