June 2015 Exascalar
Winston A. Saunders
29 Sept 2015
Synopsis
While the #1 performing supercomputer in the Top500 has not changed in the last five publications of the Top500 list (since Jun 2013), the population of the top supercomputers has been far from stagnant. The computer at the top of the efficiency mark, the #1 Green500 computer has changed with every publication.
Exascalar explores the full richness of the Top500 and Green500 supercomputers using intuitive “data visuaization” techiques. By seeing how and where changes occur, we can gain deeper insight into trends which are helpful in understanding what lies ahead.
This analysis contains the usual Exasclat plot, showing change between the June 2015 lists and the Novmber2014 lists. It also contains compares the Top Exascalar systems and looks at the role of core counts and GPU’s.
Background and notes
Supercomputer leadership requires both extreme scale and high efficiency. The Green500 and Top500 lists are excellent resources for understanding this. But they are both, well, lists, which makes it hard to recognize trends and correlations. The approach here is to leverage data visualization techniques to explore some of the richness of the combined high quality data contained in both lists. While the Top500 emphasizes highest performance, and the Gree500 emphasizes the highest efficiency, Exascalar analysis is conceived as a way to visualize supercomputer efficiency and performance in one coherent picture.
The name Exascalar originates from the goal of achieving an exaflops, \(10^{12}\) mflops, in an envelope of \(20\) MWatt. Exascalar is a logarithmic parametric indicator of progress in efficiency and performance along an iso-power line toward this goal. You can actually Google it.
While source code is made freely available on Github, I welcome suggestions for additional anlayses (or even half-baked experiments) which might be of interest. You can reach me via twitter @WinstonOnEnergy
This analysis is done independently and reflects my opinions alone. In particular, it in no way reflects opinions of my employer (Intel Corporation) or any other entity with which I’m affiliated.
June 2015 Exascalar
## Loading required package: bitopsThis section focuses on the June 2015 lists and the Exascalar ranking of the data.
Exascalar Plot
The easiest way to visualize change in the Top500 and Green500 lists is to overlay the Exascalar plots of November 2014 with that of June 2015.
In the plot below points from June 2015 are smaller red dots, points with empty blue circles are computers that are no longer on the list, and red points with blue circles around them are computers on both lists. Changes are clearly visible. While the highestperformance computer did not change, changes within the population as well as at the extremes of efficiency are evident.
Reaching back: Comparing June 2015 to June 2013
To emphasize the massive changes in the population of the top supercomputers, it’s instructive to compare not only to the last iteration, but also the last iteration where the #1 Supercomputer changed, two years ago in June 2013. In intervening years the minimum performance needed to make the Top500 has increased over 50%. In addition, the maximum efficiency has increased by about the same magnitude. This neatly emphasizes why the population holds as much (and perhaps more) interest as the top one or two systems.
## Warning: Removed 2 rows containing missing values (geom_point).
Year of Appearance
Finally, the year systems were added to the list shows that the evolution of both the Top500 and Green500 is far less uniform within the population. In the graph below the age of systems is plotted as \(age = 2016 - year.t\). The newest systems, rather than being clutered at highest efficiency and performance, are distributed more or less uniformly throughout the population. And while the cluster of high power top performance systems are getting a little long in the tooth, there are serveral newer systems with powers over 2.0 MWatt with lower than peak efficiency.
By comparison, here is the plot for the 2013 list.
As a general observation, the 2015 population, with a median age of 3 is older than the 2013 population, with a median age of 2, with a difference in means of 0.702 years.
Key Indicators June 2015 Lists
Exascalar, as visulaized by above, is descriptive of the population of supercomputers. Here ia a list of some of the key systems. Note that Exascalar spans from the highest performance to the lowest efficiency system.
| Exa Rank | Exascalar | Performance Rank | Efficiency Rank | rmax(mflops) | power(kW) | efficiency (mflops/Watt) | |
|---|---|---|---|---|---|---|---|
| Top Exascalar | 1 | -2.04 | 1 | 83 | 33862700.00 | 17808.00 | 1901.54 |
| Bottom Exascalar | 500 | -5.04 | 475 | 500 | 168600.00 | 7625.82 | 22.11 |
| Top Performance | 1 | -2.04 | 1 | 83 | 33862700.00 | 17808.00 | 1901.54 |
| Top Efficiency | 27 | -3.00 | 160 | 1 | 353820.00 | 50.32 | 7031.58 |
| Low Efficiency | 500 | -5.04 | 475 | 500 | 168600.00 | 7625.82 | 22.11 |
Key New Entrants June 2015
Of the new entrants its interesting to note in this particular year new systems occupy both the highest and lowest efficiency.
The median Exascalar of the New Computers is -3.52 compared to the median of all computers on the June 2015 list -3.84 and the November 2014 list -3.99.
| Exa Rank | Exascalar | Performance Rank | Efficiency Rank | rmax (mflops) | power (kW) | efficiency (mflops/Watt) | |
|---|---|---|---|---|---|---|---|
| Top Exascalar | 7 | -2.59 | 7 | 75 | 5536990.00 | 2834.00 | 1953.77 |
| Top Performance | 7 | -2.59 | 7 | 75 | 5536990.00 | 2834.00 | 1953.77 |
| Top Efficiency | 27 | -3.00 | 160 | 1 | 353820.00 | 50.32 | 7031.58 |
Trends
Exascalar is useful for visualizing trends in the population. The graphs below illustrate how top efficiency and performance have evolved over the last several years.
Efficiency of Top-Efficiency and Top-Performance Systems
THe trend of the Efficiency of the #1 Top500 and the T #1 Green500 lists is interesting in that is shows “fits and starts” of the Top500 systme as it jumps to new tiers of efficiency versus the stead evolution of the Green500 systems. It’s interesting to note that the gap in efficiency between the two systems appears to be at an alltime high, suggesting another jump may soon be in our future.
Exascalar of Top-Efficiency and Top-Performance Systems
We can also look at progress of the top systems along the Exascalar parameter. Note here that jumps in the Top500 correlate to when the Green500 system is closest to the Top500 curve. It’s not surprising (indeed it is expected), but the visualization helps highlight the behavior.
Technology Trends
In this section we visually explore different influencers of the supercomputer populations. Some of these graphs were “requests” and others are just interesting.
Cores Per Socket
This graph, showing the the side by side comparison of the June 2013 and current June 2015 plots shows the evolution in cores per socket as a shift to redder data points in the latter graph. This encodes the coressocket data from the Top500 list.
The side-by-side comparison here also highlights how the IBM Power systems, so dominant in both efficiency and performance, in the Novermber 2012 list, have been matched and now exceeded by alternative architectures. In particular, the role of GPU and GPU architecture will become apparent in the analysis below.
Core Count Histogram
The steady shift in core count is shown clearly in the histrogram below.
Table of Core Count mean
As a summary statistic we can also look to the median and mean core counts of each list. While the median core count has not changed in the last two years, the mean core count has increased from roughly 8 to 10.
| Date | Median Cores | Mean Cores | |
|---|---|---|---|
| 1 | Jun2013 | 8 | 7.958 |
| 2 | Jun2014 | 8 | 8.78 |
| 3 | Jun2015 | 8 | 9.848 |
Processor
The processor plays a big role in the supercomputer. Here, to simplify, I have taken the processorgeneration data and simplified it to capture only the manufacturer. The near ubiquity of Intel processors is evident.
Accelerator Co-Processor
Encoding the aceeleratorcoprocessor data from the Green500 list highlights interesting differentiation among different manufacturers very clearly. For the sake of clarity list elements have been reduced to just the manufacturer.
Total Number of Cores
Another interesting trend is the total number of cores. Since this is an extrinsic parameter, it’s best to encode the logarithm. Again, the shift up in performance, and the shift toward higher overall core count is apparent. The totalcores data from the Top500 list are used.
Accelerator Processor Cores
Here we can see where Accelerator Processors (GPUs) are making their way into the Top500 population. While high core count systems play an obvious role in high perfoormance, lower core counts have an interesting play at the high efficiency end (even if they have not yet achieved scale). Here the acceleratorcoprocessor data from the Top500 are encoded.
Summary and Conclusions
While the #1 Supercomputer has remained unchanged for two years, the technology of supercomputing is undergoing rapid evolution. Based on historical trends of the gap between the efficiency of the top peformance and the top efficiency systems, as well as the decreasing difference in Exascalar between the systems, another push to higher performance seems imminent.
Evidence for a slowdown in teh refresh of supercomputers is seen, with an increase in the average age of about 0.7 years between 2015 and 2013.
An increase in the cores per socket is evident in the overall data, while GPU architecture and total cores play an strong role in pushing to higher efficiency and greater performance.
Final Query
In looking at the Top500 and Green500 data thru the lens of Exascalar, systems with exascalar \(\epsilon\) < -4.0 are technologically of little interest. Indeed, as time evolves, they are really just roadkill. Would it be more interesting and meaningful to include more advanced systems, perhaps with lower overall performance, but higher efficiency, rather than low efficiency, low performing systems, in an overall list of leading supercomputers?