Dec30
Guess I’ll Just Do It
I’ve been putting off this first post: trying to get all of my materials set up, sort out what I want to do, and in general make everything perfect. Finally I had to face the inevitable I don’t know what I’m doing and the inarguable I’ll never get anything done unless I start and just do it. I’ll add pages and features as I get to them, and in the meantime I’ll try to keep it interesting.
HPCS
I’ve been working on the High Productivity Computer Systems (HPCS) project within Sun for the past two years. This is a project sponsored by DARPA to make super-computer systems more productive as well as faster and bigger. At the very least, it’s a noble effort and will increase our understanding of productivity. I’m in the Developer Products group and am working on developer tools for highly parallel programs.
I’m going to concentrate on Performance, Productivity, and tools, but with a bit of a different emphasis from most of the Sun blogs.
Performance
First, let’s define the term High Performance Computing (HPC). This used to be High Performance Technical Computing (HPTC), and I don’t know why it changed. The major features of HPC are floating point computation, arrays as a data structure, Fortran, and simply enormous problem size. Let’s discuss each point individually, keeping in mind that I can’t include everything in this initial post. There are unmentioned exceptions, overgeneralizations, and significant omissions from each discussion. I’ll try to cover those in later posts.
Floating Point Computation
Most HPC programs make heavy use of floating point computation. In fact a count of Floating Point Operations Per Second (FLOPS) is commonly used as a figure of merit for HPC systems. There are even one or two common applications for which this makes sense. Most HPC applications, though, are not so simple, and spend more time moving data around than actually doing floating point arithmetic. None the less, FLOPS is an easy number to measure, and is to this day the basis for inclusion in the top 500 list of supercomputers.
Arrays
Again, HPC programs tend to depend quite heavily on arrays rather than more complicated data structures. Even when there is some indirection involved, this is usually done with an array of indices. Scaling a program is usually a matter of changing the size of arrays. A great deal of discussion goes into just how these arrays are distributed in memory. Different programming models imply different data distribution.
Fortran
A skillful HPC programmer can write Fortran programs in any language, and usually does. Such programs have most of the computation in loops that iterate over arrays, doing some floating point operations on the elements. The calculations on an element are frequently independent or simply related to the calculations on other elements.
This makes it possible to vectorize or parallelize the loop to improve the performance. If the program is written in Fortran 90 or some later version, array operations provide a way to apply calculations to every element without writing any loops. In fact, it was a pleasant surprise to me just how close F90 programs are to the underlying mathematical notation. I haven’t written any Fortran programs since the 1970’s, but I’d certainly use it now if I had any computationally intensive problems to solve.
Size
HPC programs tend to be big, and they tend to have very large amounts of data. There is no such thing as “big enough” for a real HPC program. If you tell a web applications programmer that his machine just got ten times faster, he will say something like “Now I only need a tenth the number of machines to do my job.” if you tell an HPC programmer the same thing, he’ll say something more like “Now I can decrease the mesh granularity and get better answers.” In other words, system size and speed has moved from being a purely economic problem to being a technical problem. It’s a very different attitude.
Productivity
Productivity is a big and complex subject, and I’ll have lots to say about it over the future weeks. For now, I’d like to leave you with a couple of teasers. These are two keynote talks from a recent conference on programming models for HPC. They are by people who have been in this field a long time, and understand the problems very well indeed. I just have slide sets here, so they are short. Take time to look at them
[…] The kind of computing he’s talking about is called High Performance Technical Computing (HPC), and although the hardware is getting steadily cheaper, is still very hard to write the programs. When I decided to join MicroSoft in 2006, it was because they told me that we could set up a group to make writing HPC programs much easier. Then MS decided that they weren’t going to do it, because there wasn’t a big enough market. I still think they blew it. Finally, I left. […]