[Beowulf] Parallel programming skills (was Checkpointing using flash)
prentice.bisbal at rutgers.edu
Tue Sep 25 08:10:13 PDT 2012
On 09/25/2012 08:19 AM, Ellis H. Wilson III wrote:
> On a related note (I assume a majority of your users are scientists),
> regarding your or somebody else's post a bit back about how poor
> scientists are at coding -- I've witnessed the exact opposite. Now,
> this is going on limited experience and all, but when I interned at
> Argonne National Labs by Chicago I saw some absolutely amazing code
> written by people without a computer science background that ran on what
> was then one of the top supers in the country (Intrepid). The point is,
> they need to get their work done, and they know just how painful and
> long poor code will be and take. Moreover, their careers rest on the
> premise that their calculations and resultant code are correct, and they
> have deadlines like the rest of us that they have to meet, which means
> therefore their code has to complete by. My golden rule of HPC is
> therefore quite the opposite: "Never underestimate the cleverness of
> your users." Their code might do "weird" things, but it's simply
> because your framework wasn't adaptive enough. I have supreme respect
> for most of the "users" I've dealt with, but as I said before, this is
> admittedly going on limited experience and I could be an exceptional case.
I've been supporting scientific computing here in the US for over 14
years. I've worked at a DoE National Lab, A pharmaceutical company, a
large oil company, and a couple of very well-known academic
institutions, so I've supported a wide range of scientists.
From my experience, many scientists don't have very good programming
skills, or computer skills in general. They learn just enough to
complete their thesis, graduate and then work in their field, using the
same skills they learned while getting their degree for the rest of
their lives. Notice I said most, not all.
I've also met some brilliant scientists who knew every aspect of
computer science (hardware, software, operating system design, and
algorithms) better than ANY computer scientist I ever met. I've found a
good part of this variability comes from the field, and the environment.
For example, from my experience, computational physicists seem to be the
most computer-savvy, followed by computational chemists, and then the
life sciences. I've also found that the scientists at national labs seem
to be the most computer-savvy, too, with people in academia actually
being the least computer savvy researchers.
Please note I'm speaking in generalizations based on my own experience
and observations, there are plenty of exceptions to these observations.
I think these trends have significant historical components -
computational physicists were the first to use computers, so have the
longest history, and have therefore placed more value on computing and
programming, going all they back to ENIAC and the Von Neumann computer
at IAS. Also, the DoE National Labs we the first institutions to use
computing for research so, likewise, they also have cultivated a high
value on computer skills.
In industry, well, if you can't produce and be profitable, they'll
replace you with someone who can.
In academia, they are judged on their science knowledge (physics,
chemistry, etc), not their computer skills, regardless of how relevant
those skills are to their field of research. so there's not much
incentive to improve your computer skills. Although you will find some
scientists who realize the value of knowing how to use their tools for
maximum effectiveness even in this environment and become CS experts.
My two cents.
More information about the Beowulf