From hahn at mcmaster.ca Sun Apr 1 11:22:33 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> Message-ID: > Electromagnetics Research Symposium" in Verona. There appears > to be a considerable buzz now around FDTD calculations on GPUs. the very latest gen GPUs (G80 and as yet unreleased R600) make very interesting coprocessors for vector-ish calculations which can be expressed using integer or single-precision operations. > Has anyone any experience of this? How do these products stack > up against the traditional Beowulf solution? they _are_ in the spirit of Beowulf, which is all about hacking commodity hardware to suit HPC purposes. > We are planning to buy a new Beowulf in the next few months. I'm > wondering whether I should set aside some funds for GPU instead > of CPU... as with any purchase, you need to figure out what your workload needs, and how you can feed it. GPGPU requires substantial custom programming effort - there is no standardized interface (like MPI) to do it. GPGPU makes a lot of sense where you have a research project which: - has some large amount of high-level programming resources (say, a top grad student for at least 6 months). - is going to be seriously limited on normal hardware (ie, runs will take 2 years each). - has some promise of running well on GPU hardware (very SIMD, needs to fit into limited memories, integer or 32b float, etc) the speedup from a GPU is around an order of magnitude (big hand wave here). the main drawback is that effort is probably not portable to other configs, probably not to the future, and is probably in conflict with development of portable/scalable approaches (say f90/MPI). really, this issue is quite similar to the tradeoff in pursuing FPGA acceleration. in short, I think the opportunity for GPU is great if you have a pressing need which cannot be practically satisfied using the conventional approach, and you're able to dedicate an intense burst of effort at porting to the GPU. as far as I know, there are not any well-developed libraries which simply harness whatever GPU you provide, but don't require your whole program to be GPU-ized. the cost of sharing data with a GPU is significant, but blas-3 might have a high enough work-to-size ratio to make it feasible. 3d fft's might also be expressible in GPU-friendly terms (the trick would be to utilize not fight the GPU's inherent memory-access preferences.) perhaps some MCMC stuff might be SIMD-able? I doubt that sequence analysis would make much sense, since GPUs are not well-tuned to access host memory, and sequence programs are not actually that compute-intensive. I'd guess that anything involving sparse matrices would be difficult to do on a GPU. my organization will probably build a GPU-oriented cluster soon; I'm pushing for it, but I'm fearful that we might not have users who are prepared to invest the intense effort necessary to take advantage of it. I have some suspicion also that when Intel and AMD talk about greater integration between CPU and GPU, they're headed in the direction of majorly extended SSE, rather than something which still has parts called shader, vertex or texture. From hahn at mcmaster.ca Sun Apr 1 11:25:19 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <8e6393ac0703302312v1840c22fw60efa957e0365a40@mail.gmail.com> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0703302312v1840c22fw60efa957e0365a40@mail.gmail.com> Message-ID: > If you want to use GPUs for computations, I suggest that you take a look at > CUDA > (http://www.nvidia.com/cuda). The SDK is available for free and it is > using a C like syntax (so you don't need to write shader and be > familiar with OpenGL or DX9 ). there's ATI/AMD's CTM effort as well, as well as several independent ones. www.gpgpu.org is a great resource to start with. From hahn at mcmaster.ca Sun Apr 1 13:07:07 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> Message-ID: > CUDA comes with a full BLAS and FFT library (for 1D,2D and 3D transforms). I read the CUDA doc, but I guess I was focusing on the language itself. > You can have relevant speed up even for 2D transforms or for a batch of 1Ds. I assume this is only single-precision, and I would guess that for numerical stability, you must be limited to fairly short fft's. what kind of peak flops do you see? what's the overhead of shoving data onto the GPU, and getting it back? (or am I wrong that the GPU cannot do an FFT in main (host) memory? > You can offload only compute intendive parts of your code to the GPU > from C and C++ ( writing a wrapper from Fortran should be trivial). sure, but what's the cost (in time and CPU overhead) to moving data around like this? > The current generation of the hardware supports only single precision, > but there will be a double precision version towards the end of the > year. do you mean synthetic doubles? I'm guessing that the hardware isn't going to gain the much wider multipliers necessary to support doubles at the same latency as singles... > PS: I work on CUDA at Nvidia, so I may be a little biased... I did guess from the nvidia-limited nature of your reply, but thanks for confirming it. >> as far as I know, there are not any well-developed libraries which simply by "well-developed", I did also mean "runs on any GPU or at least not a single vendor"... From laytonjb at charter.net Sun Apr 1 13:15:29 2007 From: laytonjb at charter.net (Jeffrey B. Layton) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> Message-ID: <461012E1.8020006@charter.net> Mark Hahn wrote: >> The current generation of the hardware supports only single precision, >> but there will be a double precision version towards the end of the >> year. > > do you mean synthetic doubles? I'm guessing that the hardware isn't > going to gain the much wider multipliers necessary to support doubles > at the same latency as singles... The next gen of hardware will support native double precision (AFAIK). I've heard it should be out this year, but I'm sure Mass can't comment on it. Jeff From hahn at mcmaster.ca Sun Apr 1 14:17:20 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <461012E1.8020006@charter.net> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> <461012E1.8020006@charter.net> Message-ID: > The next gen of hardware will support native double precision (AFAIK). my point is that there's native and there's native. if the HW supports doubles, but they take 8x as long, then there's still a huge reason to make sure the program uses only low-precision. and 8x (WAG, of course) may actually be enough so that a 4-core, full-rate SSE CPU to beats it... From deadline at clustermonkey.net Sun Apr 1 14:39:48 2007 From: deadline at clustermonkey.net (Douglas Eadline) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> Message-ID: <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> I just posted some interesting news on Cluster Monkey. http://www.clustermonkey.net//content/view/192/1/ -- Doug From landman at scalableinformatics.com Sun Apr 1 18:50:57 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> Message-ID: <46106181.40401@scalableinformatics.com> Mark Hahn wrote: >> Has anyone any experience of this? How do these products stack >> up against the traditional Beowulf solution? > > they _are_ in the spirit of Beowulf, which is all about hacking > commodity hardware to suit HPC purposes. ... but it would be hard to fit into a 1U case, and the 200+ watt power requirements could be daunting to smaller supplies. Not that I am against GPUs as accelerators, on the contrary. Just be aware that GPUs with significant calculation capability also will require a rather significant power supply and cooling airflow. Right now, accelerated computing is in its infancy. You have host based (SSE*), and host attached GPUs, FPGAs, APUs (Accelerator Processor Units) in general such as ClearSpeed et al. You can think of clusters as accelerators in the sense that they provide a larger number of cycles per unit time to your application. There are no single APIs to bind them all though. A number of the APU people are realizing that there is significant benefit to providing acceleration behind existing popular interfaces, as it lowers the barrier to adoption and usage. If your code is designed with FFTW in mind and you have to re-organize your arrays to suit another FFT implementation, this can be both annoying for the programmer, and inefficient. Regardless of the accelerator you choose, expect *some* rewriting of code at minimum. Current GPUs are focused upon singles and ints. As Jeff noted, doubles should be coming. As Mark noted, slow doubles aren't useful. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From rgb at phy.duke.edu Sun Apr 1 21:43:13 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Which is better; 4 Mountains or 5 Hills In-Reply-To: <428810f20703310104p42d8fc81nfd77b2a1de0601@mail.gmail.com> References: <428810f20703310104p42d8fc81nfd77b2a1de0601@mail.gmail.com> Message-ID: On Sat, 31 Mar 2007, amjad ali wrote: > Hi All, > > Which one of the following two choices are better (assuming both > clusters have nearly same cost): > > 1) A 4-node cluster having 8 AMD Opteron of 2.6GHz each and 2GB RAM/node > > 1) A 5-node cluster having 10 AMD Opteron of 2.2GHz each and 2GB RAM/node What is the meaning of this word "better"? Better for what? A) 4x2.6 = 10.4 aggregate GHz (whatever that means) Can do a distributed calculation that fits into 8 GB RAM Fewer faster nodes means less Amdahlian penalty for parallel code. Probably uses less power, has slightly lower aggregate maintenace costs B) 5x2.2 = 11 aggregate GHz Can do a distributed calculation that fits into 10 GB RAM More slower nodes means a bit more flexibility and less penalty if a node is down Probably uses more power, has higher costs. Which is "better"? B) has more aggregate GHz (and presumably FLOPS by at least some measures) so it is better. It can also hold a larger partitioned problem, which for some people makes it MUCH better, and for others just wastes money. A) might well run faster for some kinds of parallel code, and it probably costs less in the long run, once (expensive) human time and infrastructure costs are accounted for. > Consider GiGE (max 1000mbps) as the interconnect. > Is there any significant performance difference between the two? Sure, maybe, for some code. For example, try running a 9 GB partitioned computation on A and oooo, slow (as it swaps to run at all). Try running a computation with lots of little messages, but there are just plain fewer of them between four nodes than for five, and A might shine. > Does the compasison depends on the MPI communication among nodes, for > a given application? It depends on EVERYTHING. The task mix, how the task(s) are executed, whether, how, and how much they intercommunicate during execution, how big they are (or might be) and whether four boxes plugged into your only available circuit doesn't blow the breaker where five does. rgb > > Thanking all in anticipation. > Regards. > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Sun Apr 1 23:24:22 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Server room design consulting In-Reply-To: <1175104216.28106.236.camel@rahl.acomp.usf.edu> References: <1175104216.28106.236.camel@rahl.acomp.usf.edu> Message-ID: On Wed, 28 Mar 2007, Daniel Majchrzak wrote: > We have a dedicated cluster room, email server room, and networking room > that have slowly evolved over the years. Due to budget constraints in > the past no one has ever done an analysis of our electricity and AC. > (We've had the facilities people in, but their analysis wasn't any > better than our own guestimates. ) When increases in either power or > cooling were necessary it was either done piecemeal or not done at all. > (We have had some orders to come up with some "zero-dollar" solutions). > Now all three rooms are about to go through some equipment expansions. > While we can make some rough estimates, and we could go to the > university's facilities people, we thought we would try to get some > funds together to hire some professional services so that it gets done > right. Has any one on the list had any experiences they would like to > share on hiring these kind of consultants. Referrals? Can anyone give > a (very) rough estimate of what we should expect to pay? My own experience with power and AC people from normal contracting firms is that they are immensely clueless about computer server room infrastructure. I haven't ever hired a consultant, but I do consulting on this from time to time. I'd guess that you'll pay perhaps $150 to $200/hour for consulting, but it might well depend on what kind of consultant you got and where you got them. Again, based on my experience with area computer management contracting houses, they are not really competent with infrastructure issues. A really good electrical contractor MIGHT have somebody. Liebert or APC might be able to refer you. An architect with experience in server room design might be your best bet. rgb > > Thanks, > > Dan > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From ballen at gravity.phys.uwm.edu Mon Apr 2 01:47:25 2007 From: ballen at gravity.phys.uwm.edu (Bruce Allen) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: Hmmm is the April 1 date purely a coincidence?? On Sun, 1 Apr 2007, Douglas Eadline wrote: > > I just posted some interesting news on Cluster Monkey. > > http://www.clustermonkey.net//content/view/192/1/ > > > -- > Doug > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > From rbw at ahpcrc.org Mon Apr 2 06:35:14 2007 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0703302312v1840c22fw60efa957e0365a40@mail.gmail.com> Message-ID: <46110692.2000007@ahpcrc.org> Mark Hahn wrote: >> If you want to use GPUs for computations, I suggest that you take a >> look at CUDA >> (http://www.nvidia.com/cuda). The SDK is available for free and it is >> using a C like syntax (so you don't need to write shader and be >> familiar with OpenGL or DX9 ). > there's ATI/AMD's CTM effort as well, as well as several independent > ones. > www.gpgpu.org is a great resource to start with. This connects back to an earlier posting of mine which drew a "dead cat bounce" for a response ... ;-) ... , but you should also definitely look at both the offerings of PeakStream and RapidMind. PeakStream (like CUDA) provides libraries and a development environment (their current focus is GPUs), but abstract the idea of co-processing one step further to a virtual machine (Mitrion-C does the same for FPGAs) connected to a master serial processor. Any additional co-processing resource (any of several flavors of GPUs, a CELL SPE, or non-rank zero,homogenous multi-cores, etc.) can provide the horse power for the data parallel accelerations. They claim that once a particular backend for the VM is available you will be able run your code without a recompile on it or any other supported backend. I think the RapidMind product is similar. Peakstream has a nice white paper on their web site and the main Rapid Mind paper is: Data-parallel Programming on the Cell BE and the GPU using RapidMind Developoment Platform by Mike McCool. The jist of my earlier posting was "Does such a data-parallel VM abstraction have a future in an HPC world of heterogeneous on and off-chip co-processors?" Its presence as part of the Mitrion-C and PeakStream programming models suggests someone with money believes as much. rbw > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -- -- Richard B. Walsh Project Manager Network Computing Services, Inc. Army High Performance Computing Research Center (AHPCRC) rbw@ahpcrc.org | 612.337.3467 > > "Making predictions is hard, especially about the future." > > Nils Bohr ----------------------------------------------------------------------- This message (including any attachments) may contain proprietary or privileged information, the use and disclosure of which is legally restricted. If you have received this message in error please notify the sender by reply message, do not otherwise distribute it, and delete this message, with all of its contents, from your files. ----------------------------------------------------------------------- From rbw at ahpcrc.org Mon Apr 2 06:45:55 2007 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> <461012E1.8020006@charter.net> Message-ID: <46110913.2040007@ahpcrc.org> Mark Hahn wrote: >> The next gen of hardware will support native double precision (AFAIK). > > my point is that there's native and there's native. if the HW supports > doubles, but they take 8x as long, then there's still a huge reason to > make sure the program uses only low-precision. and 8x (WAG, of course) > may actually be enough so that a 4-core, full-rate SSE CPU to beats it I would be surprised if they "faked" double precision is this way. GPUs are the widest thing you can get in a processor. My WAG is that they will provide true/fast 64-bit (minus the same IEEE 754 twiddles) by coalescing 32-bit ... reducing the floating point width of a given core by half, but still delivery lots of FLOPs. Especially with the G80, it makes to think of these GPUs and multi-core SIMD processors. rbw > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -- -- Richard B. Walsh Project Manager Network Computing Services, Inc. Army High Performance Computing Research Center (AHPCRC) rbw@ahpcrc.org | 612.337.3467 > > "Making predictions is hard, especially about the future." > > Nils Bohr ----------------------------------------------------------------------- This message (including any attachments) may contain proprietary or privileged information, the use and disclosure of which is legally restricted. If you have received this message in error please notify the sender by reply message, do not otherwise distribute it, and delete this message, with all of its contents, from your files. ----------------------------------------------------------------------- From amjad11 at gmail.com Sun Apr 1 02:02:53 2007 From: amjad11 at gmail.com (amjad ali) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! Message-ID: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> Hi All, Would any of you please like to share usage-experience/views/comments about Windows Compute Cluster Server 2003 based Beowulf Clusters? What in your opinion is the future of such clusters? How you compare these with the LINUX CLUSTERS? regards. From jlforrest at berkeley.edu Sun Apr 1 09:58:49 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] How to Diagnose Cause of Cluster Ethernet Errors? In-Reply-To: <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> References: <460AFFA1.6070103@berkeley.edu> <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> Message-ID: <460FE4C9.6040204@berkeley.edu> Douglas Eadline wrote: > > I am constantly amazed at how many people buy the > latest and greatest node hardware and then connect > them with a sub-optimal switch (or cheap cables), thus reducing > the effective performance of the nodes (for parallel > applications). Kind "penny wise and pound foolish" as they say. > I sincerely appreciate all the comments about my problem. I will reply to them in due time. However, I'd like to comment on this, which admittedly is off-topic from my original posting. I don't disagree with what you're saying. The problem is how to recognize "sub-optimal" equipment. For example, I see three tiers in ethernet switching hardware: 1) The low-end, e.g. Netgear, Linksys, D-link, ... 2) The mid-end, e.g. HP Procurve, Dell, SMC, ... 3) The high-end, e.g. Cisco, Foundry, ... What I, as a system manager, not as an Electrical Engineer, have trouble understanding, is what the true differences are between these levels, and, at one level, between the various vendors. These days I suspect that many of the vendors are using ASICs made by other chip companies, and the many vendors use the same ASICs. Assuming that's true, where's the added value that justifies the cost differences? Sometimes the value is in the "management" abilities of a device. I don't deny this can be a major selling point in a large enterprise environment, but in a 30-node cluster, or a small LAN, it's hard to justify paying for this. In terms of ethernet performance, once a device can handle wirespeed communication on all ports, where's the added value that justifies the added cost? I'm looking for empirical answers, which aren't always easy to find, and sometimes to understand. In the case of my cluser, it was configured and purchased before I got here, so I had nothing to do with choosing its components but I have to admit that I'm not sure what I would have done differently. Cordially, Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From mfatica at gmail.com Sun Apr 1 12:30:32 2007 From: mfatica at gmail.com (Massimiliano Fatica) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> Message-ID: <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> Mark, CUDA comes with a full BLAS and FFT library (for 1D,2D and 3D transforms). You can have relevant speed up even for 2D transforms or for a batch of 1Ds. You can offload only compute intendive parts of your code to the GPU from C and C++ ( writing a wrapper from Fortran should be trivial). The current generation of the hardware supports only single precision, but there will be a double precision version towards the end of the year. Massimiliano PS: I work on CUDA at Nvidia, so I may be a little biased... On 4/1/07, Mark Hahn wrote: > as far as I know, there are not any well-developed libraries which simply > harness whatever GPU you provide, but don't require your whole program to > be GPU-ized. the cost of sharing data with a GPU is significant, but > blas-3 might have a high enough work-to-size ratio to make it feasible. > 3d fft's might also be expressible in GPU-friendly terms (the trick would > be to utilize not fight the GPU's inherent memory-access preferences.) > perhaps some MCMC stuff might be SIMD-able? I doubt that sequence analysis > would make much sense, since GPUs are not well-tuned to access host memory, > and sequence programs are not actually that compute-intensive. I'd guess > that anything involving sparse matrices would be difficult to do on a GPU. From mfatica at gmail.com Sun Apr 1 17:53:52 2007 From: mfatica at gmail.com (Massimiliano Fatica) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> Message-ID: <8e6393ac0704011753l28e8727drb944bc1734cb7dc2@mail.gmail.com> On 4/1/07, Mark Hahn wrote: > > I assume this is only single-precision, and I would guess that for > numerical stability, you must be limited to fairly short fft's. > what kind of peak flops do you see? what's the overhead of shoving > data onto the GPU, and getting it back? (or am I wrong that the GPU > cannot do an FFT in main (host) memory? I will run some benchmark in the next days ( I usually do more than just an FFT). I remember some numbers for SGEMM (real SGEMM C=alphaA*B+beta*C), 120 Gflops on board, 80 Gflops measured from the host (with all the I/O overhead) , for N=2048. > > > You can offload only compute intendive parts of your code to the GPU > > from C and C++ ( writing a wrapper from Fortran should be trivial). > > sure, but what's the cost (in time and CPU overhead) to moving data > around like this? It depends on your chipset and from other details ( cold access, data in cache, pinned memory): it goes from around 1GB/s to 3GB/s. > > > The current generation of the hardware supports only single precision, > > but there will be a double precision version towards the end of the > > year. > > do you mean synthetic doubles? I'm guessing that the hardware isn't > going to gain the much wider multipliers necessary to support doubles > at the same latency as singles... > Can't comment on this one..... :-) Massimiliano From weikuan.yu at gmail.com Mon Apr 2 06:44:38 2007 From: weikuan.yu at gmail.com (Weikuan Yu) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] HotI 2007 Call for Papers -- Deadline (April 9) is approaching Message-ID: <461108C6.5050701@gmail.com> -------------------------------------------------------------------- Apologies if you received multiple copies of this posting. Please feel free to distribute it to those who might be interested. -------------------------------------------------------------------- Hot Interconnects 15 IEEE Symposium on High-Performance Interconnects August 22-24, 2007 Stanford University Palo Alto, California, USA Hot Interconnects is the premier international forum for researchers and developers of state-of-the-art hardware and software architectures and implementations for interconnection networks of all scales, ranging from on-chip processor-memory interconnects to wide-area networks. This yearly conference is very well attended by leaders in industry and academia. The atmosphere provides for a wealth of opportunities to interact with individuals at the forefront of this field. Themes include cross-cutting issues spanning computer systems, networking technologies, and communication protocols. This conference is directed particularly at new and exciting technology and product innovations in these areas. Contributions should focus on real experimental systems, prototypes, or leading-edge products and their performance evaluation. In addition to those subscribing to the main theme of the conference, contributions are also solicited in the topics listed below. * Novel and innovative interconnect architectures * Multi-core processor interconnects * System-on-Chip Interconnects * Advanced chip-to-chip communication technologies * Optical interconnects * Protocol and interfaces for interprocessor communication * Survivability and fault-tolerance of interconnects * High-speed packet processing engines and network processors * System and storage area network architectures and protocols * High-performance host-network interface architectures * High-bandwidth and low-latency I/O * Tb/s switching and routing technologies * Innovative architectures for supporting collective communication * Novel communication architectures to support grid computing Submission Guideline o Extended deadline: April 9th, 2007 o Notification of acceptance: May 15, 2007 o Papers need sufficient technical detail to judge quality and suitability for presentation. o Submit title, author, abstract, and full paper (six pages, double-column, IEEE format). o Papers should be submitted electronically at the specified link location found on http://www.hoti.org o For further information please see http://www.hoti.org/hoti15/cfp.html About the Conference - Conference held at the William Hewlett Teaching Center at Stanford University. - Papers selected will be published in proceedings by the IEEE Computer Society. - Presentations are 30-minute talks in a single-track format. - Online information at http://www.hoti.org GENERAL CO-CHAIRS * John W. Lockwood, Washington University in St. Louis * Fabrizio Petrini, Pacific Northwest National Laboratory TECHNICAL CO-CHAIRS * Ron Brightwell, Sandia National Laboratories * Dhabaleswar (DK) Panda, The Ohio State University LOCAL ARRANGEMENTS CHAIR * Songkrant Muneenaem, Washington University in St. Louis PANEL CHAIR * Daniel Pitt, Santa Clara University PUBLICITY CO-CHAIRS * Weikuan Yu, Oak Ridge National Laboratory PUBLICATION CHAIR * Luca Valcarenghi, Scuola Superiore Sant'Anna FINANCE CHAIR * Herzel Ashkenazi, Xilinx TUTORIAL CO-CHAIRS - TBA REGISTRATION CHAIR * Songkrant Muneenaem, Washington University in St. Louis Webmaster * Liz Rogers, LRD Group Steering Committee o Allen Baum, Intel o Lily Jow, Hewlett Packard o Mark Laubach, Broadband Physics o John Lockwood, Stanford University o Daniel Pitt, Santa Clara University Technical Program Committee * Dennis Abts Cray, Inc. * Adnan Aziz University of Texas, Austin * Alan Benner IBM * Keren Bergman Columbia University * Andrea Bianco Politecnico di Torino * Piero Castoldi Scuola Superiore Sant'Anna * Sarang Dharmapurikar Nuova Systems * Hans Eberle Sun Microsystems Laboratories * Wu-chun Feng Virginia Tech * Juan Fernandez University of Murcia * Ada Gavrilovska Georgia Institute of Technology * Paolo Giaccone Politecnico di Torino * Mitchell Gusat IBM Zurich Research Laboratory * Ron Ho Sun Microsystems Laboratories * Doan Hoang University of Technology, Sydney * D. N. (Jay) Jayasimha Intel * Isaac Keslassy Technion * Venkata Krishnan Dolphin Interconnect Solutions * Tal Lavian Nortel Networks Labs, UC Berkeley * Bill Lin University of California, San Diego * Olav Lysne Simula Research Laboratory * Pankaj Mehra HP Labs * Rami Melhem University of Pittsburgh * Cyriel Minkenberg IBM Zurich Research Laboratory * Gregory Pfister IBM * Craig Stunkel IBM T.J. Watson Research Center * Anujan Varma University of California at Santa Cruz * Zuoguo (Joe) Wu Intel From gerry.creager at tamu.edu Mon Apr 2 07:04:42 2007 From: gerry.creager at tamu.edu (Gerry Creager) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <46110913.2040007@ahpcrc.org> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> <461012E1.8020006@charter.net> <46110913.2040007@ahpcrc.org> Message-ID: <46110D7A.9090506@tamu.edu> Richard Walsh wrote: > Mark Hahn wrote: >>> The next gen of hardware will support native double precision (AFAIK). >> my point is that there's native and there's native. if the HW supports >> doubles, but they take 8x as long, then there's still a huge reason to >> make sure the program uses only low-precision. and 8x (WAG, of course) >> may actually be enough so that a 4-core, full-rate SSE CPU to beats it > I would be surprised if they "faked" double precision is this way. GPUs > are the widest thing > you can get in a processor. My WAG is that they will provide true/fast > 64-bit (minus the same > IEEE 754 twiddles) by coalescing 32-bit ... reducing the floating point > width of a given > core by half, but still delivery lots of FLOPs. Especially with the > G80, it makes to think of these > GPUs and multi-core SIMD processors. In discussions w/ Mike McCool of PeakStream at SC06, I think Mark is correct. At this time, I believe they're stiull faking DP. Look for hardware enhancements 3-4Q this calendar year. Gerry -- Gerry Creager -- gerry.creager@tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From maraz at ics.forth.gr Mon Apr 2 07:04:25 2007 From: maraz at ics.forth.gr (Manolis Marazakis) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] Ethernet break through? Message-ID: <46110D69.608@ics.forth.gr> Hmmm ... Perhaps Geyser? technology would tie nicely with the recent discovery of CPU-clock relativistic effects ? http://www.reghardware.co.uk/2007/04/01/cpu_time_dilation/ /* This link appeared in slashdot ... */ Best regards, Manolis Marazakis. -- Computer Architecture and VLSI Laboratory, Institute of Computer Science, Foundation for Research and Technology - Hellas (FORTH), Vassilika Vouton, Heraklion, Greece GR-71110. Tel: +30.2810391438, +30.2810391699 Fax: +30.2810391601 E-mail: maraz@ics.forth.gr From rbw at ahpcrc.org Mon Apr 2 07:10:15 2007 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue Mar 16 01:05:57 2010 Subject: [Beowulf] OT? GPU accelerators for finite difference time domain In-Reply-To: <46110D7A.9090506@tamu.edu> References: <1175147950.5777.13.camel@ceiriog.eclipse.co.uk> <8e6393ac0704011230o50e4071enffe5986e85e497bd@mail.gmail.com> <461012E1.8020006@charter.net> <46110913.2040007@ahpcrc.org> <46110D7A.9090506@tamu.edu> Message-ID: <46110EC7.9060708@ahpcrc.org> Gerry Creager wrote: > > Richard Walsh wrote: >> Mark Hahn wrote: >>>> The next gen of hardware will support native double precision (AFAIK). >>> my point is that there's native and there's native. if the HW supports >>> doubles, but they take 8x as long, then there's still a huge reason to >>> make sure the program uses only low-precision. and 8x (WAG, of course) >>> may actually be enough so that a 4-core, full-rate SSE CPU to beats it >> I would be surprised if they "faked" double precision is this way. GPUs >> are the widest thing >> you can get in a processor. My WAG is that they will provide true/fast >> 64-bit (minus the same >> IEEE 754 twiddles) by coalescing 32-bit ... reducing the floating point >> width of a given >> core by half, but still delivery lots of FLOPs. Especially with the >> G80, it makes to think of these >> GPUs and multi-core SIMD processors. > > In discussions w/ Mike McCool of PeakStream at SC06, I think Mark is > correct. At this time, I believe they're stiull faking DP. Look for > hardware enhancements 3-4Q this calendar year. Sorry ... Did not mean to suggest that true 64-bit was available in hardware from anyone >>today<<, only that when it comes (later this year) it will not be "faked". The graphics folks need it for better dynamic range in contrast and brightness I think. rbw -- -- Richard B. Walsh Project Manager Network Computing Services, Inc. Army High Performance Computing Research Center (AHPCRC) rbw@ahpcrc.org | 612.337.3467 > > "Making predictions is hard, especially about the future." > > Nils Bohr ----------------------------------------------------------------------- This message (including any attachments) may contain proprietary or privileged information, the use and disclosure of which is legally restricted. If you have received this message in error please notify the sender by reply message, do not otherwise distribute it, and delete this message, with all of its contents, from your files. ----------------------------------------------------------------------- From deadline at clustermonkey.net Mon Apr 2 07:34:44 2007 From: deadline at clustermonkey.net (Douglas Eadline) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] How to Diagnose Cause of Cluster Ethernet Errors? In-Reply-To: <460FE4C9.6040204@berkeley.edu> References: <460AFFA1.6070103@berkeley.edu> <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> <460FE4C9.6040204@berkeley.edu> Message-ID: <40810.192.168.1.1.1175524484.squirrel@mail.eadline.org> Jon, I hear your frustration. You are quite right that many of the ASICs are the same. Implementation is important. In terms of clusters, there are no hard and fast rules for switches. i.e. I have fund some cheap GigE switches (like the SMC 8508T) to be real performer (8 ports/Jumbo Frames for under $100) I just got an SMC CGS16 to use in my test rack. So I am a little partial to SMC at this point. However, I have not tested the CGS16 fully so it may not live up to my expectations. In the past I have found Foundry and Extreme to work quite well, and give the price you pay they should. I think the trick is to find the bargains that still perform well. As I see it, there three ways to buy a good switch: 1. Hire a consultant to help with the cluster (their experience can save money and head-aches on other issues as well) 2. Use Google and this list to see what you can find about a particular switch, but be warned most people do not push switches the way HPC users do so what is good for the back office may not be good for the cluster. (and pretty much ignore vendor data sheets) 3. Get some evaluation switches (or a least test them within the 30-day return period) for specific applications you plan to run. This is probably the best way to proceed. Unfortunately there does not seem to be an easy way to really test a switch. The easiest thing to do is to run netpipe on two ports to establish a baseline. Choose the switches that provides the best netpipe results. Then run netpipe on ports at the same time and see if there is degradation. This however is not the whole story, some performance may depend on port choice (i.e. ports may span multiple ASICs) and performance may vary. Also, to full test a switch I would assume that you would want to test every port combination while the other ports were at some constant network load. So you can probably see why it is hard to test switches. In any case, these treads on the list should help as well (quite informative): http://www.beowulf.org/archive/2006-March/015282.html http://www.beowulf.org/archive/2006-April/015295.html http://www.beowulf.org/archive/2006-April/015340.html Finally, I am open to any one who can come up with a reasonably good switch test, maybe combination of applications and synthetic tests so that we can at least eliminate the poor performers. I would like to post this kind of data on ClusterMonkey. -- Doug > Douglas Eadline wrote: > >> >> I am constantly amazed at how many people buy the >> latest and greatest node hardware and then connect >> them with a sub-optimal switch (or cheap cables), thus reducing >> the effective performance of the nodes (for parallel >> applications). Kind "penny wise and pound foolish" as they say. >> > > I sincerely appreciate all the comments about my problem. I will reply > to them in due time. However, I'd like to comment on this, which > admittedly is off-topic from my original posting. > > I don't disagree with what you're saying. The problem is how > to recognize "sub-optimal" equipment. For example, I see > three tiers in ethernet switching hardware: > > 1) The low-end, e.g. Netgear, Linksys, D-link, ... > > 2) The mid-end, e.g. HP Procurve, Dell, SMC, ... > > 3) The high-end, e.g. Cisco, Foundry, ... > > What I, as a system manager, not as an Electrical Engineer, > have trouble understanding, is what the true differences > are between these levels, and, at one level, between > the various vendors. > > These days I suspect that many of the vendors are using > ASICs made by other chip companies, and the many vendors > use the same ASICs. Assuming that's true, where's the > added value that justifies the cost differences? Sometimes > the value is in the "management" abilities of a device. > I don't deny this can be a major selling point in a > large enterprise environment, but in a 30-node cluster, > or a small LAN, it's hard to justify paying for this. > > In terms of ethernet performance, once a device > can handle wirespeed communication on all ports, > where's the added value that justifies the added > cost? I'm looking for empirical answers, which > aren't always easy to find, and sometimes to understand. > > In the case of my cluser, it was configured and purchased > before I got here, so I had nothing to do with choosing > its components but I have to admit that I'm not > sure what I would have done differently. > > Cordially, > > Jon Forrest > Unix Computing Support > College of Chemistry > 173 Tan Hall > University of California Berkeley > Berkeley, CA > 94720-1460 > 510-643-1032 > jlforrest@berkeley.edu > > !DSPAM:460fe4d2220232889862676! > -- Doug From gerry.creager at tamu.edu Mon Apr 2 08:15:31 2007 From: gerry.creager at tamu.edu (Gerry Creager) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Server room design consulting In-Reply-To: References: <1175104216.28106.236.camel@rahl.acomp.usf.edu> Message-ID: <46111E13.1020207@tamu.edu> At least in my area, Liebert does have some competent folks who could help. Tell their salesman to get one on them involved, and in general, don't take the Liebert salesman's word that he's necessarily competent. I've run across exceptions, but the last time we got burned, it was pretty bad (and, no, I wasn't involved in trusting the salesman, but that's a whole 'nother story). gerry Robert G. Brown wrote: > On Wed, 28 Mar 2007, Daniel Majchrzak wrote: > >> We have a dedicated cluster room, email server room, and networking room >> that have slowly evolved over the years. Due to budget constraints in >> the past no one has ever done an analysis of our electricity and AC. >> (We've had the facilities people in, but their analysis wasn't any >> better than our own guestimates. ) When increases in either power or >> cooling were necessary it was either done piecemeal or not done at all. >> (We have had some orders to come up with some "zero-dollar" solutions). >> Now all three rooms are about to go through some equipment expansions. >> While we can make some rough estimates, and we could go to the >> university's facilities people, we thought we would try to get some >> funds together to hire some professional services so that it gets done >> right. Has any one on the list had any experiences they would like to >> share on hiring these kind of consultants. Referrals? Can anyone give >> a (very) rough estimate of what we should expect to pay? > > My own experience with power and AC people from normal contracting firms > is that they are immensely clueless about computer server room > infrastructure. I haven't ever hired a consultant, but I do consulting > on this from time to time. I'd guess that you'll pay perhaps $150 to > $200/hour for consulting, but it might well depend on what kind of > consultant you got and where you got them. Again, based on my > experience with area computer management contracting houses, they are > not really competent with infrastructure issues. A really good > electrical contractor MIGHT have somebody. Liebert or APC might be able > to refer you. An architect with experience in server room design might > be your best bet. > > rgb > >> >> Thanks, >> >> Dan >> >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> > -- Gerry Creager -- gerry.creager@tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From rgb at phy.duke.edu Mon Apr 2 08:39:36 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] How to Diagnose Cause of Cluster Ethernet Errors? In-Reply-To: <40810.192.168.1.1.1175524484.squirrel@mail.eadline.org> References: <460AFFA1.6070103@berkeley.edu> <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> <460FE4C9.6040204@berkeley.edu> <40810.192.168.1.1.1175524484.squirrel@mail.eadline.org> Message-ID: To jump in late, let me reiterate that before jumping to any conclusions concerning proximate cause of your networking difficulty, it is best to systematically figure out what it might be. a) List points of possible failure. They are in very general terms: ethernet switch(es), cables, NICs, kernel/drivers, software. b) On the switches one can have two or three general classes of failure. The simplest and most pernicious is probably "bad ports". A port can be "bad" because its internal electronics is screwed, maybe by spot heating (resulting in intermittant problems or port death) or by the little bitty wires in the RJ45 socket getting bent or deforming over time so that solid contact is no longer made with an inserted cable. This happens -- especially if the wires aren't properly supported going into the port socket so that they exert a torgue that pressures the contact wires in a warm system over a long time. Dust or corrosion can also contribute to spotty electrical connectivity inside the port connection itself. "Bad ports" can usually be identified by the fact that any system plugged into the port has a high chance of having problems where that same system, plugged into a different port on a different switch, works flawlessly. The solution to bad ports is throw out-of-service-contract switches away, buy new ones with a four year contract on them (tier 1 or tier 2 if possible) and move on with life. Switches are cheap -- human time is expensive. c) cables can always be bad, especially if they are dangling, homemade, have been moved around a lot. Again, the little contact pins deform under certain heat/pressure circumstances. Also the PLASTIC THEY ARE MADE OF can deform under warm pressure so that they no longer seat properly in a port socket, or the little "snap" on the back wears down so that they can wiggle to where connectivity is mediocre. Note that in wiring situations with patch panels, bad cable/port connections extend recursively between system and port. "Bad cables" can be detected one of several ways. The best way is to invest in a halfway decent cable tester. The cheap one I own can be snapped into any socket with a short and RELIABLE patch cable or accept any cable being tested. It then transmits voltage on each wire pair one at a time that is reflected at the far end and lights up little LEDs that tell you if any pairs are faulting. This is good for cable breaks and bad contacts, but can "pass" marginal cables that are making contact but arcing a bit. To do better, you have to use a higher quality tester that actually puts a data signal on the line, or use e.g. a laptop and a secondary reliable system to test the secondary cabling route compared to a "known good" point-to-point (direct) cable hookup. This approach can with care help detect bad ports on switches as well, although it can be difficult to resolve problems with switches from problems with ports. d) Your problem is very unlikely to be bad NICs, but if it is it will show up when you use a known-good crossover cable to directly connect your laptop to the suspect NIC and observe significant problems with connectivity (bad ping rates, especially on ping floods). Ping is actually a fairly powerful tool, as is traceroute. netpipe is pretty good for testing interfaces as well. e) Integrated kernel/driver/card problems are far from unknown, especially for certain cards. At one point in time, for example, RTL 8139 cards were cheap and nearly ubiquitous -- and sucked unimaginably. They effectively didn't buffer incoming traffic, and one could override the kernel/driver's ability to process asynchronous arriving packets with ease. Consequently they'd "work" until one tried to send a high speed stream of small packets to one, at which point they'd basically fail to receive 9 out of 10 of them. I actually managed to get a netperf out of a 100 Mbps card of something like 1 Mbps -- the other 99% of the packets were basically dropped and either lost (UDP) or had to be retransmitted (TCP). This is a very difficult problem to diagnose. General symptoms that should make you suspect this is a problem include -- having debugged the switch and physical connection and found that it works fine for "known good" NICs on both ends. Getting decent connectivity for certain "low stress" applications or low loads, but losing more and more packets as you increase the network load on the NIC. Obvious kernel-based network error messages in /var/log/messages. And the kicker -- buy a known-good ethernet card, one you are certain works perfectly in the kernel (the listvolken can probably give you half a dozen recommendations if you don't already favor 3coms or intels or the like). Swap it into a system that is having problems and leaving everything else the same (wire, port, etc) repeat the test. If the problem goes away, it's a bad sign for the NIC. Solution is probably to just put known good NICs in the systems and stop using the (usually onboard and sucky) NIC. NICs are cheap, time is dear. Note well that you've already had excellent advice concerning autonegotiation of duplex and speed. This SHOULD NOT be a problem with pretty much any modern (unmanaged) switch and card, but old-timers remember well that it once was and might be again. Or if the switch is a managed switch, then god only knows what state it is in and debugging this becomes a major PITA. My own solution is still to dump the switch and get a new one. You can get really lovely 48 port dual power supply gigabit switches for a kilobuck or so from Dell, or you can spend a lot less and get perfectly reasonable unmanaged switches from any of a half-dozen vendors. One good way to debug things is just to get a new switch and see if it solves all your problems -- how many hours do you have to waste before a few hundred dollars worth of new hardware is cheap? f) Software (e.g. MPI flavor, PVM, userspace socket code). This too is a bitch to debug as there are a near infinity of ways to write buggy code, sometimes buggy code where the failure only occurs under certain rarely-accessed modes of operation of the software involved. The diagnosis here is one of exclusion. If ping, traceroute, netperf or netpipe, hardware network testers, card swapping, switch swapping all yield no problems but the "problem" persists for your application throughout, you should suspect that your application is somehow buggy. This is by no means impossible, depending on just what is being done with the sockets. If you conclude that it is likely to BE your application, you have only two or three possible routes. * fix it yourself * get somebody to fix it for you * throw up your hands in disgust and use a different tool to accomplish the same task. Which one works depends on whether the tool is open source or commercial, your coding skills, availability of service or support forums, contact with the developers, etc. The idea of all of the above is to set up a diagnosis tree and run down it >>systematically<< until you figure out what is wrong. This method has never failed me through numerous diagnoses of every possible mode of failure over twenty years. I've seen NICs with intermittant heat-sensitive malfunctions (and figured it out), NICs with visibly toasted components (afio), bad cables and ports galore (afio), bad switches (afio), bad drivers (afio), and bad software (afio, without saying that I was always able to fix the latter). If you proceed very systematically you can eventually end up where one answer, however unlikely it might appear, is the truth. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From smulcahy at aplpi.com Mon Apr 2 07:28:37 2007 From: smulcahy at aplpi.com (stephen mulcahy) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] How to Diagnose Cause of Cluster Ethernet Errors? In-Reply-To: <460FE4C9.6040204@berkeley.edu> References: <460AFFA1.6070103@berkeley.edu> <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> <460FE4C9.6040204@berkeley.edu> Message-ID: <46111315.5030509@aplpi.com> Hi Jon, Things I look out for in switches in general are reliability and build quality. I'd have some cheaper switches which worked but got worryingly warm to touch. The 3com switches we use in our office in general tend to be solid and don't seem to heat up as much as some of the SMCs. Having said that, I've heard good things (here mostly) about some specific SMC switches. I generally don't pay for managed switches unless I have clear needs to work with my traffic at that level. I don't have those needs for a small office or department environment. For a cluster, given the budgets typically involved, I'm inclined to err on the side of a switch with a good reputation and a more extensive feature-set then I actually need since it is such a critical piece of the picture. For clusters, the overall bandwidth of the switch is also a huge issue. It's still not clear to me how reliable manufacturers figures for switch bandwidth are though. The procurve we have in our cluster seems to be performing well, and as I said, I've heard good things about some of the SMCs (tigers?) but short of going with what others are using successfully I'm not sure. Has anyone tested a dozen switches in a lab for backplane bandwidth? I'm sure the more experience members will have more concrete pointers but maybe my comments give you a starting point - it's an interesting, and very relevant, question. -stephen Jon Forrest wrote: -- Stephen Mulcahy, Applepie Solutions Ltd, Innovation in Business Center, GMIT, Dublin Rd, Galway, Ireland. http://www.aplpi.com From deadline at clustermonkey.net Mon Apr 2 09:12:29 2007 From: deadline at clustermonkey.net (Douglas Eadline) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> Message-ID: <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> > Hi All, > > Would any of you please like to share usage-experience/views/comments > about Windows Compute Cluster Server 2003 based Beowulf Clusters? As a point of clarification, there is no such thing as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" This link may help: http://en.wikipedia.org/wiki/Beowulf_cluster > > What in your opinion is the future of such clusters? > > How you compare these with the LINUX CLUSTERS? You will not find much information on this list as it mainly focuses on Linux Beowulf style clusters. My understanding is MS is working on the "ease of use" and integration issues. You may find more information at http://www.winhpc.org/ -- Doug > > regards. > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > !DSPAM:46110fba89467511819938! > -- Doug From epaulson at cs.wisc.edu Mon Apr 2 09:45:39 2007 From: epaulson at cs.wisc.edu (Erik Paulson) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> Message-ID: <20070402164539.GA20577@swingline.cs.wisc.edu> On Mon, Apr 02, 2007 at 12:12:29PM -0400, Douglas Eadline wrote: > > Hi All, > > > > Would any of you please like to share usage-experience/views/comments > > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > > As a point of clarification, there is no such thing > as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" > This link may help: > > http://en.wikipedia.org/wiki/Beowulf_cluster > I don't know why everyone is so obsessed with saying "Your Beowulf must run an (F)OSS operating system to be a Beowulf." You can build a Beowulf out of Windows. God only knows why you'd want to, but you can. Just to invent a little bit of evidence, Thomas Sterling edited a book called "Beowulf Cluster Computing with Windows" http://www.amazon.com/Beowulf-Computing-Scientific-Engineering-Computation/dp/0262692759/ref=pd_sim_b_5/002-1371173-4594458?ie=UTF8&qid=1175531228&sr=8-1 It was actually two books - a "Beowulf Cluster Computing with Linux" and a "Beowulf Cluster Computing with Windows". 75% of the text was the same. (We wrote a chapter in it - we used the same chapter, with latex macros \iflinux and \ifwinnt for whichever book was being built) The Linux book way outsold the Windows book, and so there was no second edition of the Windows book. My guess is that most everyone had the good sense to say "Windows as the base OS for my cluster? No thanks" > > > > What in your opinion is the future of such clusters? > > > > How you compare these with the LINUX CLUSTERS? > > You will not find much information on this list as > it mainly focuses on Linux Beowulf style clusters. > The parallel programming part of this list applies to Windows as much as it applies to Linux (or FreeBSD or Darwin or HURD) -Erik From peter.st.john at gmail.com Mon Apr 2 09:55:55 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> Message-ID: Amjad, MS WIn3.11 was a GUI application on top of a small, reliable OS (DOS). Since then, however, MS has integrated the OS with the GUI, which is awkward for developing interprocess communication and booting light kernels. Since XP, there is further integration of the OS with licencing and security material, which can't be stripped off without comprimising some big MS concerns. Basically, Unix was designed to be a development environment; VMS was designed for safe and efficient DP; MS was designed for marketing (and arguably, an easier end-user experience). The main reason I know of to use MS for a server is to commoditize MS-certified admins, who are cheaper than RGB is. Even IBM uses a zillion instances of Linux over VMWare on a 390 to make a web farm. I'd be interested in a benchmark comparison of MS cluster vs linux cluster, but I pause at wondering who would bother. Of course I'd merely expect that the OS would be too big to make sense on a compute node; I'm told that Vista has an efficient kernel, but if MS stripped off the layers people would just make 1-node "clusters" to get around the licensing material, which would be counterproductive to MS. But suppose someone wanted to do a test to find out (instead of merely assuming). If the result came out well for MS, I'd want to be paid by MS; they exist for marketing, they are richer than Croesus, it would be foolish to do anything for them for free. But that would impugn objectivity. And if the result were poor for MS, nobody would care because that's our expectations (pardon me for speaking for others). So I just don't see that anyone at all is motivated to do an objective comparison. But I'd be very interested anyway in an honest attempt, if you hear of one. Salaam, Peter On 4/1/07, amjad ali wrote: > > Hi All, > > Would any of you please like to share usage-experience/views/comments > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > > What in your opinion is the future of such clusters? > > How you compare these with the LINUX CLUSTERS? > > regards. > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/d7abfb32/attachment.html From James.P.Lux at jpl.nasa.gov Mon Apr 2 10:17:41 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> Message-ID: <6.2.3.4.2.20070402100940.02fb3088@mail.jpl.nasa.gov> At 09:12 AM 4/2/2007, Douglas Eadline wrote: > > Hi All, > > > > Would any of you please like to share usage-experience/views/comments > > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > >As a point of clarification, there is no such thing >as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" >This link may help: > >http://en.wikipedia.org/wiki/Beowulf_cluster I know that most people interpret Beowulf to include only those with open source software (hence leaving out things like Windows), but I think that really, the distinguishing feature of a classic Beowulf cluster is that it uses "consumer commodity" sorts of things (e.g. inexpensive PCs intended for the consumer market). I think one could fairly say that Windows falls in the "consumer commodity" category. Now, WinCCS isn't exactly in the commodity area, but, then, neither are some of the high performance interconnects currently used, and that doesn't exclude them from discussion. And, of course, some Beowulf clusters have been built using Macs and Suns, with their respective not-entirely-open operating systems. It IS true that the "mainstream" of Beowulfery is Linux oriented, and, as Doug comments later, that IS the focus of the list. As far as Windows CCS goes, there have been a couple discussions on the list about what CCS is, how it fits in to the clustering world, and how such things might differ from the traditional Linux model. Check the archives. > > > > What in your opinion is the future of such clusters? > > > > How you compare these with the LINUX CLUSTERS? > >You will not find much information on this list as >it mainly focuses on Linux Beowulf style clusters. > >My understanding is MS is working on the "ease of use" >and integration issues. You may find more information >at http://www.winhpc.org/ > > -- > Doug > >-- >Doug James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From greg.lindahl at qlogic.com Mon Apr 2 10:18:20 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: <20070402171820.GA5081@dhcp-2-200.internal.keyresearch.com> On Sun, Apr 01, 2007 at 05:39:48PM -0400, Douglas Eadline wrote: > > I just posted some interesting news on Cluster Monkey. > > http://www.clustermonkey.net//content/view/192/1/ I can easily believe that rgb's documentation has a lot of redundancy in it... what the filter is doing is re-deriving the AI that wrote the documentation in the first place... which is probably much smaller than the document. -- g From greg.lindahl at qlogic.com Mon Apr 2 10:34:25 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> Message-ID: <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> On Mon, Apr 02, 2007 at 12:55:55PM -0400, Peter St. John wrote: > The main reason I know of to use > MS for a server is to commoditize MS-certified admins, who are cheaper than > RGB is. Not on a per-word basis! rgb is far more productive! -- g From James.P.Lux at jpl.nasa.gov Mon Apr 2 10:36:18 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070402164539.GA20577@swingline.cs.wisc.edu> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> Message-ID: <6.2.3.4.2.20070402103446.02d16ed0@mail.jpl.nasa.gov> At 09:45 AM 4/2/2007, Erik Paulson wrote: >On Mon, Apr 02, 2007 at 12:12:29PM -0400, Douglas Eadline wrote: > > > Hi All, > > > > > > Would any of you please like to share usage-experience/views/comments > > > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > > > > As a point of clarification, there is no such thing > > as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" > > This link may help: > > > > http://en.wikipedia.org/wiki/Beowulf_cluster > > > >I don't know why everyone is so obsessed with saying "Your Beowulf must run >an (F)OSS operating system to be a Beowulf." > >You can build a Beowulf out of Windows. God only knows why you'd want to, but >you can. Perhaps because you have the boxes given to you? (e.g. Cornell) Perhaps because you relish the thrill? Perhaps because you have a huge investment in applications specific software tailored to Windows that you want to use? >Just to invent a little bit of evidence, Thomas Sterling edited a book >called "Beowulf Cluster Computing with Windows" > >http://www.amazon.com/Beowulf-Computing-Scientific-Engineering-Computation/dp/0262692759/ref=pd_sim_b_5/002-1371173-4594458?ie=UTF8&qid=1175531228&sr=8-1 > >It was actually two books - a "Beowulf Cluster Computing with Linux" and a >"Beowulf Cluster Computing with Windows". 75% of the text was the same. (We >wrote a chapter in it - we used the same chapter, with latex macros \iflinux >and \ifwinnt for whichever book was being built) > >The Linux book way outsold the Windows book, and so there was no second >edition of the Windows book. My guess is that most everyone had the good >sense to say "Windows as the base OS for my cluster? No thanks" > > > > > > > > What in your opinion is the future of such clusters? > > > > > > How you compare these with the LINUX CLUSTERS? > > > > You will not find much information on this list as > > it mainly focuses on Linux Beowulf style clusters. > > > >The parallel programming part of this list applies to Windows as >much as it applies to Linux (or FreeBSD or Darwin or HURD) CP/M forever >-Erik > >_______________________________________________ >Beowulf mailing list, Beowulf@beowulf.org >To change your subscription (digest mode or unsubscribe) visit >http://www.beowulf.org/mailman/listinfo/beowulf James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From peter.st.john at gmail.com Mon Apr 2 10:45:53 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: The one node refusing to send the doc, and the other note receiving it anyway, cracked me the *** up! Thanks P.S. the Google April Fool's actually got me thinking (network via plumbing). Water conducts acoustics real well. So a free peer-to-peer network within a city, or some counties, would be easy and require no new infrastructure. I imagine the bandwidth would be weak and certainly there'd be a problem getting between cities, but I don't think the water and sewer utiltiies claim rights to the acoustic bandwidth (unlike hijacking phone lines with stuff that would interfere with existing telephony) so all free. On 4/1/07, Douglas Eadline wrote: > > > I just posted some interesting news on Cluster Monkey. > > http://www.clustermonkey.net//content/view/192/1/ > > > -- > Doug > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/6f5b8ee5/attachment.html From peter.st.john at gmail.com Mon Apr 2 10:50:28 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> Message-ID: Greg, cheaper per billable hour, I would never never say that a MCxE for any x is more valuable than RGB. RGB writes many words per minute, which an NT admin would spend many minutes per word reading... well except the good ones of course. No allusions to the Golden Mountain intended or implied. Peter On 4/2/07, Greg Lindahl wrote: > > On Mon, Apr 02, 2007 at 12:55:55PM -0400, Peter St. John wrote: > > > The main reason I know of to use > > MS for a server is to commoditize MS-certified admins, who are cheaper > than > > RGB is. > > Not on a per-word basis! rgb is far more productive! > > -- g > > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/b8ce1503/attachment.html From deadline at clustermonkey.net Mon Apr 2 11:01:44 2007 From: deadline at clustermonkey.net (Douglas Eadline) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070402164539.GA20577@swingline.cs.wisc.edu> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> Message-ID: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> > On Mon, Apr 02, 2007 at 12:12:29PM -0400, Douglas Eadline wrote: >> > Hi All, >> > >> > Would any of you please like to share usage-experience/views/comments >> > about Windows Compute Cluster Server 2003 based Beowulf Clusters? >> >> As a point of clarification, there is no such thing >> as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" >> This link may help: >> >> http://en.wikipedia.org/wiki/Beowulf_cluster >> > > I don't know why everyone is so obsessed with saying "Your Beowulf must > run > an (F)OSS operating system to be a Beowulf." Because I believe, "the art of Beowulf" has a rich history of development that is based on the original definition from "How to Build a Beowulf" by Thomas Sterling, John Salmon, Donald J. Becker and Daniel F. Savarese: ".. a collection of personal computers interconnected by widely available networking technology running anyone of several open source Unix-like operating systems. " I would not want to see it usurped by other clustering efforts. I believe that if we do not protect against revisionist history, then all of a sudden WCCS is now "Beowulf" computing. Such things, in my opinion dis-honor all the people (who I respect) that have contributed to this community. To me it is almost akin to removing author credit in open source software. A short aside. I overheard a conversation at SC-2000 about the origin of Beowulf from a MS representative "Beowulf was a copy of the Microsoft Wolfpack software. They chose that name so it would seem like Wolfpack some how". Truth is a slippery fish. Certainly Thomas Sterling can rework the definition as he pleases (he co-authored it), And I am not disparaging WCCS or any other clustering method. I just want to keep the credit where credit is due. So I stand as a defender of the faith, as it where. -- Doug > > You can build a Beowulf out of Windows. God only knows why you'd want to, > but > you can. > > Just to invent a little bit of evidence, Thomas Sterling edited a book > called "Beowulf Cluster Computing with Windows" > > http://www.amazon.com/Beowulf-Computing-Scientific-Engineering-Computation/dp/0262692759/ref=pd_sim_b_5/002-1371173-4594458?ie=UTF8&qid=1175531228&sr=8-1 > > It was actually two books - a "Beowulf Cluster Computing with Linux" and a > "Beowulf Cluster Computing with Windows". 75% of the text was the same. > (We > wrote a chapter in it - we used the same chapter, with latex macros > \iflinux > and \ifwinnt for whichever book was being built) > > The Linux book way outsold the Windows book, and so there was no second > edition of the Windows book. My guess is that most everyone had the good > sense to say "Windows as the base OS for my cluster? No thanks" > > >> > >> > What in your opinion is the future of such clusters? >> > >> > How you compare these with the LINUX CLUSTERS? >> >> You will not find much information on this list as >> it mainly focuses on Linux Beowulf style clusters. >> > > The parallel programming part of this list applies to Windows as > much as it applies to Linux (or FreeBSD or Darwin or HURD) > > -Erik > > > !DSPAM:46113337114068298414181! > -- Doug From rgb at phy.duke.edu Mon Apr 2 11:22:56 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> Message-ID: On Mon, 2 Apr 2007, Douglas Eadline wrote: >> Hi All, >> >> Would any of you please like to share usage-experience/views/comments >> about Windows Compute Cluster Server 2003 based Beowulf Clusters? > > As a point of clarification, there is no such thing > as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" > This link may help: > > http://en.wikipedia.org/wiki/Beowulf_cluster > >> >> What in your opinion is the future of such clusters? >> >> How you compare these with the LINUX CLUSTERS? Allow me to assume the Lotus position (as best as my creaky old joints permit). A few words of mantra to calm my spirit and purge the Daemons that afflict it whenever "Windows" is mentioned. A swig of beer to help calm them still more:-). There. I'm ready now. They compare with linux clusters by being: * Expensive (per seat) for licenses, if one respects that sort of thing. * Heavily commercial in application orientation. * Closed source -- if and where they don't work, you won't be able to fix them, tune them, alter them in any way. * Did I mention expensive? It is alleged that using it, it will be miraculously simpler to set up and configure to turn a pile of PCs into a cluster and then reliably run at least certain classes of largely commercial cluster applications, and that the ease of management will pay for the expense. It is my belief that this allegation is generally false, although in specific cases (e.g. shops that have no linux experience whatsoever and immensely deep pockets) it may be true when the marginal cost of a linux-skilled human (or training up one of your MSCEs to become linux-skilled) exceeds the marginal cost of a Windows cluster compared to a linux cluster. Linux clusters these days can be set up and built with a fairly minimal skillset, so the true economic marginal benefit is dubious. As in I routinely remotely advise students ranging from bright high school students in computer clubs to graduate students in various disciplines (not necessarily computer science) all over the world (for example, I currently have 2-3 groups I am directly advising in India). All of these student groups manage to craft functional linux clusters, often out of old and decrepit parts that (from what I hear) Vista would laugh at hysterically, perhaps, but never run on. If a bright high school student can manage to rig up a working linux cluster, one certainly HOPES that an MCSE can manage it on opportunity cost time in nearly all environments. Furthermore, it is quite trivial to set up linux clusters that boot diskless these days (and hard to beat diskless boot systems for easy of management), and it has finally become fairly easy to set up virtual clusters for embarrassingly parallel programs using e.g. Xen or VMware. One can go to the vmware website and -- for free -- download a windows player and virtual ready to run "instant cluster" environment and run a linux cluster node as a windows subtask. In this way once VMware itself is installed and configured as a good old Windows task there is NO configuration, installation, setup (beyond creating a virtual network so nodes can find one another). One isn't even forced to choose between Win and Lin clusters -- one can do both at the same time and simply decide which one your hardware will be today, with the caveat that node costs scale UP with Win and DOWN with Lin. With all that said, my opinion on the future of Win clusters is that they probably have one. With the advent of Universal System Virtualization (at the CPU hardware level) we are on the threshold of a new era in computing, one where the barrier between operating environments lowers to almost nothing. Microsoft has little choice but to participate in this or risk being badly hurt by free Linux environments that will run Windows as a virtual task "out of the box". They could find themselves in three years as being known as the most expensive linux desktop environment if they don't at least try to make linux one of Windows' least expensive desktop environments (that comes with order of ten to twenty thousand software packages ready to run). And in server environments the pressure is even greater -- people are eager to shrink their server environment to where hardware utilization is optimized instead of having one or more mostly idle servers per application, as is the RULE in Win server shops. Virtualize or die. So we'll just have to see where the chaos this creates takes us. I predict a wild ride for the next 60 months. Single server boxes with 4-8 cores, lots of memory, and running as many as 16 to 20 distinct server environments on top of 2-3 distinct operating systems are not out of the question. Windows cluster environment has the first 24 months of that to grow without that dominating everything, but from 36 months out my crystal ball gets very, very fuzzy. All I can say is that everybody needs to start learning to do their Xen Meditations...;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Mon Apr 2 11:35:30 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> Message-ID: On Mon, 2 Apr 2007, Greg Lindahl wrote: > On Mon, Apr 02, 2007 at 12:55:55PM -0400, Peter St. John wrote: > >> The main reason I know of to use >> MS for a server is to commoditize MS-certified admins, who are cheaper than >> RGB is. > > Not on a per-word basis! rgb is far more productive! And on the list, I work for beer and pretzels. Top that, Microsoft....;-) Off the list I do get a bit more expensive. Greg, the GEEZER daemon met my rgbbot and they ran off together. I'm down to wrting my own txet again, two fingers at a time. They did say before they left that they'd already asked AND answered all the questions that would ever be asked on the ilst, so that the list could go ahead and shut down now. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Mon Apr 2 11:37:28 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: On Mon, 2 Apr 2007, Peter St. John wrote: > The one node refusing to send the doc, and the other note receiving it > anyway, cracked me the *** up! Thanks > > P.S. the Google April Fool's actually got me thinking (network via > plumbing). Water conducts acoustics real well. So a free peer-to-peer > network within a city, or some counties, would be easy and require no new > infrastructure. I imagine the bandwidth would be weak and certainly there'd > be a problem getting between cities, but I don't think the water and sewer > utiltiies claim rights to the acoustic bandwidth (unlike hijacking phone > lines with stuff that would interfere with existing telephony) so all free. Are you seriously suggesting that people should deliberately adopt a shitty network? (:-o> rgb > > On 4/1/07, Douglas Eadline wrote: >> >> >> I just posted some interesting news on Cluster Monkey. >> >> http://www.clustermonkey.net//content/view/192/1/ >> >> >> -- >> Doug >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From deadline at clustermonkey.net Mon Apr 2 11:36:39 2007 From: deadline at clustermonkey.net (Douglas Eadline) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: <46041.192.168.1.1.1175538999.squirrel@mail.eadline.org> kind of gives new meaning to the term dropped packets ... > The one node refusing to send the doc, and the other note receiving it > anyway, cracked me the *** up! Thanks > > P.S. the Google April Fool's actually got me thinking (network via > plumbing). Water conducts acoustics real well. So a free peer-to-peer > network within a city, or some counties, would be easy and require no new > infrastructure. I imagine the bandwidth would be weak and certainly > there'd > be a problem getting between cities, but I don't think the water and sewer > utiltiies claim rights to the acoustic bandwidth (unlike hijacking phone > lines with stuff that would interfere with existing telephony) so all > free. > > On 4/1/07, Douglas Eadline wrote: >> >> >> I just posted some interesting news on Cluster Monkey. >> >> http://www.clustermonkey.net//content/view/192/1/ >> >> >> -- >> Doug >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> > > > !DSPAM:46114154123312004121031! > -- Doug From rgb at phy.duke.edu Mon Apr 2 11:58:13 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> Message-ID: On Mon, 2 Apr 2007, Douglas Eadline wrote: > ".. a collection of personal computers interconnected > by widely available networking technology running anyone of several > open source Unix-like operating systems. " > > I would not want to see it usurped by other clustering efforts. > I believe that if we do not protect against revisionist history, then > all of a sudden WCCS is now "Beowulf" computing. Such things, in my > opinion dis-honor all the people (who I respect) that have contributed > to this community. To me it is almost akin to removing author credit > in open source software. I second the motion. I should point out that this link: http://www.beowulf.org/overview/faq.html#1 Which has been there since Kragen Sitaker put it there back in oh, maybe 1997 or so, and he put it there directly from the original beowulf.org project description. There are good reasons for this to be part of the definition, as well. The Windows cluster is very likely not a >>beowulf<< for a variety of reasons, not just one. It is a form of cluster in a box, but the cluster it makes up seems very likely to be more suited for gridware than "real parallel code", in particular development code. The latter is where having an open source operating system and programming environment becomse so crucial. I can remember any number of times on this list (doubtless preserved in the archives somewhere) where serious problems with various aspects of the network were revealed and (eventually) repaired BECAUSE everything -- kernel, drivers, and application -- were open source. Try doing that with Microsoft's product. Ha. > A short aside. I overheard a conversation at SC-2000 about > the origin of Beowulf from a MS representative "Beowulf > was a copy of the Microsoft Wolfpack software. They chose > that name so it would seem like Wolfpack some how". > Truth is a slippery fish. In the hands of lying bastards it is. But Hitler fully understood the pervasive nature of the big lie, and MS uses it to perfection in their ongoing campaign of FUD. > Certainly Thomas Sterling can rework the definition as he pleases > (he co-authored it), And I am not disparaging WCCS or > any other clustering method. I just want to keep the credit > where credit is due. > > So I stand as a defender of the faith, as it were. And as a defender of the historical record against revisionism. THAT is worth fighting about. rgb > > -- > Doug > >> >> You can build a Beowulf out of Windows. God only knows why you'd want to, >> but >> you can. >> >> Just to invent a little bit of evidence, Thomas Sterling edited a book >> called "Beowulf Cluster Computing with Windows" >> >> http://www.amazon.com/Beowulf-Computing-Scientific-Engineering-Computation/dp/0262692759/ref=pd_sim_b_5/002-1371173-4594458?ie=UTF8&qid=1175531228&sr=8-1 >> >> It was actually two books - a "Beowulf Cluster Computing with Linux" and a >> "Beowulf Cluster Computing with Windows". 75% of the text was the same. >> (We >> wrote a chapter in it - we used the same chapter, with latex macros >> \iflinux >> and \ifwinnt for whichever book was being built) >> >> The Linux book way outsold the Windows book, and so there was no second >> edition of the Windows book. My guess is that most everyone had the good >> sense to say "Windows as the base OS for my cluster? No thanks" >> >> >>>> >>>> What in your opinion is the future of such clusters? >>>> >>>> How you compare these with the LINUX CLUSTERS? >>> >>> You will not find much information on this list as >>> it mainly focuses on Linux Beowulf style clusters. >>> >> >> The parallel programming part of this list applies to Windows as >> much as it applies to Linux (or FreeBSD or Darwin or HURD) >> >> -Erik >> >> >> !DSPAM:46113337114068298414181! >> > > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From landman at scalableinformatics.com Mon Apr 2 11:59:39 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> Message-ID: <4611529B.5080200@scalableinformatics.com> Robert G. Brown wrote: >> Not on a per-word basis! rgb is far more productive! > > And on the list, I work for beer and pretzels. Top that, > Microsoft....;-) You do? (rushes to get an RGB contract in place ....) > > Off the list I do get a bit more expensive. D'oh! Imported beer. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From peter.st.john at gmail.com Mon Apr 2 11:59:54 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: On 4/2/07, Robert G. Brown wrote: > > On Mon, 2 Apr 2007, Peter St. John wrote: > > > The one node refusing to send the doc, and the other note receiving it > > anyway, cracked me the *** up! Thanks > > > > P.S. the Google April Fool's actually got me thinking (network via > > plumbing). Water conducts acoustics real well. So a free peer-to-peer > > network within a city, or some counties, would be easy and require no > new > > infrastructure. I imagine the bandwidth would be weak and certainly > there'd > > be a problem getting between cities, but I don't think the water and > sewer > > utiltiies claim rights to the acoustic bandwidth (unlike hijacking phone > > lines with stuff that would interfere with existing telephony) so all > free. > > Are you seriously suggesting that people should deliberately adopt a > shitty network? > > (:-o> > > rgb Ah, it was just a pipe-dream. Among beowulf-gurus Robert you remain on the throne! (B-{|} (glasses with goatee) Peter > > > On 4/1/07, Douglas Eadline wrote: > >> > >> > >> I just posted some interesting news on Cluster Monkey. > >> > >> http://www.clustermonkey.net//content/view/192/1/ > >> > >> > >> -- > >> Doug > >> _______________________________________________ > >> Beowulf mailing list, Beowulf@beowulf.org > >> To change your subscription (digest mode or unsubscribe) visit > >> http://www.beowulf.org/mailman/listinfo/beowulf > >> > > > > -- > Robert G. Brown http://www.phy.duke.edu/~rgb/ > Duke University Dept. of Physics, Box 90305 > Durham, N.C. 27708-0305 > Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/5df6983e/attachment.html From landman at scalableinformatics.com Mon Apr 2 12:07:08 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> Message-ID: <4611545C.2050907@scalableinformatics.com> Douglas Eadline wrote: > I believe that if we do not protect against revisionist history, then [...] you mean like how now with WCCS2k+3 clustering and HPC is *now* (suddenly magically spontaneously) "mainstream" ? This is just something I personally take issue with. The entire explosive growth of clustering has driven HPC hard into the mainstream. This happened long before it was a glimmer in their eyes. 6+ years of explosive growth, going from noise in the statistics to dominating the statistics. Then along they came with WCCS2k+3. Their entry is late into the cycle. And if you listen to the comments of the senior execs, it makes one wonder how committed they are to HPC and clusters as compared to how committed they are to battling Linux. This is not to diminish their efforts. WCCS2k+3 is likely reasonably good for some subset of groups. Microsoft has some good people there, and playing with the W2k+3 x64 on our JackRabbit unit was fun. They still need a real POSIX subsystem, and hopefully, someday, they will give in, and get cygwin or mingw to be fully supported/shipping using their compilers/tools. Though I expect to see airborn and stable flight from porcine critters about the same time. Too bad, as that would likely ease adoption/porting issues. Tremendously. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From peter.st.john at gmail.com Mon Apr 2 12:23:26 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611545C.2050907@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> Message-ID: A couple weeks ago a kid (by which I mean, energetic person) who makes his living via MS products (as I often do) said that Vista solved the problem of viruses, so there won't be viruses any more. We discussed it (and he deserved some credit for patience, because my intial reaction was overtly dismissive). It turned out that what he meant was that MS has integrated Virus Scanning with the OS, so instead of downloading a dictionary of up-to-date keywords (like "An**na Kourni**kova", I still fear to type her name) for every byte in your system to be matched against, from a 3d party vendor, you will get it automatically when you do your regular, probably automatic, OS update. I remmember when (that AK virus) came out, I read the VBS and wrote up a study of it, which I mailed to a NT Admin friend of mine. HIs corporate firewall bounced it (because it had "that" name in the **subject**, there was NO attachment at all). The Firewall guy replied to me and sent it through to the intended recipient, but I was astonished at what Virus Scanners **really** are. I'm sorry, but I think anyone who would put an OS that integrates all this stuff, onto a compute node in a cluster, is a moron. I'm sorry. Peter On 4/2/07, Joe Landman wrote: > > Douglas Eadline wrote: > > > I believe that if we do not protect against revisionist history, then > > [...] > > you mean like how now with WCCS2k+3 clustering and HPC is *now* > (suddenly magically spontaneously) "mainstream" ? > > This is just something I personally take issue with. The entire > explosive growth of clustering has driven HPC hard into the mainstream. > This happened long before it was a glimmer in their eyes. 6+ years of > explosive growth, going from noise in the statistics to dominating the > statistics. Then along they came with WCCS2k+3. > > Their entry is late into the cycle. And if you listen to the comments > of the senior execs, it makes one wonder how committed they are to HPC > and clusters as compared to how committed they are to battling Linux. > > This is not to diminish their efforts. WCCS2k+3 is likely reasonably > good for some subset of groups. Microsoft has some good people there, > and playing with the W2k+3 x64 on our JackRabbit unit was fun. They > still need a real POSIX subsystem, and hopefully, someday, they will > give in, and get cygwin or mingw to be fully supported/shipping using > their compilers/tools. > > Though I expect to see airborn and stable flight from porcine critters > about the same time. Too bad, as that would likely ease > adoption/porting issues. Tremendously. > > -- > > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics LLC, > email: landman@scalableinformatics.com > web : http://www.scalableinformatics.com > phone: +1 734 786 8423 > fax : +1 734 786 8452 or +1 866 888 3112 > cell : +1 734 612 4615 > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/98665d98/attachment.html From peter.st.john at gmail.com Mon Apr 2 13:15:05 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> Message-ID: If you'd like to keep the history of Beowulfry straight for the coming generation, I urge someone to fix up the reference to "Thomas L Sterling" in the Wikipedia article, http://en.wikipedia.org/wiki/Beowulf_%28computing%29 ; apparently there are two Thomas Sterling's, and the article had had a link to the wrong one, so now there is no link. If you click on the "broken" link (to his name, in the article), it puts you in the editor to create the page (if none exists still, when you do this). Then a minimal article would just be: == Thomas Sterling == Co-author of the seminal work "How to Build a Beowulf" regarding [[Beowulf_%28computing%29| Beowulf]] strategy of cost-effective high-performance computing with commodity hardware and open-source software. That's it. Drop me a line if I can help in any way. I'm sure lot's of you could do great justice to this individual; though I spent hours doing the mindlessly-wiki-compliant article for the late mathematician Leonard Carlitz http://en.wikipedia.org/wiki/Leonard_Carlitz . Just publishing a book is plenty "notability" to satisfy the wiki standards; here you have a whole mailing list of rocket scientists to back you up. Peter "Those who do not know history are doomed to repeat it. Those who do not learn meta-history are doomed to have it reinvented on their heads." -- Euphistopheles On 4/2/07, Douglas Eadline wrote: > > > On Mon, Apr 02, 2007 at 12:12:29PM -0400, Douglas Eadline wrote: > >> > Hi All, > >> > > >> > Would any of you please like to share usage-experience/views/comments > >> > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > >> > >> As a point of clarification, there is no such thing > >> as a "Windows Compute Cluster Server 2003 based Beowulf Clusters" > >> This link may help: > >> > >> http://en.wikipedia.org/wiki/Beowulf_cluster > >> > > > > I don't know why everyone is so obsessed with saying "Your Beowulf must > > run > > an (F)OSS operating system to be a Beowulf." > > Because I believe, "the art of Beowulf" has a rich history of development > that is based on the original definition from "How to Build a Beowulf" by > Thomas Sterling, John Salmon, Donald J. Becker and Daniel F. Savarese: > > ".. a collection of personal computers interconnected > by widely available networking technology running anyone of several > open source Unix-like operating systems. " > > I would not want to see it usurped by other clustering efforts. > I believe that if we do not protect against revisionist history, then > all of a sudden WCCS is now "Beowulf" computing. Such things, in my > opinion dis-honor all the people (who I respect) that have contributed > to this community. To me it is almost akin to removing author credit > in open source software. > > A short aside. I overheard a conversation at SC-2000 about > the origin of Beowulf from a MS representative "Beowulf > was a copy of the Microsoft Wolfpack software. They chose > that name so it would seem like Wolfpack some how". > Truth is a slippery fish. > > Certainly Thomas Sterling can rework the definition as he pleases > (he co-authored it), And I am not disparaging WCCS or > any other clustering method. I just want to keep the credit > where credit is due. > > So I stand as a defender of the faith, as it where. > > -- > Doug > > > > > You can build a Beowulf out of Windows. God only knows why you'd want > to, > > but > > you can. > > > > Just to invent a little bit of evidence, Thomas Sterling edited a book > > called "Beowulf Cluster Computing with Windows" > > > > > http://www.amazon.com/Beowulf-Computing-Scientific-Engineering-Computation/dp/0262692759/ref=pd_sim_b_5/002-1371173-4594458?ie=UTF8&qid=1175531228&sr=8-1 > > > > It was actually two books - a "Beowulf Cluster Computing with Linux" and > a > > "Beowulf Cluster Computing with Windows". 75% of the text was the same. > > (We > > wrote a chapter in it - we used the same chapter, with latex macros > > \iflinux > > and \ifwinnt for whichever book was being built) > > > > The Linux book way outsold the Windows book, and so there was no second > > edition of the Windows book. My guess is that most everyone had the good > > sense to say "Windows as the base OS for my cluster? No thanks" > > > > > >> > > >> > What in your opinion is the future of such clusters? > >> > > >> > How you compare these with the LINUX CLUSTERS? > >> > >> You will not find much information on this list as > >> it mainly focuses on Linux Beowulf style clusters. > >> > > > > The parallel programming part of this list applies to Windows as > > much as it applies to Linux (or FreeBSD or Darwin or HURD) > > > > -Erik > > > > > > !DSPAM:46113337114068298414181! > > > > > -- > Doug > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070402/b9c2597a/attachment.html From rgb at phy.duke.edu Mon Apr 2 13:17:45 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611529B.5080200@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> <4611529B.5080200@scalableinformatics.com> Message-ID: On Mon, 2 Apr 2007, Joe Landman wrote: > Imported beer. ...and those fat, soft pretzels with cheezy-mustard sauce or rolled in asiago parmesan and garlic. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From James.P.Lux at jpl.nasa.gov Mon Apr 2 13:48:46 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> Message-ID: <6.2.3.4.2.20070402134556.02e9e9e8@mail.jpl.nasa.gov> At 11:01 AM 4/2/2007, Douglas Eadline wrote: > > On Mon, Apr 02, 2007 at 12:12:29PM -0400, Douglas Eadline wrote: > >> > Hi All, > >> > >Because I believe, "the art of Beowulf" has a rich history of development >that is based on the original definition from "How to Build a Beowulf" by >Thomas Sterling, John Salmon, Donald J. Becker and Daniel F. Savarese: > > ".. a collection of personal computers interconnected > by widely available networking technology running anyone of several > open source Unix-like operating systems. " > >I would not want to see it usurped by other clustering efforts. >I believe that if we do not protect against revisionist history, then >all of a sudden WCCS is now "Beowulf" computing. Such things, in my >opinion dis-honor all the people (who I respect) that have contributed >to this community. To me it is almost akin to removing author credit >in open source software. I concede that I was incorrect in my contention about Beowulf being just commodity, and not necessarily opensource. I beg forgiveness for a moment of misremembering (my copy of Sterling, et.al., is at home on my bedside table). Mistakes were made, can't we all just along, etc.etc.etc. {I do, however, retain the inclination to play devil's advocate on occasion, just because a good discussion helps crystallize the answers...} James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From James.P.Lux at jpl.nasa.gov Mon Apr 2 13:51:35 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611529B.5080200@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402173425.GA5320@dhcp-2-200.internal.keyresearch.com> <4611529B.5080200@scalableinformatics.com> Message-ID: <6.2.3.4.2.20070402134937.02fce5e8@mail.jpl.nasa.gov> At 11:59 AM 4/2/2007, Joe Landman wrote: >Robert G. Brown wrote: > >>>Not on a per-word basis! rgb is far more productive! >>And on the list, I work for beer and pretzels. Top that, >>Microsoft....;-) > >You do? (rushes to get an RGB contract in place ....) > >>Off the list I do get a bit more expensive. > >D'oh! > >Imported beer. What's considered imported??? Used to be that Coors was an imported brand in the Carolinas.. James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From ballen at gravity.phys.uwm.edu Mon Apr 2 13:58:06 2007 From: ballen at gravity.phys.uwm.edu (Bruce Allen) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] How to Diagnose Cause of Cluster Ethernet Errors? In-Reply-To: <460FE4C9.6040204@berkeley.edu> References: <460AFFA1.6070103@berkeley.edu> <40218.192.168.1.1.1175362730.squirrel@mail.eadline.org> <460FE4C9.6040204@berkeley.edu> Message-ID: Hey Jon, Fun to see you here!! I was just looking through some old Goleta pictures last week. Just for kicks have a look at these figures: http://www.lsc-group.phys.uwm.edu/beowulf/nemo/design/SMC_8508T_Performance.html This was part of a study that we did to select edge switches for the NEMO cluster. We were able to find sub-$100 switches that were wire speed up to MTUs of about 6k. There was a big difference between similar looking cheap switches from various companies. And indeed, 'under the hood' they all used integrated chip sets from a handful of chip vendors. Here are some more testing results from different edge switches: http://www.lsc-group.phys.uwm.edu/beowulf/nemo/design/switching.html (Note: our processing is embarassingly parallel, so we are primarily building compute farms. We don't need very high bandwidth very low latency connections, eg infiniband or myrinet performance.) Cheers, Bruce On Sun, 1 Apr 2007, Jon Forrest wrote: > Douglas Eadline wrote: > >> >> I am constantly amazed at how many people buy the >> latest and greatest node hardware and then connect >> them with a sub-optimal switch (or cheap cables), thus reducing >> the effective performance of the nodes (for parallel >> applications). Kind "penny wise and pound foolish" as they say. >> > > I sincerely appreciate all the comments about my problem. I will reply > to them in due time. However, I'd like to comment on this, which > admittedly is off-topic from my original posting. > > I don't disagree with what you're saying. The problem is how > to recognize "sub-optimal" equipment. For example, I see > three tiers in ethernet switching hardware: > > 1) The low-end, e.g. Netgear, Linksys, D-link, ... > > 2) The mid-end, e.g. HP Procurve, Dell, SMC, ... > > 3) The high-end, e.g. Cisco, Foundry, ... > > What I, as a system manager, not as an Electrical Engineer, > have trouble understanding, is what the true differences > are between these levels, and, at one level, between > the various vendors. > > These days I suspect that many of the vendors are using > ASICs made by other chip companies, and the many vendors > use the same ASICs. Assuming that's true, where's the > added value that justifies the cost differences? Sometimes > the value is in the "management" abilities of a device. > I don't deny this can be a major selling point in a > large enterprise environment, but in a 30-node cluster, > or a small LAN, it's hard to justify paying for this. > > In terms of ethernet performance, once a device > can handle wirespeed communication on all ports, > where's the added value that justifies the added > cost? I'm looking for empirical answers, which > aren't always easy to find, and sometimes to understand. > > In the case of my cluser, it was configured and purchased > before I got here, so I had nothing to do with choosing > its components but I have to admit that I'm not > sure what I would have done differently. > > Cordially, > > Jon Forrest > Unix Computing Support > College of Chemistry > 173 Tan Hall > University of California Berkeley > Berkeley, CA > 94720-1460 > 510-643-1032 > jlforrest@berkeley.edu > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > From mitch48 at sbcglobal.net Mon Apr 2 15:37:55 2007 From: mitch48 at sbcglobal.net (Tom Mitchell) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> Message-ID: <20070402223755.GA24691@xtl1.xtl.tenegg.com> On Mon, Apr 02, 2007 at 03:23:26PM -0400, Peter St. John wrote: > > A couple weeks ago a kid (by which I mean, energetic person) who makes his > living via MS products (as I often do) said that Vista solved the problem of > viruses, so there won't be viruses any more. We discussed it (and he > deserved some credit for patience, because my intial reaction was overtly > dismissive). Sort of off topic to the title but this implies that Vista is fair game to abuse just like any Windows product. .... Date: Fri, 30 Mar 2007 14:47:54 ======= Technical Cyber Security Alert TA07-089A Microsoft Windows ANI header stack buffer overflow Original release date: March 30, 2007 Last revised: -- Source: US-CERT Systems Affected Microsoft Windows 2000, XP, Server 2003, and Vista are affected. Applications that provide attack vectors include: but this implies that Vista .... * Vulnerability Note VU#191609 - * Microsoft Security Advisory (935423) - * Unpatched Drive-By Exploit Found On The Web - -- T o m M i t c h e l l Found me a new place to hang my hat :-) Now it got bought. From mitch48 at sbcglobal.net Mon Apr 2 16:07:21 2007 From: mitch48 at sbcglobal.net (Tom Mitchell) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> Message-ID: <20070402230721.GB24691@xtl1.xtl.tenegg.com> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: > Date: Sun, 1 Apr 2007 14:02:53 +0500 > From: "amjad ali" > To: beowulf@beowulf.org > Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > Hi All, > > Would any of you please like to share usage-experience/views/comments > about Windows Compute Cluster Server 2003 based Beowulf Clusters? > > What in your opinion is the future of such clusters? > > How you compare these with the LINUX CLUSTERS? With full consideration to "fat, soft pretzels with cheezy-mustard sauce or rolled in asiago parmesan and garlic." MS pulled a version of mpich/mvapich/MPI and ported it to windows. They also developed some library code to gateway some *inx library/system calls to windows. The root sources of MPI are public and not GPL so they can. It might be worth looking at the MS announcement -- but why bother. If you look you might think that common MPI codes would just compile and run... I have no idea I expect some will and there begins silly porting for the next... Once a set of boxes are interconnected and you have library support for MPI or another way to share data (PVM... whatever) you are off and running in the clustering world. Sadly MS has a MS specific library that abuses "standard MPI" and could quickly cause source code to surface that runs correctly or on a MS cluster but not on another OS based cluster (Linux, Solaris, Irix, AIX). I see this all the time with java script, and c, c++, and other codes where little 'features' hook you in. Some will be fooled into thinking that this is something to look at or worse something to spend money on. SUMMARY: Since you posted this on 1 Apr 2007 all I can do is giggle and wonder why I replied. Regards, mitch PS: Ask in a year but not on April fools/joke day. -- T o m M i t c h e l l Found me a new place to hang my hat :-) Now it got bought. From rgb at phy.duke.edu Mon Apr 2 17:21:37 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070402230721.GB24691@xtl1.xtl.tenegg.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <20070402230721.GB24691@xtl1.xtl.tenegg.com> Message-ID: On Mon, 2 Apr 2007, Tom Mitchell wrote: > On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >> Date: Sun, 1 Apr 2007 14:02:53 +0500 >> From: "amjad ali" >> To: beowulf@beowulf.org >> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >> >> Hi All, >> >> Would any of you please like to share usage-experience/views/comments >> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >> >> What in your opinion is the future of such clusters? >> >> How you compare these with the LINUX CLUSTERS? > > With full consideration to "fat, soft pretzels with > cheezy-mustard sauce or rolled in asiago parmesan and garlic." > > MS pulled a version of mpich/mvapich/MPI and ported it to windows. > They also developed some library code to gateway some *inx library/system > calls to windows. The root sources of MPI are public and not GPL so they can. > > It might be worth looking at the MS announcement -- but why > bother. If you look you might think that common MPI codes > would just compile and run... I have no idea I expect some will > and there begins silly porting for the next... Sure. MS did this, no doubt. And as you note below, no sooner do they get it in when they begin the borgification of MPI, just as they've borgified java, c, c++, and anything in the Universe they can sucker somebody into buying in borgified form. Borgifying MPI is the most humorous thing in the Universe, BTW, given its historical origins -- it was basically a language written (reluctantly!) by supercomputer vendors when the US government got tired of paying for all their important codes to be ported to each new generation of proprietary hardware with its proprietary low level calls. MS is doubtless trying to figure out just how much of that they can undo while building up a big enough market share and enough vendors of closed source applications written with their borgisms that they can... Oh wait. It IS GPL. Do you think that they actually read it? However, I was really referring to the other aspects of program development and performance tuning associated with using a closed source development environment. Resistance is Futile. rgb > > Once a set of boxes are interconnected and you have library > support for MPI or another way to share data (PVM... whatever) > you are off and running in the clustering world. Sadly MS > has a MS specific library that abuses "standard MPI" and could > quickly cause source code to surface that runs correctly or on a > MS cluster but not on another OS based cluster (Linux, Solaris, > Irix, AIX). I see this all the time with java script, and c, > c++, and other codes where little 'features' hook you in. > > Some will be fooled into thinking that this is something to look at > or worse something to spend money on. > > SUMMARY: > Since you posted this on 1 Apr 2007 all I can do is giggle > and wonder why I replied. > > Regards, > mitch > > PS: Ask in a year but not on April fools/joke day. > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From gerry.creager at tamu.edu Tue Apr 3 06:07:26 2007 From: gerry.creager at tamu.edu (Gerry Creager) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> Message-ID: <4612518E.9050003@tamu.edu> Peter St. John wrote: > The one node refusing to send the doc, and the other note receiving it > anyway, cracked me the *** up! Thanks > > P.S. the Google April Fool's actually got me thinking (network via > plumbing). Water conducts acoustics real well. So a free peer-to-peer > network within a city, or some counties, would be easy and require no > new infrastructure. I imagine the bandwidth would be weak and certainly > there'd be a problem getting between cities, but I don't think the water > and sewer utiltiies claim rights to the acoustic bandwidth (unlike > hijacking phone lines with stuff that would interfere with existing > telephony) so all free. In general, water conduction requires a continuous column of water unless you're willing to overdrive the signal to allow it to modulate an intermediate air column. The presence of "solids" also modifies the index of refraction and can induce standing waves and cause you to flush your signal to noise ratio. Also, the use of lift stations to continue a sanitary sewer system's contents toward their ultimate destination (A water treatment plant [central routing factility?]) would require significant effort in order to synchronize the lift station's mechanical interfaces with the modulated waveforms in the sewer pipes. For these reasons, use of a sanitary sewer system is likely impractical. Of course, one DOES have a continuous column of water in city distribution systems. This opens all sorts of possibilities for other networking benefits. First, however, let's review some of the drawbacks: 1. Most water distribution pipes are buried at some point in their transit from reservoir to user. This tends to dampen their movement and that of their relatively incompressible content, by decreasing the pipe's ability to expand and contract. Thus, dynamic range is reduced. 2. Except for those about to rupture, copper water pipe tends to be relatively rigid, and its length tends to act as a low-pass filter. Combined with the multitude of joints in 10-foot pipe sections, and ignoring for the moment the potential to use longer copper tubing extruded and packaged as 50 foot rolls, this would represent a high-order filter with a high Q value. Calculating the resonant frequency of such a filter is left to the student. 3. A large number of newer construction commercial and residential structures, as well as those which have undergone plumbing renovation in the last 10 years or so, will have internal piping of CPVC or PEX origin. The mechanical interface to the copper transmission line... er, piping... would require significant impedance matching to diminish an additional standing wave introduction that could dampen audio frequency amplitude below the threshold of detection. 4. Water mains made of virtually any material are effectively rigid when placed in the ground. Thus, water (incompressible for the sake of this discussion) would be required to store its acoustic energy within the liquid matrix, or would have to transfer that energy to to velocity modifications, thus making shaving and showing inconvenient. 5. Because we are, obviously, discussing a service oriented architecture, the concept of widespread use and availability to a large number of users should be obvious. In this regard, significant work will be required to engender a new form of liquid wave division multiplexing to allow carrying more than one signal at a time. This ties to #2 above in that, to be efficient, the filter elements should be dynamic and self-modifying. It's been my experience that water piping tends to be self-modifying only during periods of extended and significant cold weather here in Texas. However, as said pipes tend to lose their structural integrity and contents in this event, message reliability seems to suffer in the face of self-modifying filter elements. On the positive side, large lakes (surface reservoirs) could be used as capacitive elements, tending to dampen the amplitude of an over-modulated signal. Driving the signal to saturation and beyond is a noted problem for teenage users and network component salesmen. Standpipes and other overhead storage containers could serve, once modulated appropriately, as broadcasting repeaters (a switching mechanism that takes advantage of the WWDM [water wave division multiplexing] scheme has yet to be determined) to support multicast or anycast operation. Finally, complete documentation would have to be generated, and rgb or one of his clones has not stepped forward to offer such assistance. As he (or one of his clones) usually anticipates major innovations in technology and pre-writes said documentation, this casts a pall on the potential viability of the concept. gerry > On 4/1/07, *Douglas Eadline* > wrote: > > > I just posted some interesting news on Cluster Monkey. > > http://www.clustermonkey.net//content/view/192/1/ > > > -- > Doug > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Gerry Creager -- gerry.creager@tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From gerry.creager at tamu.edu Tue Apr 3 06:18:04 2007 From: gerry.creager at tamu.edu (Gerry Creager) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611545C.2050907@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> Message-ID: <4612540C.1030407@tamu.edu> I'm reminded here of an advertisement proposed, but never actually published, for Data General back in its day... "According to IBM, their entry into the minicomputer marketplace has legitimized it. The Bastards say, 'Welcome!'"... gerry Joe Landman wrote: > Douglas Eadline wrote: > >> I believe that if we do not protect against revisionist history, then > > [...] > > you mean like how now with WCCS2k+3 clustering and HPC is *now* > (suddenly magically spontaneously) "mainstream" ? > > This is just something I personally take issue with. The entire > explosive growth of clustering has driven HPC hard into the mainstream. > This happened long before it was a glimmer in their eyes. 6+ years of > explosive growth, going from noise in the statistics to dominating the > statistics. Then along they came with WCCS2k+3. > > Their entry is late into the cycle. And if you listen to the comments > of the senior execs, it makes one wonder how committed they are to HPC > and clusters as compared to how committed they are to battling Linux. > > This is not to diminish their efforts. WCCS2k+3 is likely reasonably > good for some subset of groups. Microsoft has some good people there, > and playing with the W2k+3 x64 on our JackRabbit unit was fun. They > still need a real POSIX subsystem, and hopefully, someday, they will > give in, and get cygwin or mingw to be fully supported/shipping using > their compilers/tools. > > Though I expect to see airborn and stable flight from porcine critters > about the same time. Too bad, as that would likely ease > adoption/porting issues. Tremendously. > -- Gerry Creager -- gerry.creager@tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From bill at platform.com Tue Apr 3 07:00:17 2007 From: bill at platform.com (Bill Bryce) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! Message-ID: Regarding the 'borgification' of MPI on Windows....there is an opening for them to do this. MPI did not define how you start the tasks when you launch a parallel job....it left that up to the MPI implementation. Now most MPIs use something reasonable, like say ssh - or starting a mpd ring to launch the tasks (and this is all open source), however I believe that for Microsoft MPI they wrote their own replacement for 'ssh' and also modified how the tasks are launched to ensure their MPI fits into Microsoft's security model, here is specific information on this from Microsoft's site "An important improvement that MS MPI brings to MPICH2 is in how security is managed during execution of MPI jobs. Each job is run with the credentials of the user. Credentials are only present while the job is being executed on the compute nodes and are erased when the job completes. Individual processes only have access to their logon token, and do not have access to the credentials (user name and password) that generated the token or to the credentials used by other processes." So this change effectively ties Microsoft MPI to only Microsoft Windows platforms, and the security changes are closed source. Not all of Microsoft's partners like MS MPI - when HP ships Microsoft CCS they remove MS MPI and put in HP MPI - which probably just adds to confusion on the end user side. Regards, Bill. -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Monday, April 02, 2007 8:22 PM To: Tom Mitchell Cc: beowulf@beowulf.org Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Mon, 2 Apr 2007, Tom Mitchell wrote: > On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >> Date: Sun, 1 Apr 2007 14:02:53 +0500 >> From: "amjad ali" >> To: beowulf@beowulf.org >> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >> >> Hi All, >> >> Would any of you please like to share usage-experience/views/comments >> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >> >> What in your opinion is the future of such clusters? >> >> How you compare these with the LINUX CLUSTERS? > > With full consideration to "fat, soft pretzels with > cheezy-mustard sauce or rolled in asiago parmesan and garlic." > > MS pulled a version of mpich/mvapich/MPI and ported it to windows. > They also developed some library code to gateway some *inx library/system > calls to windows. The root sources of MPI are public and not GPL so they can. > > It might be worth looking at the MS announcement -- but why > bother. If you look you might think that common MPI codes > would just compile and run... I have no idea I expect some will > and there begins silly porting for the next... Sure. MS did this, no doubt. And as you note below, no sooner do they get it in when they begin the borgification of MPI, just as they've borgified java, c, c++, and anything in the Universe they can sucker somebody into buying in borgified form. Borgifying MPI is the most humorous thing in the Universe, BTW, given its historical origins -- it was basically a language written (reluctantly!) by supercomputer vendors when the US government got tired of paying for all their important codes to be ported to each new generation of proprietary hardware with its proprietary low level calls. MS is doubtless trying to figure out just how much of that they can undo while building up a big enough market share and enough vendors of closed source applications written with their borgisms that they can... Oh wait. It IS GPL. Do you think that they actually read it? However, I was really referring to the other aspects of program development and performance tuning associated with using a closed source development environment. Resistance is Futile. rgb > > Once a set of boxes are interconnected and you have library > support for MPI or another way to share data (PVM... whatever) > you are off and running in the clustering world. Sadly MS > has a MS specific library that abuses "standard MPI" and could > quickly cause source code to surface that runs correctly or on a > MS cluster but not on another OS based cluster (Linux, Solaris, > Irix, AIX). I see this all the time with java script, and c, > c++, and other codes where little 'features' hook you in. > > Some will be fooled into thinking that this is something to look at > or worse something to spend money on. > > SUMMARY: > Since you posted this on 1 Apr 2007 all I can do is giggle > and wonder why I replied. > > Regards, > mitch > > PS: Ask in a year but not on April fools/joke day. > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From landman at scalableinformatics.com Tue Apr 3 07:00:23 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4612518E.9050003@tamu.edu> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> Message-ID: <46125DF7.1070108@scalableinformatics.com> Gerry Creager wrote: > In general, water conduction requires a continuous column of water > unless you're willing to overdrive the signal to allow it to modulate an > intermediate air column. The presence of "solids" also modifies the > index of refraction and can induce standing waves and cause you to flush > your signal to noise ratio. Also, the use of lift stations to continue What occurs to me (pre-coffee) is that one could curse their network as being made of ... uh ... (in part) fecal material ... and be literally correct. Of course such cursing would occur when "packets" didn't flow as well as they should ... -- Joe Landman landman |at| scalableinformatics |dot| com From rgb at phy.duke.edu Tue Apr 3 07:29:23 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4612518E.9050003@tamu.edu> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> Message-ID: On Tue, 3 Apr 2007, Gerry Creager wrote: > Finally, complete documentation would have to be generated, and rgb or one of > his clones has not stepped forward to offer such assistance. As he (or one > of his clones) usually anticipates major innovations in technology and > pre-writes said documentation, this casts a pall on the potential viability > of the concept. Not so. In fact, my bots finished the documentation six years ago in anticipation of this day. However, the documentation is not being >>released<< because very soon now the idea of sewer based transit will be revived (making the water-pipe idea obsolete) as you forgot one key factor. You see, we are about to discover how to encode data on DNA, which (as a direct molecular encoding) is extremely efficient -- multiple terabytes of data can be stored in a gram of pure DNA. You may not be aware of it, but excrement is about 30% dead bacteria. There is a top-secret team working hard on a special (somewhat invasive) "interface" that will reach right into your bowel and perform recombinant DNA activity in real-time to place messages right into the DNA of e.g. E. Coli. Once encoded and packetized, the message will be placed into a data buffer using traditional methods and flushed through into the fat pipe that leads to the data consumer. The consumer, in turn, will have to process the message to extract the highly redundant bacterial packets, use little-bitty shears to cut the base pairs in just the right places to extract the message while throwing away the irrelevant message headers that (we can imagine) comprise the bacteria's "time to live" data, PCR up the packet contents and decode the slightly compressed base 4 packet contents. A bit of packet resequencing and the full message can easily be reconstructed, hopefully without a handshake. As an incidental bonus, java is well known to stimulate an increase in message production! Once the message is digested, the process >>can<< be repeated the other way -- full duplex communications are certainly possible although at the moment I don't think the protocols for message collisions have been properly defined. The protocol is still a bit messy, in other words, and it isn't clear what one has to do if the message collides with a propeller-based bridging device that is designed to keep one's data flow moving (already being called "the fan" by the data teams working on the routing and retransmission of packetized messages). Another problem these teams are dealing with is the nontrivial risk that viruses can be transmitted along with the message, but antiviral solutions are readily available over the counter for a few dollars. One of the primary advantages being touted for the protocol is that it should once and for all solve the problem of SPAM. Bacteria won't eat it. Overall, the protocol is a great favorite of the "green" crowd as all packets are ultimately biodegradable and recylable, although they do result in an unfortunate co-production of methane, a well-known greenhouse gas. Anyway, these remaining issues are doubtless easy to resolve, and what the protocol lacks in latency it will make up in bandwidth. I can definitely think of several groups whose attitude toward the world could only be improved by giving them their very own messaging interface as a permanent insertion. Consumers (especially government consumers) are standing by eager to adopt the new technology. Although sending messages will require a significant infrastructure investments running network conduit from office to office and installing appropriate interface hardware in all of the office workstation chairs, the messages thus sent within, say, the Pentagon, will be extremely secure -- Islamic terrorists, for example, won't be able to touch them. Congress likes the idea because the introduction of a new (already patented, sorry) technology like this clearly benefits US hardware manufacturers hitherto ignored in the general technology boom and will doubtless require immense amounts of highly sophisticated and proprietary software that only one company in the world is in a position to write. I understand that Microsoft is working hard on building the software tools to encourage the spreading of the technology within certain fertile fields of human endeavor, and there is a rumor that they have a plan already in place to smear existing networking protocols to where nobody will want to handle them any more. I just hope that it isn't a case of Microsoft being willing to "eat its own dog-FUD" here -- I think that the ASPCA will have something to say about any attempt to use canines in place of data tech workers and interface units. Finally, although it is currently top-secret (and I only know about it because of the transluminal nature of the offspring of the GEEZER daemon and rgbbot that is actually typing this message, rgb being away on vacation and absolutely not responsible for its contents as he can easily prove -- he has long since washed his hands of the entire discussion), certain unspecified but VERY highly ranked members of our government wish to use it as a secured channel for discussing the war in Iraq. They >>will<< have a bit of difficulty with the interface insertion as I believe they are currently using the requisite socket for their head nodes, but I'm sure that this technical difficulty will soon be resolved, if necessary by increasing the width of the data path in some way. And now I I mean my bots, must stop. rgb > > gerry > >> On 4/1/07, *Douglas Eadline* > > wrote: >> >> >> I just posted some interesting news on Cluster Monkey. >> >> http://www.clustermonkey.net//content/view/192/1/ >> >> >> -- >> Doug >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Beowulf mailing list, Beowulf@beowulf.org >> To change your subscription (digest mode or unsubscribe) visit >> http://www.beowulf.org/mailman/listinfo/beowulf > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Tue Apr 3 07:58:55 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: On Tue, 3 Apr 2007, Bill Bryce wrote: > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. Security changes? To MPI itself? This puts them into a somewhat grey area with regard to the GPL, doesn't it... that viral thing. I'm sure that they can manage the remote job launch any way that they like, but doesn't that leave the MPI CODE still portable? It isn't completely crazy that the US government would intervene here if they broke MPI portability. After all, MPI exists at all primarily because of direct government intervention (unlike PVM, which exists because some Very Bright People conceived it and invented it and made it fly). Still, nothing the Borg Empire does in this regard would surprise me. rgb > > Regards, > > Bill. > > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Robert G. Brown > Sent: Monday, April 02, 2007 8:22 PM > To: Tom Mitchell > Cc: beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > On Mon, 2 Apr 2007, Tom Mitchell wrote: > >> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >>> Date: Sun, 1 Apr 2007 14:02:53 +0500 >>> From: "amjad ali" >>> To: beowulf@beowulf.org >>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >>> >>> Hi All, >>> >>> Would any of you please like to share usage-experience/views/comments >>> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >>> >>> What in your opinion is the future of such clusters? >>> >>> How you compare these with the LINUX CLUSTERS? >> >> With full consideration to "fat, soft pretzels with >> cheezy-mustard sauce or rolled in asiago parmesan and garlic." >> >> MS pulled a version of mpich/mvapich/MPI and ported it to windows. >> They also developed some library code to gateway some *inx > library/system >> calls to windows. The root sources of MPI are public and not GPL so > they can. >> >> It might be worth looking at the MS announcement -- but why >> bother. If you look you might think that common MPI codes >> would just compile and run... I have no idea I expect some will >> and there begins silly porting for the next... > > Sure. MS did this, no doubt. And as you note below, no sooner do they > get it in when they begin the borgification of MPI, just as they've > borgified java, c, c++, and anything in the Universe they can sucker > somebody into buying in borgified form. > > Borgifying MPI is the most humorous thing in the Universe, BTW, given > its historical origins -- it was basically a language written > (reluctantly!) by supercomputer vendors when the US government got tired > of paying for all their important codes to be ported to each new > generation of proprietary hardware with its proprietary low level calls. > MS is doubtless trying to figure out just how much of that they can > undo while building up a big enough market share and enough vendors of > closed source applications written with their borgisms that they can... > > Oh wait. It IS GPL. Do you think that they actually read it? > > However, I was really referring to the other aspects of program > development and performance tuning associated with using a closed source > development environment. > > Resistance is Futile. > > rgb > >> >> Once a set of boxes are interconnected and you have library >> support for MPI or another way to share data (PVM... whatever) >> you are off and running in the clustering world. Sadly MS >> has a MS specific library that abuses "standard MPI" and could >> quickly cause source code to surface that runs correctly or on a >> MS cluster but not on another OS based cluster (Linux, Solaris, >> Irix, AIX). I see this all the time with java script, and c, >> c++, and other codes where little 'features' hook you in. >> >> Some will be fooled into thinking that this is something to look at >> or worse something to spend money on. >> >> SUMMARY: >> Since you posted this on 1 Apr 2007 all I can do is giggle >> and wonder why I replied. >> >> Regards, >> mitch >> >> PS: Ask in a year but not on April fools/joke day. >> >> > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From hahn at mcmaster.ca Tue Apr 3 08:14:05 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: on the original topic, to me, Beowulf is about cluster hacking. that is, a slightly subversive re-purposing to suit HPC. all hacking is based on something like a paradox, that if you simply break convention, there's a better way to do it. so rather than buy high-end supers and mini computers, stick a bunch of glorified desktops together for the same calc. so on the topic of "why not windows", I can't really see the point - is there any value to repurposing windows? I very much doubt it would give any better performance, and windows' main advantage in the market is that you can get cheap if mediocre admins. > 'ssh' and also modified how the tasks are launched to ensure their MPI > fits into Microsoft's security model, here is specific information on > this from Microsoft's site . it's natural that if you run windows apps on a cluster, you're a site which is an msft shop and obviously want to know that the jobs fit within your AD infrastructure, etc. of course, there's nothing that requires the cluster to run windows to do this - a pure beowulf cluster _could_ use AD and SMB filesystems without any problem I know of. if your goal is really to drive a cluster from an excel spreadsheet, I don't imagine it's all that hard - especially if you simply have the spreadsheet trigger some jobs via ssh keys. no new infrastructure required. > "An important improvement that MS MPI brings to MPICH2 is in how > security is managed during execution of MPI jobs. Each job is run with > the credentials of the user. Credentials are only present while the job this is, of course, ASININE fud. msft just isn't very good at producing beowulf fud (yet, I fear). it's probably good enough fud to scare PHB's. if msft really cared about this sort of credential/token/etc stuff, they'd probably organize the cluster to provision VM's on each node, each of which only has access to a specific private VPN. such plumbing would be moderately messy to set up, and IMO not worth the trouble. From peter.st.john at gmail.com Tue Apr 3 08:16:17 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> Message-ID: [RGB]... as I believe they are currently using the requisite socket for their head nodes ... I'm spitting up in my coffee. I had just been thinking, "I hope Creager and Brown don't resonate over this, it could make a great SF novel or destroy all life as far out as Tau Epsilon". Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/54b71a9d/attachment.html From toon.knapen at fft.be Tue Apr 3 08:21:17 2007 From: toon.knapen at fft.be (Toon Knapen) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: <461270ED.8000107@fft.be> Robert G. Brown wrote: > It isn't completely crazy that the US government would intervene here if > they broke MPI portability. After all, MPI exists at all primarily > because of direct government intervention (unlike PVM, which exists > because some Very Bright People conceived it and invented it and made it > fly). It's not up to the government to decide if MS can put some product on the market or not. They can indeed refuse to finance porting code to CCS because of its incompatibilities. toon From rgb at phy.duke.edu Tue Apr 3 09:00:36 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461270ED.8000107@fft.be> References: <461270ED.8000107@fft.be> Message-ID: On Tue, 3 Apr 2007, Toon Knapen wrote: > Robert G. Brown wrote: > >> It isn't completely crazy that the US government would intervene here if >> they broke MPI portability. After all, MPI exists at all primarily >> because of direct government intervention (unlike PVM, which exists >> because some Very Bright People conceived it and invented it and made it >> fly). > > > It's not up to the government to decide if MS can put some product on the > market or not. They can indeed refuse to finance porting code to CCS because > of its incompatibilities. The way they intervened in the case of MPI (IIRC) is they said something like "we will not buy supercomputers from any vendor that does not also provide a portable message passing environment". At the time, they were "the" market for supercomputers -- at least a sine qua non of survival (given that supercomputers were munitions and controlled). So the vendors of hardware and a team of academic and lab types wrote up MPI and made sure MPI would run reasonably portably across all their big iron hardware. In the current case if the government says "we will not fund any cluster and research software development that uses non-portable constructs in their MPI implementation" I think that it would have the same effect. rgb > > toon > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From greg.lindahl at qlogic.com Tue Apr 3 10:49:40 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: <20070403174940.GD5528@dhcp-2-200.internal.keyresearch.com> On Tue, Apr 03, 2007 at 10:58:55AM -0400, Robert G. Brown wrote: > Security changes? To MPI itself? This puts them into a somewhat grey > area with regard to the GPL, doesn't it... that viral thing. MPICH is BSD licensed. Quite a few closed source MPIs are based on it. -- greg From toon.knapen at fft.be Tue Apr 3 11:12:27 2007 From: toon.knapen at fft.be (Toon Knapen) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <461270ED.8000107@fft.be> Message-ID: <4612990B.6000001@fft.be> Robert G. Brown wrote: > > In the current case if the government says "we will not fund any cluster > and research software development that uses non-portable constructs in > their MPI implementation" But their (MPI) API is still standard conforming. The way they integrated it (in the OS) is a bit different. IIUC they have their own queue-ing system on top (or integrated). But OTOH neither SGE or PBS or whatever is standard. So from a programmer's POV, they conform to the MPI standard. From the user's POV (how to launch a job or how to submit a job to a queue) there was never any standard. toon From 3lucid at gmail.com Tue Apr 3 11:26:17 2007 From: 3lucid at gmail.com (Kyle Spaans) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> Message-ID: <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> On a related, yet more serious note: Would something like this: http://www.killernic.com/KillerNic/ Be applicable to beowulf? Could the onboard Linux system help to prioritize MPI traffic? Or is it kind of pointless when you can just buy extra NICs for $10 - instead of $200 for one of these babies - to use for non-MPI traffic? From greg.lindahl at qlogic.com Tue Apr 3 12:09:01 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> Message-ID: <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> On Tue, Apr 03, 2007 at 02:26:17PM -0400, Kyle Spaans wrote: > On a related, yet more serious note: > Would something like this: > http://www.killernic.com/KillerNic/ > Be applicable to beowulf? Now that's weird: * it's an offload engine * it claims to "lower your ping" i.e. help short messages * it claims to let you play while downloading torrents #2 is hard, #3 is harder, as traffic shaping isn't very effective against torrents. -- greg From peter.st.john at gmail.com Tue Apr 3 12:38:28 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> Message-ID: Gaming driving supercomputing still makes me feel funny (but then, Ken Thompson and Belle so...). But I'm picturing ganglionic head nodes with KillerNic and playstations for compute nodes... Peter On 4/3/07, Greg Lindahl wrote: > > On Tue, Apr 03, 2007 at 02:26:17PM -0400, Kyle Spaans wrote: > > > On a related, yet more serious note: > > Would something like this: > > http://www.killernic.com/KillerNic/ > > Be applicable to beowulf? > > Now that's weird: > > * it's an offload engine > * it claims to "lower your ping" i.e. help short messages > * it claims to let you play while downloading torrents > > #2 is hard, #3 is harder, as traffic shaping isn't very effective > against torrents. > > -- greg > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/9deeee28/attachment.html From jamesjamiejones at aol.com Tue Apr 3 12:03:38 2007 From: jamesjamiejones at aol.com (matt jones) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4612A4BC.5010806@aol.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> Message-ID: <4612A50A.8020108@aol.com> An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/0e67b51a/attachment.html From bill at platform.com Tue Apr 3 13:11:46 2007 From: bill at platform.com (Bill Bryce) Date: Tue Mar 16 01:05:58 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! Message-ID: Can I ask one simple question..... Why did Microsoft have to do this (and I quote from MSDN) "Every job and task that uses Microsoft MPI must be submitted through the Microsoft job scheduler. There are several ways to submit jobs and tasks to the job scheduler:" I do not have that restriction with other MPI's such as MPICH2 for Windows from Argonne. Regards, Bill. -----Original Message----- From: John Vert [mailto:jvert@windows.microsoft.com] Sent: Tuesday, April 03, 2007 3:19 PM To: Robert G. Brown; Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! I'd like to clear up a few misconceptions on this thread. 1. There is no "MS specific library that abuses 'standard MPI'". The #1 goal for our MPI implementation is to make it as simple as possible for software vendors to port their codes to Windows and our MPI stack. That is why we based our implementation on MPICH2. The feedback we have gotten from software vendors has shown this approach was successful and we have had very few issues with MPI incompatibilities. We have not added anything outside the MPI spec. 2. MPICH2 is not licensed under GPL or under BSD. See http://www-unix.mcs.anl.gov/mpi/mpich2/license.htm Many commercial closed-source MPI implementations have been based on MPICH. Microsoft is actively working with Argonne on incorporating the changes we made back into the MPICH2 distribution. Argonne has been great to work with and really supportive of our efforts. 3. The changes we made can be loosely grouped into three areas. Performance, security, and scheduler integration. We handle the user credentials differently than Argonne's Windows implementation which is different than Argonne's Linux implementation due to some fundamental differences between the two operating systems. The scheduler integration is pretty straightforward - things like cleaning up all MPI child processes when a job is cancelled, and automatically propagating things like the allocated nodes and the number of CPUs from the scheduler to mpiexec. I don't think this is significantly different than what other job schedulers have done. 4. If you want to learn more about Windows HPC clusters, I recommend checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also happy to answer questions on this list, but frankly the S/N ratio tends to drop dramatically as soon as someone mentions Windows or Microsoft. John Vert Development Manager High Performance Computing -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Tuesday, April 03, 2007 7:59 AM To: Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Tue, 3 Apr 2007, Bill Bryce wrote: > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. Security changes? To MPI itself? This puts them into a somewhat grey area with regard to the GPL, doesn't it... that viral thing. I'm sure that they can manage the remote job launch any way that they like, but doesn't that leave the MPI CODE still portable? It isn't completely crazy that the US government would intervene here if they broke MPI portability. After all, MPI exists at all primarily because of direct government intervention (unlike PVM, which exists because some Very Bright People conceived it and invented it and made it fly). Still, nothing the Borg Empire does in this regard would surprise me. rgb > > Regards, > > Bill. > > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Robert G. Brown > Sent: Monday, April 02, 2007 8:22 PM > To: Tom Mitchell > Cc: beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > On Mon, 2 Apr 2007, Tom Mitchell wrote: > >> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >>> Date: Sun, 1 Apr 2007 14:02:53 +0500 >>> From: "amjad ali" >>> To: beowulf@beowulf.org >>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >>> >>> Hi All, >>> >>> Would any of you please like to share usage-experience/views/comments >>> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >>> >>> What in your opinion is the future of such clusters? >>> >>> How you compare these with the LINUX CLUSTERS? >> >> With full consideration to "fat, soft pretzels with >> cheezy-mustard sauce or rolled in asiago parmesan and garlic." >> >> MS pulled a version of mpich/mvapich/MPI and ported it to windows. >> They also developed some library code to gateway some *inx > library/system >> calls to windows. The root sources of MPI are public and not GPL so > they can. >> >> It might be worth looking at the MS announcement -- but why >> bother. If you look you might think that common MPI codes >> would just compile and run... I have no idea I expect some will >> and there begins silly porting for the next... > > Sure. MS did this, no doubt. And as you note below, no sooner do they > get it in when they begin the borgification of MPI, just as they've > borgified java, c, c++, and anything in the Universe they can sucker > somebody into buying in borgified form. > > Borgifying MPI is the most humorous thing in the Universe, BTW, given > its historical origins -- it was basically a language written > (reluctantly!) by supercomputer vendors when the US government got tired > of paying for all their important codes to be ported to each new > generation of proprietary hardware with its proprietary low level calls. > MS is doubtless trying to figure out just how much of that they can > undo while building up a big enough market share and enough vendors of > closed source applications written with their borgisms that they can... > > Oh wait. It IS GPL. Do you think that they actually read it? > > However, I was really referring to the other aspects of program > development and performance tuning associated with using a closed source > development environment. > > Resistance is Futile. > > rgb > >> >> Once a set of boxes are interconnected and you have library >> support for MPI or another way to share data (PVM... whatever) >> you are off and running in the clustering world. Sadly MS >> has a MS specific library that abuses "standard MPI" and could >> quickly cause source code to surface that runs correctly or on a >> MS cluster but not on another OS based cluster (Linux, Solaris, >> Irix, AIX). I see this all the time with java script, and c, >> c++, and other codes where little 'features' hook you in. >> >> Some will be fooled into thinking that this is something to look at >> or worse something to spend money on. >> >> SUMMARY: >> Since you posted this on 1 Apr 2007 all I can do is giggle >> and wonder why I replied. >> >> Regards, >> mitch >> >> PS: Ask in a year but not on April fools/joke day. >> >> > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From peter.st.john at gmail.com Tue Apr 3 13:49:11 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: John, Thank you for "... 4. If you want to learn more about Windows HPC clusters, I recommend checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also happy to answer questions on this list, but frankly the S/N ratio tends to drop dramatically as soon as someone mentions Windows or Microsoft. ..." How large is the OS on an idle (but ready) compute node in MS's cluster system? Reasonable possible responses would include that the question is ill-posed. I would appreciate any response. Here's for d/dt (S/N) > 0 Thanks, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/a22a500d/attachment.html From hahn at mcmaster.ca Tue Apr 3 14:16:51 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4612A50A.8020108@aol.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> <4612A50A.8020108@aol.com> Message-ID: > you say that, but don't PS2 and PS3 make ideal compute nodes for some > applications like rendering and 3D and 4D (time as the 4th) ? I doubt it. while gaming consoles are admirably commoditized (actually sold at a loss in some cases), they're designed with constraints different from a cluster. for instance, looking pretty, or working well in a livingroom environment. or playing music, or reading hd-dvd's, etc. as render nodes, I don't think they're all that good, though perhaps effective for the price. I would guess that for rendering, you would do the very best to use last-gen cards in the $150 range. should be able to put two in a machine - sli support not necessary. such cards have probably had a respin to give them smaller/cooler chips, as well. I'd guess that the ethernet in gaming consoles is also not great. > Xboxes make good web servers for small scale. and use less energy than a > equivalent Piii or P4 system :-) gross. webserving is so easy that tiny embedded chips to a good job, and something like a via low-power would do a great job. 95% of a game console would be wasted as a webserver. > end desktop. it's just risen in the gaming market because people want games > which need greater computational power. likewise with gaming computers, they unclear. the gaming market is driven by eyecandy, and I sometimes wonder whether, for instance, humans can even see jaggies at 60 fps. so much of the gamer market is driven by what game-geek sites say about blowups of stills of particular frames. From jamesjamiejones at aol.com Tue Apr 3 13:25:34 2007 From: jamesjamiejones at aol.com (matt jones) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Ethernet break through? off topic In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> <4612A50A.8020108@aol.com> Message-ID: <4612B83E.5030504@aol.com> > > Xboxes make good web servers for small scale. and use less > energy than a > > equivalent Piii or P4 system :-) > > gross. webserving is so easy that tiny embedded chips to a good job, > and something like a via low-power would do a great job. 95% of a game > console would be wasted as a webserver. indeed it would. 95% of any machine that is not designed to be a webserver is lost... a true webserver would be as you say just embedded chips, storage and net. little need for anything else. the admin can be done over the net. -- matt. From James.P.Lux at jpl.nasa.gov Tue Apr 3 16:46:34 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4612A50A.8020108@aol.com> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> <4612A50A.8020108@aol.com> Message-ID: <6.2.3.4.2.20070403164545.02fb9008@mail.jpl.nasa.gov> At 12:03 PM 4/3/2007, matt jones wrote: >>you say that, but don't PS2 and PS3 make ideal compute nodes for >>some applications like rendering and 3D and 4D (time as the 4th) ? >> >>a PS3 cluster is already out there somewhere... http://www.elecdesign.com/Articles/Index.cfm?AD=1&ArticleID=15229&bypass=1 >>especially if you take cost into account, a retail PS3 has the >>processing power (if you can access it) of a number of P4 >>processors or the like, and costs less! >>PS2's were a bit slow to be honest, but with HDD's and networking, >>along with Xboxes make good web servers for small scale. and use >>less energy than a equivalent Piii or P4 system :-) >>________________________________ James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/3c9c9d49/attachment.html From bill at platform.com Tue Apr 3 19:45:39 2007 From: bill at platform.com (Bill Bryce) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! References: Message-ID: Thanks John, I appreciate your response. It is what I thought but I wasn't quite sure. I looked at MPICH2 for Windows and it does have two techniques with tradeoffs as you mentioned...I hope that you do decide to open up things a little more and allow other job schedulers and MPI stacks access to the credential management. I know from experience that the current job schedulers on Windows (other than your own) do not handle credentials in a transparent manner and it would be nice if their credential management was better implemented....entering a user password and saving it...'somewhere safe' is not really an ideal solution especially when the users password changes and the job scheduler does not pickup the change. Regards, Bill Bryce Product Manager Platform Open Cluster Stack -----Original Message----- From: John Vert [mailto:jvert@windows.microsoft.com] Sent: Tue 4/3/2007 10:32 PM To: Bill Bryce; Robert G. Brown Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! The job scheduler is responsible for managing the user credentials and tokens as well as allocating CPUs to jobs. So if you are not going to use our scheduler, you need a different solution for the credential management. How do the user's processes get created with the right token? MPICH2 has a couple different solutions with different tradeoffs - I'm not sure which one you use. We have had requests for two different scenarios - enabling other MPI stacks to use the credential management of our job scheduler and enabling other job schedulers to do the credential management for our MPI stack. We are looking at both of these for the next release but they didn't make V1. So like most product decisions, we didn't "have to do this" it was just the normal tradeoff you make between features and schedules with a healthy dose of security paranoia thrown into the mix. John Vert Development Manager High Performance Computing -----Original Message----- From: Bill Bryce [mailto:bill@platform.com] Sent: Tuesday, April 03, 2007 1:12 PM To: John Vert; Robert G. Brown Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! Can I ask one simple question..... Why did Microsoft have to do this (and I quote from MSDN) "Every job and task that uses Microsoft MPI must be submitted through the Microsoft job scheduler. There are several ways to submit jobs and tasks to the job scheduler:" I do not have that restriction with other MPI's such as MPICH2 for Windows from Argonne. Regards, Bill. -----Original Message----- From: John Vert [mailto:jvert@windows.microsoft.com] Sent: Tuesday, April 03, 2007 3:19 PM To: Robert G. Brown; Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! I'd like to clear up a few misconceptions on this thread. 1. There is no "MS specific library that abuses 'standard MPI'". The #1 goal for our MPI implementation is to make it as simple as possible for software vendors to port their codes to Windows and our MPI stack. That is why we based our implementation on MPICH2. The feedback we have gotten from software vendors has shown this approach was successful and we have had very few issues with MPI incompatibilities. We have not added anything outside the MPI spec. 2. MPICH2 is not licensed under GPL or under BSD. See http://www-unix.mcs.anl.gov/mpi/mpich2/license.htm Many commercial closed-source MPI implementations have been based on MPICH. Microsoft is actively working with Argonne on incorporating the changes we made back into the MPICH2 distribution. Argonne has been great to work with and really supportive of our efforts. 3. The changes we made can be loosely grouped into three areas. Performance, security, and scheduler integration. We handle the user credentials differently than Argonne's Windows implementation which is different than Argonne's Linux implementation due to some fundamental differences between the two operating systems. The scheduler integration is pretty straightforward - things like cleaning up all MPI child processes when a job is cancelled, and automatically propagating things like the allocated nodes and the number of CPUs from the scheduler to mpiexec. I don't think this is significantly different than what other job schedulers have done. 4. If you want to learn more about Windows HPC clusters, I recommend checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also happy to answer questions on this list, but frankly the S/N ratio tends to drop dramatically as soon as someone mentions Windows or Microsoft. John Vert Development Manager High Performance Computing -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Tuesday, April 03, 2007 7:59 AM To: Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Tue, 3 Apr 2007, Bill Bryce wrote: > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. Security changes? To MPI itself? This puts them into a somewhat grey area with regard to the GPL, doesn't it... that viral thing. I'm sure that they can manage the remote job launch any way that they like, but doesn't that leave the MPI CODE still portable? It isn't completely crazy that the US government would intervene here if they broke MPI portability. After all, MPI exists at all primarily because of direct government intervention (unlike PVM, which exists because some Very Bright People conceived it and invented it and made it fly). Still, nothing the Borg Empire does in this regard would surprise me. rgb > > Regards, > > Bill. > > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Robert G. Brown > Sent: Monday, April 02, 2007 8:22 PM > To: Tom Mitchell > Cc: beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > On Mon, 2 Apr 2007, Tom Mitchell wrote: > >> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >>> Date: Sun, 1 Apr 2007 14:02:53 +0500 >>> From: "amjad ali" >>> To: beowulf@beowulf.org >>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >>> >>> Hi All, >>> >>> Would any of you please like to share usage-experience/views/comments >>> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >>> >>> What in your opinion is the future of such clusters? >>> >>> How you compare these with the LINUX CLUSTERS? >> >> With full consideration to "fat, soft pretzels with >> cheezy-mustard sauce or rolled in asiago parmesan and garlic." >> >> MS pulled a version of mpich/mvapich/MPI and ported it to windows. >> They also developed some library code to gateway some *inx > library/system >> calls to windows. The root sources of MPI are public and not GPL so > they can. >> >> It might be worth looking at the MS announcement -- but why >> bother. If you look you might think that common MPI codes >> would just compile and run... I have no idea I expect some will >> and there begins silly porting for the next... > > Sure. MS did this, no doubt. And as you note below, no sooner do they > get it in when they begin the borgification of MPI, just as they've > borgified java, c, c++, and anything in the Universe they can sucker > somebody into buying in borgified form. > > Borgifying MPI is the most humorous thing in the Universe, BTW, given > its historical origins -- it was basically a language written > (reluctantly!) by supercomputer vendors when the US government got tired > of paying for all their important codes to be ported to each new > generation of proprietary hardware with its proprietary low level calls. > MS is doubtless trying to figure out just how much of that they can > undo while building up a big enough market share and enough vendors of > closed source applications written with their borgisms that they can... > > Oh wait. It IS GPL. Do you think that they actually read it? > > However, I was really referring to the other aspects of program > development and performance tuning associated with using a closed source > development environment. > > Resistance is Futile. > > rgb > >> >> Once a set of boxes are interconnected and you have library >> support for MPI or another way to share data (PVM... whatever) >> you are off and running in the clustering world. Sadly MS >> has a MS specific library that abuses "standard MPI" and could >> quickly cause source code to surface that runs correctly or on a >> MS cluster but not on another OS based cluster (Linux, Solaris, >> Irix, AIX). I see this all the time with java script, and c, >> c++, and other codes where little 'features' hook you in. >> >> Some will be fooled into thinking that this is something to look at >> or worse something to spend money on. >> >> SUMMARY: >> Since you posted this on 1 Apr 2007 all I can do is giggle >> and wonder why I replied. >> >> Regards, >> mitch >> >> PS: Ask in a year but not on April fools/joke day. >> >> > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070403/433e01ba/attachment.html From ashley at quadrics.com Wed Apr 4 03:55:15 2007 From: ashley at quadrics.com (Ashley Pittman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: <46138413.10009@quadrics.com> Bill Bryce wrote: > Regarding the 'borgification' of MPI on Windows....there is an opening > for them to do this. MPI did not define how you start the tasks when > you launch a parallel job....it left that up to the MPI implementation. > Now most MPIs use something reasonable, like say ssh - or starting a mpd > ring to launch the tasks (and this is all open source), however I > believe that for Microsoft MPI they wrote their own replacement for > 'ssh' and also modified how the tasks are launched to ensure their MPI > fits into Microsoft's security model, here is specific information on > this from Microsoft's site You'll find this is common to all MPI's. What microsoft have done here is no different. I'd argue that ssh is also not reasonable in this day and age so however MS have done this it doesn't need to be very good to be on a par with some Linux setups. > "An important improvement that MS MPI brings to MPICH2 is in how > security is managed during execution of MPI jobs. Each job is run with > the credentials of the user. Credentials are only present while the job > is being executed on the compute nodes and are erased when the job > completes. Individual processes only have access to their logon token, > and do not have access to the credentials (user name and password) that > generated the token or to the credentials used by other processes." This is an implementation detail and on Linux not a very important one, I think the fact it's quoted here says more about expectations of MS security than anything else. It's my impression that security/credentials hasn't been a problem for most people in practice so although there are ways of doing it they aren't widely used. > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. This is also common to all closed source MPI's (Including those based on MPICH) and actually in practice common to most open source MPI's as well, it's the age old ABI issue. HP ships their MPI on *all* systems they sell, buy a Quadrics system from HP and you get HP-MPI by default. I don't think it's because their MPI is necessarily any better than our MPI on the particular platform, it's just that it's the HP-MPI and it's a HP platform. Ashley, From hahn at mcmaster.ca Wed Apr 4 05:42:49 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: <4613.86.135.104.48.1175675489.squirrel@webmail.hostme.co.uk> References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> <4612A50A.8020108@aol.com> <4613.86.135.104.48.1175675489.squirrel@webmail.hostme.co.uk> Message-ID: > You seam to be making an awful lot of assumptions about what a game > console can or can't do. Let's have some facts rather than vague > guesswork. you misinterpret generalities for ignorance. > As I understand it the PS3 has a decent GigE port, it runs a standard gigabit is marginal for anything but low-communication clusters. for parallel rendering (the topic), it would certainly have issues keeping up with the rather large scene descriptions you find in, for instance, movies. > Linux OS, YDL, and it has both a general purpose CPU, (Power5) and 6 (I the main core is indeed GP, but not very fast - intended mainly to supervise the SPU's. > think) cell processors. 8 SPU's, though for yield reasons, shipped with 7 enabled. these things are not that much better than a single shader on something like a G80. > I would have thought a repetitive application > like rendering would be ideal for a cell based system. it's pretty good, given cost tradeoffs, for its intended purpose of playing games. I don't think it would do that well for, say, big CFD, MD, etc, or even rendering things which are quite different from games (ie movies). > I do not understand your reference to SLI cards. you can operate two GPUs in most any dual 8x PCIE board; SLI is not necessary, since you don't need both cards to work on the same frame. in short, creating a beowulf-like cluster of PS3s has high hack factor, always a good thing, but is not going to make much of a dent, since the hardware is, of necessity, somewhat specialized. From ashley at quadrics.com Wed Apr 4 06:13:13 2007 From: ashley at quadrics.com (Ashley Pittman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Performance characterising a HPC application In-Reply-To: <460B520F.1000204@myri.com> References: <45F96E56.9090900@aplpi.com> <45FFDAB7.6090706@aplpi.com> <20070321060958.GA4982@localhost.localdomain> <20070323022823.GA4882@localhost.localdomain> <46083413.6080905@ahpcrc.org> <460B520F.1000204@myri.com> Message-ID: <4613A469.2070301@quadrics.com> Patrick Geoffray wrote: > I would bet that UPC could more efficiently leverage a strided or vector > communication primitive instead of message aggregation. I don't know if > GasNet provides one, I know ARMCI does. GasNet does however get extra credit for having a asynchronous collective, namely barrier. Unfortunately when you read the spec it's actually a special case asynchronous reduce which is almost impossible to optimise anything like as well as barrier which is a shame. Ashley, From rbw at ahpcrc.org Wed Apr 4 06:25:45 2007 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Performance characterising a HPC application In-Reply-To: <4613A469.2070301@quadrics.com> References: <45F96E56.9090900@aplpi.com> <45FFDAB7.6090706@aplpi.com> <20070321060958.GA4982@localhost.localdomain> <20070323022823.GA4882@localhost.localdomain> <46083413.6080905@ahpcrc.org> <460B520F.1000204@myri.com> <4613A469.2070301@quadrics.com> Message-ID: <4613A759.8000703@ahpcrc.org> Ashley Pittman wrote: > Patrick Geoffray wrote: > >> I would bet that UPC could more efficiently leverage a strided or vector >> communication primitive instead of message aggregation. I don't know if >> GasNet provides one, I know ARMCI does. >> > > GasNet does however get extra credit for having a asynchronous > collective, namely barrier. Unfortunately when you read the spec it's > actually a special case asynchronous reduce which is almost impossible > to optimise anything like as well as barrier which is a shame. > All, Berkeley's recent paper on the some optimization techniques that they have applied within their UPC compiler emphasizes reducing and hoisting shared pointer references, maximally split-phased reads and writes (implying that GASnet can do what Ashley suggests), and aggregating- coalescing communication. I have only read half of it, but no mention of pipelining or pseudo-vector operations (too bad) ... this would seem to be harder to do as it would required whole loop analysis. You may be interested in Googling for it: "Communication Optimizations for Fine-Grained UPC Applications" Wei-Yu Chen Costin Iancu Kathy Yelick Regards, rbw -- Richard B. Walsh Project Manager Network Computing Services, Inc. Army High Performance Computing Research Center (AHPCRC) rbw@ahpcrc.org | 612.337.3467 > > "Making predictions is hard, especially about the future." > > Nils Bohr ----------------------------------------------------------------------- This message (including any attachments) may contain proprietary or privileged information, the use and disclosure of which is legally restricted. If you have received this message in error please notify the sender by reply message, do not otherwise distribute it, and delete this message, with all of its contents, from your files. ----------------------------------------------------------------------- From christian.bell at qlogic.com Wed Apr 4 07:00:35 2007 From: christian.bell at qlogic.com (Christian Bell) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Performance characterising a HPC application In-Reply-To: <4613A469.2070301@quadrics.com> References: <45F96E56.9090900@aplpi.com> <45FFDAB7.6090706@aplpi.com> <20070321060958.GA4982@localhost.localdomain> <20070323022823.GA4882@localhost.localdomain> <46083413.6080905@ahpcrc.org> <460B520F.1000204@myri.com> <4613A469.2070301@quadrics.com> Message-ID: <20070404140035.GZ20133@pathscale.com> On Wed, 04 Apr 2007, Ashley Pittman wrote: > GasNet does however get extra credit for having a asynchronous > collective, namely barrier. Unfortunately when you read the spec it's > actually a special case asynchronous reduce which is almost impossible > to optimise anything like as well as barrier which is a shame. GASNet's named split-phase barriers cater to a language-level feature in UPC to generate mismatches if the name doesn't match, which can be a useful debugging tool. The same primitive has a flags option to make it unnamed, which probably has the properties you want (being split-phase without the need to reduce a value across processors). This one could be implemented over a faster split-phase notify/wait if the interconnect can offer one (although yes, this would require bending the rules somewhat at the language-level since one local proc doing what it thinks is an unnamed barrier would have to assume that all of its peers also used an unnamed barrier). . . christian -- christian.bell@qlogic.com (QLogic SIG, formerly Pathscale) From ashley at quadrics.com Wed Apr 4 07:06:05 2007 From: ashley at quadrics.com (Ashley Pittman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Performance characterising a HPC application In-Reply-To: <4613A759.8000703@ahpcrc.org> References: <45F96E56.9090900@aplpi.com> <45FFDAB7.6090706@aplpi.com> <20070321060958.GA4982@localhost.localdomain> <20070323022823.GA4882@localhost.localdomain> <46083413.6080905@ahpcrc.org> <460B520F.1000204@myri.com> <4613A469.2070301@quadrics.com> <4613A759.8000703@ahpcrc.org> Message-ID: <4613B0CD.7090309@quadrics.com> Richard Walsh wrote: > Ashley Pittman wrote: >> Patrick Geoffray wrote: >> >>> I would bet that UPC could more efficiently leverage a strided or vector >>> communication primitive instead of message aggregation. I don't know if >>> GasNet provides one, I know ARMCI does. >>> >> GasNet does however get extra credit for having a asynchronous >> collective, namely barrier. Unfortunately when you read the spec it's >> actually a special case asynchronous reduce which is almost impossible >> to optimise anything like as well as barrier which is a shame. >> > All, > > Berkeley's recent paper on the some optimization techniques that they have applied within their UPC compiler emphasizes reducing and hoisting shared pointer references, maximally split-phased reads and writes (implying that GASnet can do what Ashley suggests), and aggregating- > coalescing communication. I have only read half of it, but no mention of pipelining or pseudo-vector operations (too bad) ... this would seem to be harder to do as it would required whole loop analysis. I was thinking of gasnet_barrier_notify() and gasnet_barrier_wait()/gasnet_barrier_try() specifically which allow you to pipeline useful code with barriers. It's a surprisingly useful trick and one that I hope will become commonplace in future. Ashley, From ashley at quadrics.com Wed Apr 4 07:33:43 2007 From: ashley at quadrics.com (Ashley Pittman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Performance characterising a HPC application In-Reply-To: <20070404140035.GZ20133@pathscale.com> References: <45F96E56.9090900@aplpi.com> <45FFDAB7.6090706@aplpi.com> <20070321060958.GA4982@localhost.localdomain> <20070323022823.GA4882@localhost.localdomain> <46083413.6080905@ahpcrc.org> <460B520F.1000204@myri.com> <4613A469.2070301@quadrics.com> <20070404140035.GZ20133@pathscale.com> Message-ID: <4613B747.3050907@quadrics.com> Christian Bell wrote: > On Wed, 04 Apr 2007, Ashley Pittman wrote: >> GasNet does however get extra credit for having a asynchronous >> collective, namely barrier. Unfortunately when you read the spec it's >> actually a special case asynchronous reduce which is almost impossible >> to optimise anything like as well as barrier which is a shame. > > GASNet's named split-phase barriers cater to a language-level feature > in UPC to generate mismatches if the name doesn't match, which can be > a useful debugging tool. The same primitive has a flags option to > make it unnamed, which probably has the properties you want (being > split-phase without the need to reduce a value across processors). > This one could be implemented over a faster split-phase notify/wait > if the interconnect can offer one So it does, it's clearly been a while since I looked at this, the GASNET_BARRIERFLAG_ANONYMOUS implys exactly the semantics I would have preferred, I take it all back. "Named barriers" as you say are incredibly useful as you say and not just for debugging, I'm not trying to beat on them but they are effectively AllReduce when it comes to implementation them which is significantly slower/harder to do. > (although yes, this would require > bending the rules somewhat at the language-level since one local proc > doing what it thinks is an unnamed barrier would have to assume that > all of its peers also used an unnamed barrier). Looking at it carefully I don't think any rules would actually be broken, in particular passing both GASNET_BARRIERFLAG_ANONYMOUS and GASNET_BARRIERFLAG_MISMATCH would appear to not be allowed so you could use a different barrier implementation for the fast (anonymous) case and the named case. It is as you say the language bending it's own rules here. Ashley. From ashley at quadrics.com Wed Apr 4 08:27:56 2007 From: ashley at quadrics.com (Ashley Pittman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Re: Performance characterising a HPC application In-Reply-To: <460A8420.8080200@aplpi.com> References: <45F96E56.9090900@aplpi.com> <460A8420.8080200@aplpi.com> Message-ID: <4613C3FC.1020608@quadrics.com> stephen mulcahy wrote: > Hi, > > As a follow on to my previous mail, I've gone ahead and run the Intel > MPI Benchmarks (v3.0) on this cluster and gotten the following results - > I'd be curious to know how they compare to other similar clusters. You don't say what hardware you are using but from reading one of your original mails "a 20 node opteron 270 (2.0GHz dual core, 4GB ram, diskless) cluster with gigabit ethernet interconnect" You should be aware that the PingPong and PingPing tests in IMB ping messages between processes 0 and 1 in the job, it's likely that these are on the same node so you are measuring shared memory performance. Given it reports a bandwidth figure of 607MB/s I doubt very much it's going over the ethernet. You should re-run these two tests using one processer per node over two nodes to get the network speeds, you can then use the ratio of nodes to cpus of a typical job to determine how much weight to attach to the different results. None of the figures you have posted strike me as being that great, I've commented on some of the individual ones below. > Also, I'm trying to determine which parts of the IMB results are most > important for me It depends on the application and what you use. PingPong is probably most important. > my understanding is that PingPong is a good measure > of overall latency and bandwidth between individual nodes in the cluster. Yes. > Am I correct in thinking that Bcast and Reduce are good indicators of > the performance of the cluster in terms of sending and receiving data > from the head node to the compute nodes? My guess is that the other > benchmarks are not as relevant to me since they measure performance for > various types of inter-node traffic rather than the one-to-many pattern > exhibited by my application. It depends. If you are using one-to-many patterns in the app then code it to use Bcast, creating the requisite sub communicators as needed and then the Bcast benchmark worth looking at. > #--------------------------------------------------- > # Intel (R) MPI Benchmark Suite V3.0, MPI-1 part > #--------------------------------------------------- > #--------------------------------------------------- > # Benchmarking PingPong > # #processes = 2 > # ( 78 additional processes waiting in MPI_Barrier) > #--------------------------------------------------- > #bytes #repetitions t[usec] Mbytes/sec > 0 1000 27.63 0.00 > 1 1000 28.56 0.03 > 2 1000 28.65 0.07 > 4 1000 28.72 0.13 > 8 1000 27.03 0.28 > 16 1000 28.63 0.53 > 32 1000 28.79 1.06 > 64 1000 28.72 2.13 > 128 1000 28.61 4.27 > 256 1000 28.75 8.49 > 512 1000 27.90 17.50 > 1024 1000 27.55 35.45 > 2048 1000 29.70 65.77 > 4096 1000 86.92 44.94 > 8192 1000 87.85 88.93 > 16384 1000 91.98 169.88 > 32768 1000 105.01 297.60 > 65536 640 149.88 417.00 > 131072 320 312.52 399.98 > 262144 160 547.92 456.27 > 524288 80 998.77 500.62 > 1048576 40 2008.35 497.92 > 2097152 20 3407.78 586.89 > 4194304 10 6583.70 607.56 *Assuming* this is for shared memory these are pretty dreadful figures, for reference we get the following when we run the test intra-node, as you can see they are strikingly different. #--------------------------------------------------- # Benchmarking PingPong # #processes = 2 #--------------------------------------------------- #bytes #repetitions t[usec] Mbytes/sec 0 1000 1.05 0.00 1 1000 1.03 0.92 2 1000 1.03 1.86 4 1000 1.09 3.49 8 1000 1.05 7.28 16 1000 1.03 14.83 32 1000 1.05 29.08 64 1000 1.16 52.61 128 1000 1.27 96.34 256 1000 1.44 169.75 512 1000 1.76 278.16 1024 1000 2.36 414.15 2048 1000 3.51 556.56 4096 1000 5.25 744.18 8192 1000 5.89 1326.33 16384 1000 7.85 1989.66 32768 1000 12.77 2447.58 65536 640 23.85 2620.20 131072 320 44.14 2831.98 262144 160 85.17 2935.16 524288 80 359.69 1390.10 1048576 40 1207.04 828.47 2097152 20 2407.75 830.65 4194304 10 4792.27 834.68 > #--------------------------------------------------- > # Benchmarking PingPing > # #processes = 2 > # ( 78 additional processes waiting in MPI_Barrier) > #--------------------------------------------------- As above really. > #---------------------------------------------------------------- > # Benchmarking Reduce_scatter > # #processes = 80 > #---------------------------------------------------------------- > #bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] > 0 1000 1.15 1.17 1.15 > 4 1000 2.79 116.45 37.74 > 8 1000 2.88 146.80 28.30 > 16 1000 2.89 174.36 31.96 > 32 1000 3.23 302.12 45.65 > 64 1000 3.12 408.95 93.14 > 128 1000 3.07 576.51 232.83 > 256 1000 3.10 923.28 738.29 > 512 1000 1090.20 1092.25 1091.05 > 1024 1000 1212.27 1217.47 1214.91 > 2048 1000 1300.73 1306.18 1302.84 > 4096 1000 1474.26 1476.34 1475.58 > 8192 1000 1935.20 1936.17 1935.86 > 16384 1000 2562.77 2563.99 2563.63 > 32768 1000 3874.37 3876.27 3875.87 > 65536 640 73380.90 73387.62 73385.88 > 131072 320 350385.89 350418.67 350407.98 > 262144 160 12655.13 12828.11 12671.15 > 524288 80 23142.21 23554.27 23348.91 > 1048576 40 41440.35 41724.03 41584.49 > 2097152 20 58607.35 59571.41 59087.99 > 4194304 10 94975.40 100692.30 97813.92 There is something very wrong with these figures, you should never get a jump in performance like that, also 2.79 for Reduce_scatter isn't possible, I'd recompile IMB with checking on and check for errors first. > #---------------------------------------------------------------- > # Benchmarking Bcast > # #processes = 80 > #---------------------------------------------------------------- > #bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] > 0 1000 0.06 0.10 0.07 > 1 1000 2559.63 2642.39 2577.71 > 2 1000 2577.59 2660.47 2592.78 > 4 1000 2541.65 2624.19 2561.87 > 8 1000 2538.07 2579.05 2556.78 > 16 1000 2540.35 2581.47 2558.62 > 32 1000 2539.32 2580.52 2557.10 > 64 1000 2539.23 2580.50 2555.27 > 128 1000 2539.12 2581.62 2553.66 > 256 1000 2585.36 2627.14 2588.85 > 512 1000 141.55 142.39 141.99 > 1024 1000 208.98 210.17 209.78 > 2048 1000 227.81 228.95 228.45 > 4096 1000 293.99 318.13 306.15 > 8192 1000 486.10 487.18 486.73 > 16384 1000 799.65 801.21 800.74 > 32768 1000 1483.64 1486.06 1485.42 > 65536 640 3192.47 3199.19 3197.68 > 131072 320 6314.70 6341.48 6335.46 > 262144 160 12469.44 12554.68 12532.73 > 524288 80 9770.92 10413.19 10318.85 > 1048576 40 18792.78 20762.40 20533.59 > 2097152 20 33849.45 42141.25 41535.32 > 4194304 10 65966.61 81472.99 79850.54 Why is 512 so much quicker than 8? The reduce figures showed the same issue and it's something I'd want to look at. Ashley, From peter.st.john at gmail.com Wed Apr 4 09:44:50 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: John, Thanks for your reply, and taking the trouble to grab those measurements. My own HPC cluster is only in my head (where it performs poorly!) so I can't return the favor directly. But like many good answers we can narrow down some discussion/thinking points. I'll spare you stories of booting SVr4 on 512K :-) -- it didn 't work well, "vi temp" hung, I had to get expansion memory for my 286. Of course unices can be made arbitrarily tiny, but my feeling is that 20% of RAM is not unreasonable these days. However, we'd want to think about what resources are consumed (presumably, disk?) or functionality lost (presumably, none?) when the OS does not have the RAM to resize itself larger (i.e., what is the cost of using only 153MB when only 512 is available, and it would use 400 if it could get it? presumably disk paging?). This ties into the related item, thanks very much for introducing it into the thread, of disk usage. This would tie the footprint question into broader networking issues in the case of diskless compute nodes (the virtual memory expected by the OS is remote) and the disk usage time for diskful nodes. I hope someone sitting next to an actual terminal to an actual cluster can handily post something similar for comparison. Of course it's not apples-to-apples; unless someone with a Beowulif of Macs wants to chime in :-) But the comparison can make a table-of-contents for a discussion. Engineers build things with constraints: climate, terrain, budget, stresses, loads....and their employer's business model. I have no animosity to engineers building things with constraints. Peter On 4/3/07, John Vert wrote: > > It depends how you measure the size. Here are some measurements I just > made. > > > > On a 2GB RAM machine out of the box, a single process can get a 1.6GBminimum working set by calling SetProcessWorkingSetSize(). ( > 1.6GB of memory resident and available for use without page faulting). So > you might say the OS is 400MB. But on a 1GB machine, you can get 750MB, so > you might say the OS is 250MB. And on a 512MB machine you can get 359MB, so > now the OS is 153MB. In all cases you could probably get a bit more > depending on what your process is doing and whether you want to tweak the > configuration any. (e.g. if you're not doing cached file I/O, the file > cache will shrink under memory pressure) > > > > The disk footprint of the OS is about 1.75GB plus another 2GB for the > paging file. > > > > John Vert > > Development Manager > > High Performance Computing > > > > *From:* Peter St. John [mailto:peter.st.john@gmail.com] > *Sent:* Tuesday, April 03, 2007 1:49 PM > *To:* John Vert > *Cc:* Robert G. Brown; beowulf@beowulf.org; Bill Bryce > *Subject:* Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > > > John, > > Thank you for > > "... > > 4. If you want to learn more about Windows HPC clusters, I recommend > checking out www.microsoft.com/hpc and www.windowshpc.net . I'm also happy > to answer questions on this list, but frankly the S/N ratio tends to drop > dramatically as soon as someone mentions Windows or Microsoft. > ..." > > > > How large is the OS on an idle (but ready) compute node in MS's cluster > system? Reasonable possible responses would include that the question is > ill-posed. I would appreciate any response. > > > > Here's for d/dt (S/N) > 0 > > Thanks, > > Peter > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070404/39c30d44/attachment.html From wrankin at ee.duke.edu Thu Apr 5 13:05:55 2007 From: wrankin at ee.duke.edu (Bill Rankin) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] OT: Looking for HPC Headhunter/Recruiter Message-ID: Good afternoon folks, Pardon the interruption but if anyone has a good referral to a headhunter who has good working knowledge of jobs in the HPC arena (esp. clusters) I would appreciate it if you could drop me a line. Yeah, I'm looking. <:-) We now return you to your regularly scheduled RGB rant. -bill -- Bill Rankin, Ph.D. Director, Cluster and Grid Technology Group Center for Computational Science, Engineering and Medicine Duke University wrankin@duke.edu ph: (919) 660-6561 fx: (919) 660-7070 From ross.miller at safetynetsolutions.com Mon Apr 2 13:04:43 2007 From: ross.miller at safetynetsolutions.com (Miller Ross) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611545C.2050907@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> Message-ID: <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> The fundamental problem we as an industry face is that the consistent pattern of MS is to adopt and expand and then break compatibility with other folks technologies. Actually, I don't have a problem with this competition, the problem is when any company claims an idea as their own, when in fact it is not theirs. This pattern has happened again and again, and is part of the business model of the company as well as a cultural issue within that company. In short MS is a community in and of itself, and very often (though certainly not always), this community does not share or play well with others. AND I think we also all recognize that this is a very old battle going back way beyond the birth of Microsoft to the 60's and proprietary versus non-proprietary business models. However, as I see it they have a severe problem -- a Flat stock curve oscillating between roughly $25 and $30 for the last 5 years. That is indicative of a stable business model, but I think we all know that without growth, shrinkage ;), is inevitable. Being behind the curve consistently is what is doing them in, the copy and adopt model doesn't work in the current business climate: Java -> Response C# Google -> Response MS-Search IPod -> various MP3 players and a new format, WMA Itunes store -> on-line music sales with DRM Amiga -> Win 95 (windows gets multi-tasking for real (well, sorta kinda anyway)) AIM -> MSN Messenger and of course Beowulf -> HPC ((outcome to be determined)) But let's look at Early MS software CPM -> DOS (a big upgrade) ((yes, I wrote code for both)) DOS -> DOS with background tasking (printer drivers) DOS -> Windows (a big upgrade) PC-Write -> Word MS-Project 1.0 Lotus -> Excel Here we seen more genuine innovation, not simple 'mee too'. Though I would say the Lotus to Excel is more arguable if it was a big step forward. Unless MS starts to really embrace innovation and not put flight simulators into Excel, while the actual spreadsheet program will produce erroneous results (yes, I used the version that had the flight sim easter egg, and had to by hand check my quotes before giving them to clients and catch simple addition errors)). That said, we as an industry do owe Microsoft one significant debt. The standardization of Microcomputer hardware. Up until DOS the world was so fragmented it was impossible to launch anything in the software, or even specialized hardware arena using economies of scale. With the world standardizing on the PC platform with DOS stability and real desktop productivity gains became possible. Alan Greenspan addressed congress in 1997. Having watched the address I was personally impressed with his statement that the US. per-capita GDP was dramatically higher than any other nation at that time, because the US had a PC on pretty much every worker's desk that needed access to a PC. Microsoft brought the US that standardization, and those productivity gains. To the best of my recollection it went. U.S. 1.0 (normalized) Japan .92 Germany .87 UK same range and it went WAY down from there. That same low cost microcomputer is what we base our Beowulf clusters on today, essentially a more reliable beefier version of a desktop PC. Just as we don't want Beowulf to be overridden by 'adopt, assimilate, and expand', we should give the credit where credit is due that we are not using Commodore 64's, TI's, Atari's, or DEC- Rainbows as cluster nodes. Ross On Apr 2, 2007, at 3:07 PM, Joe Landman wrote: > Douglas Eadline wrote: > >> I believe that if we do not protect against revisionist history, then > > [...] > > you mean like how now with WCCS2k+3 clustering and HPC is *now* > (suddenly magically spontaneously) "mainstream" ? > > This is just something I personally take issue with. The entire > explosive growth of clustering has driven HPC hard into the > mainstream. This happened long before it was a glimmer in their > eyes. 6+ years of explosive growth, going from noise in the > statistics to dominating the statistics. Then along they came with > WCCS2k+3. > > Their entry is late into the cycle. And if you listen to the > comments of the senior execs, it makes one wonder how committed > they are to HPC and clusters as compared to how committed they are > to battling Linux. > > This is not to diminish their efforts. WCCS2k+3 is likely > reasonably good for some subset of groups. Microsoft has some good > people there, and playing with the W2k+3 x64 on our JackRabbit unit > was fun. They still need a real POSIX subsystem, and hopefully, > someday, they will give in, and get cygwin or mingw to be fully > supported/shipping using their compilers/tools. > > Though I expect to see airborn and stable flight from porcine > critters about the same time. Too bad, as that would likely ease > adoption/porting issues. Tremendously. > > -- > > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics LLC, > email: landman@scalableinformatics.com > web : http://www.scalableinformatics.com > phone: +1 734 786 8423 > fax : +1 734 786 8452 or +1 866 888 3112 > cell : +1 734 612 4615 > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf From toon at moene.indiv.nluug.nl Mon Apr 2 14:28:03 2007 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> Message-ID: <46117563.5030605@moene.indiv.nluug.nl> Douglas Eadline wrote: > I would not want to see it usurped by other clustering efforts. The standard way of protecting this intellectual property is to obtain a trademark. (ducks). -- Toon Moene - e-mail: toon@moene.indiv.nluug.nl - phone: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands At home: http://moene.indiv.nluug.nl/~toon/ Who's working on GNU Fortran: http://gcc.gnu.org/ml/gcc/2007-01/msg00059.html From ryanw at windows.microsoft.com Mon Apr 2 21:02:06 2007 From: ryanw at windows.microsoft.com (Ryan Waite) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4611545C.2050907@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu><38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> Message-ID: <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> Yeah, that marketing slogan wasn't too great since it sounds like we're solely taking HPC mainstream. Ouch. Of course, that isn't the case. Where would people on this list place the credit for HPC going mainstream? If I had to pick one source, which is unfair to do, it would be the rich array of applications that exist, from BLAST to Fluent, from Ganglia to LSF, from PERL to Matlab. Thanks for the feature requests, Joe. Ryan Waite Group Program Manager - HPC Microsoft Corporation ryanw@microsoft.com -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Joe Landman Sent: Monday, April 02, 2007 12:07 PM To: Douglas Eadline Cc: Erik Paulson; beowulf@beowulf.org Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! Douglas Eadline wrote: > I believe that if we do not protect against revisionist history, then [...] you mean like how now with WCCS2k+3 clustering and HPC is *now* (suddenly magically spontaneously) "mainstream" ? This is just something I personally take issue with. The entire explosive growth of clustering has driven HPC hard into the mainstream. This happened long before it was a glimmer in their eyes. 6+ years of explosive growth, going from noise in the statistics to dominating the statistics. Then along they came with WCCS2k+3. Their entry is late into the cycle. And if you listen to the comments of the senior execs, it makes one wonder how committed they are to HPC and clusters as compared to how committed they are to battling Linux. This is not to diminish their efforts. WCCS2k+3 is likely reasonably good for some subset of groups. Microsoft has some good people there, and playing with the W2k+3 x64 on our JackRabbit unit was fun. They still need a real POSIX subsystem, and hopefully, someday, they will give in, and get cygwin or mingw to be fully supported/shipping using their compilers/tools. Though I expect to see airborn and stable flight from porcine critters about the same time. Too bad, as that would likely ease adoption/porting issues. Tremendously. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From jvert at windows.microsoft.com Tue Apr 3 12:18:46 2007 From: jvert at windows.microsoft.com (John Vert) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: I'd like to clear up a few misconceptions on this thread. 1. There is no "MS specific library that abuses 'standard MPI'". The #1 goal for our MPI implementation is to make it as simple as possible for software vendors to port their codes to Windows and our MPI stack. That is why we based our implementation on MPICH2. The feedback we have gotten from software vendors has shown this approach was successful and we have had very few issues with MPI incompatibilities. We have not added anything outside the MPI spec. 2. MPICH2 is not licensed under GPL or under BSD. See http://www-unix.mcs.anl.gov/mpi/mpich2/license.htm Many commercial closed-source MPI implementations have been based on MPICH. Microsoft is actively working with Argonne on incorporating the changes we made back into the MPICH2 distribution. Argonne has been great to work with and really supportive of our efforts. 3. The changes we made can be loosely grouped into three areas. Performance, security, and scheduler integration. We handle the user credentials differently than Argonne's Windows implementation which is different than Argonne's Linux implementation due to some fundamental differences between the two operating systems. The scheduler integration is pretty straightforward - things like cleaning up all MPI child processes when a job is cancelled, and automatically propagating things like the allocated nodes and the number of CPUs from the scheduler to mpiexec. I don't think this is significantly different than what other job schedulers have done. 4. If you want to learn more about Windows HPC clusters, I recommend checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also happy to answer questions on this list, but frankly the S/N ratio tends to drop dramatically as soon as someone mentions Windows or Microsoft. John Vert Development Manager High Performance Computing -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Tuesday, April 03, 2007 7:59 AM To: Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Tue, 3 Apr 2007, Bill Bryce wrote: > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. Security changes? To MPI itself? This puts them into a somewhat grey area with regard to the GPL, doesn't it... that viral thing. I'm sure that they can manage the remote job launch any way that they like, but doesn't that leave the MPI CODE still portable? It isn't completely crazy that the US government would intervene here if they broke MPI portability. After all, MPI exists at all primarily because of direct government intervention (unlike PVM, which exists because some Very Bright People conceived it and invented it and made it fly). Still, nothing the Borg Empire does in this regard would surprise me. rgb > > Regards, > > Bill. > > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Robert G. Brown > Sent: Monday, April 02, 2007 8:22 PM > To: Tom Mitchell > Cc: beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > On Mon, 2 Apr 2007, Tom Mitchell wrote: > >> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >>> Date: Sun, 1 Apr 2007 14:02:53 +0500 >>> From: "amjad ali" >>> To: beowulf@beowulf.org >>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >>> >>> Hi All, >>> >>> Would any of you please like to share usage-experience/views/comments >>> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >>> >>> What in your opinion is the future of such clusters? >>> >>> How you compare these with the LINUX CLUSTERS? >> >> With full consideration to "fat, soft pretzels with >> cheezy-mustard sauce or rolled in asiago parmesan and garlic." >> >> MS pulled a version of mpich/mvapich/MPI and ported it to windows. >> They also developed some library code to gateway some *inx > library/system >> calls to windows. The root sources of MPI are public and not GPL so > they can. >> >> It might be worth looking at the MS announcement -- but why >> bother. If you look you might think that common MPI codes >> would just compile and run... I have no idea I expect some will >> and there begins silly porting for the next... > > Sure. MS did this, no doubt. And as you note below, no sooner do they > get it in when they begin the borgification of MPI, just as they've > borgified java, c, c++, and anything in the Universe they can sucker > somebody into buying in borgified form. > > Borgifying MPI is the most humorous thing in the Universe, BTW, given > its historical origins -- it was basically a language written > (reluctantly!) by supercomputer vendors when the US government got tired > of paying for all their important codes to be ported to each new > generation of proprietary hardware with its proprietary low level calls. > MS is doubtless trying to figure out just how much of that they can > undo while building up a big enough market share and enough vendors of > closed source applications written with their borgisms that they can... > > Oh wait. It IS GPL. Do you think that they actually read it? > > However, I was really referring to the other aspects of program > development and performance tuning associated with using a closed source > development environment. > > Resistance is Futile. > > rgb > >> >> Once a set of boxes are interconnected and you have library >> support for MPI or another way to share data (PVM... whatever) >> you are off and running in the clustering world. Sadly MS >> has a MS specific library that abuses "standard MPI" and could >> quickly cause source code to surface that runs correctly or on a >> MS cluster but not on another OS based cluster (Linux, Solaris, >> Irix, AIX). I see this all the time with java script, and c, >> c++, and other codes where little 'features' hook you in. >> >> Some will be fooled into thinking that this is something to look at >> or worse something to spend money on. >> >> SUMMARY: >> Since you posted this on 1 Apr 2007 all I can do is giggle >> and wonder why I replied. >> >> Regards, >> mitch >> >> PS: Ask in a year but not on April fools/joke day. >> >> > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From asingh at ideeinc.com Tue Apr 3 13:39:32 2007 From: asingh at ideeinc.com (Amrik Singh) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] NFS & Scaling issues Message-ID: <4612BB84.6010707@ideeinc.com> Hi, We are running a cluster of 180 diskless compute nodes. 60 of them have 32 bit AMD Semptron processors and rest are dual core AMD Athelon 64 bit processors. 32 bit machines have 10/100 mbps and rest have gigabit ethernet cards. We have four file servers, each hosting around 3.5TB on SATA drives connected to 3Ware RAID controller cards configured on RAID 10 array. These file servers are exporting the drives through NFS. Each file server is running 265 daemons for nfsd. The file servers are mainly hosting large number of small files ranging from 256KB to 2 MB. The compute nodes are primarily doing a search through these files, so there is lot's of reading and some writing to the file servers. Recently we started noticing very high (70-90%) wait states on the file servers when compute nodes. We have tried to optimize the NFS through increasing the number of daemons and the rsize and wsize but to no avail. Can someone point us in the right direction as to how we should be trying to troubleshoot this problem. PS: All the nodes are running SuSE 10.0 and servers are running SuSE10.0 and 10.1 and all the drives are formatted with reiserfs. thanks -- Amrik From jvert at windows.microsoft.com Tue Apr 3 19:32:25 2007 From: jvert at windows.microsoft.com (John Vert) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: The job scheduler is responsible for managing the user credentials and tokens as well as allocating CPUs to jobs. So if you are not going to use our scheduler, you need a different solution for the credential management. How do the user's processes get created with the right token? MPICH2 has a couple different solutions with different tradeoffs - I'm not sure which one you use. We have had requests for two different scenarios - enabling other MPI stacks to use the credential management of our job scheduler and enabling other job schedulers to do the credential management for our MPI stack. We are looking at both of these for the next release but they didn't make V1. So like most product decisions, we didn't "have to do this" it was just the normal tradeoff you make between features and schedules with a healthy dose of security paranoia thrown into the mix. John Vert Development Manager High Performance Computing -----Original Message----- From: Bill Bryce [mailto:bill@platform.com] Sent: Tuesday, April 03, 2007 1:12 PM To: John Vert; Robert G. Brown Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! Can I ask one simple question..... Why did Microsoft have to do this (and I quote from MSDN) "Every job and task that uses Microsoft MPI must be submitted through the Microsoft job scheduler. There are several ways to submit jobs and tasks to the job scheduler:" I do not have that restriction with other MPI's such as MPICH2 for Windows from Argonne. Regards, Bill. -----Original Message----- From: John Vert [mailto:jvert@windows.microsoft.com] Sent: Tuesday, April 03, 2007 3:19 PM To: Robert G. Brown; Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! I'd like to clear up a few misconceptions on this thread. 1. There is no "MS specific library that abuses 'standard MPI'". The #1 goal for our MPI implementation is to make it as simple as possible for software vendors to port their codes to Windows and our MPI stack. That is why we based our implementation on MPICH2. The feedback we have gotten from software vendors has shown this approach was successful and we have had very few issues with MPI incompatibilities. We have not added anything outside the MPI spec. 2. MPICH2 is not licensed under GPL or under BSD. See http://www-unix.mcs.anl.gov/mpi/mpich2/license.htm Many commercial closed-source MPI implementations have been based on MPICH. Microsoft is actively working with Argonne on incorporating the changes we made back into the MPICH2 distribution. Argonne has been great to work with and really supportive of our efforts. 3. The changes we made can be loosely grouped into three areas. Performance, security, and scheduler integration. We handle the user credentials differently than Argonne's Windows implementation which is different than Argonne's Linux implementation due to some fundamental differences between the two operating systems. The scheduler integration is pretty straightforward - things like cleaning up all MPI child processes when a job is cancelled, and automatically propagating things like the allocated nodes and the number of CPUs from the scheduler to mpiexec. I don't think this is significantly different than what other job schedulers have done. 4. If you want to learn more about Windows HPC clusters, I recommend checking out www.microsoft.com/hpc and www.windowshpc.net. I'm also happy to answer questions on this list, but frankly the S/N ratio tends to drop dramatically as soon as someone mentions Windows or Microsoft. John Vert Development Manager High Performance Computing -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Tuesday, April 03, 2007 7:59 AM To: Bill Bryce Cc: beowulf@beowulf.org Subject: RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Tue, 3 Apr 2007, Bill Bryce wrote: > So this change effectively ties Microsoft MPI to only Microsoft Windows > platforms, and the security changes are closed source. Not all of > Microsoft's partners like MS MPI - when HP ships Microsoft CCS they > remove MS MPI and put in HP MPI - which probably just adds to confusion > on the end user side. Security changes? To MPI itself? This puts them into a somewhat grey area with regard to the GPL, doesn't it... that viral thing. I'm sure that they can manage the remote job launch any way that they like, but doesn't that leave the MPI CODE still portable? It isn't completely crazy that the US government would intervene here if they broke MPI portability. After all, MPI exists at all primarily because of direct government intervention (unlike PVM, which exists because some Very Bright People conceived it and invented it and made it fly). Still, nothing the Borg Empire does in this regard would surprise me. rgb > > Regards, > > Bill. > > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Robert G. Brown > Sent: Monday, April 02, 2007 8:22 PM > To: Tom Mitchell > Cc: beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > On Mon, 2 Apr 2007, Tom Mitchell wrote: > >> On Sun, Apr 01, 2007 at 02:02:53PM +0500, amjad ali wrote: >>> Date: Sun, 1 Apr 2007 14:02:53 +0500 >>> From: "amjad ali" >>> To: beowulf@beowulf.org >>> Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! >>> >>> Hi All, >>> >>> Would any of you please like to share usage-experience/views/comments >>> about Windows Compute Cluster Server 2003 based Beowulf Clusters? >>> >>> What in your opinion is the future of such clusters? >>> >>> How you compare these with the LINUX CLUSTERS? >> >> With full consideration to "fat, soft pretzels with >> cheezy-mustard sauce or rolled in asiago parmesan and garlic." >> >> MS pulled a version of mpich/mvapich/MPI and ported it to windows. >> They also developed some library code to gateway some *inx > library/system >> calls to windows. The root sources of MPI are public and not GPL so > they can. >> >> It might be worth looking at the MS announcement -- but why >> bother. If you look you might think that common MPI codes >> would just compile and run... I have no idea I expect some will >> and there begins silly porting for the next... > > Sure. MS did this, no doubt. And as you note below, no sooner do they > get it in when they begin the borgification of MPI, just as they've > borgified java, c, c++, and anything in the Universe they can sucker > somebody into buying in borgified form. > > Borgifying MPI is the most humorous thing in the Universe, BTW, given > its historical origins -- it was basically a language written > (reluctantly!) by supercomputer vendors when the US government got tired > of paying for all their important codes to be ported to each new > generation of proprietary hardware with its proprietary low level calls. > MS is doubtless trying to figure out just how much of that they can > undo while building up a big enough market share and enough vendors of > closed source applications written with their borgisms that they can... > > Oh wait. It IS GPL. Do you think that they actually read it? > > However, I was really referring to the other aspects of program > development and performance tuning associated with using a closed source > development environment. > > Resistance is Futile. > > rgb > >> >> Once a set of boxes are interconnected and you have library >> support for MPI or another way to share data (PVM... whatever) >> you are off and running in the clustering world. Sadly MS >> has a MS specific library that abuses "standard MPI" and could >> quickly cause source code to surface that runs correctly or on a >> MS cluster but not on another OS based cluster (Linux, Solaris, >> Irix, AIX). I see this all the time with java script, and c, >> c++, and other codes where little 'features' hook you in. >> >> Some will be fooled into thinking that this is something to look at >> or worse something to spend money on. >> >> SUMMARY: >> Since you posted this on 1 Apr 2007 all I can do is giggle >> and wonder why I replied. >> >> Regards, >> mitch >> >> PS: Ask in a year but not on April fools/joke day. >> >> > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From robin at workstationsuk.co.uk Wed Apr 4 01:31:29 2007 From: robin at workstationsuk.co.uk (Robin Harker) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Ethernet break through? In-Reply-To: References: <9FA59C95FFCBB34EA5E42C1A8573784F76C0A3@mtiexch01.mti.com> <8CC53E49-E29F-4C45-B7A9-C82E46A663D2@myri.com> <51788.192.168.1.1.1175463588.squirrel@mail.eadline.org> <4612518E.9050003@tamu.edu> <5a1205b30704031126w1fe96571m5d6b06e146be95c0@mail.gmail.com> <20070403190901.GA6056@dhcp-2-200.internal.keyresearch.com> <4612A4BC.5010806@aol.com> <4612A50A.8020108@aol.com> Message-ID: <4613.86.135.104.48.1175675489.squirrel@webmail.hostme.co.uk> Mark, You seam to be making an awful lot of assumptions about what a game console can or can't do. Let's have some facts rather than vague guesswork. As I understand it the PS3 has a decent GigE port, it runs a standard Linux OS, YDL, and it has both a general purpose CPU, (Power5) and 6 (I think) cell processors. I would have thought a repetitive application like rendering would be ideal for a cell based system. I do not understand your reference to SLI cards. I do agree that a PS2 is probable not a good render node, but then again it's ~4 years old. Regards Robin >> you say that, but don't PS2 and PS3 make ideal compute nodes for >> some >> applications like rendering and 3D and 4D (time as the 4th) ? > > I doubt it. while gaming consoles are admirably commoditized (actually > sold at a loss in some cases), they're designed with constraints different > from a cluster. for instance, looking pretty, or working well in a > livingroom environment. or playing music, or reading hd-dvd's, etc. > > as render nodes, I don't think they're all that good, though perhaps > effective for the price. I would guess that for rendering, you would do > the very best to use last-gen cards in the $150 range. should be able to > put two in a machine - sli support not necessary. such cards have > probably > had a respin to give them smaller/cooler chips, as well. > > I'd guess that the ethernet in gaming consoles is also not great. > >> Xboxes make good web servers for small scale. and use less energy >> than a >> equivalent Piii or P4 system :-) > > gross. webserving is so easy that tiny embedded chips to a good job, > and something like a via low-power would do a great job. 95% of a game > console would be wasted as a webserver. > >> end desktop. it's just risen in the gaming market because people >> want games >> which need greater computational power. likewise with gaming >> computers, they > > unclear. the gaming market is driven by eyecandy, and I sometimes wonder > whether, for instance, humans can even see jaggies at 60 fps. so much of > the gamer market is driven by what game-geek sites say about blowups of > stills of particular frames. > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > Robin Harker Workstations UK Ltd DDI: 01494 787710 Tel: 01494 724498 From hahn at mcmaster.ca Fri Apr 6 16:36:02 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu><38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> Message-ID: > Where would people on this list place the credit for HPC going > mainstream? If I had to pick one source, which is unfair to do, it would I believe there is just one source, perfectly fair: cheap hardware. I don't think any of the software is all that wonderful, or has it changed that much over the years. and the user doesn't really care anyway about, for instance, MPI. or, for that matter, whether the compute node runs linux or windows. From hahn at mcmaster.ca Fri Apr 6 16:46:48 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: > The job scheduler is responsible for managing the user credentials and > tokens as well as allocating CPUs to jobs. So if you are not going to I guess I'm a little puzzled by the value of the credentials. I presume these refer to the fork of kerberos that yall use, right? I can see how the user's credentials might be relevant to the scheduler (assuming you want restricted access to certain queues, etc.) but that's just the scheduler - the actual compute nodes shouldn't really care. using the credentials for file access seems kind of iffy to me, since it means that a whole cluster will be pounding on the corp's fileservers, which are almost certainly not equipped to handle that sort of thing. not to mention that this assumes that the compute nodes will each be neworked directly into the corp's lan. if the credentials are enabling, say, DB access, the same concern of bandwidth/topology would apply. to me, the concern with credentials seems to reflect an assumption of relatively small clusters and jobs. perhaps I'm misunderstanding, since as I said, ms-kerb is outside my experience. From landman at scalableinformatics.com Fri Apr 6 19:56:46 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu><38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <4617086E.2070207@scalableinformatics.com> Mark Hahn wrote: >> Where would people on this list place the credit for HPC going >> mainstream? If I had to pick one source, which is unfair to do, it would > > I believe there is just one source, perfectly fair: cheap hardware. Cheap and powerful hardware. COTS hardware was/is on a power curve (power delivered to applications) that non-COTS technologies could not match. Economies and customer demands of scale can do surprising things to capabilities and focus. The "ah-ha" moment for me was some time in 97 or 98 when I ran my old molecular dynamics code on a Pentium II or III (don't remember precisely), and it was within a factor of 2 in performance of the big RISC boxes I used, which cost far more than 10x more. Then the informatics codes around the 1999 time frame were faster on the Pentia than the RISC boxes. Cheaper and faster. This combination rarely loses. > I don't think any of the software is all that wonderful, or has it > changed that much over the years. and the user doesn't > really care anyway about, for instance, MPI. > > or, for that matter, whether the compute node runs linux or windows. On the contrary, I think (at least RGB has convinced me) that end users *do* care if costs scale linearly with number of processors. This provides sometimes significant barriers to them adopting more computational power. Costs in this case include acquisition and maintenance. If you set up some sort of diskless booting via Warewulf/Perceus/..., or highly simplified install via Rocks, or use a Scyld-like SSI-type model, the maintenance costs are going to be low in general. If you use something that requires a license per node, and additional costs due to security requirements, then your costs will likely scale linearly per node, and the maintenance costs will also scale like this. We know lots of users that stop at smaller clusters, not due to the cost and complexity of setting them up. Rather, they stop at particular sizes due almost entirely to the cost of the application software per seat. Most end users (at least in the commercial world) buy clusters to run pre-existing codes. Most ISVs like deploying to fixed "supportable" configurations. They also like it when a customer has a fixed budget, that more of the budget is spent with them rather than the linear scale products. Customers don't really care about the OS on the machine. They just care whether or not it will work well, how much it is going to cost them, and what the support issues will be. Its not what it is, its what you do with it. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 cell : +1 734 612 4615 From landman at scalableinformatics.com Fri Apr 6 20:09:20 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] NFS & Scaling issues In-Reply-To: <4612BB84.6010707@ideeinc.com> References: <4612BB84.6010707@ideeinc.com> Message-ID: <46170B60.90805@scalableinformatics.com> Hi Amrik: Amrik Singh wrote: > Hi, > > We are running a cluster of 180 diskless compute nodes. 60 of them have > 32 bit AMD Semptron processors and rest are dual core AMD Athelon 64 > bit processors. 32 bit machines have 10/100 mbps and rest have gigabit > ethernet cards. We have four file servers, each hosting around 3.5TB on > SATA drives connected to 3Ware RAID controller cards configured on RAID > 10 array. These file servers are exporting the drives through NFS. Each > file server is running 265 daemons for nfsd. > > The file servers are mainly hosting large number of small files ranging > from 256KB to 2 MB. The compute nodes are primarily doing a search > through these files, so there is lot's of reading and some writing to > the file servers. > > Recently we started noticing very high (70-90%) wait states on the file > servers when compute nodes. We have tried to optimize the NFS through > increasing the number of daemons and the rsize and wsize but to no avail. > > Can someone point us in the right direction as to how we should be > trying to troubleshoot this problem. You might want to look at the read patterns. > > PS: All the nodes are running SuSE 10.0 and servers are running SuSE10.0 > and 10.1 and all the drives are formatted with reiserfs. Hmmm... I remember Reiser has had a problem in the past when file systems get full or nearly so. There are file tail optimizations you might want to turn off, as well as use noatime for mounts. I might suggest turning to a better file system for your servers (if possible, it might not be a trivial undertaking), but even then that might not be responsible. Grab a copy of atop (google for it), run it on your file server. See if it is the file system that is problematic (disk devices running near 80% or higher capacity for reads/writes all the time). Other possibilities are your file access patterns, what the file server is doing itself, whether or not your networks are being flooded with small packets (see if your csw is very high, or the number of interrupts or packets are very high). Joe > > > thanks > -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 cell : +1 734 612 4615 From asabigue at fing.edu.uy Sat Apr 7 04:37:41 2007 From: asabigue at fing.edu.uy (Ariel Sabiguero Yawelak) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> Message-ID: <46178285.7080602@fing.edu.uy> Miller Ross wrote: > <...> > > That said, we as an industry do owe Microsoft one significant debt. > The standardization of Microcomputer hardware. I am not so sure. Microsoft deals (at least dealt) with software, not hardware. I am not completely sure if there is someone who shall get the credit or if it was just a coincidence that the PC the is the "winner". If someone must get some credit, that shall be given to IBM and Donald Estridge. Donald was responsible for building the PC based on an open architecture and to use components (software and hardware) from outside of IBM. Open architecture and off-the-shelf components made the IBM PC architecture becoming the de-facto standard. I believe that these are the same fundaments of beowulf, why the PC grew so much during this period, and a good example of how open standards promote innovation. Ariel > Up until DOS the world was so fragmented it was impossible to launch > anything in the software, or even specialized hardware arena using > economies of scale. With the world standardizing on the PC platform > with DOS stability and real desktop productivity gains became > possible. Alan Greenspan addressed congress in 1997. Having watched > the address I was personally impressed with his statement that the US. > per-capita GDP was dramatically higher than any other nation at that > time, because the US had a PC on pretty much every worker's desk that > needed access to a PC. Microsoft brought the US that standardization, > and those productivity gains. > > To the best of my recollection it went. > > U.S. 1.0 (normalized) > Japan .92 > Germany .87 > UK same range > and it went WAY down from there. > > That same low cost microcomputer is what we base our Beowulf clusters > on today, essentially a more reliable beefier version of a desktop > PC. Just as we don't want Beowulf to be overridden by 'adopt, > assimilate, and expand', we should give the credit where credit is due > that we are not using Commodore 64's, TI's, Atari's, or DEC- Rainbows > as cluster nodes. > > Ross > From hahn at mcmaster.ca Sat Apr 7 11:16:08 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46178285.7080602@fing.edu.uy> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <46178285.7080602@fing.edu.uy> Message-ID: >> That said, we as an industry do owe Microsoft one significant debt. >> The standardization of Microcomputer hardware. > I am not so sure. Microsoft deals (at least dealt) with software, not > hardware. msft, with intel and often leading vendors, have driven a long series of "standards", such as pc97 or pc99 (why is your ps2 keyboard connector purple?) . there's also a defacto standardization caused by whatever drivers are shipped (before any oemization). > if it was just a coincidence that the PC the is the "winner". If someone > must get some credit, that shall be given to IBM and Donald Estridge. > Donald was responsible for building the PC based on an open architecture I remember the 80's, and while there were some aspects we'd call open, "open" was definitely not the dominant meme of the industry. consider, for instance, the epic battle regarding bios cloning. it's also true that the PC was predated by other other open standards, perhaps even more open (heck, how about the S-100?) > and to use components (software and hardware) from outside of IBM. Open I would certainly agree that if IBM had been its normal anal-retentive self, then its products would not have been the starting point of the PC revolution. the result may have been better or worse, though. obviously, IBM was responsible for wiring the video buffer at 640K, for instance. > so much during this period, and a good example of how open standards > promote innovation. ethernet is a far better example of how open standards contributed to innovation. along with the IETF RFC process. beowulf would certainly not exist in any recognizable form without ethernet and IETF, which made possible the web-based internet of the 90's. hardware got cheap and commoditized (so any idiot could buy parts, snap them together, and have a working computer). my memory is that the 486 was really the first PC that had enough performance to be compelling to people used to sparc, mips, hp-pa and alpha risc chips. if you could match a sparcstation for a small fraction of the price, it drew attention. I would still claim that ethernet was a critical enabling feature, though (and Don Becker played a huge role in bringing it to Linux). by the time of the P5/60, it was pretty clear that ia32 owned the world. attack of the killer micros and all that... From jlforrest at berkeley.edu Fri Apr 6 16:57:18 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: Message-ID: <4616DE5E.4030706@berkeley.edu> One thing I've noticed about 64-bit computing in general is that it's being oversold. The **only** reason for running in 64-bit mode is if you need the additional address space. Indeed, for some apps this is critical and 64-bit computing solves a real problem. For apps that don't need the extra address space, the benefits of the additional registers in x86-64 are nearly undone by the need to move more bits around, so 32-bit and 64-bit modes are pretty much a push. When you add the additional difficulty of getting 64-bit drivers and what-not, I don't think it's worth messing with 64-bit computing for apps that don't need the address space. One additional way 64-bit computing is being oversold is that there aren't now, and maybe never will be, any human written program that requires more than 32 bits for the instruction segment of the program. It's simply too complex for a human, or a group of humans, to write this much code. Again, note that this says nothing about the data segment of a program. Also, people tell me that there are programs that were generated by other programs that are larger than 32 bits. I've never seen one, but maybe they exist, and that's what I'm talking about human written programs. Cordially, -- Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From dev.sandip at gmail.com Sat Apr 7 23:39:02 2007 From: dev.sandip at gmail.com (Sandip Dev) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Linux cluster for my college Message-ID: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> We are planning to set up a linux cluster on trail basis in our college.Since its merely a trail,we have been alloted PIII PCs with 128 MB Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this hardware support a cluster? Moreover which distro/software to begin with? If this trial succeeds we will get whatever hardware we want.But I just want this trial to succeed.Please help me. Regards, Sandip Dev, Student,SVNIT, (www.svnit.ac.in) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070408/c17b3455/attachment.html From James.P.Lux at jpl.nasa.gov Sun Apr 8 10:16:50 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.co m> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: <6.2.3.4.2.20070408101259.02febeb8@mail.jpl.nasa.gov> At 11:39 PM 4/7/2007, Sandip Dev wrote: > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with > 128 MB Ram and 40 GB HD on a 100 MBps LAN.All i want to know > is,will this hardware support a cluster? Yes. I built a cluster with VIA 533 MHz motherboards (diskless) using an ancient P2 (or maybe it was a PPro) as the headnode, and wireless LAN as the interconnect. Not exactly a ball-of-fire in the speed category (We could probably compete effectively for the "bottom500" list...) but fun to fool with, and worked for what I needed. >Moreover which distro/software to begin with? You might want to start with one of the "liveCD" styles of distros... You might also want to do "net boot" if your P3s can do that (do they have PXE?).. it's a lot easier if you only have to make changes in one place, and then just reboot all the nodes. > If this trial succeeds we will get whatever hardware we want.But > I just want this trial to succeed.Please help me. Define your "success criteria" carefully... James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From hahn at mcmaster.ca Sun Apr 8 10:28:39 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4616DE5E.4030706@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> Message-ID: > add the additional difficulty of getting 64-bit drivers for windows, at least. 64b-ness was never much of an issue for linux. > and what-not, I don't think it's worth messing with 64-bit > computing for apps that don't need the address space. I think you underestimate the number of jobs that can effectively use more than 2GB/proc, and which can make excellent use of having twice as many registers. not to mention the fact that the kernel likes having a big-flat address space, even if procs get by with 32b. 32b procs run rather well on 64b systems - you get small pointers and you don't get those annoying extra registers. just compile -m32. but I'm curious whether you have some data to back up the assertion that carrying around the extra bits is a significant cost. are you doing something involving lots of pointers (sparse matrices, perhaps or something with graphs/trees)? > One additional way 64-bit computing is being oversold > is that there aren't now, and maybe never will be, any > human written program that requires more than 32 bits > for the instruction segment of the program. It's simply I've heard it said that some DB's have surprisingly large text. From hahn at mcmaster.ca Sun Apr 8 11:02:21 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with 128 MB > Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this hardware > support a cluster? Moreover which distro/software to begin with? If this > trial succeeds we will get whatever hardware we want.But I just want this > trial to succeed.Please help me. I say you should keep it simple. just install some basic distro like centos on all the nodes (just a minimal install - you DEFINITELY don't want all the gui/user/etc junk). make sure you have ssh, but you can skip everything desktop-related. install mpich. be happy. if you're familiar with Linux, ssh and mpich, this should take not much time at all. things like monitoring and management are wonderful things, and necessary for any significant-sized cluster, but certainly not for a proof-of-concept. with 128M PIII's, the examples you'll be able to demonstrate will be quite small, but still believable if you avoid extras. regards, mark hahn. From landman at scalableinformatics.com Sun Apr 8 18:44:34 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4616DE5E.4030706@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> Message-ID: <46199A82.5030201@scalableinformatics.com> Hi Jon Jon Forrest wrote: > One thing I've noticed about 64-bit computing in general > is that it's being oversold. The **only** reason I disagree (with oversold). > for running in 64-bit mode is if you need the additional > address space. Indeed, for some apps this is critical and No. The extra registers make compiler optimization work better (lower register pressure). The flat memory model (doing away with segmented registers) simplifies addressing, and reduces the work of the processor for every physical address calculation (no more segment + offset operations to perform *before* pushing out the address onto the wires). 64 bit provides more registers and a simpler addressing mechanism. Well at least the x64 (AMD64, x86_64,...). > 64-bit computing solves a real problem. For apps that > don't need the extra address space, the benefits of > the additional registers in x86-64 are nearly undone > by the need to move more bits around, so 32-bit > and 64-bit modes are pretty much a push. When you I would love to see your data for this. Please note that I have quite a bit of data that contradicts this assertion (e.g. directly measured performance data, wall clock specifically of identical programs compiled to run in 32 and 64 bit mode on the same physical machine, running identical input decks). This is older data, from 2004. c.f. http://www.amd.com/us-en/assets/content_type/DownloadableAssets/dwamd_SI_rlsdWP1.0_.pdf but it is still relevant, and specifically, directly addresses the assertions. > add the additional difficulty of getting 64-bit drivers > and what-not, I don't think it's worth messing with 64-bit > computing for apps that don't need the address space. Which OS are you using? We haven't had 64 bit driver availability issues since late 2004, for Linux. For windows this may be different. For JackRabbit and our Windows 2003 x64 Server tests, we found it very difficult to find correct 64 bit Windows drivers (from Intel) for a network card. We eventually found it, using many google searches, trial and error, and some lucky guesses. > One additional way 64-bit computing is being oversold > is that there aren't now, and maybe never will be, any > human written program that requires more than 32 bits > for the instruction segment of the program. It's simply This is a bold assertion. Sort of like the "no program will ever use more than 640k of memory" made by a computing luminary many moons ago. > too complex for a human, or a group of humans, to write > this much code. Again, note that this says nothing landman@dualcore:~> cat hi.f90 program hi print *,"hi" end landman@dualcore:~> pgf90 hi.f90 landman@dualcore:~> ./a.out hi landman@dualcore:~> ls -alF a.out -rwxr-xr-x 1 landman users 2302794 Apr 8 21:38 a.out* 2.3 MB for hi. > about the data segment of a program. Also, people tell > me that there are programs that were generated by other > programs that are larger than 32 bits. I've never seen > one, but maybe they exist, and that's what I'm talking > about human written programs. I am sorry, but I think this may be an artificial strawman. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From landman at scalableinformatics.com Sun Apr 8 18:57:42 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] They appear to be reading beowulf .... Message-ID: <46199D96.6020702@scalableinformatics.com> C.f. http://news.zdnet.com/2100-3513_22-6174266.html ... especially quoting the eminently quotable RGB. Obviously Microsoft declares victory, or at least success. What marketing organization would admit failure (and keep their jobs)? What I would like to see is evidence of uptake. Real numbers, not marketing numbers. Our take has always been that we would support it if our customers asked us for it. We have proactively asked customers about it, and have not had interest expressed in it. From windows shops. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From landman at scalableinformatics.com Sun Apr 8 20:59:40 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4619B330.1000803@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> Message-ID: <4619BA2C.6070909@scalableinformatics.com> Jon Forrest wrote: > Joe Landman wrote: > >> No. The extra registers make compiler optimization work better (lower >> register pressure). The flat memory model (doing away with segmented >> registers) simplifies addressing, and reduces the work of the processor >> for every physical address calculation (no more segment + offset >> operations to perform *before* pushing out the address onto the wires). > > Right. I was talking about the difference in running an application > that fits in 32-bits in a 64-bit environment. There's a flat memory > when it runs this way, or when it runs in a 32-bit environment. ??? Flat memory is non-segmented by definition. Would you care to point out the flat memory addressing mode on x86 which can access all 4GB ram? I am sure I missed it. [...] > The tests I made myself were non-HPC, e.g. building large software > packages. But, I'm a reasonable person and I'll be glad to > modify my statement to say that 64-bit computing is oversold > in non-HPC markets. For example, when you look at pretty much Ok, here is where I guess I don't understand your point in posting this to an (obviously) HPC list then. Is it oversold in the gamer market? In the DB market? I have found very few exceptions 64 bit being faster. There are some, but not many. The arguments leveled by those claiming that you lose half your cache data density have been demonstrated not to be a significant concern for quite a number of real world apps. > any AMD-based computer these days, and compare it to what > was available ~2 years ago (I'm not sure of the exact date), what > difference do you see on the front panel? You'll see "AMD Athlon" > in both cases, but now you also see "64". On the majority > of computers being sold, this makes no difference. (HPC users ... and your point is .... that the 64 appendage makes no difference when you are running the chip in 32 bit mode (e.g. windows)? > are different). I bet most people think that since 64 is bigger > than 32 then a 64-bit computer is "better". Yet, this isn't the ok. I might suggest avoiding conflating marketing with technology. Also note that Athlon64 units do run noticably faster than Athlon units of similar clock speed in 32 bit mode. The desktop I am typing this on now is a 1.6 GHz Athlon 64 currently running a windows XP install, and is noticably (significantly) faster than the system it replaced (a 2 GHz Athlon). The minimal benchmarks I have performed indicate 30% ballpark on most everything, with some operations tending towards 2x faster. > case for them, especially if they're using a modern version of > Windows, which is what the original posting was about. These days you > also see "X2" which is a different kettle of fish and is, if anything, > being undermarketed. Undermarketed? Not the way I see it (see the Intel ads on TV) [...] >> This is a bold assertion. Sort of like the "no program will ever use >> more than 640k of memory" made by a computing luminary many moons ago. > > Bill Gates says he never said that. In any case, most of that was > due to the architectural inferiority of the x86 at the time. > What I'm talking about is a real limit in the complexity of > what a human, or group of humans, can create. Please name of > piece of software, free or commercial, that needs more > than a 32-bit address space for its instruction space. > As far as I know, there isn't any such thing. Not even close. [...] >>> about the data segment of a program. Also, people tell >>> me that there are programs that were generated by other >>> programs that are larger than 32 bits. I've never seen >>> one, but maybe they exist, and that's what I'm talking >>> about human written programs. >> >> I am sorry, but I think this may be an artificial strawman. > > If so, I don't see it. If my statement is true, that is that You set up an argument for the sole purpose of knocking it down. "no 32bit address needed for instruction text" ... "a real limit in the complexity" ... Of course, no program I have seen is ever *just* instruction text. There are symbols. Data sections, other sections. Whether or not ld can link anything that large is not something I am directly aware of, not having done it myself. My point with fortran is that the size of the binary (which is more relevant than the size of the simple instruction text) may in fact be quite large. And I still claim that your assertion is bold and difficult to support or contradict. Way back in the good old days of DOS, we regularly generated binaries that were 700+kB in size. Had to use overlays and linker tricks to deal with it. This was around the time of DOS 6. OS2 was a breath of fresh air, my codes could hit 2-3 MB without too much pain (binary size). Had 16 MB on the machine, so it was "plenty of room". I was not however of the opinion that 16 MB was all I would ever need. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From hahn at mcmaster.ca Sun Apr 8 22:42:43 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46199A82.5030201@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> Message-ID: > landman@dualcore:~> cat hi.f90 > program hi > print *,"hi" > end > landman@dualcore:~> pgf90 hi.f90 > landman@dualcore:~> ./a.out > hi > landman@dualcore:~> ls -alF a.out > -rwxr-xr-x 1 landman users 2302794 Apr 8 21:38 a.out* interesting - my x86_64 (pgf90 6.1-6) generates just 1.2 MB, of which 77KB is text (according to 'file'). >> about the data segment of a program. Also, people tell >> me that there are programs that were generated by other >> programs that are larger than 32 bits. I've never seen I think it's true that 2GB is large for text, but it's also not terribly relevant. I don't see much problem caused by 64 instruction addresses, since things like libraries seem to get along with 32b fixups (mcmodel=small). I have no idea what windows does. From landman at scalableinformatics.com Sun Apr 8 23:08:57 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4619C620.4010409@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> <4619C620.4010409@berkeley.edu> Message-ID: <4619D879.7030101@scalableinformatics.com> Jon Forrest wrote: > Joe Landman wrote: > >> ??? Flat memory is non-segmented by definition. Would you care to >> point out the flat memory addressing mode on x86 which can access all >> 4GB ram? I am sure I missed it. > > I'll be happy to withdraw this comment. I stand corrected on this ... x386 and higher allowed for up to 4GB per segment by virtue of the 32 bit E** registers. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From greg.lindahl at qlogic.com Sun Apr 8 23:44:43 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4616DE5E.4030706@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> Message-ID: <20070409064443.GA6305@localhost.localdomain> On Fri, Apr 06, 2007 at 04:57:18PM -0700, Jon Forrest wrote: > For apps that > don't need the extra address space, the benefits of > the additional registers in x86-64 are nearly undone > by the need to move more bits around, Man, good thing you noticed. We'll have to change the default on our compiler, and re-run all those benchmarks in 32-bit mode. I wonder why nobody ever noticed before? -- greg From landman at scalableinformatics.com Mon Apr 9 05:44:15 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461A1D97.6090104@moene.indiv.nluug.nl> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461A1D97.6090104@moene.indiv.nluug.nl> Message-ID: <461A351F.6090304@scalableinformatics.com> Toon Moene wrote: >> 2.3 MB for hi. > > Yeach. This is Debian AMD64 GNU/Linux, using > > $ gfortran --version > GNU Fortran 95 (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21) > $ gfortran hi.f > $ ls -l a.out > -rwxr-xr-x 1 toon toon 9754 2007-04-09 13:01 a.out > > Doesn't pgf90 allow dynamic run time libraries ? Yes. gfortran is considerably better in terms of size of binary. My point was simply that compilation tends to produce significantly more program text/symbols than one might expect. 4 GB is possible, though as I think the argument is a straw man, I won't press it. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From gdjacobs at gmail.com Mon Apr 9 06:47:39 2007 From: gdjacobs at gmail.com (Geoff Jacobs) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: <461A43FB.9080401@gmail.com> Sandip Dev wrote: > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with 128 > MB Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this > hardware support a cluster? Moreover which distro/software to begin > with? If this trial succeeds we will get whatever hardware we want.But > I just want this trial to succeed.Please help me. For this size of cluster, the distro you use doesn't matter particularly. I would assume the important part of the trial is a test subset of your target application, no? Choose your distro based on compatibility with the application(s), your familiarity level, compiler requirements, etc. Configure NFS, password-less ssh, install MPICH, replicate the password files, and (fingers crossed) Bob's your uncle. You probably don't want to be using a source-based distro like Gentoo, though. Once you have the basics licked, you can move on to something like Warewulf and other management stuff if you need to start looking larger scale. -- Geoffrey D. Jacobs From mwill at penguincomputing.com Mon Apr 9 09:04:21 2007 From: mwill at penguincomputing.com (Michael Will) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! Message-ID: <433093DF7AD7444DA65EFAFE3987879C33D9E9@orca.penguincomputing.com> I saw Spec benchmarks published with portions of it compiled for 32bit for best results. But the bottom line is that default in 64bit is faster and some special cases could benefit from 32bit... Michael Sent from my GoodLink synchronized handheld (www.good.com) -----Original Message----- From: Greg Lindahl [mailto:greg.lindahl@qlogic.com] Sent: Sunday, April 08, 2007 11:50 PM Pacific Standard Time To: Beowulf Mailing List Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Fri, Apr 06, 2007 at 04:57:18PM -0700, Jon Forrest wrote: > For apps that > don't need the extra address space, the benefits of > the additional registers in x86-64 are nearly undone > by the need to move more bits around, Man, good thing you noticed. We'll have to change the default on our compiler, and re-run all those benchmarks in 32-bit mode. I wonder why nobody ever noticed before? -- greg _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/1609227c/attachment.html From greg.lindahl at qlogic.com Mon Apr 9 09:14:02 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <433093DF7AD7444DA65EFAFE3987879C33D9E9@orca.penguincomputing.com> References: <433093DF7AD7444DA65EFAFE3987879C33D9E9@orca.penguincomputing.com> Message-ID: <20070409161402.GA4673@localhost.localdomain> On Mon, Apr 09, 2007 at 09:04:21AM -0700, Michael Will wrote: > I saw Spec benchmarks published with portions of it compiled for > 32bit for best results. But the bottom line is that default in 64bit > is faster and some special cases could benefit from 32bit... Oh, you want a *serious* answer! I don't know for SPECcpu2006, but with SPECcpu2000, a couple of the integer benchmarks ran faster in 32-bit, and none of the floating point benchmarks. Our compiler run 2x faster as a 32-bit binary, because it has very bad cache behavior when pointers get twice as large -- it uses a lot of them. But the general rule is that code is 15% faster in 64-bit, thanks to the extra registers and better calling convention. And a few codes (especially multi-precision integer codes like crypto) benefit from 64-bit integers. -- greg From peter.st.john at gmail.com Mon Apr 9 09:41:37 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] They appear to be reading beowulf .... In-Reply-To: <46199D96.6020702@scalableinformatics.com> References: <46199D96.6020702@scalableinformatics.com> Message-ID: Quoting from that link, > ...And even in a market with cultural barriers, pragmatism can win the day. That was the case for Matt Wortman, director of computational biology and information technology at the University of Cincinnati's Genome Research Institute. His group already has Linux clusters, but picked Windows CCS for a 14-node cluster that runs simulation software to analyze drugs' molecular behavior. It integrated easily with researchers' computers, 95 percent of which run Windows, he said. "I don't care if it's Microsoft or Scyld, (Linux cluster software from Penguin Computing)," Wortman said. "I want to make it easier for the average biologist to find new drugs." < This suggests, to me, confusing the user interface with the operating system; but my own soapbox is that MS, by designing for marketing and (arguably) ease-of-use (as opposed to designing for DP, like VMS, or for developing, like Unix) integrates the OS with the UI and identifies these two things in the mind of the consumer (like identifying a pretty girl with a brand of razor blades). The UI can make it easy for a biologist to **operate** the software. Unix makes it easier to configure, scale, and adapt the software. VMS might make it easier for the nodes to perform at their maximum. For the biologist to get the best results, we want him to be effective, and we want the compute nodes to be effective, and the budget to be effective. Making the UI easy for the users is not the only or biggest component of the OS decision. I'm sure Matlab runs on MSWin, but I'm also sure that all of them, from FORTRAN to MAXSYMA to Maple to whatever y'all are inventing in your labs right now, run on Unix. That's where all these things were and are invented...including the "Windows" UI technology (which MIT named XWindows in honor of Xerox) itself. Peter On 4/8/07, Joe Landman wrote: > > C.f. http://news.zdnet.com/2100-3513_22-6174266.html ... especially > quoting the eminently quotable RGB. > > Obviously Microsoft declares victory, or at least success. What > marketing organization would admit failure (and keep their jobs)? > > What I would like to see is evidence of uptake. Real numbers, not > marketing numbers. Our take has always been that we would support it if > our customers asked us for it. We have proactively asked customers > about it, and have not had interest expressed in it. From windows shops. > > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics LLC, > email: landman@scalableinformatics.com > web : http://www.scalableinformatics.com > phone: +1 734 786 8423 > fax : +1 734 786 8452 or +1 866 888 3112 > cell : +1 734 612 4615 > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/5f96f462/attachment.html From peter.st.john at gmail.com Mon Apr 9 10:04:22 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> Message-ID: On 4/2/07, Miller Ross wrote: > > > That said, we as an industry do owe Microsoft one significant debt. > The standardization of Microcomputer hardware. Up until DOS the > world was so fragmented ... Ross, We owe that debt to IBM, which decided on Open Architecture for their PC. (And making up, some, for the EBCDIC fiasco). The big winner in the deal was Microsoft, who got to keep rights to the OS they brought in; which would be like McDonald's outsourcing the french-fries and cokes (no matter how efficiently you make a hamburger, it can't have the huge profit margin of the 99-cent items). The payoff for IBM came later, difusely, and gradually; e.g., putting a zillion instances of linux on a 390 with VMWare, which would never have been possible without open architecture (and open softare). One might wonder what would have happened if instead IBM had called Fred Brooks, the curmudgeonly manger of OS360 who went to UNC: "Hey Fred, It's Big Blue here. Here's an assembler from Intel for their 8088 chip and a truckload of prototype microcomputers. Could you get a couple grad students together and write us an OS, by September? here's a check for a million dollars". Of course we'd be nowhere, because IBM would have fallen down marketing their OS as they did marketing their PC, they just weren't ready for dealing with the mass consumer market. MS contributed marketing. That's it. That's enough. Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/6b2d3eaa/attachment.html From peter.st.john at gmail.com Mon Apr 9 10:29:56 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4619BA2C.6070909@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: [regarding the 640K RAM "barrier" of the first IBM PC] > > > > Bill Gates says he never said that. In any case, most of that was > > due to the architectural inferiority of the x86 at the time. I'm sorry, I've lost track of who is quoting whom :-) but I just wanted to clarify this about the 640K limit. The Intel 80286 was fine, I ran SVr5 on it with a memory card giving me 1536 KB. However, to run DOS, I had to configure the memory past 640K as a Virtual Disk. I believe the issue was the location in memory of reserved space, stuck at the back end of the (640K) range, making it awkward to allocate memory to an application skipping over a reserved segment. So this was purely a MSDOS issue, had nothing to do with limitations of the 8088 (first IBM PC) much less the 80286 (on which we ran unix; an option then was Motorolla 680x0 for some x). The sufficiency of 640K was merely implied by the design of MSDOS, although I think IBM would have done better to go after engineering workstations and just port unix themselves (but that happened quickly because of Open Architecture). Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/88825c28/attachment.html From peter.st.john at gmail.com Mon Apr 9 10:33:45 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: d'uh, not that anyone cares but I meant SVr4 (not SVr5), unix System V Release Four. But it might have been r3.4 at first, I dont remember now. Peter On 4/9/07, Peter St. John wrote: > > > [regarding the 640K RAM "barrier" of the first IBM PC] > > > > > > > Bill Gates says he never said that. In any case, most of that was > > > due to the architectural inferiority of the x86 at the time. > > > I'm sorry, I've lost track of who is quoting whom :-) but I just wanted to > clarify this about the 640K limit. The Intel 80286 was fine, I ran SVr5 on > it with a memory card giving me 1536 KB. However, to run DOS, I had to > configure the memory past 640K as a Virtual Disk. > > I believe the issue was the location in memory of reserved space, stuck at > the back end of the (640K) range, making it awkward to allocate memory to an > application skipping over a reserved segment. So this was purely a MSDOS > issue, had nothing to do with limitations of the 8088 (first IBM PC) much > less the 80286 (on which we ran unix; an option then was Motorolla 680x0 for > some x). > > The sufficiency of 640K was merely implied by the design of MSDOS, > although I think IBM would have done better to go after engineering > workstations and just port unix themselves (but that happened quickly > because of Open Architecture). > > Peter > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/5a5bf86b/attachment.html From hahn at mcmaster.ca Mon Apr 9 10:46:35 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:05:59 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: > I believe the issue was the location in memory of reserved space, stuck at 640K is the location of the video buffer. this is a hardware issue, not sw. the cpu could address up to 1M (+64k, iirc.) From john.hearns at streamline-computing.com Mon Apr 9 11:00:27 2007 From: john.hearns at streamline-computing.com (John Hearns) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> Message-ID: <461A7F3B.3010308@streamline-computing.com> Peter St. John wrote: > > > MS contributed marketing. That's it. That's enough. > I disagree, strangely enough. Bob Brown has mentioned in this thread that the 'tipping point' for him came with the PII or PIV when code ran faster than big RISC machines. I'll throw into the mix that nearly all 'big science' applications at the time ran on VMS or mainframe OSes, or supercomputers (Cray or CDC). At the time, Unix was seen by scientists as an OS for longhairs and computer science types. Along comes the Intel i386 architecture, and just as importantly for scientific computing, the DEC Alpha. Scientists see a big price/performance gain with the Alpha architecture. But what did they both run - Windows NT. And remember that NT was developed by the man hired by Microsoft from DEC. So at one point in history there was a prospect of having a unified OS running on everything from the desktop (i386) through to the Alpha powered job farms to the MIPS powered big SMP machines. Purely personal opinion, but I believe that Microsoft missed a big trick by dropping Alpha support for NT. The other tipping point came with SunOS/Solaris. A C compiler came bundled with SunOS (I'm not sure of my facts here, but as I recall free for academic use). With the switch to Solaris the compiler became a paid-for extra. In my experience, it caused the group I Was working with to look seriously at the Gnu C compiler for the first time, and with it the rest of the GNU tools. -- John Hearns Senior HPC Engineer Streamline Computing, The Innovation Centre, Warwick Technology Park, Gallows Hill, Warwick CV34 6UW Office: 01926 623130 Mobile: 07841 231235 From peter.st.john at gmail.com Mon Apr 9 11:39:23 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call it a software issue of failing to work around the hardware issue. Obviously the hardware was not a show-stopper. But it was the 286 I did this on, not the earlier 8088, which I don't think could reasonably have been expected to run unix; but the original comment regarded the 80286. Peter On 4/9/07, Mark Hahn wrote: > > > I believe the issue was the location in memory of reserved space, stuck > at > > 640K is the location of the video buffer. this is a hardware issue, not > sw. > the cpu could address up to 1M (+64k, iirc.) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/10e5a83b/attachment.html From peter.st.john at gmail.com Mon Apr 9 12:00:03 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461A7F3B.3010308@streamline-computing.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: On 4/9/07, John Hearns wrote: > Peter St. John wrote: > > > > > > MS contributed marketing. That's it. That's enough. > > > I disagree, strangely enough. Actually Visual Basic is very very good at what it does; my generalization is certainly not absolutely fair. However, I don't see which of your points below credits to MS. > Bob Brown has mentioned in this thread that the 'tipping point' for him > came with the PII or PIV when code ran faster than big RISC machines. MS marketing + IBM Open Architecture + competition in hardware, commoditized the intel/clone platform, and that was great, sure. But I think MS's main contribution to that was the marketing. > I'll throw into the mix that nearly all 'big science' applications at > the time ran on VMS or mainframe OSes, or supercomputers (Cray or CDC). > At the time, Unix was seen by scientists as an OS for longhairs and > computer science types. Unix wasn't designed for data processing (as VMS was, I'd say) but for development. For example, it was weak in the early development of OLTP and real-time processing generally, but ended up into embedded processing because it could be made so small. > Along comes the Intel i386 architecture, and just as importantly for > scientific computing, the DEC Alpha. Scientists see a big > price/performance gain with the Alpha architecture. > But what did they both run - Windows NT. And remember that NT was > developed by the man hired by Microsoft from DEC. Yeah NT can be considered an offshoot of VMS, but spoiled somewhat with the integration into the OS of the UI, the way VMS integrates the Thick FS into the OS (sorta, thru a Record Management System layer that really you could skip if you wanted). But unix was ported to the intel/clone platform much earlier. I ran System V on my 286 when DOS was level 3.2. Before Windows 3.11; which was a good design because the UI was an application running on DOS, unlike later 98 which intergrated the UI with the OS. > So at one point in history there was a prospect of having a unified OS > running on everything from the desktop (i386) through to the Alpha > powered job farms to the MIPS powered big SMP machines. > Purely personal opinion, but I believe that Microsoft missed a big trick > by dropping Alpha support for NT. We did have one OS running on most everything then: unix. Unix ran on DEC, HP, intel/clone, etc, before NT, and with few meaningful exceptions (brief foray with Alpha), NT only runs on the one hardware line to this day. Which btw makes it harder to design a chip for anything other than compatibility with old chips. One thing I thought was very cool about DEC is that they supported your choice of VMS or Unix on the PDP11 (predecessor to VAX alpha). This was about the time Casette Basic was starting to look primitive :-) > The other tipping point came with SunOS/Solaris. > A C compiler came bundled with SunOS (I'm not sure of my facts here, but > as I recall free for academic use). With the switch to Solaris the > compiler became a paid-for extra. In my experience, it caused the group > I Was working with to look seriously at the Gnu C compiler for the first > time, and with it the rest of the GNU tools. When I borrowed my brother-in-law's Sun workstation (he was at Sun then) I typed "cc whatever.c -o whatever" and got a license-permissions error. I was flabbergasted. I asked him about it and his face dropped like a rock. He said he'd opposed it, but they wanted to track who needed C enough to ask, vs Java. So I learned to make a java object, and invoke an "interpreter" to compile it into a program. Which said "hello world" and then I went to bed, exhausted :-) I guess I just don't see which of these things you consider MS contributions? Peter > -- > John Hearns > Senior HPC Engineer > Streamline Computing, > The Innovation Centre, Warwick Technology Park, > Gallows Hill, Warwick CV34 6UW > Office: 01926 623130 Mobile: 07841 231235 > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/13c1d45c/attachment.html From James.P.Lux at jpl.nasa.gov Mon Apr 9 13:41:27 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: <6.2.3.4.2.20070409132508.02fa15a0@mail.jpl.nasa.gov> At 10:46 AM 4/9/2007, Mark Hahn wrote: >>I believe the issue was the location in memory of reserved space, stuck at > >640K is the location of the video buffer. this is a hardware issue, not sw. >the cpu could address up to 1M (+64k, iirc.) 8086/8088 can address 1M (20 bits).. 16 bit segment register*16+ 16 bit offset.. There is a trick, though, because the processor put out which of the 4 memory spaces was being accessed on the high order address lines during some of the later clocks (T2, etc.), so you could use that to have different physical memory spaces (each 1 M long) addressed for Alt Data, Code, Stack, and Data. FFFF:10 is actually the same as 0:0 and, as I recall, wouldn't result in a trap if you used addresses that "wrapped around" (the address generator didn't feed the carry bit to the trap generator). That is, you could use FF00:1000 as a legal address But, after all, who'd ever use more than 64K of code, stack, or data space for a single task... you were just going to port your 8080 code over anyway, right? When a more virtual memory scheme came in(286), and the segment register became more of a pointer to a table, things did change a bit, and it might have generated a trap if you wrapped around physical memory. At least you had a larger virtual address space (2G) that mapped into 16M of physical address space. James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From amacater at galactic.demon.co.uk Mon Apr 9 13:51:59 2007 From: amacater at galactic.demon.co.uk (Andrew M.A. Cater) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461A7F3B.3010308@streamline-computing.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: <20070409205159.GA11233@galactic.demon.co.uk> On Mon, Apr 09, 2007 at 07:00:27PM +0100, John Hearns wrote: > Peter St. John wrote: > > > > > >MS contributed marketing. That's it. That's enough. > > > I disagree, strangely enough. > > Bob Brown has mentioned in this thread that the 'tipping point' for him > came with the PII or PIV when code ran faster than big RISC machines. > > I'll throw into the mix that nearly all 'big science' applications at > the time ran on VMS or mainframe OSes, or supercomputers (Cray or CDC). > At the time, Unix was seen by scientists as an OS for longhairs and > computer science types. > Late model Crays now run Linux or cut down versions thereof :) I regard a Cray much as I do a nuclear reactor. What goes on in the middle is frighteningly fast and potentially dangerous to hang around or think of too much, but, in the same way that the reactor has lead and 16 feet of concrete between you and hot stuff, the Cray I/O is _so_ bad by comparison to its internals that you're tolerably well insulated :) Also, the number of expert Cray admins is very small, their knowledge is specialist and recherche and you literally can't do without them and their insider knowledge when there are any problems - sounds very like a nuclear reactor admin team :) > Along comes the Intel i386 architecture, and just as importantly for > scientific computing, the DEC Alpha. Scientists see a big > price/performance gain with the Alpha architecture. I found an Alpha workstation under a desk once and was able to borrow it for home use for a while - it had been sat unused for some years because people were concerned during Y2K and had disconnected it from the network. Nicest machine I've used - but a bugger to get parts for :) DECs were bulletproof, I swear: the only thing I've ever lifted that was comparable was an RCA AR88D receiver. [Ex WWII Lend Lease and v. heavy]. > But what did they both run - Windows NT. And remember that NT was > developed by the man hired by Microsoft from DEC. > So at one point in history there was a prospect of having a unified OS > running on everything from the desktop (i386) through to the Alpha > powered job farms to the MIPS powered big SMP machines. > Purely personal opinion, but I believe that Microsoft missed a big trick > by dropping Alpha support for NT. > No, it was never pitched at the right market. NT 3.1 or so was too early to do anything useful with 64 bits. > > The other tipping point came with SunOS/Solaris. > A C compiler came bundled with SunOS (I'm not sure of my facts here, but > as I recall free for academic use). With the switch to Solaris the > compiler became a paid-for extra. In my experience, it caused the group > I Was working with to look seriously at the Gnu C compiler for the first > time, and with it the rest of the GNU tools. > I have a colleague who's still swearing at that one whenever he recalls it :) One of my first comments via his blog to Ian Murdock when he joined Sun was that he should make it easy to install a full GNU toolchain on all versions of Solaris. Luckily, when I last did this, I had an earlier version of GCC to fall back on - but the process of building GCC 4.01 on Solaris, though very instructive, was still bondage and discipline. [You know it's bad when you have to build newer versions of tar, gzip and sed just to be able to deal with the GNU source code tarball :) ] In my case, the tipping point to Unix-like OS's came when I realised that I couldn't easily get hold of AT&T kit, couldn't find someone who would sell me Coherent - and a copy of one of the PC magazines had a cut down Slackware on it in 1994 :) A day later and I had it installed - and the rest is relative history. > Andy > John Hearns From ajt at rri.sari.ac.uk Mon Apr 9 13:37:41 2007 From: ajt at rri.sari.ac.uk (Tony Travis) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: <461AA415.2090805@rri.sari.ac.uk> Peter St. John wrote: > Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call it > a software issue of failing to work around the hardware issue. Obviously > the hardware was not a show-stopper. > > But it was the 286 I did this on, not the earlier 8088, which I don't > think could reasonably have been expected to run unix; but the original > comment regarded the 80286. Hello, Peter. People have very short memories! Minix runs fine on an 8088: http://www.neonbox.org/minix_laptop/index.html I replaced a pdp11/34 running Unix version 7 with an 8086 running Minix! Hmm... I wonder if anyone remembers Amoeba? Tony. -- Dr. A.J.Travis, | mailto:ajt@rri.sari.ac.uk Rowett Research Institute, | http://www.rri.sari.ac.uk/~ajt Greenburn Road, Bucksburn, | phone:+44 (0)1224 712751 Aberdeen AB21 9SB, Scotland, UK. | fax:+44 (0)1224 716687 From landman at scalableinformatics.com Mon Apr 9 14:27:15 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070409205159.GA11233@galactic.demon.co.uk> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> <20070409205159.GA11233@galactic.demon.co.uk> Message-ID: <461AAFB3.40301@scalableinformatics.com> Andrew M.A. Cater wrote: > I have a colleague who's still swearing at that one whenever he > recalls it :) One of my first comments via his blog to Ian Murdock when > he joined Sun was that he should make it easy to install a full GNU > toolchain on all versions of Solaris. Luckily, when I last did this, > I had an earlier version of GCC to fall back on - but the process of > building GCC 4.01 on Solaris, though very instructive, was still > bondage and discipline. [virtual coffee hits real screen] > > [You know it's bad when you have to build newer versions of > tar, gzip and sed just to be able to deal with the GNU source code > tarball :) ] Yup. Had to do that already. Wasn't amused by this. I keep hearing how this is "better". Yeah. Ok. FWIW: Nextenta is Solaris inside of a Ubuntu wrapper. Haven't loaded it yet, plan to at some point. FWIW2: (and not a sun shill here) Studio 11 is *free*. Doesn't seem to generate as good (e.g. fast) code for linux as its commercial competitors under linux (nor for that matter is the code it generates under Solaris as fast as the code other compilers generate under Linux, at least for my test cases) -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From James.P.Lux at jpl.nasa.gov Mon Apr 9 14:42:43 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> At 11:39 AM 4/9/2007, Peter St. John wrote: >Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call >it a software issue of failing to work around the hardware issue. >Obviously the hardware was not a show-stopper. > >But it was the 286 I did this on, not the earlier 8088, which I >don't think could reasonably have been expected to run unix; No problem running unix on a 8086.. (Don't forget Xenix on 8086 or 80286) Or, for that matter, on a 68000. (maybe the 68010 with the mmu or the 020 with virtual memory?.. I seem to remember DRAM refresh by 64 NOPs on the timer tick) From peter.st.john at gmail.com Mon Apr 9 14:54:28 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461AA415.2090805@rri.sari.ac.uk> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> <461AA415.2090805@rri.sari.ac.uk> Message-ID: Tony, I should have said, ** I ** wouldn't have reasonably expected unix to run on an 8088 at the time (System V booted with 512K on a 286 but "vi temp" hung so I had to expand memory). At the time I was unaware of any versions besides Berkeley and AT&T. Now of course even IBM can boot linux on a wrist-watch (but the power supply is ungainly). At the time 8-bit word seemed inadequate; I could not find a way to buy quantity one 3B2 from Ma, so motorolla (e.g. Fortune 32:16 nice box in '83 made me want a unix workstation for home) and intel 286 seemed like only options I could find. But I didn't know about usenet back then. When was Minix ported to 8088? Some people kept PDP11s running for a pretty long time :-) Peter On 4/9/07, Tony Travis wrote: > > Peter St. John wrote: > > Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call it > > a software issue of failing to work around the hardware issue. Obviously > > the hardware was not a show-stopper. > > > > But it was the 286 I did this on, not the earlier 8088, which I don't > > think could reasonably have been expected to run unix; but the original > > comment regarded the 80286. > > Hello, Peter. > > People have very short memories! Minix runs fine on an 8088: > > http://www.neonbox.org/minix_laptop/index.html > > I replaced a pdp11/34 running Unix version 7 with an 8086 running Minix! > > Hmm... I wonder if anyone remembers Amoeba? > > Tony. > -- > Dr. A.J.Travis, | mailto:ajt@rri.sari.ac.uk > Rowett Research Institute, | http://www.rri.sari.ac.uk/~ajt > Greenburn Road, Bucksburn, | phone:+44 (0)1224 712751 > Aberdeen AB21 9SB, Scotland, UK. | fax:+44 (0)1224 716687 > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/6ed7b079/attachment.html From peter.st.john at gmail.com Mon Apr 9 15:07:03 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> Message-ID: Jim, You are right, Xenix was available for the 286 in 85 (I bought my 286 in late 83 I think) but was available for the 8088 (the CPU on the IBM PC, everyone says 8086 because of the sequence 80286, 80386...; I think the 80186 was a special purpose processor that turned out to be more apt for pursuing a CPU line than the 8088 was) in 83. At the time it seemed the conventional wisdom was (to develop on unix) to invest in the bigger word and I was oblivious to people running Xenix on a base PC, but apparently they did, at least it was available. I remember, I think rather later, I looked up the command "uucp" in the Xenix programming manual (idle curiousity how they would define the acronym) and it said "Xenix to Xenix Copy", suggesting this would only communicate with another Xenix machine (they had expunged the word "unix" from their docs). I don't think I ever would have considered paying money for that. Ken is one of the nicest guys I've met and that's just insulting. Peter On 4/9/07, Jim Lux wrote: > > At 11:39 AM 4/9/2007, Peter St. John wrote: > >Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call > >it a software issue of failing to work around the hardware issue. > >Obviously the hardware was not a show-stopper. > > > >But it was the 286 I did this on, not the earlier 8088, which I > >don't think could reasonably have been expected to run unix; > > > > No problem running unix on a 8086.. (Don't forget Xenix on 8086 or > 80286) Or, for that matter, on a 68000. (maybe the 68010 with the > mmu or the 020 with virtual memory?.. I seem to remember DRAM refresh > by 64 NOPs on the timer tick) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/e085b250/attachment.html From mwill at penguincomputing.com Mon Apr 9 15:10:19 2007 From: mwill at penguincomputing.com (Michael Will) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: Message-ID: <433093DF7AD7444DA65EFAFE3987879C3BA8C9@orca.penguincomputing.com> I used minix on 286 briefly until I found out about the 3-floppy MCC linux distribution that included gcc around 1992 ;-) I never got to play with amoeba since both Minix and Amoeba where not free at the time linux came out - did you? http://www.cs.vu.nl/pub/amoeba/amoeba.html shows an 80 node amoeba cluster based on sparc and seems to be free now. Michael ________________________________ From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Peter St. John Sent: Monday, April 09, 2007 2:54 PM To: Tony Travis Cc: Beowulf Mailing List Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! Tony, I should have said, ** I ** wouldn't have reasonably expected unix to run on an 8088 at the time (System V booted with 512K on a 286 but "vi temp" hung so I had to expand memory). At the time I was unaware of any versions besides Berkeley and AT&T. Now of course even IBM can boot linux on a wrist-watch (but the power supply is ungainly). At the time 8-bit word seemed inadequate; I could not find a way to buy quantity one 3B2 from Ma, so motorolla ( e.g. Fortune 32:16 nice box in '83 made me want a unix workstation for home) and intel 286 seemed like only options I could find. But I didn't know about usenet back then. When was Minix ported to 8088? Some people kept PDP11s running for a pretty long time :-) Peter On 4/9/07, Tony Travis wrote: Peter St. John wrote: > Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call it > a software issue of failing to work around the hardware issue. Obviously > the hardware was not a show-stopper. > > But it was the 286 I did this on, not the earlier 8088, which I don't > think could reasonably have been expected to run unix; but the original > comment regarded the 80286. Hello, Peter. People have very short memories! Minix runs fine on an 8088: http://www.neonbox.org/minix_laptop/index.html I replaced a pdp11/34 running Unix version 7 with an 8086 running Minix! Hmm... I wonder if anyone remembers Amoeba? Tony. -- Dr. A.J.Travis, | mailto:ajt@rri.sari.ac.uk Rowett Research Institute, | http://www.rri.sari.ac.uk/~ajt Greenburn Road, Bucksburn, | phone:+44 (0)1224 712751 Aberdeen AB21 9SB, Scotland, UK. | fax:+44 (0)1224 716687 _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/0463fafc/attachment.html From peter.st.john at gmail.com Mon Apr 9 15:32:51 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <433093DF7AD7444DA65EFAFE3987879C3BA8C9@orca.penguincomputing.com> References: <433093DF7AD7444DA65EFAFE3987879C3BA8C9@orca.penguincomputing.com> Message-ID: I got my 286 in late 83, didn't get a HD for it and install unix until maybe 85 or 6. I paid money for Microport (not to be confused with Micropro, which had OS9, a unix work-alike for real time, not to be confused with OS2...) which licensed System V (and Im' thinking SVr3.4, I must have the books somewhere). It was less than half the price of Xenix, but Xenix was available. It was around 92 that I switched to linux when I got my 486. Up to that time I had made do with a 386 daughter-card for my 286, which was what brought my memory all the way up to 1536K, woo hoo; I had been 512 + 256 + 128 = 896, I think, by populating up the motherboard (which came with 512, the original IBM AT) and an expansion/extension card (I dont remember which was which now). The distro was a box full of many > 3 floppies, Slackware? that seemed better than downloading over my phone line, which was maybe 1200 baud then? I mean, bps :-) Tony? Peter On 4/9/07, Michael Will wrote: > > I used minix on 286 briefly until I found out about the 3-floppy MCC > linux distribution that included gcc around 1992 ;-) > > I never got to play with amoeba since both Minix and Amoeba where not free > at the time linux came out - did you? > > http://www.cs.vu.nl/pub/amoeba/amoeba.html shows an 80 node amoeba cluster > based on sparc and seems to be free now. > > Michael > ------------------------------ > *From:* beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] *On > Behalf Of *Peter St. John > *Sent:* Monday, April 09, 2007 2:54 PM > *To:* Tony Travis > *Cc:* Beowulf Mailing List > *Subject:* Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > > Tony, > I should have said, ** I ** wouldn't have reasonably expected unix to run > on an 8088 at the time (System V booted with 512K on a 286 but "vi temp" > hung so I had to expand memory). At the time I was unaware of any versions > besides Berkeley and AT&T. Now of course even IBM can boot linux on a > wrist-watch (but the power supply is ungainly). At the time 8-bit word > seemed inadequate; I could not find a way to buy quantity one 3B2 from Ma, > so motorolla ( e.g. Fortune 32:16 nice box in '83 made me want a unix > workstation for home) and intel 286 seemed like only options I could find. > But I didn't know about usenet back then. > > When was Minix ported to 8088? Some people kept PDP11s running for a > pretty long time :-) > Peter > > > On 4/9/07, Tony Travis wrote: > > > > Peter St. John wrote: > > > Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call > > it > > > a software issue of failing to work around the hardware issue. > > Obviously > > > the hardware was not a show-stopper. > > > > > > But it was the 286 I did this on, not the earlier 8088, which I don't > > > think could reasonably have been expected to run unix; but the > > original > > > comment regarded the 80286. > > > > Hello, Peter. > > > > People have very short memories! Minix runs fine on an 8088: > > > > http://www.neonbox.org/minix_laptop/index.html > > > > I replaced a pdp11/34 running Unix version 7 with an 8086 running Minix! > > > > > > Hmm... I wonder if anyone remembers Amoeba? > > > > Tony. > > -- > > Dr. A.J.Travis, | mailto:ajt@rri.sari.ac.uk > > Rowett Research Institute, | http://www.rri.sari.ac.uk/~ajt > > Greenburn Road, Bucksburn, | phone:+44 (0)1224 712751 > > Aberdeen AB21 9SB, Scotland, UK. | fax:+44 (0)1224 716687 > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org > > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/22d02077/attachment.html From James.P.Lux at jpl.nasa.gov Mon Apr 9 18:02:19 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> Message-ID: <6.2.3.4.2.20070409172211.0300a508@mail.jpl.nasa.gov> At 03:07 PM 4/9/2007, Peter St. John wrote: >Jim, >You are right, Xenix was available for the 286 in 85 (I bought my >286 in late 83 I think) but was available for the 8088 (the CPU on >the IBM PC, everyone says 8086 because of the sequence 80286, >80386...; I think the 80186 was a special purpose processor that >turned out to be more apt for pursuing a CPU line than the 8088 was) in 83. The 80186 and 80188 were higher integration versions of the 8086 and 8088 with onchip peripherals (such as memory decodes, interrupt controller, counter/timer, etc.) You can still buy 80188 based single board computers for industrial control kinds of applications. The 8086 had a heyday in the MultiBus era, but because it's a 16 bit bus it made for expensive hardware (had to have two 8bit wide proms to boot from, and 16 bit wide ram, etc.).. the 8088 was designed to address this for low cost systems (e.g. the IBM PC), especially because it could use all the 8080 and 8085 peripherals (8259, 8255, 8250, 8253, etc.) for which there was extensive design, programming, and layout expertise. I can't recall, off hand, whether the 8088 had the same hooks that the 8086 had for multiprocessor systems (more correctly, whether the iAPX/88 and iAPX/86 system architectures...) In a fascinating connection with clusters, the original Intel iPSC/1 Hypercube used ethernet as the interconnect among the processors, and used 286 SBCs as the nodes. Each node had multiple Ethernet interfaces, depending on the cube order (i.e. an order 3 cube had 4 interfaces for each node.. 3 to other nodes and 1 to the cube manager). The iPSC/2 moved up to 386/387 nodes (and i960 or i860 nodes) and a more specialized interconnect. Both ran Xenix or Sys.V, as I recall. The basic programming model was message passing between processes, and the "cube manager" (what would be called a head node in Beowulfery) would be used to fire up processes on nodes. The iPSC/1 was in early 80s... I think the first hypercube was at CalTech a few years earlier, and may have been an early instance of the use of MPI. I had a "sugarcube" demo unit for a few months, which was a 4 processor box, with the cube manager separate. Enough to fool around with message passing and such. From rgb at phy.duke.edu Tue Apr 10 05:47:48 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu><38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <74735BF202608043B11025A9FAA9438905881214@WIN-MSG-21.wingroup.windeploy.ntdev.microsoft.com> Message-ID: On Mon, 2 Apr 2007, Ryan Waite wrote: > Yeah, that marketing slogan wasn't too great since it sounds like we're > solely taking HPC mainstream. Ouch. Of course, that isn't the case. > > Where would people on this list place the credit for HPC going > mainstream? If I had to pick one source, which is unfair to do, it would > be the rich array of applications that exist, from BLAST to Fluent, from > Ganglia to LSF, from PERL to Matlab. Sorry, I just spent the weekend getting snowed and sleeted on while camping at the beach -- froze my **s off, but caught many blues out of the surf and had a great time. So this is bizzarely time-delayed (doubly so, from the datestamp above). But I couldn't resist. In my opinion, the place where HPC went COTS/mainstream even before "the beowulf" was when Vaidy Sunderam and Al Geist (in Jack Dongarra's group IIRC) wrote PVM at Oak Ridge in 1989. Before that people used clusters of workstations as rsh-based or sneakernet job distribution systems (did a bit of both myself, sneakernet back into the early-to-mid 80's with original IBM PCs and our first 286 boxes, the latter with Sun 4/110's, 386i's, SGI Irises, on thinwire). And I claim nothing special there -- anybody with big computing needs and multiple computers to work with has been using "clustering" basically forever, all the way back to Enigma days (which wouldn't be a bad place to begin in the non-COTs category). With PVM, building a distributed parallel supercomputer made out of COTS computers using an OTC network for IPCs was reduced to basically selecting unixoid (tcp/ip/rsh supporting) hardware and cabling it together, or (as in my case) "discovering" that an existing network of workstations already installed so that researchers could have compute resources and access to the rapidly growing list of resources available via "the Internet" (capital "I") WAS de facto a parallel supercomputer which for a variety of tasks could easily give you far, far better performance, for "free", than $12M Cray vector machines where if you begged nicely they'd give you "100 hours of Cray time" as if that would make everything wonderful. Frankly, PVM was the sine qua non of the original beowulf. One could easily build immensely large compute clusters out of Sun workstations -- I got up to roughly 130 machines all over Duke's campus working 24 hours a day on a major compute project back in 1995 -- or pretty much any other Unix boxes. One could even use a heterogeneous network of machines -- Vaidy demonstrated this with some lovely graphs when he gave a talk at a workshop I attended back in 1992 (I think) where he showed parallel speedup in a computation using a Cray, a pile of DECstations, some Sun workstations, and a few other things I cannot remember (maybe AIX boxen?) all at once. WITH PVM, building the original beowulf was a matter of adding network drivers to the linux kernel and working to speed up the current somewhat pedestrian 10 Mbps ethernet channels available for PCs until faster networks came along (as they always do). Without PVM, there would have been no point, as only true masochists would ever write a "real parallel program" on top of raw sockets. Applications are not what made parallel supercomputers mainstream at all. Most researchers have always written their own code, or used a set of code that was originally written by a single group and then disseminated throughout a community as a community resource. These applications could not have been written without a base to write on, and without PVM originally to "create" (or enable) a true COTS cluster with message passing capabilities, there would have been no hardware base to motivate the porting of MPI to open the door to moving all of the existing MPI applications to COTS clusters (well, there would have been, sure, but it would have been delayed by years instead of an idea waiting to happen). Anyway, the short history above is why the marketspeak implying that the advent of Windows Clusters somehow signals clustering becomming "mainstream" is so (amusingly) offensive to long time clustering people. Clustering was "mainstream" back in the days of Von Neumann and Alan Turing. Gene Amdahl wrote a seminal paper on clustering back in 1967. To quote from Greg Pfister's book (as quoted on wikipedia:-) "Virtually every press release from DEC mentioning clusters says 'DEC, who invented clusters...'. IBM didn't invent them either. >>Customers<< invented clusters, as soon as they couldn't fit all their work on one computer, or needed a backup. The date of the first is unknown, but it would be surprising if it wasn't in the 1960s, or even late 1950s." emphasis my own -- and note that it is clearly documented that sneakernet as a clustering technique dates back at least to Enigma and the mid-1940's. Clustering has clearly been "mainstream" in the research world at least since 1992-1993 (when PVM-evangelists spread the word that it was now possible to write portable programs for TCP/IP based virtual "supercomputers" on COTS hardware) and it has been IDENTIFIABLY mainstream with its own "brand name" since the beowulf project created the beowulf list and people who were co-inventing it on their own anyway all joined together to create a powerful and consistent support network to ease the way for newbies. The top500 list over the last decade or so (much as I generally deplore it) clearly marks out "a" visible history of the process of clustering achieving dominance, and there is hardly a mention of Microsoft on it anywhere. rgb > > Thanks for the feature requests, Joe. > > Ryan Waite > Group Program Manager - HPC > Microsoft Corporation > ryanw@microsoft.com > > -----Original Message----- > From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] > On Behalf Of Joe Landman > Sent: Monday, April 02, 2007 12:07 PM > To: Douglas Eadline > Cc: Erik Paulson; beowulf@beowulf.org > Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > > Douglas Eadline wrote: > >> I believe that if we do not protect against revisionist history, then > > [...] > > you mean like how now with WCCS2k+3 clustering and HPC is *now* > (suddenly magically spontaneously) "mainstream" ? > > This is just something I personally take issue with. The entire > explosive growth of clustering has driven HPC hard into the mainstream. > This happened long before it was a glimmer in their eyes. 6+ years of > > explosive growth, going from noise in the statistics to dominating the > statistics. Then along they came with WCCS2k+3. > > Their entry is late into the cycle. And if you listen to the comments > of the senior execs, it makes one wonder how committed they are to HPC > and clusters as compared to how committed they are to battling Linux. > > This is not to diminish their efforts. WCCS2k+3 is likely reasonably > good for some subset of groups. Microsoft has some good people there, > and playing with the W2k+3 x64 on our JackRabbit unit was fun. They > still need a real POSIX subsystem, and hopefully, someday, they will > give in, and get cygwin or mingw to be fully supported/shipping using > their compilers/tools. > > Though I expect to see airborn and stable flight from porcine critters > about the same time. Too bad, as that would likely ease > adoption/porting issues. Tremendously. > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Tue Apr 10 06:30:07 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: On Sun, 8 Apr 2007, Sandip Dev wrote: > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with 128 MB > Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this hardware > support a cluster? Moreover which distro/software to begin with? If this > trial succeeds we will get whatever hardware we want.But I just want this > trial to succeed.Please help me. This is a FAQ -- search the list archives and you will find it answered and discussed numerous times. In the meantime, the short answer is: a) Yes, you can make a perfectly lovely cluster out of this. b) Using any linux distro -- certainly any of the mainstream ones: FC, Debian, Ubuntu, SuSE/Novell, RHEL. c) The simplest way to start is "likely" to be using a cluster support package, e.g. OSCAR, warewulf, and/or a cluster-oriented distribution e.g. scientific linux. However, you can make a perfectly gangbusters cluster using FC-whatever kickstart installed to the nodes, or a Debian cluster, or a.... as long as the distribution installs on the nodes (ideally in a fully automated, PXE/DHCP-driven way) and the distribution provides PVM and MPI in one or more flavors, install linux, the compilers, and PVM/MPI and presto chango, your pile of PCs is an instant cluster. rgb > > Regards, > Sandip Dev, > Student,SVNIT, > (www.svnit.ac.in) > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From gdjacobs at gmail.com Tue Apr 10 06:41:52 2007 From: gdjacobs at gmail.com (Geoff Jacobs) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461AAFB3.40301@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> <20070409205159.GA11233@galactic.demon.co.uk> <461AAFB3.40301@scalableinformatics.com> Message-ID: <461B9420.2080306@gmail.com> Joe Landman wrote: > FWIW2: (and not a sun shill here) Studio 11 is *free*. Doesn't seem to > generate as good (e.g. fast) code for linux as its commercial > competitors under linux (nor for that matter is the code it generates > under Solaris as fast as the code other compilers generate under Linux, > at least for my test cases) How comprehensive is the implementation of each element of Forte 11 (esp FORTRAN)? -- Geoffrey D. Jacobs From rgb at phy.duke.edu Tue Apr 10 07:51:17 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46199A82.5030201@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> Message-ID: On Sun, 8 Apr 2007, Joe Landman wrote: >> 64-bit computing solves a real problem. For apps that >> don't need the extra address space, the benefits of >> the additional registers in x86-64 are nearly undone >> by the need to move more bits around, so 32-bit >> and 64-bit modes are pretty much a push. When you > > I would love to see your data for this. Please note that I have quite a > bit of data that contradicts this assertion (e.g. directly measured > performance data, wall clock specifically of identical programs > compiled to run in 32 and 64 bit mode on the same physical machine, > running identical input decks). This is older data, from 2004. c.f. > http://www.amd.com/us-en/assets/content_type/DownloadableAssets/dwamd_SI_rlsdWP1.0_.pdf > but it is still relevant, and specifically, directly addresses the > assertions. > >> add the additional difficulty of getting 64-bit drivers >> and what-not, I don't think it's worth messing with 64-bit >> computing for apps that don't need the address space. ... >> One additional way 64-bit computing is being oversold >> is that there aren't now, and maybe never will be, any >> human written program that requires more than 32 bits >> for the instruction segment of the program. It's simply > > This is a bold assertion. Sort of like the "no program will ever use > more than 640k of memory" made by a computing luminary many moons ago. > >> too complex for a human, or a group of humans, to write >> this much code. Again, note that this says nothing I totally agree with Joe on this issue. The "ideal" computer would have an infinite, flat address space, totally transparent to the user. Want to address memory location FF 0A BB 79 C3 12 93 54 6A 19 1D DA? (or simply have 2^90 \approx 10^27 data objects to manage)? The memory should be there, flat, transparent. Further, the "ideal" computer has a discretized binary representation of floating point numbers that is as close as possible to the real numbers they approximate for a variety of excellent numerical reasons. I remember reading any number of places how single precision floating point numbers were perfectly adequate for doing any sort of meaningful computation. I remember learning the hard way just how wrong this assertion is -- how much using double precision improves a long-running numerical computation both by slowing the rate of accumulation of the inevitable round-off errors and by admitting much larger exponents without having to manage them "by hand". I remember the joy of discovering IEEE 80 bit arithmetic in the venerable 8087, with more precision even than double. I remember how much FASTER native 80 bit arithmetic and then truncating to doubles is compared to doing double precision using library routines on top of an 8-bit or 16-bit or even 32-bit CPU. >> about the data segment of a program. Also, people tell >> me that there are programs that were generated by other >> programs that are larger than 32 bits. I've never seen >> one, but maybe they exist, and that's what I'm talking >> about human written programs. I don't understand how you could possibly imagine this to be true. I do numerical spin simulations on lattices in D dimensions. An N-dimensional spin (where N is not necessarily equal to D) is typically represented by 1-(N-1) real numbers (e.g. spherical polar angles). In addition any give spin may have other internal coordinates. To represent a spin therefore requires minimally order 4*N bytes for an ordinary 32-bit float representation of the spin coordinates, more likely 8*N bytes if one sensibly uses double precision coordinates. For 3D spins say 24 bytes per site. One then wishes to do simulations on the largest lattices possible. The constraint on lattice size is generally a mix of how much memory can hold and CPU speed, noting well that for cubic lattices the number of sites scales like L^D where L is the cube length in units of cartesian-indexed "sites". A 32 bit machine can address at most 4 GB of memory; in general purpose OS implementations this is generally reduced by the requirements of running the OS itself and a VM system to 3 GB (at least in a single data structure, without swapping). Well, if I put my 24 byte spins on a 1000x1000x1000 lattice I'm already up to 24 GB of memory. If I'm working on D=4 spaces or D=5 spaces, then a mere 100x100x100x100x100 lattice is 24x10^10 or 240 GB in size. Here the speed of doing arithmetic in 64 bits native AND the larger address space of 64 bit machines are absolutely essential to even play the game. This isn't an isolated (if specific) example. There is a vast range of memory-size bound problems, some of which have modest CPU requirements but an absolute necessity to be able to efficiently address large memory spaces. So much so that there have been cluster computing development efforts that focus on building very large flat memory models at the expense of computing speed -- the Trapeze project at Duke, for example. Here the point isn't do do lots of computation in parallel -- the application may even be single threaded. The parallel computer exists solely to provide the illusion of a vast reasonably flat memory space. There are other groups in the physics department here who would routinely buy 16+ GB machines (which obviously require 64 bit OS and hardware) if only they could afford all that memory as their computations easily scale out that far. They generally can afford only one or two "large memory" machines (which are still much more expensive than 2-4 GB machines as the price premium on really large memory sticks persists) but they'd LOVE to go large. Personally I "wish" that they'd done the dual core thing entirely differently. Instead of having two completely independent 64 bit cores per CPU, they might have built a 128-bit core with a hardware floating point execution pathway that permitted it to be transparently broken down into 4 32 bit parallel pathways, 2 64 bit pathways, 1 96 bit and 1 32 bit pathway, or 1 128 bit pathway, with entirely transparent flat memory access out to 128 bits, and with hardware implementation of 128 bit integer or 128 bit floating point arithmetic (on down). Leave it to a mix of the CPU, the OS, the compiler, and the application to decide how to pipeline and allocate the available ALUs, registers, cache lines, etc. to the needs of the program. But I'm not terribly worried. This to some extent describes the cell architecture, with some slop as to just where the ganging together of smaller logic units into larger ones occurs. And lots of very smart people are working on this -- smarter than me for sure -- and doubtless have far better ideas. Stating that there is no need for 64 bit architectures and that 32 bits is enough for anyone is basically equivalent to stating "the systems engineers working for AMD and Intel and IBM and Motorola are complete idiots". This is simply not the case. they aren't idiots, they are brilliant, and the simple fact of the matter is that 64 bit systems are faster, smarter, bigger, better than 32 bit systems. When AMD's opteron was first released, it was noted that it was the fastest >>32 bit<< architecture available at the time, because it was in aggregate faster to do 32 bit arithmetic (especially where a lot of that 32 bit arithmetic de facto involved 64 bit floats) on a 64 bit machine than it was on a 32 bit machine. I have watched from the days of the Z80 and 8088 (8 bit internal, 16 bit segmented address space) through the 8086 (16 bit internal, 16 bit segmented address space) through the 186 (very short except as a programmable device CPU), 286, 386, 486 (including the crippled SX), pentium, pentium pro, etc... with similar progress by AMD and nearly forgotten Intel competitors (Cyrix?) and completely different progression by Motorola with its FLAT 68000 memory space right on up to the Opteron, the 64-bit Xeon, the Athlon 64. With sundry side trips into Sparc, MIPS, and other workstation CPU architectures on the side, BTW. The process has been from the beginning been driven by a voracious public eager to take advantage of bigger address spaces, faster arithmetic and so on associated with larger data pathways. I fully expect to see 128 bit CPUs become a standard in the next decade, unless the cell approach does indeed represent a paradigm shift away from the notion of a "central" processing unit at all and we see instead on-the-fly reconfigurable multiprocessing units that can gang together to 128 bits (or even more) if that's what you need or can equally well function as a cluster of N 32 bit "thread execution units", where the OS kernel becomes basically a cluster operating system with a dynamic "cluster" or processing and memory resources interconnected by what amounts to a network. rgb > > I am sorry, but I think this may be an artificial strawman. > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From landman at scalableinformatics.com Tue Apr 10 08:42:51 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461B9420.2080306@gmail.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> <20070409205159.GA11233@galactic.demon.co.uk> <461AAFB3.40301@scalableinformatics.com> <461B9420.2080306@gmail.com> Message-ID: <461BB07B.7030001@scalableinformatics.com> Geoff Jacobs wrote: > Joe Landman wrote: > >> FWIW2: (and not a sun shill here) Studio 11 is *free*. Doesn't seem to >> generate as good (e.g. fast) code for linux as its commercial >> competitors under linux (nor for that matter is the code it generates >> under Solaris as fast as the code other compilers generate under Linux, >> at least for my test cases) > How comprehensive is the implementation of each element of Forte 11 (esp > FORTRAN)? Don't know, haven't used it extensively for codes. Tried their C/C++ compiler. Took quite a bit of effort to get it to work with the autoconf/gnu toolchain. Almost gave up (e.g. it wasn't worth the effort anyway). GCC gave about the same performance in some cases, and about 20% less in others. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From jmdavis1 at vcu.edu Tue Apr 10 09:00:52 2007 From: jmdavis1 at vcu.edu (Mike Davis) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461BB07B.7030001@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> <20070409205159.GA11233@galactic.demon.co.uk> <461AAFB3.40301@scalableinformatics.com> <461B9420.2080306@gmail.com> <461BB07B.7030001@scalableinformatics.com> Message-ID: <461BB4B4.8020306@vcu.edu> Also FWIW, We've limited our Studio 9,10,11 usage to Solaris thus far. The performance under Solaris 10 on both sparc and x64 has thus far been superior to Gnu. I haven't done a side by side of the same app with both linux and Solaris. Joe Landman wrote: > Geoff Jacobs wrote: > >> Joe Landman wrote: >> >> >>> FWIW2: (and not a sun shill here) Studio 11 is *free*. Doesn't seem to >>> generate as good (e.g. fast) code for linux as its commercial >>> competitors under linux (nor for that matter is the code it generates >>> under Solaris as fast as the code other compilers generate under Linux, >>> at least for my test cases) >> >> How comprehensive is the implementation of each element of Forte 11 (esp >> FORTRAN)? > > > Don't know, haven't used it extensively for codes. Tried their C/C++ > compiler. Took quite a bit of effort to get it to work with the > autoconf/gnu toolchain. Almost gave up (e.g. it wasn't worth the effort > anyway). GCC gave about the same performance in some cases, and about > 20% less in others. > > > From 06002352 at brookes.ac.uk Tue Apr 10 09:24:05 2007 From: 06002352 at brookes.ac.uk (Mitchell Wisidagamage) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <6.2.3.4.2.20070408101259.02febeb8@mail.jpl.nasa.gov> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> <6.2.3.4.2.20070408101259.02febeb8@mail.jpl.nasa.gov> Message-ID: <461BBA25.9000306@brookes.ac.uk> > > You might want to start with one of the "liveCD" styles of distros... > I'm playing around with a liveCD cluster these days (BCCD to be specific) along with VMWare. LiveCD clusters have auto discovery of nodes and extremely compact and fast (since they run on memory) and can set-up a cluster in minutes. For what sort of applications are they used for? Due to the fact that machines need to be configured (authentication,mount filesystems,install libs..etc) each time they are booted they seem to be very "inconvenient?" and have limited scope in the real world. Regards, Mitchell From rgb at phy.duke.edu Tue Apr 10 12:45:19 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461A7F3B.3010308@streamline-computing.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: On Mon, 9 Apr 2007, John Hearns wrote: > I disagree, strangely enough. > > Bob Brown has mentioned in this thread that the 'tipping point' for him came > with the PII or PIV when code ran faster than big RISC machines. > > I'll throw into the mix that nearly all 'big science' applications at the > time ran on VMS or mainframe OSes, or supercomputers (Cray or CDC). > At the time, Unix was seen by scientists as an OS for longhairs and computer > science types. > > Along comes the Intel i386 architecture, and just as importantly for > scientific computing, the DEC Alpha. Scientists see a big price/performance > gain with the Alpha architecture. > But what did they both run - Windows NT. And remember that NT was developed > by the man hired by Microsoft from DEC. > So at one point in history there was a prospect of having a unified OS > running on everything from the desktop (i386) through to the Alpha powered > job farms to the MIPS powered big SMP machines. > Purely personal opinion, but I believe that Microsoft missed a big trick by > dropping Alpha support for NT. > > The other tipping point came with SunOS/Solaris. > A C compiler came bundled with SunOS (I'm not sure of my facts here, but as I > recall free for academic use). With the switch to Solaris the compiler became > a paid-for extra. In my experience, it caused the group I Was working with to > look seriously at the Gnu C compiler for the first time, and with it the rest > of the GNU tools. As I mentioned earlier today (and several days late into the discussion) I think that the development of PVM was ultimately more important than hardware architecture. People had long since found ways of building clusters out of any compute resource that they had more than one of, even lacking a network between them. A network (something that was nearly perfectly co-developed with Unix -- to the extent where one has to wonder if Unix was every more than "the operating system that happened to be the basis for the scalably designed, WA-routable network";-) just made it easier for EP codes, but provided no real threat to big-iron vector machines. It was PVM that enabled true message passing parallel code to be written that made a pile of machines (be they Alphas, simple PCs, Sun workstations, Decstation 3100's, Cray YMPs, SGI Irises, IBM RS6000s) into "a parallel supercomputer" that exhibited clear near-linear speedup for a wide range of medium to coarse grained computations back in the early to mid 90's (which was still very late 386, late 486, and early to mid Pentium days, where floating point operations sucked on the Pentium -- giving rise to a previously ignorable company called American Micro Devices whose "5" chip had floating point that sucked less, and cost less, recall). However you are dead right about MS and Sun missing any number of boats. MS knifed IBM over OS/2 (which was a decently designed OS that might have given Unix a real run for its money) and hence lost out on all the technology and advances that OS/2 had going for it. Sun -- well, where to begin. They had the world to give away, and proceeded to give it away. It wasn't just unbundling their c compiler -- everything about Solaris was evil, and unnecessarily evil at that. SunOS went from outperforming the hardware it was running on and being one of the smoothest multitasking systems I ever used to being Slow-aris, with the broken scheduler from hell and a complete inability to talk and chew gum at the same time. And Sun had a perfectly viable X86 Unix in the early 90's and chose to sell it for exorbitant amounts to protect its SPARC line instead of mass-market it at $50 a seat while seducing PC developers to write their first true multitasking "windowing" applications for sunview or X. If they had done the latter, Linux might never have come into existence, NT might have been MS's last non-Unix operating system version, and I might be bemoaning Sun's world-dominating monopoly. I must say, though, that you have the wrong idea about scientists. My kids regularly make fun of my near-complete lack of hair, not it excessive length...;-) rgb > > > > > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From hahn at mcmaster.ca Tue Apr 10 13:29:02 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: > It was PVM that enabled true message passing parallel code to be written > that made a pile of machines (be they Alphas, simple PCs, Sun I'm not disagreeing, but wonder why PVM is basically extinct now. that is, why was MPI considered an improvement/replacement? > MS knifed IBM over OS/2 (which was a decently designed OS that might > have given Unix a real run for its money) and hence lost out on all the I worked on OS/2, and it was no peach on the inside, so to speak ;) it's hard to speculate about what-ifs on a system which had barely shaken off its initial hw target (12 MHz 286!) by the time it was dropped. From mwill at penguincomputing.com Tue Apr 10 14:26:32 2007 From: mwill at penguincomputing.com (Michael Will) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: Message-ID: <433093DF7AD7444DA65EFAFE3987879C3BA911@orca.penguincomputing.com> As far as I remember PVM has some overhead to facilitate message passing in a heterogenous environment that MPI does not have. I had a customers sample code that required some data to be moved from the headnode to a compute node and while on PVM they would get about 50MB/s MPI could transfer about 100MB/s across gigabit ethernet. Michael -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Mark Hahn Sent: Tuesday, April 10, 2007 1:29 PM To: Robert G. Brown Cc: Beowulf Mailing List Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! > It was PVM that enabled true message passing parallel code to be > written that made a pile of machines (be they Alphas, simple PCs, Sun I'm not disagreeing, but wonder why PVM is basically extinct now. that is, why was MPI considered an improvement/replacement? > MS knifed IBM over OS/2 (which was a decently designed OS that might > have given Unix a real run for its money) and hence lost out on all > the I worked on OS/2, and it was no peach on the inside, so to speak ;) it's hard to speculate about what-ifs on a system which had barely shaken off its initial hw target (12 MHz 286!) by the time it was dropped. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From amacater at galactic.demon.co.uk Tue Apr 10 14:26:29 2007 From: amacater at galactic.demon.co.uk (Andrew M.A. Cater) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: <20070410212629.GA8220@galactic.demon.co.uk> On Tue, Apr 10, 2007 at 03:45:19PM -0400, Robert G. Brown wrote: > I must say, though, that you have the wrong idea about scientists. My > kids regularly make fun of my near-complete lack of hair, not it > excessive length...;-) > > rgb > Sorry, we based our ideas on Greg Lindahl :) Once again, this list is full of useful historical viewpoints and a mine of information and experience. It got cited on the Debian user list this morning by me as a model - this just to let you all know what was said: If you want to see a very good example of a mailing list with a high signal-noise ratio, technical excellence and lots of interesting sidelights - the beowulf list (archives of www.beowulf.org) is always worth a browse. The major players there are real experts - even the off-topic stuff is interesting and thought-provoking Debian released a new stable release over the Easter weekend, so I've been busy helping test stuff out and answering user questions. I come here to this list for civilized company, knowledge and expertise and am never disappointed. I can find English folk song experts, radio amateurs, physicists and mathematicians scattered all over the 'Net - but I can't find a similar erudite and cultured _community_ in a single list anywhere else. All the best, Andy > Robert G. Brown http://www.phy.duke.edu/~rgb/ > Duke University Dept. of Physics, Box 90305 From greg.lindahl at qlogic.com Tue Apr 10 15:04:06 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <20070410212629.GA8220@galactic.demon.co.uk> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> <20070410212629.GA8220@galactic.demon.co.uk> Message-ID: <20070410220406.GA9173@localhost.localdomain> On Tue, Apr 10, 2007 at 09:26:29PM +0000, Andrew M.A. Cater wrote: > Sorry, we based our ideas on Greg Lindahl :) I'm not the only one! http://www.improbable.com/projects/hair/hair-club-top.html rgb just needs a good wig. -- greg From landman at scalableinformatics.com Tue Apr 10 15:18:00 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: <461C0D18.4080803@scalableinformatics.com> Mark Hahn wrote: >> It was PVM that enabled true message passing parallel code to be written >> that made a pile of machines (be they Alphas, simple PCs, Sun > > I'm not disagreeing, but wonder why PVM is basically extinct now. > that is, why was MPI considered an improvement/replacement? Marketing by the MPI groups? More to the point, it looked like MPI had significant backing. I don't know if it was "better" in a technical sense (though some folks do argue strongly about MPI being "better"). > >> MS knifed IBM over OS/2 (which was a decently designed OS that might >> have given Unix a real run for its money) and hence lost out on all the > > I worked on OS/2, and it was no peach on the inside, so to speak ;) It was nice in that I could run huge (at the time) jobs on my 16 MB ram PC. I remember running on one of those "sparc" units and this PC, and the PC was about the same speed (molecular dynamics). The brand new R3000 shiny SGI Indigos with a whopping 192 MB of ram and a huge 1 GB disk, these were great to work with. Much faster. Could do 100 time steps of MD in a week. FWIW, I can do about 1 time step in less than 20 seconds on my laptop (AMD Athlon 64 2GHz chip)... > it's hard to speculate about what-ifs on a system which had barely > shaken off its initial hw target (12 MHz 286!) by the time it was dropped. heh... OS2 was quite a bit better than win31. Somewhat better/more stable than w95/w98. It just never caught on in any meaningful sense. And about the time it tried to pick up momentum, Linux got to be installable/useful. I remember my first laptop (self purchased) had a 75 MHz pentium processor, a 2 GB disk, and I ran windows, OS2 and Linux (RH in the pre-5 days) on there. Pretty soon after that, OS2 fell off. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From James.P.Lux at jpl.nasa.gov Tue Apr 10 15:21:31 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: <6.2.3.4.2.20070410143435.030ce730@mail.jpl.nasa.gov> At 01:29 PM 4/10/2007, Mark Hahn wrote: >>It was PVM that enabled true message passing parallel code to be written >>that made a pile of machines (be they Alphas, simple PCs, Sun > >I'm not disagreeing, but wonder why PVM is basically extinct now. >that is, why was MPI considered an improvement/replacement? I will speculate that it's because "message passing" as a interprocess and interprocessor communications model came around first, and had a bit more traction. There are always those hardy souls willing to partition the problem themselves as opposed to some sort of magic virtual memory/machine model doing it for them. And, there were extensive libraries to divvy up the work using a message passing layer underneath. Why.. sitting here on a shelf in my office (as a curiosity with which to beat young whippersnappers/early career hires who think they've just invented such things) is a set of lecture notes and manuals for CCLISP (ah yes.. the glory days of LISP for everything) as well as all the math libraries for cluster computation for matrix math, etc. Those are all based on much earlier message passing work of the DECnet era.. I have a textbook on distributed computing from 1980 (Weitzman, "Distributed Micro/Minicomputer Systems") which describes all those schemes (DEC PCL-11B? IBM DPPX? TRW DDP? etc.), both shared memory and message passing oriented (note that this book explicitly doesn't talk about supercomputers, which, obviously, also had shared memory/crossbars/etc). It even mentions that new-fangled experimental interconnect from Xerox called Ethernet, although it talks about a variety of variants that might fix some of the observed problems.... Anyway.. the message passing schemes would work over "really slow" links (albeit at reduced performance) and would at least be functional for decomposing a problem. I would cringe to think of trying to do a PVM style task with 56,700 bps interconnects between processors. Remember, you're dealing with node processors the like of PDP11s and 1 microsecond cycle times (I'd have to drag out my 11/70 manual to see for sure).. Pushing out a megabit per second on ANY interface would have been a chore. James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From becker at scyld.com Tue Apr 10 15:52:52 2007 From: becker at scyld.com (Donald Becker) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Survey on beowulf.org -- and a drawing for a video iPod Message-ID: Survey on beowulf.org -- video iPod For those who haven't checked out the beowulf.org web pages recently, we've put up a survey about HPC. You can find the link on the home page of http://beowulf.org or go directly to the survey at http://www.beowulf.org/community/survey.php As an incentive, legitimate surveys (no automated submissions please!) completed by April 20 will be entered into a drawing for the gadget de jour -- a video iPod. Good luck -- Donald Becker becker@scyld.com Penguin Computing / Scyld Software www.penguincomputing.com www.scyld.com Annapolis MD and San Francisco CA From tjrc at sanger.ac.uk Sun Apr 8 09:09:05 2007 From: tjrc at sanger.ac.uk (Tim Cutts) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: On 8 Apr 2007, at 7:39 am, Sandip Dev wrote: > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with > 128 MB > Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this > hardware > support a cluster? Yes, although it without gigabit it will be a bit poor for MPI codes. Fine for embarrassingly parallel stuff though, or just testing the ideas out. > Moreover which distro/software to begin with? Whatever distro you are more comfortable with. I've used Red Hat in the past - these days I use Debian. There are also plenty of more cluster oriented flavours, like Rocks. But at the end of the day it doesn't really matter. If your school supports a particular Linux distro, go with what local expertise you have. Given its good support for non-European languages, you might want to try Ubuntu. Tim From toon at moene.indiv.nluug.nl Sun Apr 8 10:30:41 2007 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4616DE5E.4030706@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> Message-ID: <461926C1.3000802@moene.indiv.nluug.nl> Jon Forrest wrote: > One thing I've noticed about 64-bit computing in general > is that it's being oversold. The **only** reason > for running in 64-bit mode is if you need the additional > address space. I wouldn't be to sure of that. Recently I became interested in the divisors of a given natural number. The obvious Fortran program is: integer*8 n, i, j, k character*80 arg call getarg(1,arg) read(arg, '(i20)') n i = n j = 1 k = i / j do while (k .ge. j) if (i .eq. k * j) print*, k, j j = j + 1 k = i / j end do end [ OK, this isn't purely Fortran 77, but close enough that most compilers will do the *right thing* ] I tried it on 32 bit Linux (at work) and my laptop (AMD64 Linux), using gfortran 4.1. The execution time difference was about 2 orders of magnitude. -- Toon Moene - e-mail: toon@moene.indiv.nluug.nl - phone: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands At home: http://moene.indiv.nluug.nl/~toon/ Who's working on GNU Fortran: http://gcc.gnu.org/ml/gcc/2007-01/msg00059.html From rmiguel at usmp.edu.pe Sun Apr 8 12:02:41 2007 From: rmiguel at usmp.edu.pe (Richard Miguel San Martin) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] distributed storage Message-ID: <1176058961.46193c515a284@mail.usmp.edu.pe> Hi All, I have a beowulf cluster build with PIV PCS. We are needing a space of storage where put data related with a project of medical images. My question is about which is the best way for to deploy a distributed storage .. I mean, I thought to use distributed filesystems or perhaps an application for to build the distributed storage. Can someone say me your oppinion about it? Thanks in advance -- Richard Miguel -------------------------------------------------------- ~ | . . | "In a world without fences ... who needs (0 0 ) | GATES?" / \/ \ | // \\ | /(( _ ))\ | oo0 0oo | ---------------------------------------------------------- ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From robin at workstationsuk.co.uk Sun Apr 8 12:53:55 2007 From: robin at workstationsuk.co.uk (Robin Harker) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46178285.7080602@fing.edu.uy> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <46178285.7080602@fing.edu.uy> Message-ID: <1973.86.149.14.227.1176062035.squirrel@webmail.hostme.co.uk> And after IBM built it they tried very hard to keep Compaq, et al out from this Open Architecture. > Miller Ross wrote: >> <...> >> >> That said, we as an industry do owe Microsoft one significant debt. >> The standardization of Microcomputer hardware. > I am not so sure. Microsoft deals (at least dealt) with software, not > hardware. > I am not completely sure if there is someone who shall get the credit or > if it was just a coincidence that the PC the is the "winner". If someone > must get some credit, that shall be given to IBM and Donald Estridge. > Donald was responsible for building the PC based on an open architecture > and to use components (software and hardware) from outside of IBM. Open > architecture and off-the-shelf components made the IBM PC architecture > becoming the de-facto standard. > > I believe that these are the same fundaments of beowulf, why the PC grew > so much during this period, and a good example of how open standards > promote innovation. > > Ariel > >> Up until DOS the world was so fragmented it was impossible to launch >> anything in the software, or even specialized hardware arena using >> economies of scale. With the world standardizing on the PC platform >> with DOS stability and real desktop productivity gains became >> possible. Alan Greenspan addressed congress in 1997. Having watched >> the address I was personally impressed with his statement that the US. >> per-capita GDP was dramatically higher than any other nation at that >> time, because the US had a PC on pretty much every worker's desk that >> needed access to a PC. Microsoft brought the US that standardization, >> and those productivity gains. >> >> To the best of my recollection it went. >> >> U.S. 1.0 (normalized) >> Japan .92 >> Germany .87 >> UK same range >> and it went WAY down from there. >> >> That same low cost microcomputer is what we base our Beowulf clusters >> on today, essentially a more reliable beefier version of a desktop >> PC. Just as we don't want Beowulf to be overridden by 'adopt, >> assimilate, and expand', we should give the credit where credit is due >> that we are not using Commodore 64's, TI's, Atari's, or DEC- Rainbows >> as cluster nodes. >> >> Ross >> > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > Robin Harker Workstations UK Ltd DDI: 01494 787710 Tel: 01494 724498 From jlforrest at berkeley.edu Sun Apr 8 17:29:27 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> Message-ID: <461988E7.70904@berkeley.edu> Mark Hahn wrote: >> add the additional difficulty of getting 64-bit drivers > > for windows, at least. 64b-ness was never much of an issue for linux. The title of the message I was responding to was about Win64, but your point is valid. Switching to 64-bits is much easier in the Linux world than the WinXX world. > I think you underestimate the number of jobs that can effectively > use more than 2GB/proc, and which can make excellent use of having > twice as many registers. not to mention the fact that the kernel likes > having a big-flat address space, even if procs get by with 32b. I made no claims about the number of apps that need more address space than they get in the 32-bit world, only that additional address space is the only reason for making the switch. I admittedly don't have references at hand about my claim about the advantages of the additional registers being reduced by the increased number of bits. This came up several years ago when x86-64 was first coming out. If somebody has newer data I'll be the first to retract my claim. I think it makes intuitive sense, though. > 32b procs run rather well on 64b systems - you get small pointers > and you don't get those annoying extra registers. just compile -m32. Again, I agree that on Linux this is true, but in a Win64 cluster the additional difficulties of switching to 64-bits make this more of a problem. > I've heard it said that some DB's have surprisingly large text. I worked as a SQLServer developer at Sybase, and in the Postgres research group at UCB and I never noticed this. Maybe things have changed since then. The one area where I've heard text gets quite large is in CAD simulations, but much of the code isn't human written. For yuks, I recently added up the text size of *every* executable program and library I found on my Fedora Core 6 system. I did not do a very good job at weeding out the dups caused by symbolic links and the like, but the sum total of *all* these text segments would have easily fit in a 32-bit address space. -- Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From jlforrest at berkeley.edu Sun Apr 8 17:41:28 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461926C1.3000802@moene.indiv.nluug.nl> References: <4616DE5E.4030706@berkeley.edu> <461926C1.3000802@moene.indiv.nluug.nl> Message-ID: <46198BB8.6030605@berkeley.edu> Toon Moene wrote: > I wouldn't be to sure of that. Recently I became interested in the > divisors of a given natural number. I suspect small programs can be written to show almost any kind of interesting behavior. [program snipped] > The execution time difference was about 2 orders of magnitude. My guess is that this is a perfect x86-64 program. Everything fits in the cache, and there are enough registers so that all the variables fit in registers. The experiment I tried was to build several fairly large programs on Fedora Core 6 32 bit and then 64 bit using exactly the same hardware. This is clearly not the same kind of test as your example but it's more relevant as a systems-level comparison. If I remember, I built mysql and apache. This took about the same amount of time in both modes. I suspect that the Beowulf crowd would have lots of experience with 32-bits vs 64-bits question, so I'd welcome additional comments, especially dealing with situations where programs *don't* need the additional address space of the 64-bit model. Cordially, -- Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From jlforrest at berkeley.edu Sun Apr 8 20:29:52 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:06:00 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46199A82.5030201@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> Message-ID: <4619B330.1000803@berkeley.edu> Joe Landman wrote: > No. The extra registers make compiler optimization work better (lower > register pressure). The flat memory model (doing away with segmented > registers) simplifies addressing, and reduces the work of the processor > for every physical address calculation (no more segment + offset > operations to perform *before* pushing out the address onto the wires). Right. I was talking about the difference in running an application that fits in 32-bits in a 64-bit environment. There's a flat memory when it runs this way, or when it runs in a 32-bit environment. > I would love to see your data for this. Please note that I have quite a > bit of data that contradicts this assertion (e.g. directly measured > performance data, wall clock specifically of identical programs > compiled to run in 32 and 64 bit mode on the same physical machine, > running identical input decks). This is older data, from 2004. c.f. > http://www.amd.com/us-en/assets/content_type/DownloadableAssets/dwamd_SI_rlsdWP1.0_.pdf > but it is still relevant, and specifically, directly addresses the > assertions. The tests I made myself were non-HPC, e.g. building large software packages. But, I'm a reasonable person and I'll be glad to modify my statement to say that 64-bit computing is oversold in non-HPC markets. For example, when you look at pretty much any AMD-based computer these days, and compare it to what was available ~2 years ago (I'm not sure of the exact date), what difference do you see on the front panel? You'll see "AMD Athlon" in both cases, but now you also see "64". On the majority of computers being sold, this makes no difference. (HPC users are different). I bet most people think that since 64 is bigger than 32 then a 64-bit computer is "better". Yet, this isn't the case for them, especially if they're using a modern version of Windows, which is what the original posting was about. These days you also see "X2" which is a different kettle of fish and is, if anything, being undermarketed. >> add the additional difficulty of getting 64-bit drivers >> and what-not, I don't think it's worth messing with 64-bit >> computing for apps that don't need the address space. > > Which OS are you using? We haven't had 64 bit driver availability > issues since late 2004, for Linux. For windows this may be different. I agree 100%. I should have been clearer. >> One additional way 64-bit computing is being oversold >> is that there aren't now, and maybe never will be, any >> human written program that requires more than 32 bits >> for the instruction segment of the program. It's simply > > This is a bold assertion. Sort of like the "no program will ever use > more than 640k of memory" made by a computing luminary many moons ago. Bill Gates says he never said that. In any case, most of that was due to the architectural inferiority of the x86 at the time. What I'm talking about is a real limit in the complexity of what a human, or group of humans, can create. Please name of piece of software, free or commercial, that needs more than a 32-bit address space for its instruction space. As far as I know, there isn't any such thing. Not even close. >> too complex for a human, or a group of humans, to write >> this much code. Again, note that this says nothing > > landman@dualcore:~> cat hi.f90 > program hi > print *,"hi" > end > landman@dualcore:~> pgf90 hi.f90 > landman@dualcore:~> ./a.out > hi > landman@dualcore:~> ls -alF a.out > -rwxr-xr-x 1 landman users 2302794 Apr 8 21:38 a.out* I just tried the same program on a 64-bit RHEL4-based release using gfortran 4.1.2. My results are different. First I compiled hi.f90 with the "-c" option and got # size hi.o text data bss dec hex filename 185 0 0 185 b9 hi.o After linking it to get an executable I got # size a.out text data bss dec hex filename 1893 592 8 2493 9bd a.out > 2.3 MB for hi. But I'm not sure what any of this proves, other than gfortran produces smaller code in this case. I suspect the difference is in the runtime library that gets linked in. >> about the data segment of a program. Also, people tell >> me that there are programs that were generated by other >> programs that are larger than 32 bits. I've never seen >> one, but maybe they exist, and that's what I'm talking >> about human written programs. > > I am sorry, but I think this may be an artificial strawman. If so, I don't see it. If my statement is true, that is that no human written software program will ever outgrow a 32-bit address space **for text** then that says something important about what's important and what's not important. Note that I'm not saying that a modern processor should be some kind of hybrid mutant with 32-bit text pointers and 64-bit data pointers. That would be ridiculous. Cordially, -- Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From snj78 at comcast.net Sun Apr 8 20:36:27 2007 From: snj78 at comcast.net (snj78@comcast.net) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Help with HPL Makefile Message-ID: <040920070336.8388.4619B4BB00047280000020C42216549976C8C906029C@comcast.net> Hello all, I am trying to configure HPL on a beowulf cluster that I have put together for a senior project at Weber State University, and I am having a little bit of trouble. First of all, about the cluster: 4-node diskless cluster Fedora Core 6 - 64 bit version Intel Pentium D dual core processors MSI 965 motherboards Right now I have one node doing a net boot with syslinux and would like to do a benchmark before I put the rest of the nodes together, so I can have a performance comparison with subsequent nodes. I have installed the following packages on my system for HPL: openmpi-1.1-7.fc6.x86_64.rpm openmpi-devel-1.1-7.fc6.x86_64.rpm openmpi-libs-1.1-7.fc6.x86_64.rpm lapack-3.1.0-4.fc6.x86_64.rpm Blas-3.1.0-4.fc6.x86_64.rpm atlas-4.6.0-11.fc6.x86_64.rpm cblas.tgz hpl.tgz I may have installed more packages than necessary but I didn't think it would hurt. Everything has installed successfully but I can't get the makefile. down. I simply just don't understand enough of it to build it correctly. I just keep getting 'Make.inc' errors. The Makefile that I have attempted is below, called Make.Beowulf. I just used a generic makefile from the setups directory and attempted to supply some paths to the libraries but to no avail. I have tried to find documentation explaining more clearly how everything should be setup but nothing in lay-man terms, hence the errors. A few questions: What should my arch be? Does that even matter? Does it have to be x86_64? I realize I have to supply paths to the BLAS and MPI headers and libraries but exactly which libraries and header files? The compiler I am using is mpicc which is just linked to gcc, but shouldn't that compiler supply the links to the correct libraries and header files? The MPlib parameter points to libmpich.a so I installed mpich2 but that didn't give me a libmpich.a directory so what should I use there? Also, I am not using an network file systems so am I correct in assuming that all of the libraries need to be on each of the nodes? If so, I need to know exactly where to put them, and again, I believe they would need to be put into the exact same location, so the problem is, which libraries and header files exactly? (as to save precious RAM on each of the nodes). I realize I may be asking a lot but the end of the semester is just around the corner. I appreciate any help that you may give me ahead of time. Thanks. Stephen Jenkins snj78@comcast.net Make.Beowulf SHELL = /bin/sh # CD = cd CP = cp LN_S = ln -s MKDIR = mkdir RM = /bin/rm -f TOUCH = touch # # ---------------------------------------------------------------------- # - Platform identifier ------------------------------------------------ # ---------------------------------------------------------------------- # ARCH = Linux_x86_64 # # ---------------------------------------------------------------------- # - HPL Directory Structure / HPL library ------------------------------ # ---------------------------------------------------------------------- # TOPdir = $(HOME)/hpl INCdir = $(TOPdir)/include BINdir = $(TOPdir)/bin/$(ARCH) LIBdir = $(TOPdir)/lib/$(ARCH) # HPLlib = $(LIBdir)/libhpl.a # # ---------------------------------------------------------------------- # - Message Passing library (MPI) -------------------------------------- # ---------------------------------------------------------------------- # MPinc tells the C compiler where to find the Message Passing library # header files, MPlib is defined to be the name of the library to be # used. The variable MPdir is only used for defining MPinc and MPlib. # MPdir = /usr/include/openmpi MPinc = -I$/usr/include/include MPlib = $(MPdir)/lib/libmpich.a # # ---------------------------------------------------------------------- # - Linear Algebra library (BLAS or VSIPL) ----------------------------- # ---------------------------------------------------------------------- # LAinc tells the C compiler where to find the Linear Algebra library # header files, LAlib is defined to be the name of the library to be # used. The variable LAdir is only used for defining LAinc and LAlib. # LAdir = $(HOME)/netlib/ARCHIVES/Linux_PII LAinc = LAlib = $(LAdir)/libcblas.a $(LAdir)/libatlas.a # # ---------------------------------------------------------------------- # - F77 / C interface -------------------------------------------------- # ---------------------------------------------------------------------- # You can skip this section if and only if you are not planning to use # a BLAS library featuring a Fortran 77 interface. Otherwise, it is # necessary to fill out the F2CDEFS variable with the appropriate # options. **One and only one** option should be chosen in **each** of # the 3 following categories: # # 1) name space (How C calls a Fortran 77 routine) # # -DAdd_ : all lower case and a suffixed underscore (Suns, # Intel, ...), [default] # -DNoChange : all lower case (IBM RS6000), # -DUpCase : all upper case (Cray), # -DAdd__ : the FORTRAN compiler in use is f2c. # # 2) C and Fortran 77 integer mapping # # -DF77_INTEGER=int : Fortran 77 INTEGER is a C int, [default] # -DF77_INTEGER=long : Fortran 77 INTEGER is a C long, # -DF77_INTEGER=short : Fortran 77 INTEGER is a C short. # # 3) Fortran 77 string handling # # -DStringSunStyle : The string address is passed at the string loca- # tion on the stack, and the string length is then # passed as an F77_INTEGER after all explicit # stack arguments, [default] # -DStringStructPtr : The address of a structure is passed by a # Fortran 77 string, and the structure is of the # form: struct {char *cp; F77_INTEGER len;}, # -DStringStructVal : A structure is passed by value for each Fortran # 77 string, and the structure is of the form: # struct {char *cp; F77_INTEGER len;}, # -DStringCrayStyle : Special option for Cray machines, which uses # Cray fcd (fortran character descriptor) for # interoperation. # F2CDEFS = # # ---------------------------------------------------------------------- # - HPL includes / libraries / specifics ------------------------------- # ---------------------------------------------------------------------- # HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) # # - Compile time options ----------------------------------------------- # # -DHPL_COPY_L force the copy of the panel L before bcast; # -DHPL_CALL_CBLAS call the cblas interface; # -DHPL_CALL_VSIPL call the vsip library; # -DHPL_DETAILED_TIMING enable detailed timers; # # By default HPL will: # *) not copy L before broadcast, # *) call the BLAS Fortran 77 interface, # *) not display detailed timing information. # HPL_OPTS = -DHPL_CALL_CBLAS # # ---------------------------------------------------------------------- # HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES) # # ---------------------------------------------------------------------- # - Compilers / linkers - Optimization flags --------------------------- # ---------------------------------------------------------------------- # CC = /usr/bin/gcc CCNOOPT = $(HPL_DEFS) CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops # # On some platforms, it is necessary to use the Fortran linker to find # the Fortran internals used in the BLAS library. # LINKER = /usr/bin/g77 LINKFLAGS = $(CCFLAGS) # ARCHIVER = ar ARFLAGS = r RANLIB = echo # # ---------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/00aa9de6/attachment.html From jlforrest at berkeley.edu Sun Apr 8 21:50:40 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4619BA2C.6070909@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> Message-ID: <4619C620.4010409@berkeley.edu> Joe Landman wrote: > ??? Flat memory is non-segmented by definition. Would you care to > point out the flat memory addressing mode on x86 which can access all > 4GB ram? I am sure I missed it. I'll be happy to withdraw this comment. > Ok, here is where I guess I don't understand your point in posting this > to an (obviously) HPC list then. Is it oversold in the gamer market? In > the DB market? I meant in general, although as far as I know, there aren't any 64-bit games for the Windows PCs. There are definitely DBs and other server products (e.g. Microsoft Exchange) that require a 64-bit version of Windows. > ... and your point is .... that the 64 appendage makes no difference > when you are running the chip in 32 bit mode (e.g. windows)? Right. > ok. I might suggest avoiding conflating marketing with technology. Also > note that Athlon64 units do run noticably faster than Athlon units of > similar clock speed in 32 bit mode. The desktop I am typing this on now > is a 1.6 GHz Athlon 64 currently running a windows XP install, and is > noticably (significantly) faster than the system it replaced (a 2 GHz > Athlon). The minimal benchmarks I have performed indicate 30% ballpark > on most everything, with some operations tending towards 2x faster. I don't doubt this, but this is because the version of Athlon you're running, that has the magic number "64" on it, is just a better processor than the one you had before. The same would have been true if they had called it the Super Mega Athlon 2 Turbo++ . >> case for them, especially if they're using a modern version of >> Windows, which is what the original posting was about. These days you >> also see "X2" which is a different kettle of fish and is, if anything, >> being undermarketed. > Undermarketed? Not the way I see it (see the Intel ads on TV) I was thinking of Microsoft. > You set up an argument for the sole purpose of knocking it down. "no > 32bit address needed for instruction text" ... "a real limit in the > complexity" ... > > Of course, no program I have seen is ever *just* instruction text. There > are symbols. Data sections, other sections. Whether or not ld can link > anything that large is not something I am directly aware of, not having > done it myself. My point was that the text segment is created by a human and is limited in size by human abilities. Except for constants in the program, the data segment is synthetic and is created algorithmically or in some other way that doesn't expand in complexity as the size of the data expands. > And I still claim that your assertion is bold and difficult to support > or contradict. Right. I'd be happy to label it a conjecture. Like any conjecture, all it takes is one example to be proved false. > Way back in the good old days of DOS, we regularly > generated binaries that were 700+kB in size. Had to use overlays and > linker tricks to deal with it. This was around the time of DOS 6. OS2 > was a breath of fresh air, my codes could hit 2-3 MB without too much > pain (binary size). Had 16 MB on the machine, so it was "plenty of > room". I was not however of the opinion that 16 MB was all I would ever > need. That's because you, and many other people, could handle the complexity of creating a program with 2-3MB, or even 16MB, of text. But, there's a wall up there somewhere that you, and everybody else, will hit long before you get to 4GB of text. Cordially, Jon Forrest Unix Computing Support College of Chemistry Univ. of Cal. Berkeley 173 Tan Hall Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From toon at moene.indiv.nluug.nl Mon Apr 9 04:03:51 2007 From: toon at moene.indiv.nluug.nl (Toon Moene) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46199A82.5030201@scalableinformatics.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> Message-ID: <461A1D97.6090104@moene.indiv.nluug.nl> Joe Landman wrote: > landman@dualcore:~> cat hi.f90 > program hi > print *,"hi" > end > landman@dualcore:~> pgf90 hi.f90 > landman@dualcore:~> ./a.out > hi > landman@dualcore:~> ls -alF a.out > -rwxr-xr-x 1 landman users 2302794 Apr 8 21:38 a.out* > > 2.3 MB for hi. Yeach. This is Debian AMD64 GNU/Linux, using $ gfortran --version GNU Fortran 95 (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21) $ gfortran hi.f $ ls -l a.out -rwxr-xr-x 1 toon toon 9754 2007-04-09 13:01 a.out Doesn't pgf90 allow dynamic run time libraries ? -- Toon Moene - e-mail: toon@moene.indiv.nluug.nl - phone: +31 346 214290 Saturnushof 14, 3738 XG Maartensdijk, The Netherlands At home: http://moene.indiv.nluug.nl/~toon/ Who's working on GNU Fortran: http://gcc.gnu.org/ml/gcc/2007-01/msg00059.html From TPierce at rohmhaas.com Mon Apr 9 05:36:51 2007 From: TPierce at rohmhaas.com (Thomas H Dr Pierce) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] They appear to be reading beowulf .... In-Reply-To: <46199D96.6020702@scalableinformatics.com> Message-ID: Dear Beowulf, And the amusing part of the article is the standard marketing path: paraphrasing... "Windows CCE is pretty good now, but when Version 2.0 comes out, it will be really useful... " And I can only wonder at the marketing being prepared for Version 3.0 which is when Microsoft historically might have effective software (Windows 3.0, Office 3.0, etc...) ------ Sincerely, Tom Pierce Joe Landman Sent by: beowulf-bounces@beowulf.org 04/08/2007 09:57 PM Please respond to landman@scalableinformatics.com To beowulf cc Subject [Beowulf] They appear to be reading beowulf .... C.f. http://news.zdnet.com/2100-3513_22-6174266.html ... especially quoting the eminently quotable RGB. Obviously Microsoft declares victory, or at least success. What marketing organization would admit failure (and keep their jobs)? ... _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070409/ac172e16/attachment.html From mafunk at nmsu.edu Mon Apr 9 10:30:32 2007 From: mafunk at nmsu.edu (Matt Funk) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] debugging In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> Message-ID: <200704091130.33134.mafunk@nmsu.edu> Hi, i hope this is the right mailing list to post to... Anyway, i was wondering if i could get some advice/direction on how to debug my mpich program. I am running on a scyld configuration. What i am trying right now is the following: mpirun -dbg=gdb -nolocal -np 32 exec which starts the debugger in which i go run args which then start the program. However, it doesn't get very far until it just sits there. When i ps all the processes are defunced. When i do the same thing except mpirun -dbg=gdb -nolocal -np 1 exec and run it in the debugger, the program starts running well. The reason i want to run on 32 processor though, is that it takes (on 32 procs) several hours till my program crashes. Also, i would like to be able to keep the conditions under which it crashes intact as much as possible (i.e. run on 32 procs rather than 1). Does anyone have any advice? I am open to try out other things as well if possible. I am just starting to learn debugger techniques for a parallel program. thanks mat From robin at workstationsuk.co.uk Tue Apr 10 01:37:28 2007 From: robin at workstationsuk.co.uk (Robin Harker) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <4619BA2C.6070909@scalableinformatics.com> <6.2.3.4.2.20070409140242.02e6d950@mail.jpl.nasa.gov> Message-ID: <1155.86.135.104.48.1176194248.squirrel@webmail.hostme.co.uk> Northern Telecom sold a multi-user XENIX system. It was 8086 based based using Multibus. Circa 1984 Robin > At 11:39 AM 4/9/2007, Peter St. John wrote: >>Well, I could run unix with all 1536K, but not MS/PCDOS 3.2. So call >>it a software issue of failing to work around the hardware issue. >>Obviously the hardware was not a show-stopper. >> >>But it was the 286 I did this on, not the earlier 8088, which I >>don't think could reasonably have been expected to run unix; > > > > No problem running unix on a 8086.. (Don't forget Xenix on 8086 or > 80286) Or, for that matter, on a 68000. (maybe the 68010 with the > mmu or the 020 with virtual memory?.. I seem to remember DRAM refresh > by 64 NOPs on the timer tick) > > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > Robin Harker Workstations UK Ltd DDI: 01494 787710 Tel: 01494 724498 From dev.sandip at gmail.com Tue Apr 10 07:16:38 2007 From: dev.sandip at gmail.com (Sandip Dev) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Linux cluster for my college In-Reply-To: References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: <577b3d950704100716g241a0b8ft4d5f5f2351faaf85@mail.gmail.com> Thanks a lot...We are going ahead with it..Wish me luck On 4/8/07, Mark Hahn wrote: > > > We are planning to set up a linux cluster on trail basis in our > > college.Since its merely a trail,we have been alloted PIII PCs with 128 > MB > > Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this > hardware > > support a cluster? Moreover which distro/software to begin with? If > this > > trial succeeds we will get whatever hardware we want.But I just want > this > > trial to succeed.Please help me. > > I say you should keep it simple. just install some basic distro like > centos on all the nodes (just a minimal install - you DEFINITELY don't > want all the gui/user/etc junk). make sure you have ssh, but you can > skip everything desktop-related. install mpich. be happy. > > if you're familiar with Linux, ssh and mpich, this should take not much > time > at all. things like monitoring and management are wonderful things, and > necessary for any significant-sized cluster, but certainly not for a > proof-of-concept. with 128M PIII's, the examples you'll be able to > demonstrate will be quite small, but still believable if you avoid extras. > > regards, mark hahn. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070410/fbbb29b6/attachment.html From dev.sandip at gmail.com Tue Apr 10 07:19:32 2007 From: dev.sandip at gmail.com (Sandip Dev) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Re: Linux cluster for my college In-Reply-To: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> Message-ID: <577b3d950704100719k1d3a8308w59e4cdc242316282@mail.gmail.com> Thanks a lot...We are going ahead with it..Wish me luck...Will probably use debian...I dont wish to use a live cd thing..U dont learn anything that way...Will use MPI,PVM,Openmosix and OSCAR On 4/8/07, Sandip Dev wrote: > > We are planning to set up a linux cluster on trail basis in our > college.Since its merely a trail,we have been alloted PIII PCs with 128 MB > Ram and 40 GB HD on a 100 MBps LAN.All i want to know is,will this > hardware support a cluster? Moreover which distro/software to begin with? > If this trial succeeds we will get whatever hardware we want.But I just > want this trial to succeed.Please help me. > > Regards, > Sandip Dev, > Student,SVNIT, > (www.svnit.ac.in) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070410/a643ab5b/attachment.html From jlforrest at berkeley.edu Tue Apr 10 09:16:16 2007 From: jlforrest at berkeley.edu (Jon Forrest) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> Message-ID: <461BB850.3090108@berkeley.edu> Robert G. Brown wrote: > I totally agree with Joe on this issue. The "ideal" computer would have > an infinite, flat address space, totally transparent to the user. Want > to address memory location FF 0A BB 79 C3 12 93 54 6A 19 1D DA? (or > simply have 2^90 \approx 10^27 data objects to manage)? The memory > should be there, flat, transparent. I agree completely. > I don't understand how you could possibly imagine this to be true. I do > numerical spin simulations on lattices in D dimensions. An > N-dimensional spin (where N is not necessarily equal to D) is typically > represented by 1-(N-1) real numbers (e.g. spherical polar angles). In > addition any give spin may have other internal coordinates. To > represent a spin therefore requires minimally order 4*N bytes for an > ordinary 32-bit float representation of the spin coordinates, more > likely 8*N bytes if one sensibly uses double precision coordinates. For > 3D spins say 24 bytes per site. [] > > This isn't an isolated (if specific) example. There is a vast range of > memory-size bound problems, some of which have modest CPU requirements > but an absolute necessity to be able to efficiently address large memory > spaces. The examples you give are very good reasons why there is a clear need for more than 32-bits of address space for data. Again, I agree completely. But, if you read my original posting, this is not what I'm talking about. I'm talking about the need for more than 32-bits of address space for instructions (a.k.a "text"). Why don't you run the "size" command on any of your big programs and report back what you find. My guess is that they'll be in the order of 10s of megabytes. (I agree that this method doesn't show the true run-time text-size requirements but it's a good first attempt). > Personally I "wish" that they'd done the dual core thing entirely > differently. Instead of having two completely independent 64 bit cores > per CPU, they might have built a 128-bit core with a hardware floating > point execution pathway that permitted it to be transparently broken > down into 4 32 bit parallel pathways, 2 64 bit pathways, 1 96 bit and 1 > 32 bit pathway, or 1 128 bit pathway, with entirely transparent flat > memory access out to 128 bits, and with hardware implementation of 128 > bit integer or 128 bit floating point arithmetic (on down). Leave it to > a mix of the CPU, the OS, the compiler, and the application to decide > how to pipeline and allocate the available ALUs, registers, cache lines, > etc. to the needs of the program. Like I've said before, I'm not seriously proposing changing any existing architectures to have 32-bit text pointers and 64-bit data pointers, along with the corresponding changes to internal pathways. > But I'm not terribly worried. This to some extent describes the cell > architecture, with some slop as to just where the ganging together of > smaller logic units into larger ones occurs. And lots of very smart > people are working on this -- smarter than me for sure -- and doubtless > have far better ideas. Stating that there is no need for 64 bit > architectures and that 32 bits is enough for anyone is basically > equivalent to stating "the systems engineers working for AMD and Intel > and IBM and Motorola are complete idiots". This is simply not the case. Again, you misread my original claim. > they aren't idiots, they are brilliant, and the simple fact of the > matter is that 64 bit systems are faster, smarter, bigger, better than > 32 bit systems. After some private email exchanges with various list members, I no longer claim that, in general, there is no speed difference between a program that fits in 32-bits when it's run in 32-bit mode as compared to being run in 64-bit mode. Instead, I believe that it would be instructive to run important programs in both environments to see what comes out. But I stand firm on my claim that no human, or group of humans, can write a program that requires more than 32-bits of text space. Cordially, -- Jon Forrest Unix Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest@berkeley.edu From landman at scalableinformatics.com Tue Apr 10 16:36:05 2007 From: landman at scalableinformatics.com (Joe Landman) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] debugging In-Reply-To: <200704091130.33134.mafunk@nmsu.edu> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <200704091130.33134.mafunk@nmsu.edu> Message-ID: <461C1F65.1060805@scalableinformatics.com> Matt Funk wrote: > Does anyone have any advice? I am open to try out other things as well if > possible. I am just starting to learn debugger techniques for a parallel > program. Core dumps are your friends (though most linux distros turn them off by default now). That and compiling your code with symbols, and debugging enabled. I rather enjoy (ok, I really don't) pulling up crashed binaries, loading up the core, and walking the stack backwards to see the real reason for the crash. Oddly enough I am doing that now for an MPI code (sigh). > > thanks > mat > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics LLC, email: landman@scalableinformatics.com web : http://www.scalableinformatics.com phone: +1 734 786 8423 fax : +1 734 786 8452 or +1 866 888 3112 cell : +1 734 612 4615 From James.P.Lux at jpl.nasa.gov Tue Apr 10 16:51:24 2007 From: James.P.Lux at jpl.nasa.gov (Jim Lux) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Survey on beowulf.org -- and a drawing for a video iPod In-Reply-To: References: Message-ID: <6.2.3.4.2.20070410165033.030637f8@mail.jpl.nasa.gov> At 03:52 PM 4/10/2007, Donald Becker wrote: >Survey on beowulf.org -- video iPod > >For those who haven't checked out the beowulf.org web pages recently, >we've >put up a survey about HPC. > >You can find the link on the home page of > http://beowulf.org >or go directly to the survey at > http://www.beowulf.org/community/survey.php > >As an incentive, legitimate surveys (no automated submissions please!) (even if the automation is from a cluster?.. where's your sense of sport? ) >completed by April 20 will be entered into a drawing for the gadget de >jour -- a video iPod. > >Good luck James Lux, P.E. Spacecraft Radio Frequency Subsystems Group Flight Communications Systems Section Jet Propulsion Laboratory, Mail Stop 161-213 4800 Oak Grove Drive Pasadena CA 91109 tel: (818)354-2075 fax: (818)393-6875 From asingh at ideeinc.com Mon Apr 9 14:13:29 2007 From: asingh at ideeinc.com (Amrik Singh) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] NFS & Scaling issues In-Reply-To: <46170B60.90805@scalableinformatics.com> References: <4612BB84.6010707@ideeinc.com> <46170B60.90805@scalableinformatics.com> Message-ID: <461AAC79.6020908@ideeinc.com> Thanks for the reply Joe.... atop seems to be a really cool tool that would be very helpful once I get a chance to patch the kernel on file servers (for process level disk usage and ethernet usage information.). I am setting up a test cluster to reproduce the problem. Would post updates as I find more info... Amrik Joe Landman wrote: > Hi Amrik: > > Amrik Singh wrote: >> Hi, >> >> We are running a cluster of 180 diskless compute nodes. 60 of them >> have 32 bit AMD Semptron processors and rest are dual core AMD >> Athelon 64 bit processors. 32 bit machines have 10/100 mbps and rest >> have gigabit ethernet cards. We have four file servers, each hosting >> around 3.5TB on SATA drives connected to 3Ware RAID controller cards >> configured on RAID 10 array. These file servers are exporting the >> drives through NFS. Each file server is running 265 daemons for nfsd. >> >> The file servers are mainly hosting large number of small files >> ranging from 256KB to 2 MB. The compute nodes are primarily doing a >> search through these files, so there is lot's of reading and some >> writing to the file servers. >> >> Recently we started noticing very high (70-90%) wait states on the >> file servers when compute nodes. We have tried to optimize the NFS >> through increasing the number of daemons and the rsize and wsize but >> to no avail. >> >> Can someone point us in the right direction as to how we should be >> trying to troubleshoot this problem. > > You might want to look at the read patterns. > >> >> PS: All the nodes are running SuSE 10.0 and servers are running >> SuSE10.0 and 10.1 and all the drives are formatted with reiserfs. > > Hmmm... I remember Reiser has had a problem in the past when file > systems get full or nearly so. There are file tail optimizations you > might want to turn off, as well as use noatime for mounts. I might > suggest turning to a better file system for your servers (if possible, > it might not be a trivial undertaking), but even then that might not > be responsible. > > Grab a copy of atop (google for it), run it on your file server. See > if it is the file system that is problematic (disk devices running > near 80% or higher capacity for reads/writes all the time). > > Other possibilities are your file access patterns, what the file > server is doing itself, whether or not your networks are being flooded > with small packets (see if your csw is very high, or the number of > interrupts or packets are very high). > > Joe > >> >> >> thanks >> > From kilian at stanford.edu Tue Apr 10 17:30:58 2007 From: kilian at stanford.edu (Kilian CAVALOTTI) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Help with HPL Makefile In-Reply-To: <040920070336.8388.4619B4BB00047280000020C42216549976C8C906029C@comcast.net> References: <040920070336.8388.4619B4BB00047280000020C42216549976C8C906029C@comcast.net> Message-ID: <200704101730.58719.kilian@stanford.edu> Hi Stephen, On Sunday 08 April 2007 08:36:27 pm snj78@comcast.net wrote: > What should my arch be? Does that even matter? That's just a mnemonic thing, so if you build several versions with different MPI implementations, compilers or options, you can distinguish them. > I realize I have to supply paths to the BLAS and MPI headers and > libraries but exactly which libraries and header files? Basically a BLAS library and MPI libs. > The compiler I > am using is mpicc which is just linked to gcc, but shouldn't that > compiler supply the links to the correct libraries and header files? Only the ones the compiler itself needs, but it cannot guess where are the libs you want to link your program to. > Also, > I am not using an network file systems so am I correct in assuming that > all of the libraries need to be on each of the nodes? Right, unless you statically compile your program. > # ---------------------------------------------------------------------- > # - HPL Directory Structure / HPL library ------------------------------ > # ---------------------------------------------------------------------- > # > TOPdir = $(HOME)/hpl Be sure this TOPdir is the one in which you uncompressed the HPL source code. > MPdir = /usr/include/openmpi > MPinc = -I$/usr/include/include > MPlib = $(MPdir)/lib/libmpich.a You can try to replace the last line with: MPlib = -L$(MPdir)/lib/ The rest of the Makefile seems fine. What errors exactly do you get? Cheers, -- Kilian From hahn at mcmaster.ca Tue Apr 10 20:19:08 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] debugging In-Reply-To: <461C1F65.1060805@scalableinformatics.com> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <200704091130.33134.mafunk@nmsu.edu> <461C1F65.1060805@scalableinformatics.com> Message-ID: >> Does anyone have any advice? I am open to try out other things as well if >> possible. I am just starting to learn debugger techniques for a parallel >> program. we bought Allinea DDT, which is pretty good. I think it's a good illustration that the parallel part of debugging doesn't have to be hard. getting parallel logic is still nontrivial, as often serial is. > Core dumps are your friends (though most linux distros turn them off by while post-mortem approaches are quite valuable (though I probably use strace/ltrace more frequently), they do not compare to being able to walk your program through its paces, monitoring variables along the way, even messing with the parallelism. > default now). That and compiling your code with symbols, and debugging it is unfortunate that compiling with debugging will normally disable a good number of valuable optimizations. I emphasize to users that production code (ie, which consumes, say >1k cpu hours) needs to be maximally tweaked. anything else is stealing from other users. From greg.lindahl at qlogic.com Tue Apr 10 21:18:32 2007 From: greg.lindahl at qlogic.com (Greg Lindahl) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] debugging In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <200704091130.33134.mafunk@nmsu.edu> <461C1F65.1060805@scalableinformatics.com> Message-ID: <20070411041832.GA9972@localhost.localdomain> On Tue, Apr 10, 2007 at 11:19:08PM -0400, Mark Hahn wrote: > it is unfortunate that compiling with debugging will normally disable > a good number of valuable optimizations. Actually, it's pretty common that compiling with debugging won't disable optimization, but whether or not you can symbolically debug the code you want to is unlikely... you have to stay out of all the loop nests, because code that the compiler has transformed just doesn't look like the source anymore. Then again, I've never had a positive experience with a symbolic debugger in my life, so I don't miss them. Instead, I add print statements, and now I know only too much how that can alter optimizations and cause bugs to become heisenbugs. -- greg From rgb at phy.duke.edu Tue Apr 10 21:37:52 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <39703.192.168.1.1.1175530349.squirrel@mail.eadline.org> <20070402164539.GA20577@swingline.cs.wisc.edu> <38539.192.168.1.1.1175536904.squirrel@mail.eadline.org> <4611545C.2050907@scalableinformatics.com> <346E33EA-5D6B-46BC-A3F8-C14BA1A0D338@safetynetsolutions.com> <461A7F3B.3010308@streamline-computing.com> Message-ID: On Tue, 10 Apr 2007, Mark Hahn wrote: >> It was PVM that enabled true message passing parallel code to be written >> that made a pile of machines (be they Alphas, simple PCs, Sun > > I'm not disagreeing, but wonder why PVM is basically extinct now. > that is, why was MPI considered an improvement/replacement? Be careful. Last time I suggested it was "basically extinct", this was (somewhat forcefully) pointed out to me: http://pvmmpi07.lri.fr/ As of last year, there were still quite a lot of papers on PVM. I think PVM is still widely used in Europe, and a lot of "old hand" cluster people, myself included still prefer it (given their druthers). It is much more a part of the ORIGINAL beowulfs and related linux clusters than MPI. MPI had a completely distinct history: www.csm.ornl.gov/pvm/PVMvsMPI.ps It was developed first for real supercomputers, and only ported to COTS TCP/IP clusters (with a "network" communications channel being introduced) once TCP/IP clusters had been "invented" (noting well that the first TCP/IP cluster probably postdated the invention of TCP/IP by about fifteen minutes:-) and were regularly being built (mostly to run EP or PVM programs). PVM was from the beginning about being able to configure and control a "virtual machine" -- the core idea behind the original beowulf, by the way. MPI was intended to be a parallel programming library to run on an existing big iron system which required no "configuration" (e.g. node selection or other VM setup). They differ fairly strongly today in these arenas, although of course time and open source has reduced the differences. MPI's two primary "advantages" over time have been -- much better vendor support for native (non-TCP/IP) advanced network devices (PVM at one point had a native myrinet driver IIRC, but pretty much could never run any of the other advanced networks except as TCP/IP networks, which added a pretty huge latency hit per message from the TCP stack); a larger code base inherited from the old big iron supercomputers and more programmers moving from those supercomputers who were skilled at coding in it. More recently it sounds (from list discussions) like MPI has also gained some real technical advantages in the actual library internals, but you'd need to ask an MPI expert for an opinion there, not me. >> MS knifed IBM over OS/2 (which was a decently designed OS that might >> have given Unix a real run for its money) and hence lost out on all the > > I worked on OS/2, and it was no peach on the inside, so to speak ;) > it's hard to speculate about what-ifs on a system which had barely > shaken off its initial hw target (12 MHz 286!) by the time it was dropped. People I knew down here who worked on it (out in the park) thought pretty highly of it (probably about 2.0 or later versions of it -- the early 286-only versions were pretty rough as I understand it) but I myself never used it or NT -- I used Win 3.[0,1,2] at home, and while I used Win 95 for a brief time a) it sucked; b) I'd long been using and adminstering Unix networks, and had started using linux at home. Mind you, linux only ran with infinite pain on my small-memory 486 -- I barely could get slackware installed on an 8 MB system, although it supposedly COULD be installed on 4 -- but I got an AMD 5 system with (IIRC) 16 whole MB of memory and it ran charmlike on it. It did not suck -- it ran every bit as nicely as the Suns I had been using for many years at that point, and came with X and compilers and everything, for free. I never looked back. So my "decently designed" is definitely hearsay. The "knifed" part is more than JUST hearsay, though -- I met a bunch of IBMers at the Linux Expo over in Raleigh back in 1999 or thereabouts (I was running the Extreme Linux booth, and met Doug and Forrest and a bunch of other linux geek types there for the first time in human person:-). The IBMers HATED Microsoft, and were actually quite fond of linux. The reasons given were that MS had screwed IBM royally over OS/2, costing lots of their friends their jobs and depressing a whole section of the company. Linux was perceived as being a long term road to revenge -- as IBM gradually shrank down its residual OS/2 business they were working to replace it with linux. To this day, IBM is the only company that I know of that has advertised linux on TV. And then there is the MS settlement with IBM in an OS/2 antitrust action ($750 million, two years ago) where for that much money to exchange hands in a SETTLEMENT, one imagines that there was ample evidence of knife-wound blood -- so call it "heavily supported" hearsay. rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Wed Apr 11 00:07:16 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <4619B330.1000803@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> Message-ID: On Sun, 8 Apr 2007, Jon Forrest wrote: > The tests I made myself were non-HPC, e.g. building large software > packages. But, I'm a reasonable person and I'll be glad to > modify my statement to say that 64-bit computing is oversold > in non-HPC markets. For example, when you look at pretty much > any AMD-based computer these days, and compare it to what > was available ~2 years ago (I'm not sure of the exact date), what > difference do you see on the front panel? You'll see "AMD Athlon" > in both cases, but now you also see "64". On the majority > of computers being sold, this makes no difference. (HPC users > are different). I bet most people think that since 64 is bigger > than 32 then a 64-bit computer is "better". Yet, this isn't the The primary difference is that a 64-bit computer is FASTER. Measurably, and on a remarkably wide range of applications. True, most of them don't matter much on a desktop, especially a desktop running nothing but a web browser or email, but if the user is doing ANY task with a significant CPU-bound computational component or a fairly wide class of memory bound operations those tasks will complete faster, with less "stress" on the CPU (improving multitasking and interactive latency by virtue of completing those little chorelets faster. Note well that on really bleeding edge operating systems (e.g. Vista) the speed is required just to make the base operating system and windowing environment usable, according to what I hear. The ability to address much larger memory segments is another major difference, and it matters to many people as I pointed out before. Just because you personally have only rarely or even never observed a system that has "filled up" available memory on a system with 4 GB installed doesn't mean that it has never happened or cannot happen -- it happens all the time in the HPC community. Forget large lattices of spins -- think about a SMALL set of only (say) 100 two level atoms in a fully quantum entangled state. There are 2^100 degrees of freedom in such a system. An entire 64 bit memory space isn't ENOUGH to hold its description. You seem to be differentiating between 64 bits of address space just for the actual programs and 64 bit architectures in general. This doesn't make much sense to me. Modern computers more often than not use a flat memory model (even where segmentation or virtualization of memory spaces is supported) and just what any given piece of memory contains is up to the operating system (and possibly BIOS) -- it isn't a feature of the processor itself, with some memory labelled "data" and other memory labelled "code" (unless the processor is a new, 64 bit processor that lets you tag memory pages in precisely that way). Program data is often intertwined with code, and it is easy (at least in C) to write code that will do horrible things to your program code by overwriting it. This of course is the origin of buffer-overwrite attacks and much hair pulling over deep bugs. So the assertion that "programs will never be written that use more than 32 bits of code address space" (an assertion that will almost certainly be proven false by time, barring the landing of an asteroid or the like that wipes out all of humankind in the next five or ten years) is not a good argument against "64 bit processors". There is a law out there somewhere that says something like "programs expand to fill the available space" on any given architecture, and I suspect that it is a correct one. Indeed, Windows Vista seems hell-bent on proving that programs are currently expanding FASTER than the available (affordable) space. It also ignores the many performance and security benefits of e.g. * having wider registers * more registers * direct hardware support for e.g. 64 bit integers * a larger virtual memory space * a larger physical address space * no-execute bits to tag data memory vs code memory pages to prevent buffer overwrite attacks (see above). * the ability to perform certain kinds of code virtualization all at a more or less constant cost. This isn't about "marketing" at all, as the Athlon 64 is dirt cheap at <$100 retail, $170 for dual core -- compared to a whopping $50 for their regular (32 bit) processor. I see no massive margins there, given that the older processor has long since played out its R&D costs and is doubtless end-of-lifed. In as little as one more year, we may see the last 32 bit processors go away forever, like their 8 and 16 bit forefathers. It is about performance and competitive market advantage. The opteron simply blows away any 32 bit processor on the market in ways that have a direct impact on overall system performance (however ignorable those improvements are for certain classes of task that already leave a system "mostly idle"). So does the dirt-cheap AMD 64. > case for them, especially if they're using a modern version of > Windows, which is what the original posting was about. These days you Modern as in Vista? I though Vista was supposed to be trans-demanding on hardware -- so much so that it is a complete pig on older hardware. It openly calls for a 64 bit CPU in its hardware planning documentation, although it grudgingly allows that it might, if it feels like it, run on older 32 bit CPUs, if you don't care about not having a "premium experience" (that is, its desktop interface won't work or will run like treacle). > also see "X2" which is a different kettle of fish and is, if anything, > being undermarketed. > >>> add the additional difficulty of getting 64-bit drivers >>> and what-not, I don't think it's worth messing with 64-bit >>> computing for apps that don't need the address space. >> >> Which OS are you using? We haven't had 64 bit driver availability >> issues since late 2004, for Linux. For windows this may be different. > > I agree 100%. I should have been clearer. I should note that modern 64-bit CPUs have been very deliberately designed so that one doesn't ever "mess with 64-bit computing" at all, unless you are a compiler designer or OS programmer or maybe in a few cases a writer/maintainer of a library. You boot the system. It runs. You load an application. It runs. You compile an application from source. It runs (or not, but it probably has nothing to do with the nature of the CPU if it doesn't). What exactly has to be "messed with"? The 64 bit CPUs will execute 32 bit programs -- the hardest single thing associated with making this happen transparently to the user is to ensure that there are 32 bit (DL) libraries available to those dynamically linked programs, requiring that a lot of libraries have to be installed twice, once for 32 bit programs and once for 64 bit programs. But a user shouldn't have to know much about this. Linux does it nearly perfectly transparently, and so does (to my own experience) Windows. Even a programmer doesn't have to know much about it -- to "port" my personal code from "32 bit" sources in an NFS mounted directory to a 64 bit opteron the first time was something like cd programsrc make clean; make and then run the program. Whoa, it works. So smoothly it is hard to TELL that it is running as a 64 bit application, except by looking at the libraries it links to and noting that it runs twice as fast. >>> One additional way 64-bit computing is being oversold >>> is that there aren't now, and maybe never will be, any >>> human written program that requires more than 32 bits >>> for the instruction segment of the program. It's simply >> >> This is a bold assertion. Sort of like the "no program will ever use >> more than 640k of memory" made by a computing luminary many moons ago. > > Bill Gates says he never said that. In any case, most of that was > due to the architectural inferiority of the x86 at the time. > What I'm talking about is a real limit in the complexity of > what a human, or group of humans, can create. Please name of > piece of software, free or commercial, that needs more > than a 32-bit address space for its instruction space. > As far as I know, there isn't any such thing. Not even close. And you may be correct. Or not -- recall not all code is written by humans, and programs writing programs may well have exceeded your limits already in some computer science or mathematics department. Also remember that groups of humans have systematically filled every currently available amount of memory within a matter of years for decades now, so regularly that one can probably predict fairly accurately the DATE where this will not be true by simply extrapolating a regular curve. Finally, remember that complexity is a funny thing -- there are whole CLASSES of application that have yet to be written, including programs that e.g. explore various branches of mathematics via exhaustive search or do voice/pattern recognition that may well take a lot more lines of code than you currently anticipate. But I won't argue, as I don't have a specific counterexample and don't feel like writing a piece of perl that could generate C code in a matter of minutes that would, if compiled, exceed a 32 bit CPU's capacity. For one thing, I personally don't have an 8 GB 64 bit system to test the code on. For another, that doesn't matter because that's not the point of 64 bit CPUs. The point is to let computers operate on much larger memory spaces, regardless of whether the contents of that memory is code, heap, stack, data, cache, buffer, or any mix of the above being overwritten with something unexpected by an errant pointer. The point is to move data around in bigger chunks, faster. The point is to perform all sorts of arithmetical and logical operations on 64 bit operands faster. The point is to go faster, at what is ultimately constant cost to the consumer, just like all the other Moore's law advances over the last forty or so years. > If so, I don't see it. If my statement is true, that is that > no human written software program will ever outgrow > a 32-bit address space **for text** then that says something > important about what's important and what's not important. > Note that I'm not saying that a modern processor should > be some kind of hybrid mutant with 32-bit text pointers > and 64-bit data pointers. That would be ridiculous. Given the conditional, this of course is impossible to argue with;-) Either point. If your statement is true (and conceding that it might still BE true, or nearly so) that truth would be interesting information. Either way, a "hybrid mutant" isn't ridiculous -- it is a standard feature in modern CPU design already. The Opteron, for example, can only address 48 bits worth of memory, not 64, because nobody can imagine how to put EVEN ~2.5 x 10^14 bytes of memory on a single CPU's memory pathway at the moment. This is just as "ridiculous" for data as 32 bits is for code -- more so, actually, as it INCLUDES code and data symmetrically in a flat memory model. It is just better to engineer for far more than we'll need for a decade or so and then grow into our hundred terabyte systems with room to spare rather than spend a lot of money on a processor design only to have to change it in two or three years because "nobody could imagine" how a program could need more than (fill in the blank with any resource assertion of the past)...;-) rgb -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From rgb at phy.duke.edu Wed Apr 11 00:17:06 2007 From: rgb at phy.duke.edu (Robert G. Brown) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <46198BB8.6030605@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> <461926C1.3000802@moene.indiv.nluug.nl> <46198BB8.6030605@berkeley.edu> Message-ID: On Sun, 8 Apr 2007, Jon Forrest wrote: > The experiment I tried was to build several fairly large programs > on Fedora Core 6 32 bit and then 64 bit using exactly the same > hardware. This is clearly not the same kind of test as your > example but it's more relevant as a systems-level comparison. > If I remember, I built mysql and apache. This took about the same > amount of time in both modes. Building programs isn't a particularly good benchmark for anything but building programs, for all that people tend to use make as a benchmark anyway. RUNNING programs (other than make and gcc) is much better. For one thing, compilers tend to basically be parsing engines processing text, with very little advantage available due to e.g. optimizing loops or streaming in double precision data to do a complex numerical computation on. > I suspect that the Beowulf crowd would have lots of experience > with 32-bits vs 64-bits question, so I'd welcome additional > comments, especially dealing with situations where programs > *don't* need the additional address space of the 64-bit model. Sure. 64-bits = "good". 32-bits = "bad", in nearly all CPU-intensive (especially floating point intensive) applications that don't thrash the cache. Cache thrashing is actually more expensive IIRC for 64 bit systems than it is for 32 bit systems because the penalty for a cache miss is higher. For pretty much everything else, though, goood. And even where it is bad it isn't THAT bad because its good at least partly compensates. Size of the program isn't terribly important to this conclusion, as Toone's miniprogram clearly shows. You don't need a lot of code to do a long-running double precision loop. rgb > > Cordially, > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu From tjrc at sanger.ac.uk Wed Apr 11 01:40:38 2007 From: tjrc at sanger.ac.uk (Tim Cutts) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Survey on beowulf.org -- and a drawing for a video iPod In-Reply-To: References: Message-ID: On 10 Apr 2007, at 11:52 pm, Donald Becker wrote: > > > Survey on beowulf.org -- video iPod > > For those who haven't checked out the beowulf.org web pages recently, > we've > put up a survey about HPC. > > You can find the link on the home page of > http://beowulf.org > or go directly to the survey at > http://www.beowulf.org/community/survey.php I find the initial question essentially impossible to answer. The whole point of beowulf computing is that you can build it out of almost anything. Most of the software is community driven. The ideas are community driven. The leaders are the community, not any one company. The hardware is commodity. I can't even name one when pushed to; I don't use any cluster software I had to buy, and the vendors who made my hardware are pretty much incidental - I could have used anyone. And that answer to the first question makes the subsequent questions even more impossible to answer. Looks like I don't get the video iPod. :-) Tim From peter.st.john at gmail.com Wed Apr 11 09:12:24 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:01 2010 Subject: Fwd: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> Message-ID: I'm a dingbat, I "replied" instead of "reply-to-all-ed" Peter ---------- Forwarded message ---------- From: Peter St. John Date: Apr 11, 2007 11:51 AM Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! To: Jon Forrest Jon, On 4/10/07, Jon Forrest wrote: > > > But I stand firm on my claim that no human, or group > of humans, can write a program that requires more than > 32-bits of text space. I like to say that proving a theorem is alot like writing a program. I don't know about the biggest software projects, but the Classification of Finite Simple Groups was huge; from Wiki: "In all, the work comprises tens of thousands of pages in 500 journal articles by some 100 authors." My thumbnail guesstimate of how much bytes are in a typical journal page of mathematics (maybe less than AMSTeX source, but more than 2k of plaintext prose, because all the symbols have to be expressed as abbreivations...) suggests that this work, done by humans, amounts to more than 4 GB. I'm not clear, myself, about the "infinite flat address space", as I want my data space to be a bit more structured (in my view of the C source, say) and don't want to care about how it looks to the compiler (as long as the compiler is happy). However, the killer app to me is what RGB mentioned; I know and love numbers that don't fit in one 32-bit word. Peter Cordially, > -- > Jon Forrest > Unix Computing Support > College of Chemistry > 173 Tan Hall > University of California Berkeley > Berkeley, CA > 94720-1460 > 510-643-1032 > jlforrest@berkeley.edu > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070411/8cc31b91/attachment.html From peter.st.john at gmail.com Wed Apr 11 09:39:38 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Survey on beowulf.org -- and a drawing for a video iPod In-Reply-To: References: Message-ID: I considered writing in "The University of Tokyo" :-) but felt I was missing a boat somewhere. I couldn't find the link on the top page, or on the community page, but used the "direct" link. Maybe you had meant some article indicating the context? A questionaire comparing vendors of particular things seems fine to me, but the question as stated seemed a bit broad compared to the list of specific companies in the followup. And I can pass on the iPod, I can't do anything without either chalk, or a 100+ key keyboard. My phone chuckles at me. Peter On 4/11/07, Tim Cutts wrote: > > > On 10 Apr 2007, at 11:52 pm, Donald Becker wrote: > > > > > > > Survey on beowulf.org -- video iPod > > > > For those who haven't checked out the beowulf.org web pages recently, > > we've > > put up a survey about HPC. > > > > You can find the link on the home page of > > http://beowulf.org > > or go directly to the survey at > > http://www.beowulf.org/community/survey.php > > I find the initial question essentially impossible to answer. The > whole point of beowulf computing is that you can build it out of > almost anything. Most of the software is community driven. The > ideas are community driven. The leaders are the community, not any > one company. The hardware is commodity. I can't even name one when > pushed to; I don't use any cluster software I had to buy, and the > vendors who made my hardware are pretty much incidental - I could > have used anyone. > > And that answer to the first question makes the subsequent questions > even more impossible to answer. > > Looks like I don't get the video iPod. :-) > > Tim > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070411/eccc9cfa/attachment.html From mathog at caltech.edu Wed Apr 11 09:48:22 2007 From: mathog at caltech.edu (David Mathog) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Re: debugging Message-ID: Matt Funk wrote: > The reason i want to run on 32 processor though, is that it takes (on 32 > procs) several hours till my program crashes. Also, i would like to be able > to keep the conditions under which it crashes intact as much as possible > (i.e. run on 32 procs rather than 1). > > Does anyone have any advice? This is pretty general... My advice is to be sure the code is absolutely as clean and standard compliant as possible before you touch a debugger. That means add -Wall -pedantic -std=c99 (for gcc, or as appropriate for your compiler) and don't stop until every bit of it compiles without a single warning. Then run it through valgrind or the equivalent and fix every memory problem it finds. Then, and only then, try your long run again. If you're lucky this will fix the problem and you won't have to debug anything. Also a comment - if your program crashes pretty much by definition it is not doing sufficient error checking. Rather than "kaboom!" a well written program will emit an "could not allocate memory" or "invalid pointer" message and then exit gracefully. Yes, I know that level of error checking is often left out of inner loops for speed reasons. Assuming that your code has a fairly fast cycle, so that several hours represents many, many cycles, you're almost certainly looking for either an invalid memory access, a memory leak, or running some loop counter past the end of the loop (for instance, via an unhandled condition.) Valgrind can help you find some of these. If it does any file IO you might also be using up all the file descriptors. (Saw that once in a version of NCBI BLAST, where it kept opening a gi file and not closing it before the next open.) If all of that fails, and you have easy access to another cluster with a completely different architecture, try building and running there. Often a subtle problem on one CPU type stands out like a sore thumb on another. If all that fails, then Joe is probably right, start with the dumps and work backwards to at least find out where in the code the crash is taking place. Or run each instance with strace, but be sure to log the output to for each compute node to a local file on that node. Then you can put in print statements in the relevant locations and run again. Just don't be surprised if, if the code is optimized, those print statements themselves "cure" the problem. Regards, David Mathog mathog@caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech From gdjacobs at gmail.com Wed Apr 11 12:35:40 2007 From: gdjacobs at gmail.com (Geoff Jacobs) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> Message-ID: <461D388C.2080703@gmail.com> According to most public information, the AMD chips receive a modest performance boost when going to 64 bit in most situations. Core 2 chips suffer a small penalty when going 64 bit. I suspect this is due to the cache hit rate suffering which, with a traditional north bridge, would have proportionally more effect. Vista is really pushing 64 bit, but that's partly due to the new driver policy. 64 bit Vista has more, harder to bypass, DRM as compared to 32 bit. Every driver has to be blessed by Microsoft. -- Geoffrey D. Jacobs From gdjacobs at gmail.com Wed Apr 11 12:43:07 2007 From: gdjacobs at gmail.com (Geoff Jacobs) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461BB850.3090108@berkeley.edu> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> Message-ID: <461D3A4B.5060201@gmail.com> Jon Forrest wrote: > The examples you give are very good reasons why there is > a clear need for more than 32-bits of address space for > data. Again, I agree completely. But, if you read my original > posting, this is not what I'm talking about. I'm talking > about the need for more than 32-bits of address space for > instructions (a.k.a "text"). Why don't you run the "size" > command on any of your big programs and report back what > you find. My guess is that they'll be in the order > of 10s of megabytes. (I agree that this method doesn't > show the true run-time text-size requirements but > it's a good first attempt). Um, that would require distinct memory pools or some weird internal mapping scheme. Not nice. Much better architecturally to have no difference between pointers to instructions and pointers to data. -- Geoffrey D. Jacobs From gdjacobs at gmail.com Wed Apr 11 12:53:02 2007 From: gdjacobs at gmail.com (Geoff Jacobs) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Re: Linux cluster for my college In-Reply-To: <577b3d950704100719k1d3a8308w59e4cdc242316282@mail.gmail.com> References: <577b3d950704072339g39393b3dk7e8ff66b561bf6fd@mail.gmail.com> <577b3d950704100719k1d3a8308w59e4cdc242316282@mail.gmail.com> Message-ID: <461D3C9E.3080407@gmail.com> Sandip Dev wrote: > Thanks a lot...We are going ahead with it..Wish me luck...Will probably > use debian...I dont wish to use a live cd thing..U dont learn anything > that way...Will use MPI,PVM,Openmosix and OSCAR OSCAR on Debian still isn't very mature, so you might want to look at a different solution/distro. OpenMOSIX is interesting for a small sized cluster. With MPI on such a machine, the process migration should be completely automatic. Nice. Kind of limited in which kernels it can use, though, which isn't good. -- Geoffrey D. Jacobs From peter.st.john at gmail.com Wed Apr 11 12:59:00 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461D3A4B.5060201@gmail.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> <461D3A4B.5060201@gmail.com> Message-ID: On 4/11/07, Geoff Jacobs wrote: > > Jon Forrest wrote: > > The examples you give are very good reasons why there is > > a clear need for more than 32-bits of address space for > > data. Again, I agree completely. But, if you read my original > > posting, this is not what I'm talking about. I'm talking > > about the need for more than 32-bits of address space for > > instructions (a.k.a "text"). Why don't you run the "size" > > command on any of your big programs and report back what > > you find. My guess is that they'll be in the order > > of 10s of megabytes. (I agree that this method doesn't > > show the true run-time text-size requirements but > > it's a good first attempt). > > Um, that would require distinct memory pools or some weird internal > mapping scheme. Not nice. Much better architecturally to have no > difference between pointers to instructions and pointers to data. Hear hear! For self-adapting softare you *can't* distinguish instructions from data. That may sound over-specialized but I invite you to consider DNA and what it does: instruct enzymes to modify DNA. An awful lot comes out of that process. So I don't think it's just von Neumann's whim. Peter -- > Geoffrey D. Jacobs > > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070411/4cdb429d/attachment.html From hahn at mcmaster.ca Wed Apr 11 13:18:18 2007 From: hahn at mcmaster.ca (Mark Hahn) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461D388C.2080703@gmail.com> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <4619B330.1000803@berkeley.edu> <461D388C.2080703@gmail.com> Message-ID: > performance boost when going to 64 bit in most situations. Core 2 chips > suffer a small penalty when going 64 bit. I suspect this is due to the Core2 has some micro-optimizations which are only enabled for 32b mode, which is an interesting choice. AMD tends to be biased towards HPC or at least serverish stuff, which is 64b-oriented... > cache hit rate suffering which, with a traditional north bridge, would > have proportionally more effect. I can't think of negative effects of x86_64 other than possible higher memory overhead for apps that use lots of pointers. compilers are a typical example, but perhaps some sparse matrix stuff, too? code density in x86_64 mode is not much different in mcmodel=small (for instance, it's not the case that every offset in branch and memory reference instructions become giant...) it'll be most interesting to see how AMD has optimized Barcelona... From rbw at ahpcrc.org Wed Apr 11 14:42:52 2007 From: rbw at ahpcrc.org (Richard Walsh) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> <461D3A4B.5060201@gmail.com> Message-ID: <461D565C.5090907@ahpcrc.org> Peter St. John wrote: > On 4/11/07, *Geoff Jacobs* > wrote: > > Jon Forrest wrote: > > The examples you give are very good reasons why there is > > a clear need for more than 32-bits of address space for > > data. Again, I agree completely. But, if you read my original > > posting, this is not what I'm talking about. I'm talking > > about the need for more than 32-bits of address space for > > instructions ( a.k.a "text"). Why don't you run the "size" > > command on any of your big programs and report back what > > you find. My guess is that they'll be in the order > > of 10s of megabytes. (I agree that this method doesn't > > show the true run-time text-size requirements but > > it's a good first attempt). > > Um, that would require distinct memory pools or some weird internal > mapping scheme. Not nice. Much better architecturally to have no > difference between pointers to instructions and pointers to data. > > > Hear hear! For self-adapting softare you *can't* distinguish > instructions from data. That may sound over-specialized but I invite > you to consider DNA and what it does: instruct enzymes to modify DNA. > An awful lot comes out of that process. So I don't think it's just von > Neumann's whim. On the opposite side of the argument, that code/instruction set (self-modified for say about a billion years [and recombined, and mutated]) is only ~3 billion base pairs in length. If we liberally give each base pair a byte, that's only 3 billion bytes. We can quibble about what an instruction is, but also about how much of the code is used and what would be the reduced instruction set equivalent. On the other hand, claiming that producing an application with more than 4 Gbytes of instructions is impossible seems foolish, unless there is prize money involved, of course ... ;-) ... rbw -- Richard B. Walsh Project Manager Network Computing Services, Inc. Army High Performance Computing Research Center (AHPCRC) rbw@ahpcrc.org | 612.337.3467 > > "Making predictions is hard, especially about the future." > > Nils Bohr ----------------------------------------------------------------------- This message (including any attachments) may contain proprietary or privileged information, the use and disclosure of which is legally restricted. If you have received this message in error please notify the sender by reply message, do not otherwise distribute it, and delete this message, with all of its contents, from your files. ----------------------------------------------------------------------- From peter.st.john at gmail.com Wed Apr 11 15:08:12 2007 From: peter.st.john at gmail.com (Peter St. John) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> Message-ID: My friend Landon sent me the link I needed to say "math proves that math is equivalent to programming"; it actually says something more like "there is an isomorphism between proofs of deductive logic and constructions of Lambda Calculus" but even that I'm saying badly, the link is http://en.wikipedia.org/wiki/Curry-Howard_isomorphism Gratifyingly, the intro has the phrase "Proofs are Programs" :-) On second thought my guestimate of the size of a program that might be produced from the Clasification of Finite Simple Groups is excessive. I'm thinking 100MB would be more fair order of magnitude than 1GB. I don't mean that the possible huge size of source code justifies 64 bit words, it's enough, to me, that a huge integer can fit into a register, for number theory. But yes we humans can indeed create vast grotesque edifices of scholarship without the necessity of machine generation. But the machines will, of course, win in the end :-) Peter On 4/11/07, Peter St. John wrote: > > Jon, > > On 4/10/07, Jon Forrest wrote: > > > > > > But I stand firm on my claim that no human, or group > > of humans, can write a program that requires more than > > 32-bits of text space. > > > I like to say that proving a theorem is alot like writing a program. I > don't know about the biggest software projects, but the Classification of > Finite Simple Groups was huge; from Wiki: > "In all, the work comprises tens of thousands of pages in 500 journal > articles by some 100 authors." > My thumbnail guesstimate of how much bytes are in a typical journal page > of mathematics (maybe less than AMSTeX source, but more than 2k of plaintext > prose, because all the symbols have to be expressed as abbreivations...) > suggests that this work, done by humans, amounts to more than 4 GB. > I'm not clear, myself, about the "infinite flat address space", as I want > my data space to be a bit more structured (in my view of the C source, say) > and don't want to care about how it looks to the compiler (as long as the > compiler is happy). However, the killer app to me is what RGB mentioned; I > know and love numbers that don't fit in one 32-bit word. > Peter > > Cordially, > > -- > > Jon Forrest > > Unix Computing Support > > College of Chemistry > > 173 Tan Hall > > University of California Berkeley > > Berkeley, CA > > 94720-1460 > > 510-643-1032 > > jlforrest@berkeley.edu > > _______________________________________________ > > Beowulf mailing list, Beowulf@beowulf.org > > To change your subscription (digest mode or unsubscribe) visit > > http://www.beowulf.org/mailman/listinfo/beowulf > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.scyld.com/pipermail/beowulf/attachments/20070411/fd3055f6/attachment.html From 3lucid at gmail.com Wed Apr 11 19:19:48 2007 From: 3lucid at gmail.com (Kyle Spaans) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> <461D3A4B.5060201@gmail.com> Message-ID: <5a1205b30704111919n64c3bbd4uef323621bf60db5@mail.gmail.com> I know this is way old by now but: A quick example of a game that is 64-bit enabled: Far Cry. When in WinXP64, with proper drivers, and Far Cry properly patched, I remember some people claiming performance increases under 20%, or at very least prettier graphics at the same performance. And (according to my so-far basic understanding) weren't the early Athlon 64s 'faster' at the same clock speed mostly because of the integrated memory controller and less because of the 64-bit'ness (in general of course)? From larry.stewart at sicortex.com Wed Apr 11 19:28:43 2007 From: larry.stewart at sicortex.com (Larry Stewart) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: <461D565C.5090907@ahpcrc.org> References: <4616DE5E.4030706@berkeley.edu> <46199A82.5030201@scalableinformatics.com> <461BB850.3090108@berkeley.edu> <461D3A4B.5060201@gmail.com> <461D565C.5090907@ahpcrc.org> Message-ID: <461D995B.3050903@sicortex.com> An inconclusive datapoint: The largest programs I know about (text segment size) are compiled simulators. These things turn chip logic into programs whose execution does whatever. RAMs aren't compiled like this, but logic is. For our chips, the text segment sizes are in the 20 megabyte area. Of course our chips are small compared to Cell or Itanium or ... I suppose compiled simulators are a way of turning data into instructions! Compiled simulators are also fun because they break most CPU designer's ideas about instruction bandwidth. They can be one huge basic block with no branches. So our 20 MB text segment runs at about 100 simulation cycles per second, using 2 GB/s of I stream bandwidth and the rest D stream recording state changes. (This is Cadence NC-Verilog, if I have this right.) It isn't HPC exactly, but we do have several hundred CPUs running simulation programs all the time. -L PS The resulting chip IS for HPC of course :-) From agshew at gmail.com Wed Apr 11 20:45:58 2007 From: agshew at gmail.com (Andrew Shewmaker) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Cell programming In-Reply-To: <200703201003.01506.twilcox@terrasoftsolutions.com> References: <200703201003.01506.twilcox@terrasoftsolutions.com> Message-ID: On 3/20/07, Tim Wilcox wrote: > It seems to me a good starting point is to divide a problem, say a CFD, into > larger sections at the MPI layer and then a smaller division of the subset on > the individual Cell processor. This poses the issue of message passing > between disparate SPUs. Any input on the characteristics of how this may > work and perform? Hi Tim, Have you read A Synchronous Mode MPI Implementation on the Cell BE? Architecture http://www.cs.fsu.edu/research/reports/TR-061215.pdf or MPI microtask for programming the Cell Broadband Engine? processor http://www.research.ibm.com/journal/sj/451/ohara.html I don't see any code available to download for those. Cell SDK 2.1 includes the Accelerator Library and Framework (ALF), and the documentation looks good. I don't think it handles messages between SPUs, but it does help out with things like double buffering. I don't think that Sequoia http://sequoia.stanford.edu transfers data between SPUs either. Dr. Dobbs has an article on optimizing Breadth First Search on the Cell. The authors describe how they implement an all-to-all exchange amongst the SPUs. http://www.ddj.com/dept/64bit/197801624?pgno=4 -- Andrew Shewmaker From dkondo at lri.fr Wed Apr 11 01:27:43 2007 From: dkondo at lri.fr (Derrick Kondo) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] [CFP] EuroPVM/MPI'07 Message-ID: <60ec14620704110127g1985647fk41c707142e2c62f2@mail.gmail.com> ************************************************************************ *** *** *** CALL FOR PAPERS *** *** *** ************************************************************************ EuroPVM/MPI 2007 14th European PVMMPI Users' Group Meeting Paris, France, September 30 - October 3, 2007 web: http://www.pvmmpi07.org e-mail: chairs@pvmmpi07.org organized by Project Grand-Large (http://grand-large.lri.fr/index.php/Accueil) from INRIA Futurs (http://www-futurs.inria.fr) BACKGROUND AND TOPICS PVM (Parallel Virtual Machine) and MPI (Message Passing Interface) have evolved into the standard interfaces for high-performance parallel programming in the message-passing paradigm. EuroPVM/MPI is the most prominent meeting dedicated to the latest developments of PVM and MPI such as new support tools, implementation and applications using these interfaces. The EuroPVM/MPI meeting naturally encourages discussions of new message-passing and other parallel and distributed programming paradigms beyond MPI and PVM. The 14th European PVM/MPI Users' Group Meeting will be a forum for users and developers of PVM, MPI, and other message-passing programming environments. Through the presentation of contributed papers, vendor presentations, poster presentations and invited talks, attendees will have the opportunity to share ideas and experiences to contribute to the improvement and furthering of message-passing and related parallel programming paradigms. Topics of interest for the meeting include, but are not limited to: * PVM and MPI implementation issues and improvements * Latest extensions to PVM and MPI * PVM and MPI for high-performance computing, clusters and grid environments * New message-passing and hybrid parallel programming paradigms * Interaction between message-passing software and hardware * Fault tolerance in message-passing programs * Performance evaluation of PVM and MPI applications * Tools and environments for PVM and MPI * Algorithms using the message-passing paradigm * Applications in science and engineering based on message-passing This year special emphasis will be put on large-scale issues, such as those related to hardware and interconnect techologies, or the potential or demonstrated shortcomings of PVM or MPI. As in the preceding years, the special session 'ParSim' will focus on numerical simulation for parallel engineering environments. EuroPVM/MPI 2007 will also hold the new 'Outstanding Papers' session introduced in 2006, where the best papers selected by the program committee will be presented. SUBMISSION INFORMATION Contributors are invited to submit a full paper as a PDF (or Postscript) document not exceeding 8 pages in English (2 pages for poster abstracts and Late and Breaking Results). The title page should contain an abstract of at most 100 words and five specific keywords. The paper needs to be formatted according to the Springer LNCS guidelines [2]. The usage of LaTeX for preparation of the contribution as well as the submission in camera ready format is strongly recommended. Style files can be found at the URL [2]. New work that is not yet mature for a full paper, short observations, and similar brief announcements are invited for the poster session. Contributions to the poster session should be submitted in the form of a two-page abstract. All these contributions will be fully peer reviewed by the program committee. Submissions to the special session 'Current Trends in Numerical Simulation for Parallel Engineering Environments' (ParSim 2007) are handled and reviewed by the respective session chairs. For more information please refer to the ParSim website [1]. All accepted submissions are expected to be presented at the conference by one of the authors, which requires registration for the conference. IMPORTANT DATES Submission of full papers and poster abstracts May 7th, 2007 Notification of authors June 11th, 2007 Camera-ready papers July 2nd, 2007 Submission of Late and Breaking Results September 15th, 2007 Tutorials September 30th, 2007 Conference October 1st-3rd, 2007 For up-to-date information, visit the conference web site at http//www.pvmmpi07.org. PROCEEDINGS In addition, selected papers of the conference, including those from the 'Outstanding Papers' session, will be considered for publication in a special issue of Parallel Computing in an extended format. GENERAL CHAIR * Jack Dongarra (University of Tennessee) PROGRAM CHAIRS * Franck Cappello (INRIA Futurs) * Thomas Herault (Universite Paris Sud-XI / INRIA Futurs) PROGRAM COMMITTEE George Almasi, (IBM, USA) Ranieri Baraglia, (CNUCE Institute, Italy) Richard Barrett, (ORNL, USA) Gil Bloch, (Mellanox, Israel) George Bosilca (Univeristy of Tennesse, USA) Hakon Bugge, (Scali, Norway) Franck Cappello, (University of Paris-Sud, France) Brian Coghlan, (Trinity College Dublin, Ireland) Yiannis Cotronis, (University of Athens, Greece) Jose Cunha, (New University of Lisbon, Portugal) Marco Danelutto, (University of Pisa, Italy) Luiz DeRose, (Cray, USA) Frederic Desprez, (INRIA, France) Erik D'Hollander, (University of Ghent, Belgium) Beniamino Di Martino, (Second University of Naples, Italy) Jack Dongarra, (University of Tennessee, USA) Edgar Gabriel, (University of Houston, USA) Al Geist, (OakRidge National Laboratory, USA) Patrick Geoffray, (Myricom, USA) Andrzej Goscinski, (Deakin University, Australia) Richard L. Graham, (ORNL, USA) William Gropp, (Argonne National Laboratory, USA) Rolf Hempel, (DLR - German Aerospace Center, Germany) Thomas Herault (INRIA/LRI, France) Yutaka Ishikawa (University of Tokyo, Japan) Rainer Keller, (HLRS, Germany) Stefan Lankes, (RWTH Aachen, Germany) Erwin Laure, (CERN, Switzerland) Laurent Lefevre, (INRIA, France) Greg Lindahl, (Pathscale, USA) Thomas Ludwig, (University of Heidelberg, Germany) Ewing Rusty Lusk, (Argonne National Laboratory, USA) Tomas Margalef (University Autonoma de Barcelona, Spain) Jean-Fran?ois M?haut (IMAG, France) Bernd Mohr (Forschungszentrum J?lich, Germany) Matthias M?ller, (Dresden University of Technology, Germany) Raymond Namyst (University of Bordeaux, France) Salvatore Orlando, (University of Venice, Italy) Christian Perez (IRISA, France) Fabrizio Petrini, (PNNL, USA) Neil Pundit, (Sandia National Laboratories, USA) Thomas Rauber, (Universit?t Bayreuth, Germany) Casiano Rodriguez-Leon, (University de La Laguna, Spain) Martin Schulz, (Lawrence Livermore National Laboratory, USA) Jeffrey Squyres, (Open System hs LAB, Indiana) Bernard Tourancheau, (University of Lyon / INRIA, France) Jesper Larsson Tr?ff, (C&C Research Labs, NEC Europe, Germany) Carsten Trinitis, (Technische Universit?t M?nchen, Germnay) Roland Wismueller, (University Siegen, Germany) Felix Wolf, (Forschungszentrum J?lich, Germany) Joachim Worringen, (C&C Research Labs, NEC Europe, Germany) CONFERENCE VENUE The conference will be held in the historical, cultural and economic center of Paris, the capital of France. The city, which is renowned for its neo-classical architecture, hosts many museums and galleries and has an active nightlife. The symbol of Paris is the 324 metre (1,063 ft) Eiffel Tower on the banks of the Seine. Dubbed "the City of Light" (la Ville Lumiere) since the 19th century, Paris is regarded by many as one of the most beautiful and romantic cities in the world. It is also the most visited city in the world with more than 30 million foreign visitors per year. Paris is easily reachable from any European capital and most of the large European, American and Asian cities. It is an ideal starting point for visiting european institutes and cities. REFERENCES [1] ParSim 2007: http://wwwbode.in.tum.de/Par/arch/events/parsim07/ [2] Springer Guidelines: http://www.springer.de/comp/lncs/authors.html From smulcahy at aplpi.com Wed Apr 11 02:09:13 2007 From: smulcahy at aplpi.com (stephen mulcahy) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Survey on beowulf.org -- and a drawing for a video iPod In-Reply-To: References: Message-ID: <461CA5B9.7070101@aplpi.com> Tim Cutts wrote: > I find the initial question essentially impossible to answer. The whole > point of beowulf computing is that you can build it out of almost > anything. Most of the software is community driven. The ideas are > community driven. The leaders are the community, not any one company. > The hardware is commodity. I can't even name one when pushed to; I > don't use any cluster software I had to buy, and the vendors who made my > hardware are pretty much incidental - I could have used anyone. > > And that answer to the first question makes the subsequent questions > even more impossible to answer. > > Looks like I don't get the video iPod. :-) As a relative newcomer to the HPC scene I'm inclined to agree that there is no clear hpc leader to my mind. But in the interests of getting into the draw I sat and pondered the question for a few minutes and then opted for the company name that bobbed on the surface of my mind :) Arguably my follow-on responses are of limited value in light of that. -stephen -- Stephen Mulcahy, Applepie Solutions Ltd, Innovation in Business Center, GMIT, Dublin Rd, Galway, Ireland. http://www.aplpi.com From m.janssens at opencfd.co.uk Wed Apr 11 02:42:09 2007 From: m.janssens at opencfd.co.uk (Mattijs Janssens) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] debugging In-Reply-To: <200704091130.33134.mafunk@nmsu.edu> References: <428810f20704010202k15df1d28m1a12a7d0c3facb92@mail.gmail.com> <200704091130.33134.mafunk@nmsu.edu> Message-ID: <200704111042.09827.m.janssens@opencfd.co.uk> Haven't used the debug facility in mpich so cannot comment on that one. But can you just start the job as usual and then on the node that crashes use gdb -pid XXX ? I tend to use lam and there you can use an 'application scheme' file which basically tells where to start what executable. You can use it to start a gdb session in a separate window for every process. Not recommendable for 32 processor jobs but excellent for development. A sample application scheme to start 'executableName' on two processors: xterm -e /bin/sh -c "gdb -command gdbCommands executableName 2>&1 | tee processor0.log; read dummy" xterm -e /bin/sh -c "gdb -command gdbCommands executableName 2>&1 | tee processor1.log; read dummy" The gbCommands file contains the gdb commands: run arg1 arg2 where It must be possible to do something similar with mpich. Mattijs On Monday 09 April 2007 18:30, Matt Funk wrote: > Hi, > > i hope this is the right mailing list to post > to... > > Anyway, i was wondering if i could get some > advice/direction on how to debug my mpich > program. I am running on a scyld configuration. > What i am trying right now is the following: > > mpirun -dbg=gdb -nolocal -np 32 exec > > which starts the debugger in which i go > run args > > which then start the program. However, it > doesn't get very far until it just sits there. > When i ps all the processes are defunced. > > When i do the same thing except mpirun -dbg=gdb > -nolocal -np 1 exec and run it in the debugger, > the program starts running well. > > The reason i want to run on 32 processor > though, is that it takes (on 32 procs) several > hours till my program crashes. Also, i would > like to be able to keep the conditions under > which it crashes intact as much as possible > (i.e. run on 32 procs rather than 1). > > Does anyone have any advice? I am open to try > out other things as well if possible. I am just > starting to learn debugger techniques for a > parallel program. > > thanks > mat > _______________________________________________ > Beowulf mailing list, Beowulf@beowulf.org > To change your subscription (digest mode or > unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf -- Mattijs Janssens OpenCFD Ltd. 9 Albert Road, Caversham, Reading RG4 7AN. Tel: +44 (0)118 9471030 Email: M.Janssens@OpenCFD.co.uk URL: http://www.OpenCFD.co.uk From schnorr at gmail.com Wed Apr 11 09:34:56 2007 From: schnorr at gmail.com (Lucas Schnorr) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] SBAC-PAD 2007: Call for Papers Message-ID: We apologize if you received multiple copies of this posting. Please feel free to distribute it to those who might be interested. =========================================================== CALL FOR PAPERS SBAC-PAD 2007 19th International Symposium on Computer Architecture and High Performance Computing Gramado, Brazil October 24-27, 2007 http://www.sbc.org.br/sbac/2007/ Sponsors: Brazilian Computer Society IEEE Computer Society (pending) =========================================================== SBAC-PAD is an annual international conference series, the first of which was held 20 years ago, in 1987. Each conference has traditionally presented new developments and high performance applications, as well as the latest trends in computer architecture and parallel and distributed technologies. This year, the symposium returns to Gramado, Rio Grande do Sul, Brazil. The conference will be held at the Hotel "Serra Azul," located in the heart of downtown Gramado and within walking distance of the main tourist attractions. SBAC_PAD 2007 will be sponsored by the Brazilian Computer Society, the IEEE Computer Society (approval pending) through the Technical Committees on Computer Architecture (TCCA) and Scalable Computing (TCSC), and IFIP. Authors are invited to submit manuscripts that present original unpublished research in all areas of computer architecture and high performance computing. Work focusing on applications or emerging technologies is especially welcome. Topics of interest include, but are not limited to: - Application-specific Architectures - Benchmarking, Performance Measurements and Analysis - Cache and Memory Architectures - Embedded Systems - Fault-Tolerant Architectures and Systems - Grid, Cluster, and Peer-to-Peer Computing - High Performance Applications and Parallel and Distributed Algorithms - Interconnection Networks, Routing, and Communication - Languages, Compilers and Tools for Parallel and Distributed Programming - Load Balancing and Scheduling - Microarchitecture - Operating Systems and Virtualization - Parallel and Distributed Architectures - Pervasive and Heterogeneous Computing - Real World Applications and Case Studies - Reconfigurable Systems SBAC-PAD invites manuscripts of original research, written in English and not exceeding 8 pages in double column following the IEEE Conference style, to be submitted in PDF format. The proceeding will be published by the IEEE Computer Society Press. A blind submission and review process will be used, therefore papers should be submitted without the authors' names on the manuscript. Moreover, authors should take any reasonable precautions to hide their identity. Citations to the authors' own prior work can be included, but references should be in the third person. Please check the SBAC-PAD 2007 web site for further paper submission and format information: http://www.sbc.org.br/sbac/2007/ The authors of selected papers will also be invited to submitted extended versions of their work for possible publication in a special issue of the International Journal of Parallel Programming. Important Dates --------------- - Paper Submission deadline May 28, 2007 - Author Notification July 09, 2007 - Conference October 24-27, 2007 General Chair ------------- Philippe O. Alexandre Navaux (UFRGS, Brazil) Program Co-chairs ----------------- Jean-Luc Gaudiot (UCI, USA) Vinod Rebello (UFF, Brazil) Steering Committee ------------------ Alberto F. de Souza (UFES, Brazil) Cl?udio L. Amorim (UFRJ, Brazil) Jairo Panetta (INPE, Brazil) Jean-Luc Gaudiot (UCI, USA) L?ria M. Sato (USP, Brazil) Philippe O. Alexandre Navaux (UFRGS, Brazil) Rajkumar Buyya (UMelb, Australia) Siang W. Song (USP, Brazil) Organizing Committee -------------------- Adenauer Yamin (UCPEL, Brazil) Denise Gr?ne Ewald (CESUP, UFRGS, Brazil) Lourdes Tassinari (UFRGS, Brazil) Lucas Schnorr (UFRGS, Brazil) Nicolas Maillard (UFRGS, Brazil) Philippe Navaux (UFRGS, Brazil) Tatiana Santos (UNISC, Brazil) Vinod Rebello (UFF, Brazil) Wagner Meira (UFMG, Brazil) Program Committee ----------------- Alba Melo (UnB, Brazil) Alberto De Souza (UFES, Brazil) Alex Nicolau (University of California, Irvine, USA) Alfredo Goldman (USP, Brazil) Alvaro Coutinho (UFRJ, Brazil) Bertil Folliot (University Pierre et Marie Curie, Paris VI, France) Bruno Schulze (LNCC, Brazil) Celso Mendes (University of Illinois, USA) Cesar De Rose (PUCRS, Brazil) Christophe Cerin (Universit Paris 13, France) Claudio Amorim (UFRJ, Brazil) Claudio Geyer (UFRGS, Brazil) Cristina Boeres (UFF, Brazil) D. Janakiram (Indian Institute of Technology, Madras India) David Kaeli (Northeastern University, USA) Denis Trystram (IMAG, France) Edil Fernandes (UFRJ, Brazil) Edson C?ceres (UFMS, Brazil) Eduardo Bergamini (INPE, Brazil) Felipe Fran?a (UFRJ, Brazil) Francisco Brasileiro (UFCG, Brazil) Frank Dehne (Carleton University, Canada) Gabriel P. Silva (UFRJ, Brazil) Geoffrey Fox (Indiana University, USA) Jack Dongarra (University of Tennessee, USA) Jairo Panetta (INPE/CPTEC, Brazil) Jean-Luc Gaudiot (University of California, Irvine, USA) Jos? Fortes (University of Florida, USA) Jose Moreira (IBM, USA) Jos? Saito (UFSCar, Brazil) Kuan-Ching Li (Providence University, Taiwan) Lalit Patnaik (Indian Institute of Science, India) Laurence Yang (St Francis Xavier University, Canada) Liria Sato (USP, Brazil) Lucia Catabriga (UFES, Brazil) Lucia Drummond (UFF, Brazil) Luiz Barroso (Google, USA) Luiz DeRose (Cray Inc., USA) Luiza Mourelle (UERJ, Brazil) Maria Clicia de Castro (UERJ, Brazil) Mario Nemirovsky (ConSentry Networks, USA) Mateo Valero (Technical University of Catalonia, Spain) Nader Bagherzadeh (University of California, Irvine, USA) Noemi Rodrigues (PUC-Rio, Brazil) Orlando Loques (UFF, Brazil) Osni Marques (Lawrence Berkeley National Laboratory, USA) Peter Rounce (University College London, UK) Philippe Navaux (UFRGS, Brazil) Priscila M V Lima (UFRJ, Brazil) Rafael Lins (UFPE, Brazil) Rajkumar Buyya (University of Melbourne, Australia) Renato Silva (LNCC, Brazil) Ricardo Bianchini (Rutgers University, USA) Ricardo Corr?a (UFC, Brazil) Ron Perrott (Queen's University Belfast, UK) Ronaldo Goncalves (Universidade Estadual de Maring?, Brazil) S?rgio Kofuji (USP, Brazil) Siang Song (USP, Brazil) Srikumar Venugopal (University of Melbourne, Australia) Valmir Barbosa (UFRJ, Brazil) Viktor Prasanna (University of Southern California, USA) Vinod Rebello (UFF, Brazil) Wagner Meira, Jr. (UFMG, Brazil) Yale Patt (University of Texas at Austin, USA) From Todd.Henderson at L-3Com.com Wed Apr 11 11:31:40 2007 From: Todd.Henderson at L-3Com.com (Henderson, T Todd @ IS) Date: Tue Mar 16 01:06:01 2010 Subject: Why is 64 bit faster. RE: [Beowulf] Win64 Clusters!!!!!!!!!!!! In-Reply-To: Message-ID: <5A340D9566F8C14EBFFBDF69158F07D502B56D3E@gvlexch03.is.l-3com.com> O.k. I'm going to ask, what I'm sure is a stupid question, actually 2. 1.) Why is a 64 bit cpu faster? I had assumed the main benefit was the memory that could be addressed, obviously a bad assumption. 2.) If you install the 32 bit version of an os, say linux, instead of the 64 bit version and then run 32 bit apps, do you get the speedup? Thanks, Todd -----Original Message----- From: beowulf-bounces@beowulf.org [mailto:beowulf-bounces@beowulf.org] On Behalf Of Robert G. Brown Sent: Wednesday, April 11, 2007 2:17 AM To: Jon Forrest Cc: Beowulf Mailing List Subject: Re: [Beowulf] Win64 Clusters!!!!!!!!!!!! On Sun, 8 Apr 2007, Jon Forrest wrote: > The experiment I tried was to build several fairly large programs on > Fedora Core 6 32 bit and then 64 bit using exactly the same hardware. > This is clearly not the same kind of test as your example but it's > more relevant as a systems-level comparison. > If I remember, I built mysql and apache. This took about the same > amount of time in both modes. Building programs isn't a particularly good benchmark for anything but building programs, for all that people tend to use make as a benchmark anyway. RUNNING programs (other than make and gcc) is much better. For one thing, compilers tend to basically be parsing engines processing text, with very little advantage available due to e.g. optimizing loops or streaming in double precision data to do a complex numerical computation on. > I suspect that the Beowulf crowd would have lots of experience with > 32-bits vs 64-bits question, so I'd welcome additional comments, > especially dealing with situations where programs > *don't* need the additional address space of the 64-bit model. Sure. 64-bits = "good". 32-bits = "bad", in nearly all CPU-intensive (especially floating point intensive) applications that don't thrash the cache. Cache thrashing is actually more expensive IIRC for 64 bit systems than it is for 32 bit systems because the penalty for a cache miss is higher. For pretty much everything else, though, goood. And even where it is bad it isn't THAT bad because its good at least partly compensates. Size of the program isn't terribly important to this conclusion, as Toone's miniprogram clearly shows. You don't need a lot of code to do a long-running double precision loop. rgb > > Cordially, > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@phy.duke.edu _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf From snj78 at comcast.net Wed Apr 11 21:02:03 2007 From: snj78 at comcast.net (snj78@comcast.net) Date: Tue Mar 16 01:06:01 2010 Subject: [Beowulf] Help with HPL Makefile Message-ID: <041220070402.9464.461DAF3B000AA4E1000024F82215567074C8C906029C@comcast.net> Thanks for your replies everyone. I feel like I am understanding things a little better. One thing that I realized but not sure if this makes a difference is that I have installed an openmpi package and an mpich package so could that cause problems using the mpicc compiler and linking the different packages/libraries? I have rebuilt my make file and something interesting that I have noticed is that no matter what I change I still get the same error. I am always getting the following error: Makefile:47: Make.inc: No such file or directory *** No rule to make target 'Make.inc'. Stop I use the following command in my top directory, which is /hpl: make arch=Beowulf while my makefile is Make.Beowulf. I am still at a loss at can't figure out what to do next so I really the help you guys have given me, even though this may seem trivial. The make file and error log follow: Make.Beowulf SHELL = /bin/sh # CD = cd CP = cp LN_S = ln -s MKDIR = mkdir RM = /bin/rm -f TOUCH = touch # ARCH = Linux_x86_64 # - HPL Directory Structure / HPL library ------------------------------ TOPdir = $(HOME)/hpl INCdir = $(TOPdir)/include BINdir = $(TOPdir)/bin/$(ARCH) LIBdir = $(TOPdir)/lib/$(ARCH) # HPLlib = $(LIBdir)/libhpl.a # - Message Passing library (MPI) -------------------------------------- MPdir = /usr MPinc =-I$(MPdir)/include/openmpi MPlib = $(MPdir)/lib64/openmpi/libmpi.so # - Linear Algebra library (BLAS or VSIPL) ----------------------------- LAdir = /usr/lib64 LAinc =-I$(MPdir)/include LAlib = $(LAdir)/libblas.so.3 $(LAdir)/atlas/libblas.so.3 # F2CDEFS = # HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) # HPL_OPTS = -DHPL_CALL_CBLAS # ---------------------------------------------------------------------- HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES) # - Compilers / linkers - Optimization flags --------------------------- CC = mpicc CCNOOPT = CCFLAGS = $(HPL_DEFS) -pipe -O3 -funroll-loops # LINKER = mpicc LINKFLAGS = $(CCFLAGS) # ARCHIVER = ar ARFLAGS = r RANLIB = echo Error File ( Condensed to show the last portion with the error code ) make -f Make.top refresh_tst arch=Beowulf make[1]: Entering directory `/hpl' cp makes/Make.matgen testing/matgen/Beowulf/Makefile cp makes/Make.timer testing/timer/Beowulf/Makefile cp makes/Make.pmatgen testing/pmatgen/Beowulf/Makefile cp makes/Make.ptimer testing/ptimer/Beowulf/Makefile cp makes/Make.ptest testing/ptest/Beowulf/Makefile make[1]: Leaving directory `/hpl' make -f Make.top build_src arch=Beowulf make[1]: Entering directory `/hpl' ( cd src/au