[Beowulf] RE: [Bioclusters] FPGAin bioinformatics clusters (again?)

[Mike Davis]

I agree in principle and I've had these talks with several post-docs and 
PI's, however, the assembly of a genome is not a trivial undertaking, 
and we have found no better software than that already mentioned. The 
issue is that it just wasn't designed for the sizes that we are now 
dealing with.

My point is that in bioinformatics, sequencing is just the first part of 
the problem. The assembly, annotation, array information, mass spec, and 
more is also part of the equation. It is important to remember also, as 
I mentioned at a BioGrids workshop last week, that much of 
bioinformatics software has been developed and successfully used not 
because it was the best, but because it was good enough. What I believe 
that the field is experiencing now is reaching the phase where 
optimization and even recoding will become important to find the 
solutions that we are starting to look for.

I personally am happy to listen to any ideas that might improve the process.

Mike Davis





bella at carolina.rr.com wrote:

> Mike Davis wrote:
>
>> But BLAST is only a small part and argueably the easiest part of 
>> genomics work. The advantages of parallelization and/or smp come into 
>> play when attempting to assemble the genome. Phred/Phrap can do the 
>> work but starts to slow even large machines when your talking 50k+ of 
>> sequences (which it wants to be in one folder). A quiz for  the Unix 
>> geeks out there, what happens when a folder has 50,000 files in it. 
>> Can you say SLOOOOOOOOOWWWW?
>>
>> Mike Davis
>>
> Sorry... I just couldn't let this one go by.  And no offense meant to 
> anyone but...
>
> Many times I have found users and application folks making 
> inordinately and (in my opinion) unacceptably large numbers of files 
> in sub-directories on one of "my" UNIX or Linux boxes.
> I simply gently take them aside and have a little "prayer meetin'" 
> with them.  There is always a way to fix this kind of problem by 
> consulting with the applications folks, and helping them see a better 
> way.  That's why God made "mkdir (2)".
>
> In my opinion, if this "Phred/Phrap" thingy (about which I KNOW 
> NOTHING - all disclaimers apply) _absolutely_  requires one to place 
> 50,000 (or more) files in a single sub-directory... and therefore is 
> slow... the application is simply broken.  Contact the developers, or 
> get the source... and we'll go fix it.
>
> My 1 & 1/2 cents worth.
>
> Arthur Bell
> Senior UNIX/Linux System Administrator
>
>