[Beowulf] Accelerator for data compressing
lindahl at pbm.com
Thu Oct 2 14:55:02 PDT 2008
On Thu, Oct 02, 2008 at 05:40:31PM -0400, Joe Landman wrote:
> I have heard of some "xml accelerators" in the
> past (back when XML was considered a good buzzword) that did on-the-fly
Well, given how wordy the tags are, simply compressing those is
inexpensive and is a big win, if they're a large % of the data.
To get back to the question that was asked, (1) no hardare compresser
compresses smaller than a good software compresser (hardware
compressors tend to be faster but can't compress as well), and (2)
sounds like your data can be compressed in an embarrassingly parallel
fashion, so on a quad-core box you might find that you can keep up
with your link with software compression in parallel.
More information about the Beowulf