I am investigating a couple of compression libraries, and comparing both their compression and speed. zlib is what we use now, and is moderately fast and gives pretty good compression. fastlz is blindingly fast, but doesn’t compress quite as well, and the code isn’t “stable”. lzma is one that I just started looking at, and my initial tests were abysmal. It gave the best compression ratio, but compression took over sixty times as long as fastlz (0.27 seconds vs. 16.47 seconds for the same 11 MB file). I posted a question on their forum asking what I was doing wrong, and got this reply (emphasis mine):
What do you compress and why do you need it faster?
Excuse me? Your algorithm runs an order of magnitude slower than the others I’m looking at, and you are seriously asking why I need it faster? To his credit, the suggestion he gave me did speed it up so that it was about 3.2 seconds; still the slowest of the bunch, but at least it’s now acceptable. And it did still have the best compression ratio. I’m just stunned that any software guy would ask such a question.