How does Google’s Page Speed lossless image compression work?

If you’re really interested in the technical details, check out the source code: png_optimizer.cc jpeg_optimizer.cc webp_optimizer.cc For PNG files, they use OptiPNG with some trial-and-error approach // we use these four combinations because different images seem to benefit from // different parameters and this combination of 4 seems to work best for a large // … Read more

Hadoop: compress file in HDFS?

For me, it’s lower overhead to write a Hadoop Streaming job to compress files. This is the command I run: hadoop jar $HADOOP_HOME/contrib/streaming/hadoop-streaming-0.20.2-cdh3u2.jar \ -Dmapred.output.compress=true \ -Dmapred.compress.map.output=true \ -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec \ -Dmapred.reduce.tasks=0 \ -input <input-path> \ -output $OUTPUT \ -mapper “cut -f 2” I’ll also typically stash the output in a temp folder in case something … Read more

Compression formats with good support for random access within archives? [closed]

Take a look at dictzip. It is compatible with gzip and allows coarse random access. An excerpt from its man page: dictzip compresses files using the gzip(1) algorithm (LZ77) in a manner which is completely compatible with the gzip file format. An extension to the gzip file format (Extra Field, described in 2.3.1.1 of RFC … Read more

Gzip versus minify

Very simple to test. I took your js, put them in different files and ran gzip -9 on them. Here’s the result. This was done on a WinXP machine running Cygwin and gzip 1.3.12. -rwx—— 1 xxxxxxxx mkgroup-l-d 88 Apr 30 09:17 expanded.js.gz -rwx—— 1 xxxxxxxx mkgroup-l-d 81 Apr 30 09:18 minified.js.gz Here’s a further … Read more

What is a good Java library to zip/unzip files? [closed]

I know its late and there are lots of answers but this zip4j is one of the best libraries for zipping I have used. Its simple (no boiler code) and can easily handle password protected files. import net.lingala.zip4j.exception.ZipException; import net.lingala.zip4j.core.ZipFile; public static void unzip(){ String source = “some/compressed/file.zip”; String destination = “some/destination/folder”; String password = … Read more

How are zlib, gzip and zip related? What do they have in common and how are they different?

Short form: .zip is an archive format using, usually, the Deflate compression method. The .gz gzip format is for single files, also using the Deflate compression method. Often gzip is used in combination with tar to make a compressed archive format, .tar.gz. The zlib library provides Deflate compression and decompression code for use by zip, … Read more