How to Build DPI Products? (Part XV - DPI for Compress HTTP w/o Decompression)

 
A research by Anat Bremler-Barr (pictured - see also here) and Shimrit Tzur David from The Interdisciplinary Center, Hertzelia, Israel, David Hay, The Hebrew University, Jerusalem, Israel and Yaron Koral, TelAviv University, Tel Aviv, Israel.

See "Decompression-Free Inspection: DPI for Shared Dictionary Compression over HTTP" - here.

ABSTRACT

.. The portion of compressed traffic of overall Internet traffic is constantly increasing. This paper focuses on traffic compressed using shared dictionary. Unlike traditional compression algorithms, this compression method takes advantage of the inter response redundancy (e.g., almost the same data is sent over and over again) as in nowadays dynamic Data. Shared Dictionary Compression over HTTP (SDCH), introduced by Google in 2008, is the first algorithm of this type. SDCH works well with other compression algorithm (as Gzip), making it even more appealing. Performing DPI on any compressed traffic is considered hard, therefore today’s security tools either do not inspect compressed data, alter HTTP headers to avoid compression, or decompress the traffic before inspecting it.

We present a novel pattern matching algorithm that inspects SDCH-compressed traffic without decompressing it first. Our algorithm relies on offline inspection of the shared dictionary, which is common to all compressed traffic, and marking auxiliary information on it to speed up the online DPI inspection. We show that our algorithm works near the rate of the compressed traffic, implying a speed gain of SDCH’s compression ratio (which is around 40%). We also discuss how to deal with SDCH compression over Gzip compression, and show how to perform regular expression matching with about the same speed gain.