The DPI Story – Part II – DPI for Traffic/Bandwidth Management

In my previous post I listed some of the solutions associated with DPI. Nevertheless, the most common use of DPI today is for traffic (or bandwidth) management, in public networks – usually serving residential subscribers.

The roots for these solutions take us back to the ‘90s, when enterprise class products for traffic management appeared. Used to be called QoS (Quality of Service), and later Wan Optimization solutions, these products allowed IT mangers to control how their Wan connections will behave.

Typically, Wan links (either private, VPN or Internet) were slow – just few Mbps – and overloaded during business hours. IT mangers wanted a simple way to tell the network which applications are critical, prioritize or guarantee their performance while slowing down other applications. The payback (ROI) was very quick – as bandwidth was expensive and the demand for bandwidth was always on the rise.

Around the turn of the century, broadband services became widely available to consumers at home and replaced the slow dialup connections. At the same time, new services started to emerge – and one of the most appealing service was music sharing/downloading (Napster was the first). Regardless of the question of the legality of copyrighted content sharing – these music downloads (and uploads) generated unprecedented load on ISPs uplinks and last-mile connections. It was quickly became clear that without control, any upgrade to the network will be 100% loaded, as more subscribers will just download and share more content.

Since enterprise Wan Optimization solutions were available, they found their way to the ISP market. They were required to do what they had done for enterprise networks – prioritize critical traffic (for example Web browsing) to provide the subscribers a good QoE (Quality of Experience) and slow-down other types of traffic (music download, later known as P2P applications), to avoid network upgrades (links and infrastructure). And so they did, and the ROI was great.

The early adaptors were usually smaller ISPs, serving a relatively small number of subscribers or operators in regions were bandwidth was extremely expensive or not available. P2P applications were still simple to detect, and the main need was for a simple-to-use appliance rather than the sophisticated DPI solution we see today. Enterprise class products were “good enough” to help these early adaptors.

During 10 years of deployment of bandwidth management solutions in ISP netwroks, the market has gone through a number of changes, resulting in with today’s solutions:

• P2P file-sharing applications became very popular, and are used to download and upload music, video and other large files. During the years, the P2P applications (Emule, BitTorrent and others – collectively described as “bandwidth hungry” applications) consumed 60-70% of the internet capacity.
• Detecting the bandwidth hungry applications became a major engineering challenge, as their creators found many ways to avoid control. A number of techniques had to be developed to identify these applications – creating the core of DPI.
• The network capacity has grown in 3 orders of magnitude. DPI solutions today support throughput of 10s of Gbps vs. few 10s of Mbps
• New applications, such as real-time voice and video-streaming appeared creating high expectations for QoE by the subscribers
• Traffic management solutions are expected to be integrated with other network elements and services, such as subscriber management systems, policy management servers and others services elements. All together they should create a cost-effective and well-managed solution solving the service provider business needs.

As a result of the above trends and changes, Wan Optimization and DPI products and solutions are no longer the same. While the basic core technology may look the same, the differences are significant enough to expect separate product lines (and in most cases, vendors) for each market.

More on DPI for bandwidth management on the next post.