Sandvine: Streaming Video is 70% (Netflix 37%) of NA Fixed Traffic; BitTorrent is 4.4%

A new study by Sandvine focusing on Africa, the Middle East, and North America (based on data from a selection of Sandvine’s customers), finds that:

• "Real-Time Entertainment (streaming video and audio) traffic now accounts for over 70% of North American downstream traffic in the peak evening hours on fixed access networks. Five years ago it accounted for less than 35%
   
• Netflix (37.1%), YouTube (17.9%), and Amazon Video (3.1%), the top three sources of video traffic on fixed access networks in North America, all saw an increase in traffic share over the levels observed earlier in the year
   
• With the growth of video, BitTorrent share continues to see a decline in fixed access bandwidth share, and now accounts for only 5% of total traffic in North America. Last year during the same period it accounted for over 7%". 

See "Sandvine: Over 70% Of North American Traffic Is Now Streaming Video And Audio" - here.

Sweden: ISPs Cannot Be Forced to Block Pirate Bay

THE LOCAL SE reports that "In a landmark decision, a Swedish court on Friday ruled that the country's internet service providers cannot be forced to block controversial Swedish file-sharing site Pirate Bay. After considering the case for almost a month, the District Court of Stockholm ruled that copyright holders could not make Swedish ISP Bredbandsbolaget block Pirate Bay"
 
Related post - "How will Pirate Bay Avoid Blocking?" - here.

"The court found that Bredbandsbolaget's operations do not amount to participation in the copyright infringement offences carried out by some of its 'pirate' subscribers.

The action was brought by Universal Music, Sony Music, Warner Music, Nordisk Film and the Swedish Film Industry, who teamed up in a lawsuit last year designed to force Bredbandsbolaget to block the site. They claimed that, unless it blocks Pirate Bay, Bredbandsbolaget should be held responsible for the copyright infringements of its customers". 

See "Swedish court: 'We cannot ban Pirate Bay'" - here.

BitTorrent Clients May Generate DDoS Attacks, Boosting Bandwidth x120

BitTorrent, the old ISP challenger, is now creating a new threat - a way to generate DDoS attacks.

Source: P2P File-Sharing in Hell:

 Exploiting BitTorrent 

Vulnerabilities to Launch Distributed 

Reflective DoS Attacks (here)

Ernesto reports to TorrentFreak that "New research shows that BitTorrent clients and BitTorrent Sync can be exploited for Denial of Service attacks. With the help of the popular file-sharing protocol an attacker can reflect and amplify traffic through fellow file-sharers, boosting the original bandwidth 120 times. With dozens of millions of active users at any given point in the day the BitTorrent protocol is a force to be reckoned with.

While BitTorrent swarms are relatively harmless, a new paper published by City University London researcher Florian Adamsky reveals that there’s potential for abuse.

The paper shows that various BitTorrent protocols can be used to amplify Denial of Service attacks.

BitTorrent Inc has been notified about the vulnerabilities and patched some in a recent beta release. For now, however, uTorrent is still vulnerable to a DHT attack. Vuze was contacted as well but has yet to release an update according to the researcher.

For users of BitTorrent-based software there is no security concern other than the fact that people are participating in a DDoS attack without their knowledge. The vulnerability mostly leads to a lot of wasted bandwidth"

See "BITTORRENT CAN BE EXPLOITED FOR DOS ATTACKS, RESEARCH WARNS" - here.

Sandvine: NA Internet is Concentrated Around 3 Content Providers

Mobile Vs. Fixed: While Ericsson that sees "Facebook and 4 Other Apps" (here) in mobile networks, Sandvine sees (in North America's Fixed networks) much higher usage stats for Netflix, and claims that the internet is used to serve Netflix, Google and Facebook.

In its "latest Global Internet Phenomena Report focusing on North America and Latin America. The report is based on data from a selection of Sandvine’s 250-plus communications service provider (CSP) customers" Sandvine finds that:

• Netflix continues to slowly increase its domination of North American fixed networks, accounting for 36.5% of downstream traffic in the peak evening hours
   
• In Latin American, two companies, Facebook and Google, now control over 60% of total mobile traffic in the region

• BitTorrent continues to see a decline in fixed access bandwidth share, and now accounts for only 6.3% of total traffic in North America, and 8.5% in Latin America
   
• Netflix’s recent decision to encrypt their service will result in the majority of Internet traffic in Latin America and North America being encrypted in 2016 (see "Sandvine: Netflix Brings us to the Encrypted Age" - here)

See "Sandvine: In The Americas, Netflix + Google + Facebook = The Internet?" - here.

"The Interview" Downloads: Sony Vs. Torrent

While "The Interview" had modest distribution and revenues in movie theaters, it gained nice on line traction, mainly in the legitimate (paid) rental and purchase - but also in file sharing:

• Liana B. Baker report for Reuters that "Sony Pictures said on Sunday that the "The Interview" had been purchased or rented online more than 2 million times, generating more than $15 million in the first four days after the controversial comedy's wide theatrical release was shelved.
  This would rank the film, which angered North Korea and triggered a cyberattack against the studio, as the No. 1 online movie ever released by Sony Pictures, the company said in a statement.
  The film has also brought in $2.8 million in the limited theatrical run that began Christmas Day in more than 300 mostly independent theaters, according to tracking firm Rentrak".
  
  See "'The Interview' Becomes Sony's No. 1 Online Movie Of All Time" - here.
  

• ERNESTO, TorrentFreak, reported that "The unexpected release of The Interview is making headlines around the world, but for now only people inside the U.S. can see the film. Perhaps unsurprisingly, 200,000 people have already circumvented this restriction by turning to torrent sites where the film appeared just an hour after its official release. Even The Pirate Bay joined in and started pointing people to the movie as well".
  
  See "The Interview is a Pirate hit with 200K Downloads" - here. 

P2P Traffic Shaping is Still There; South Korea Leads

 
A recent report in TorrentFreak shows that DPI and traffic shaping is still a popular practice, globally. The report also finds that South Korea and Japan, known for their high speed broadband connections, are in the top 5 countries where ISPs are limiting the use of file sharing.

".. fresh data from the Google-backed Measurement Lab, which provides new insight into the BitTorrent throttling practices of ISPs all over the world. The data show that many ISPs still interfere with file-sharing traffic, but to varying degrees. Hundreds of ISPs all over the world limit and restrict BitTorrent traffic on their networks. Unfortunately, this is something that most of these companies are quite secretive about".

See "Is Your ISP Messing With BitTorrent Traffic? Find Out" - here.

BitTorrent's Live Streaming Goes Live

Yet another challenge for carriers (or DPI vendors) ? 

Andrew Orlowski reports to The Register that "BitTorrent’s live streaming protocol has finally emerged into the daylight after years of development. The broadcast data is assigned to small groups - "clubs" - which then share the stream with a UDP protocol. Congestion control is added at the last hop. There’s more detail in this excellent technical post on BitTorrent Inc’s engineering blog.

Source: The BitTorrent Engineering Blog

.. Putting high quality, low cost transmission tools in the hands of individuals and small groups is an exciting prospect. Particularly since the potholed public internet isn't capable of high quality video broadcasting - you need to use a CDN (Content Delivery Network), or even better, own one. Google kicked the net neutrality campaign in the nuts three years ago, and today operates the largest private network in the world, to carry its YouTube traffic".

See "BitTorrent opens kimono, gets out one-to-many streaming tool" - here.

F5 Announces DPI; 320Gbps/Chassis

As mentioned yesterday (see "[Light Reading]: Will F5's DPI have DPI? PCRF?" - here) F5 announced today its Policy Policy Enforcement Manager (PEM). " .. the newest member of its growing portfolio of solutions for fixed and mobile service providers .. generally available now".

"With BIG-IP® Policy Enforcement Manager (PEM)  service providers now have the ability to classify traffic and enforce policy based on applications and subscriber behavior. BIG-IP PEM also provides granular reports on subscriber and application flows, and it integrates with online charging subsystems, giving operators real-time credit control

BIG-IP PEM gives you the ability to classify traffic into several categories of applications and protocols. These are some examples of the types of apps and protocols BIG-IP PEM supports:

• P2P: BitTorrent, Gnutella
• VoIP: SIP, Skype, Yahoo!, Jabber
• Web: HTTP, HTTPS, FTP, YouTube, Facebook
• Streaming: HTTP Streaming, RTSP, HTTP Audio

 .. BIG-IP PEM is a NEBS-compliant platform .. achieve up to 320 gigabits per second L7 throughput, support up to 72 million concurrent connections, and serve as many as 96 million subscribers within a single chassis—helping you manage traffic with fewer servers"

See "F5 announces BIG-IP Policy Enforcement Manager (PEM)" - here.

Research: OpenDPI Performance; How many Packets Needed to Identify Applications?

 
New research by Jawad Khalife and Amjad Hajjar (pictured), Lebanese University/Faculty of Engineering, IT department, Beirut, Lebanon and Jesús Díaz-Verdejo, University of Granada/Department of Signal Processing, Telematics and Communication, Granada, Spain looks at the performance of OpenDPI (see also "nDPI Supports Skype, Whatsapp and Netflix" - here).

Abstract

The identification of the nature of the traffic flowing through a TCP/IP network is a relevant target for traffic engineering and security related tasks. Despite the privacy concerns it arises, Deep Packet Inspection (DPI) is one of the most successful current techniques. Nevertheless, the performance of DPI is strongly limited by computational issues related to the huge amount of data it needs to handle, both in terms of number of packets and the length of the packets. One way to reduce the computational overhead with identification techniques is to sample the traffic being monitored. This paper addresses the sensitivity of OpenDPI, one of the most powerful freely available DPI systems, with sampled network traffic. Two sampling techniques are applied and compared: the per-packet payload sampling, and the per-flow packet sampling. Based on the obtained results, some conclusions are drawn to show how far DPI methods could be optimised through traffic sampling.

One finding relates to the number of packet the classifier needs to see in order to correctly classify applications, which significantly affect the system performance:

 "The average packet detection number in the dataset is shown in Fig. 6  for most common protocols. Some protocols like iMESH and Bittorrent, show higher values than other protocols. We validated the fact that the presence of most deviation is due to flows that were under course during the start of the capture. Most protocols averages were below 10 packets .. As a result for per-flow sampling, studied in this section, inspecting the first 4 to 10 packets of a flow (as DPI input for inspection) could maintain the flow classification accuracy at high levels ranging from 90% to 99%. 

In choosing the appropriate value of Nmin 

for the 

classifier, two situations should be distinguished

according to the classification target:

If the target is to classify only one specific protocol, 

N

min could be easily specified according to Fig. 6 (e.g. for 

HTTP, Nmin=4). In this case, the classifier would inspect only the minimum number of packets, necessary for flow classification. However, if the target is to classify all protocols, which is the most common situation, Nmin should be assigned the maximum value of the average packet detection number (Nmin=10) in order to classify most protocols. In this case, and for protocols whose average packet detection number is lower than Nmin, the classifier would inspect more packets than necessary.
 
See "Performance of OpenDPI in Identifying Sampled Network Traffic" - here.

[Ofcom] UK Traffic Management; P2P Might be Reduced by 99%

In its 2012 update, Ofcom, the UK regulator reports also on the status of traffic management in the UK. Ofcom says  [see also "UK: Broadband ISPs (except Sky) Still Limit P2P; Some Prioritize VoIP!" - here]:

• We looked at the traffic management polices used by fixed and mobile operators and found that there is often significant variation in fixed ISPs’ and mobile operators’ traffic management practices. While most apply traffic management in some form, others do not and use the fact that they have no traffic management as a part of their promotional message (e.g. advertising their service as ‘truly unlimited’ in terms of both data use and throttling of certain services). 
   
• Most commonly, ISPs tend to apply traffic management to P2P services such as BitTorrent. P2P traffic is most likely to have restrictions placed on it because of the way it works: the nature of P2P software is such that it increases data usage along a network to fill whatever capacity is available. That, and the fact that P2P downloads are not typically as time-dependent as other types of application mean that many ISPs and mobile operators find that controlling P2P traffic is an approach to keep down network infrastructure costs. 

• The impact of traffic management on users of P2P services can be significant – in some cases speeds of P2P traffic can be reduced to a fraction of the speed of other traffic on the network. We asked operators to estimate the impact of traffic management restrictions. Although most told us that such calculations were too difficult to make, others told us that the impact could be significant, e.g. one fixed ISP said the estimated impact of its traffic management policies was to reduce P2P traffic to 1% of what it would otherwise be at peak times. 

• P2P is not the only type of traffic which is subject to traffic management. For example, O2 restricts the speed of video traffic on its lowest priced broadband package to 0.8Mbit/s, and offers higher-priced broadband packages that do not use traffic management for video traffic.
   
• In some cases, particular types of traffic are given greater priority and consumers who particularly value a type of service can choose packages that prioritise such traffic. For instance, Plusnet offers a premium broadband service for an additional £5 a month which gives priority to gaming traffic and traffic for virtual private networks (VPN).

See more - "Infrastructure Report - 2012 Update" - here. 

DPI Deployments [151]: 3UK Identifies and Controls File Sharing, Tethering and High Data Use

    

While 3 UK mobile data plans are "all you can eat" plans, the UK operator decided that some restrictions should apply. 3's network is "designed for data" but "it’s a fact that no mobile operator has an unlimited amount of network capacity, so while we’ll continue to invest, we also have a responsibility to make sure the network we already have is used wisely".

3 UK launched recently a new management tool, TrafficSense, which "..looks at types of data traffic and behaviour that stop other users from enjoying a fair allocation of our network. TrafficSense™ will only manage a very small percentage of users on our network – and the vast majority will actually benefit from higher speeds and a better internet experience".

TrafficSense identifies "File sharing, Tethering and High data use". See " About TrafficSense" - here.

File Sharing includes "peer-to-peer download services like BitTorrent or Napster [see "Napster, the Service that Started DPI, is Gone" -here] and downloading and sharing files using certain sites specifically set up to share files". See more below. 

Pirate Pay Stops Illegal File Transfers - How?

 
Elena Shipilova reports to Russia Beyond the Headlines that "The creators of Pirate Pay, a Perm-based start-up, say they can stop files from being illegally downloaded from torrent networks .. the technology prevents file sharing in torrent networks".

   

Andrei Klimenko, founder and CEO of Pirate Pay describes the method they use: "In December 2011 the film “Vysotsky. Thanks to God I’m Alive” came out in movie theaters, and for a month after its opening, Pirate Pay protected the film on torrent networks .. We used a number of servers to make a connection to each and every p2p client that distributed this film. Then Pirate Pay sent specific traffic to confuse these clients about the real I.P. addresses of other clients and to make them disconnect from each other .. Not all the goals were reached. But nearly 50,000 users did not complete their downloads" (see more- here).
 
"This company’s successful defense of the film brought in its first big payday. Company officials will not discuss exact earnings, but said that projects will cost clients between approximately $12,000 and $50,000 depending on the resources needed to mount a defense".
 
A year ago Microsoft invested $100,000 in the compnay (see "Microsoft вручил грант первому стартапу, ставшему победителем конкурса Фонда посевного финансирования компании" - here).
 
See "Russian innovators pursue prototype to prevent piracy" - here.

Bell Canada and Rogers are still on the World's Top Throttlers List

  

MLabs, a project "funded by the U.S. National Science Foundation" (see "Syracuse University Research on Deep Packet Inspection" - here), published updated information on ISPs found to be throttling data, through DPI, for BitTorrent files sharing traffic.

".. we show the 10 or 11 ISPs who had the highest percentage of tests that yielded a positive result; that is, a result indicating that DPI was used to throttle or block BitTorrent. If an ISP’s result on the chart is 93%, for example, it means that 93 percent of the tests their users conducted using Glasnost produced a result that shows that the BitTorrent application was singled out and manipulated by the ISP using deep packet inspection technology. The chart shows the two-letter country code and the name of the network operator".

Among the the top 10 throttlers for the first quarter of 2012 are two Canadian operators, who said recently they will stop throttling P2P traffic on March 2012 - will we see them the next time?

• Bell Canada - see "Bell Canada will Stop Shape "diminishing" P2P Traffic on March 2012" - here 
• Rogers - see "Rogers Stops DPI/Traffic Management" - here.

See " The world’s top bittorrent manipulators" - here.

Syracuse University Research on Deep Packet Inspection

   

Syracuse University has now a website ("The Network is aware" - here) that ".. features the ongoing activities and results of research investigating whether deep packet inspection is changing the way the Internet is governed .. We analyze DPI deployments that generated political, legal and regulatory conflicts. We explore how its capabilities led to strategic interactions among network operators pursuing their business interests, government agencies seeking control, activists fighting for privacy or net neutrality, politicians and regulators responding to publicity, legislators and courts resolving disputes"
 

Analytical model of co-production

 of technology and governance

used in the case studies.

From Bendrath & Mueller (2011)



The research is led by Milton Mueller (pictured) , Principal Investigator, with a team of 5 researches (here) and "funded [here] by the U.S. National Science Foundation, SBER Division, Program on Science, Technology and Society".

See also "Milton Mueller vs. Dan Kaminsky on ISP Traffic Management Detection" - here.

Among other things, the site features [rather old] data from MLab (here, supported by Google), that uses "crowdsourced network monitoring data" to detect possible use of DPI by ISPs for BitTorrent throttling.

Salient Federal Solutions: IPv6 Threats can be Eliminated Using DPI

  
Salient Federal Solutions reports "real-world incidents of IPv6 attacks based on the emerging protocol's tunneling capabilities, routing headers, DNS broadcasting and rogue routing announcements. The company asserts that all of these threats can be eliminated with the use of IPv6-enabled deep packet inspection tools, which it (here) and other network vendors sell".

See "IPv6 eyed by torrent users to avoid network throttling" - here.

Jeremy Duncan (pictured), senior director and IPv6 network architect for Salient Federal Systems, said: "IPv6 tunneling gives attackers a green light to penetrate networks .. uTorrent, which is an IPv6-capable freeware client for the BitTorrent peer-to-peer protocol that's used to share large files such as music and movies runs very well over Teredo, and that the BitTorrent community is discovering IPv6 as a way of avoiding network congestion controls that are used by ISPs to manage BitTorrent traffic on IPv4 networks"

The slides below are taken from Jeremy's presentation "IPv6 Is Here. Is Your Network Secure?" at the 7th Annual GFIRST National Conference, held in Nashville earlier this month.

The 2 parts presentation is available here and here.

 

Research Concludes: "BitTorrent is Good for Tier1 ISPs"

 

A new report  by John S. Otto, Mario A. Sánchez, David R. Choffnes, Fabián E. Bustamante (pictured) from Northwestern University and Georgos Siganos from Telefónica Research  presents a comprehensive view of BitTorrent - "using data from a representative set of 500,000 users sampled over a two year period, located in 169 countries and 3,150 networks".

See "On Blind Mice and the Elephant - Understanding the Network Impact of a Large Distributed System" - here and below.

In their conclusions, the authors say that "We found that despite its global reach, BitTorrent is able to remain local for large portions of its traffic. Further, our results show that most traffic generated by BitTorrent users stays at or below tier 3 .. Most BitTorrent traffic flows over cost-free paths and that it generates substantial revenue potential for many higher tier ISPs. We also highlighted the importance of the temporal pattern behind the generated traffic under the common 95th-percentile charging model"

Tesco[UK] Fair Use Policy - Pay More or be Terminated!

  

Tesco Broadband [UK] has a common traffic management policy for its fixed broadband service: "Tesco constantly monitors the way in which our customers use our broadband services. In particular, at peak times, we will look for (and restrict) non-time-critical traffic, such as Bit Torrent, other peer to peer file sharing applications and on-line storage services ,,  during the peak hours ..we may slow down specific services to make the shared usage of our network fair for all customers .. This policy ensures that we can deliver a great service to all our customers at all times and we never have to limit the customers who are using the internet for day to day time-critical transactions, such as normal surfing, e-mailing, on-line shopping and banking, using BBC/Sky iPlayer applications, gaming or making on-line phone calls via companies like Skype".

However, Tesco's fair use policy seems to be unique (here):

"We regularly monitor and review our customer’s collective and average monthly usage to set our fair usage limit (FUL) at a level that will not affect the majority (at least 95%) of our customers. Currently the FUL is set at 100GB per month.  If a customer regularly downloads in excess of the FUL, we take the following steps:
 
1. When we first notice that a customer has exceeded the FUL, we contact the customer to bring the matter to their attention. We will ask the customer to modify their use and/or give them the opportunity to move onto a ‘Super-user tariff’ (see our Price List for details). [I can’t find it there or elsewhere]

2. If the customer declines to move to the Super-user Tariff but continues to exceed the FUL for a further two consecutive months, we will suspend or terminate the customer’s service.
 

NI Deployments (72): StarHub (Singapore) Selected Neuralitic for Usage Profiling and Monetization

     

Neuralitic Systems announced a ".. a new deal with Singapore’s StarHub. The mobile operator will utilize Neuralitic’s SevenFlow™ software suite to understand the data usage profile of its customers and monetize data traffic".

See "Neuralitic Expands its Customer Base with StarHub" - here.

StarHub, Singapore’s 2nd largest mobile operator, had 2.145M subscribers at the end of Q1, 2011 (see chart).

Joanna Chan, VP of Personal Solutions, StarHub said: “The insights provided by SevenFlow help us in understanding customers' data usage pattern so that we can develop products and initiatives that better cater to the needs of specific customer segments.”

Neuralitic’s SevenFlow is a Mobile Data Intelligence solution that gives marketing professionals and other decision-makers the critical insights they need to improve marketing ROI based on actual subscriber usage characteristics. Its Data Bus component extracts 100% of all mobile network data from a copy of traffic across an industry-standard network interface. The Data Bus also includes customizable business rules to filter data on an as-needed basis, and channels the data to the Knowledge Portal.

Among other functions, the Knowledge Portal presents protocol usage information on a wide range of distinct protocols (http, BitTorrent, ftp, etc.); and helps users measure the impact of particular protocols on the mobile network. See also a patent application - "A METHOD AND SYSTEM FOR ANALYSING A MOBILE OPERATOR DATA NETWORK" - here.

[update] Does Netflix Replace P2P File Sharing?

 

A while ago I posted a Wired article that used data from Arbor Networks and Sandvine to analyze if Netflix replaces P2P file sharing as the most common way to consume Internet video (here). The information from the two DPI vendors showed very different levels of P2P file sharing (19.2% according to Sandvine and 8% as reported by Arbor).

Now Sandvine tries to explain why its data is more reliable.  Matt Tooley (picture), VP of Consulting Solutions, says in the company's Blog that the reason is Sandvine' ability to detect encrypted BitTorrent traffic.

Indeed, encryption was always the main challenge to DPI vendors when trying to identify P2P file sharing traffic, or other application that ISPs do not like (such as Skype). See "DPI Research: New Features are Needed (Encryption, User Profiling)" - here. 

  

"..  “BitTorrent (regular)” and “BitTorrent (UDP)” are trivial to identify, whereas the encrypted and uTP varieties require very sophisticated traffic identification techniques.  Perhaps this reality can explain the discrepancy between the numbers provided by Sandvine and those provided by other organizations. It’s possible that our P2P Filesharing numbers (which included all varieties of BitTorrent) were being compared against only the easily-detected protocols".

See "Sandvine’s Take on Netflix’s Impact on P2P File Sharing" - here.

BitTorrent Works with Content Producers for "content 'discoverability’, distribution and monetisation"

  

Natalie Apostolou reports for The Register on BitTorrent's CEO Eric Klinker (picture)speech at the MipTV event in Cannes, France.

“our big challenge is that our technology gets used for things that don’t make us many friends in certain parts of the world, ISPs and media to be specific"

"Over the last 18 months, the company has been working with content producers on ways to leverage the BitTorrent audience for content “discoverability’, distribution and monetisation .. BitTorrent is also working with Facebook but has yet to reveal details. One area that they may work on together could be the shift from downloads to streaming"

Klinker said:“There’s no question that streaming is a user experience that people want: YouTube proved it, and YouTube proved that a streaming piracy site is what people want, but fortunately Google took it over just in time”

See "BitTorrent plays nice with content" - here.

See also "BitTorrent CEO Prepares for the No Net Neutrality Future - Power to the People!" - here.