How does Netflix Use Big Data to Improve QoE?

Nirmal Govind, Director, Streaming Science & Algorithms, Netflix, explains how "we use big data for deep analysis and predictive algorithms to help provide the best experience for our members ..  the types of problems we're solving [I am quoting 2 of the items]

• Understanding the impact of QoE on user behavior

To improve the streaming experience, we look at QoE metrics that are likely to have an impact on user behavior. One metric of interest is the rebuffer rate, which is a measure of how often playback is temporarily interrupted while more data is downloaded from the server to replenish the local buffer on the client device. Another metric, bitrate, refers to the quality of the picture that is served/seen - a very low bitrate corresponds to a fuzzy picture. There is an interesting relationship between rebuffer rate and bitrate. Since network capacity is limited, picking too high of a bitrate increases the risk of hitting the capacity limit, running out of data in the local buffer, and then pausing playback to refill the buffer. What’s the right tradeoff?

• Determining what movies and shows to cache on the edge servers based on member viewing behavior [see "Netflix CDN Customers have More Fun" - here]

A set of big data problems also exists on the content delivery side. Open Connect is Netflix's own content delivery network that allows ISPs to directly connect to Netflix servers at common internet exchanges, or place a Netflix-provided storage appliance (cache) with Netflix content on it at ISP locations. The key idea here is to locate the content closer (in terms of network hops) to our members to provide a great experience.

One of several interesting problems here is to optimize decisions around content caching on these appliances based on the viewing behavior of the members served. With millions of members, a large catalog, and limited storage capacity, how should the content be cached to ensure that when a member plays a particular movie or show, it is being served out of the local cache/appliance?

See "Optimizing the Netflix Streaming Experience with Data Science" - here.