20-05-2014, 14:05 #1
[EN] Apple Negotiating Paid Interconnect Deals With ISPs For Their Own CDNDan Rayburn | Tuesday May 20, 2014
In February I blogged about a new group formed inside of Apple last year, tasked with building out their own CDN to deliver Apple software updates, apps and other Apple related content. Since my post, Apple has been very busy with their build out deploying a lot of boxes running Apache Traffic Server and buying a ton of transit, co-location, wavelengths and other infrastructure services. Their CDN is quickly growing, and it won’t be long before we start seeing a portion of their content getting delivered from their new CDN.
As part of their build out, Apple is currently negotiating paid interconnection deals with some of the largest ISPs in the U.S. I’m not going to disclose which ISPs they are talking to and what deals they have already done, but it’s interesting to note that with all the talk lately of net neutrality, peering and interconnect relationships, Apple isn’t out in the market making any complaints. While Netflix has used the media, consumers and lawmakers to try and argue that CDNs should get as much peering as they want, at no charge, Apple doesn’t seem to agree with that sentiment. If they do, they certainly aren’t complaining in any public forum.
At a time when interconnection deals are getting so much exposure, Apple hasn’t used it as an opportunity to argue about the current business models of how networks connect with one another. Much like Microsoft, Google, Facebook, Pandora, Ebay and other content owners that have already built out their own CDNs, Apple appears to see paid interconnect deals as simply part of the costs associated with building out their own CDN network. To date, no other content owner or content syndicator that has built out their own CDN has complained of the current business models or argued about doing mutually beneficial interconnect deals between networks. If interconnect deals are such a problem in the industry, or a threat as many make it out to be, you don’t see the ones who actually have to pay for these deals complaining, other than Netflix.
Some might suggest that the reason for this is that Apple is not as big as Netflix since according to Sandvine data, Apple takes up only 2% of total Internet traffic at peak while Netflix takes up 34%. While that’s true, when Apple releases a new operating system for the iPhone, like they did last year, their iOS 7 and app downloads accounted for nearly 40% of all the traffic inside ISP’s networks, almost overnight. So Apple does push a lot of traffic at times and the more devices they sell, the larger their traffic grows.
What Apple is working on aligns with what all of the other big content syndicators out there have already built, which is a considerable amount of their own distributed origin infrastructure, for both large and small objects. Part of Apple’s reasoning for building their own CDN is because of performance issues with iCloud, with Apple wanting to have more control over the end-user experience. Apple already controls the hardware, the OS and the iTunes/App store platforms. Right now Apple controls the entire customer experience, except for the way content is delivered to their devices, so it’s only natural that a company of their size would build out their own CDN.
For all the talk in the media about how bad paid interconnect deals are for the Internet, this is how the Internet was able to grow over the last twenty years and how services got to the scale and performance that they are today. Without these interconnect deals taking place, the Internet would not operate as well and fewer services would be available in the market. I think we should rely on those who actually have to build out these CDNs and pay the costs associated with doing so to tell us whether or not the current way of doing business need to be changed. But so far, out of those who have built their own CDNs to deliver content including Netflix, Microsoft, Apple, Pandora, Yahoo, Ebay, Facebook, Amazon and others, only Netflix is complaining.
There is only one company that I can think of that could bring even more exposure to the interconnect topic than Netflix, to try to get the current business models changed, and that would be Apple. So far, they haven’t done that, have not complained to the media or to the public and are currently signing and negotiating paid interconnect deals with ISPs. So it’s just another example you have to look at when some make the statement that paid interconnect deals are bad for everyone involved. It seems that other than Netflix, those companies paying and signing these deals don’t seem to have a problem with them.
21-05-2014, 14:17 #2
Chart Shows Which Content Owners Have Direct Interconnect Deals With ISPsDan Rayburn | Wednesday May 21, 2014
With all the confusion in the media around the topic of interconnects, I thought it would be helpful to show exactly which content owners have done direct interconnects with ISPs. While some members of the media want to claim that content is exchanged on the Internet in a “secretive nature”, the fact is, it’s not a secret at all. If you know how and where to look, you can easily see what is taking place between networks. All of the information I provide in the chart below is from public information that anyone can lookup simply by knowing the networks Autonomous System number (AS) and looking at their peers. You don’t have to be a networking engineer to look at BGP relationships and see what is taking place. It’s all public info.
The search results won’t come right out and say which relationships are settlement free versus settlement-based, but you can infer from traditional business relationships and public peering policies, who’s paying whom. Not to mention, the companies who pay for these interconnect relationships will tell you which ISPs they are paying, and which one’s they aren’t, if you have a good relationship with them. Anyone involved in building out networks for a living understands how this has worked since the late 90′s. It is not a secret. Many members of the media simply don’t have any insight into these deals because many of them don’t cover infrastructure topics for a living. But just because they don’t know themselves, does not mean that these deals happen in secret.
Based on the AS lookups I did, here are the companies that have direct interconnects with various ISPs. I didn’t look at every ISP out there or every content owner, simply some of the larger ones.
What you can see on the chart is that all of the major content owners have negotiated direct business relationships with multiple ISPs for their CDN. There are various business and technical reasons why they pick one ISP over another, but there are a lot of these deals in the market and have been for many, many years. Some ISPs in the U.S. have dozens of paid interconnect deals in place. Multiply that by the number of ISPs in the U.S., and the number of paid interconnect deals is probably close to 100. So for Netflix’s CEO to call their paid interconnect deal with Comcast “unprecedented,” is intentionally misleading.
Each of the X’s in my chart shows that large content providers are unafraid to connect with large ISPs, most have been doing so longer then Netflix, and none have found the need to seek intervention. Of course that does not tell you the size of the deal, how much the ISP is charging or any of the other business terms, but no one should argue or debate that these deals aren’t common, continue to happen and are how networks, especially in the U.S., have connected to one another for almost twenty years.
Part of the argument by many around interconnects is that it costs the ISPs almost nothing to turn up ports on their network, or they say that ISPs have “unlimited capacity”. Neither would be accurate. There is a direct cost to the ISP to add ports and backhaul infrastructure to bring traffic all the way into their network. For anyone to say that the costs should now shift to the ISP because it’s “cheap” to them, or “doesn’t cost them much”, it’s not a valid argument. If that’s the argument some want to use, then by the same token I could say that if I have a letter to deliver to someone’s house and the postman is already going there, then it costs them next to nothing to deliver my letter and my cost should be free. It’s simply not a logical argument.
Put it another way. What if Google, Apple, Netflix, Facebook, Ebay and Microsoft all said to the same ISP that they want unlimited peering capacity, in multiple cities in the U.S. Would anyone think that would be fair to the ISP and that they should have to spend the money to fulfill all of those requests, at no cost to the content owner? Would this entitlement give major content owners special treatment over a smaller startup competitor? I think most would agree that’s not fair, yet that is what many are suggesting should happen. And using the argument that the ISP is already charging consumers for access to the Internet is also not valid. Your connection from your ISP is about broadband Internet. It is not about the service infrastructure cost for third parties.
Historically companies like Netflix have always paid for their transit connectivity, or Internet delivery costs. The 60% of broadband customers who do not use Netflix, should not suddenly have to pay for the network transit/interconnect costs of others because Netflix feels their Internet costs should now shift to all ISP customers. Perhaps someone wants to use Hulu, Amazon or consume no video at all. Each of these companies has a cost for their service infrastructure and paid interconnect deals are not increasing that cost. These costs are staying where they have always been, included within the specific service. Saying that it is new or unprecedented is just not true.