Just to play devils advocate here, but has anyone fully considered the impact on ISPs that this massive increase in demand is having on them. Especially the hated Cable companies who are seeing customers turn their backs on their own TV products whilst shouting and demanding more and more bandwidth so they can watch (and pay instead) NetFlix et al.
NetFlix and YouTube do not pay the ISPs anything. ISPs are not some cloud based company selling software, they are a utility company with Billions of $$ of investment in physical cables spanning every street in every country. In nearly every instance, the initial investment in this infrastructure was for a completely different technology (telephone or TV) from a time way before internet. The industry is now at a point where demand for bandwidth is simply outpacing technological innovation and these networks can't take the strain.
Think about the work involved in doubling Broadband speeds over say 2-3 years, in line with demand. In the past this involved installing some new equipment in the central office or exchange. Then this wasn't enough so the equipment had to be installed in the street, costly, especially as the number required was so much higher. Now we're at the point where the copper Coax or phone-line can't physically do the job anymore and it needs replacing, ideally with Fibre. Imagine the work and cost involved in doing this task. It's a hugely labour intensive task to dig up every street and there are many layers of red-tape and bureaucracy that simply weren't there during the initial build out. Once you've done that, you then need to buy vast amounts of brand new equipment and back-office software to run these new systems, plus hire in a load of new talent.
Now imagine doing this for every single home in the US or Europe (not just Kansas).
Now imagine doing this without increasing your prices.
Now imagine doing this whilst some key revenue streams are also being taken away from you.
Then think about the payback period on this investment... how long will all that Fibre even be used before wireless speeds increase and half your customer base decide they don't even want a fixed line anymore because they can get 1Gbps on their mobile? That 4K Video stream is still only 30mbps! 10 years perhaps?
I'm not surprised the ISPs are fighting back, they're putting in all the money and taking all of the risk for little to no reward.
Now... to speak my own mind. The sooner the old guard realise their time is up and adapt to fit the new demand the better, for customers and for the ISPs. Cable companies in particular have a very difficult time coming to terms with the fact that they're now just a data pipe. People don't want to buy their TV or their programming. People want internet and on the internet you can get everything, including TV and Movies. ISPs need to start investing in their networks even if it means we have to pay a bit more, because it won't come cheap, and stop pretending they're still relevant. But throttling the guy selling the better widget is just petty and benefits no-one.
The big difference is that each player is already paying for bandwidth at their end. Google, Netflix, myself (PortableApps.com), we all pay for bandwidth to host and serve videos, binary downloads, etc. And then the home user is paying on their end.
But, ISPs charge their customers for 'unlimited' bandwidth, a lie, and then have ghost caps, throttling, etc. The only person being dishonest here is the ISP. And having all the major content providers on-board with calling them out on throttling will keep them from doing it since it will 1. make their customers angry/consider switching and 2. cost them quite a bit in support hours. So, they'll be forced to switch to a more honest set of advertising, billing, and bandwidth counting based on individual usage.
Granted, we'll still have to put up with their monopolies, lack of common carrier rules (so your home VOIP phone is offline for weeks and you can't do anything about it), poor customer service, etc. Realistically, we'll need wireless to come up with better plans to force them to be competitive. Living in NYC and having only a single cable company as your option (unless you want incredibly slow DSL that was down 1 week of my 1 month trial) is kind of silly at this point.
> The big difference is that each player is already paying for bandwidth at their end
We are really only paying for bandwidth to/from our providers. Everything in-between is best effort, nebulous peering arrangements, etc.
> But, ISPs charge their customers for 'unlimited' bandwidth
I doubt you can find a single residential Internet provider who advertises unlimited bandwidth at this point. The marketing for 'unlimited' started to counter dial-up services like AOL who billed usage by connection duration not bandwidth usage.
> having all the major content providers on-board with calling them out on throttling will keep them from doing it
Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up. Your video is buffering because Netflix & Google want YOU to pay more for Internet service"
I doubt you can find a single residential Internet provider who advertises unlimited bandwidth at this point.
Nation-wide? Maybe. But there are plenty of ISPs that offer unlimited bandwidth as a differentiating feature. See sonic.net, which covers most of CA[1].
> Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up. Your video is buffering because Netflix & Google want YOU to pay more for Internet service"
Haha, good luck with that on increasingly HTTPS-only services.
"You need to configure your browser to trust our SSL certificate authority as part of your account creation and router setup process....oh by the way we are also going to use it to inspect your SSL connections."
>We are really only paying for bandwidth to/from our providers. Everything in-between is best effort, nebulous peering arrangements, etc.
I don't know about you, but I'm certainly not. I fully intend to be provided connectivity to the wider Internet at the speed advertised. The money I'm giving to them pays for the peering arrangements that let that happen. Let me put it another way, I wouldn't pay anything for a connection that's just to/from Comcast with no wider connectivity.
>Way more likely ISPs will do a little deep packet inspection and re-write the page to have a message along the lines of "Netflix & Google are the reason your rates keep going up.
Netflix and Google can just use HTTPS if that happens. I believe Google already does.
Interestingly, in Australia we have had Data Caps for a significant amount of time (2002 I believe was when Telstra introduced the 3gb Cable Cap).
I know it's not US based, but looking at ISP's and their offerings elsewhere, there are options available. For example I have an Unlimited ADSL2+ connection from TPG (www.tpg.com.au). The trend locally seems to be ISP's offering more data and more value for money (Unlimited in some cases, but 500GB / 1TB plans for marginal fees per month)
I'm also with TPG and can't fathom having a cap, much less the typically tiny ones offered. Recently I noticed that Optus is now offering an unlimited account for a reasonable price. I've heard anecdotally from people working there that it would be cheaper for Optus to do away with the caps because of the internal overhead that tracking and accounting requires.
Doubling broadband speeds every 2-3 years? if only they actually did that, i might have some sympathy for them. I get about 15Mbit. This is the same speed i have gotten for approximately the last 6 years. the only thing that is doubling every 2-3 years is the amount they bill me.
>NetFlix and YouTube do not pay the ISPs anything.
bullshit. You think google and netflix have teams of ninjas that go and splice wires into your ISPs lines? There are long standing agreements regarding how the exchange of data happens between Tier-1 providers, and how much it costs to send data to somebody else's network. Netflix is not getting a free ride. The consumer ISPs are trying to get paid twice.
Doubling the infrastructure required to deliver customer bandwidth demands -- not speeds. For example 2 years ago maybe the average customer had utilization between 8-11PM of 4Mbit/sec. Today it's maybe 8Mbit/sec. That requires your ISP to deliver 2x as much bandwidth during those hours regardless if the speed package has changed or not.
There was some similar discussion in Germany last year. The major ISP announced to throttle monthly traffic above 70 GB in the future. Funnily enough they told that 70 GB are enough for every user while telling everyone that Internet traffic has a high increase every year.
Fact is, that their backbones aren't on any limit at all. The real bottleneck of the Internet connection is actually the last mile, the subscriber line. And that cable is exclusive per customer, so no shared medium and throttling wouldn't improve anything than making it possible to earn more money by selling premium contracts. To be fair, this is different for Internet over the TV cable which is indeed a shared medium. Still, throttling on the basis of a monthly limit doesn't make sense since it is reset for the beginning of every month.
Oh, and also that ISP was the only one not interested in doing a direct peering with Google's local CDNs for free. In France Google is actually paying for getting YouTube better routed.
The important thing is that these ISPs just try to make money on both ends of the cable while actually arguing with the power of the opponents. "Hey enduser, here you'll get YouTube! - Hey Google, wanna have more satisfied users?" And that they are often actively lying for achieving that.
It's not (only) that 1Gbps to the home, it also that fat pipe per street, the even fatter pipe per neighborhood, etc.
Especially the latter pipes need to get a lot fatter, the more people use that 1Gbps bandwidth.
Look at it this way: roads to houses haven't increased in bandwidth in the past century or so, but investments in highways have. Why? People started using 'their' bandwidth. They don't walk to work, but use a car that takes more room, and also travel over larger distances.
Funny story about that 1gbps needing fiber. In fact, a DOCSIS 3.1 modem at 24x8 will easily hit 1-1.5gbps with half the investment necessary as it would to roll out today under DOCSIS 3.0. So expect to magically see cable start competing with Google Fiber if it actually takes off! ;-)
Any increase in average utilization is an insignificant cost compared to the elephant in the room: Broadband speeds have been essentially stagnant for 10+ years if you look at tech development vs what gets deployed. The ISPs have been reaping the savings that faster backbone technology brings and not passing them on to customers, while failing to deploy improvements in last mile connectivity tech.
Actually the tech would have gotten better much faster if its development hadn't stalled due to low demand. If you plot a Moore's law type from the 90's, we should be getting 10 gigabits at home by now.
The data caps are artificial scarcity to create even more profit or drive people to their own video providers.
Cities have decided to only allow 2 sets of cables to everyones house (and the FCC decided not to make them share between providers) so there is a defacto duopoly - competition is nearly non-existant.
Hence they know they can get away with double and triple dipping. Nexflix has to pay more - guess who they pass it on to?
Keep in mind that generating traffic is nowadays really easy.
For 50k$, you can buy 50 servers with 2x10Gb cards, and generate a constant 1 terabit per second of video streaming data. No monthly fee besides electric bill.
So the "content" provider offers the ISP to host 50 of its cache servers and now the ISP has 1 more terabit of data to distribute over its whole network.
NetFlix and YouTube do not pay the ISPs anything. ISPs are not some cloud based company selling software, they are a utility company with Billions of $$ of investment in physical cables spanning every street in every country. In nearly every instance, the initial investment in this infrastructure was for a completely different technology (telephone or TV) from a time way before internet. The industry is now at a point where demand for bandwidth is simply outpacing technological innovation and these networks can't take the strain.
Think about the work involved in doubling Broadband speeds over say 2-3 years, in line with demand. In the past this involved installing some new equipment in the central office or exchange. Then this wasn't enough so the equipment had to be installed in the street, costly, especially as the number required was so much higher. Now we're at the point where the copper Coax or phone-line can't physically do the job anymore and it needs replacing, ideally with Fibre. Imagine the work and cost involved in doing this task. It's a hugely labour intensive task to dig up every street and there are many layers of red-tape and bureaucracy that simply weren't there during the initial build out. Once you've done that, you then need to buy vast amounts of brand new equipment and back-office software to run these new systems, plus hire in a load of new talent.
Now imagine doing this for every single home in the US or Europe (not just Kansas). Now imagine doing this without increasing your prices. Now imagine doing this whilst some key revenue streams are also being taken away from you.
Then think about the payback period on this investment... how long will all that Fibre even be used before wireless speeds increase and half your customer base decide they don't even want a fixed line anymore because they can get 1Gbps on their mobile? That 4K Video stream is still only 30mbps! 10 years perhaps?
I'm not surprised the ISPs are fighting back, they're putting in all the money and taking all of the risk for little to no reward.
Now... to speak my own mind. The sooner the old guard realise their time is up and adapt to fit the new demand the better, for customers and for the ISPs. Cable companies in particular have a very difficult time coming to terms with the fact that they're now just a data pipe. People don't want to buy their TV or their programming. People want internet and on the internet you can get everything, including TV and Movies. ISPs need to start investing in their networks even if it means we have to pay a bit more, because it won't come cheap, and stop pretending they're still relevant. But throttling the guy selling the better widget is just petty and benefits no-one.