OPINON: Web 2.slow

By Paul Sweeting -- Video Business, 11/21/2007

The report, which is available on the Nemertes Web site [free registration required], paints a sobering picture of a “coming bandwidth crunch” that could stifle the innovation needed for content owners to effectively monetize their content online.

Though other researchers have plowed this field before, both predicting and debunking fears of an Internet meltdown, the Nemertes team took a different approach to projecting the demand for bandwidth, which, if you buy it, leads to a forecast of much higher theoretical traffic loads in the fairly near term than other studies have predicted.

Critically, the report also looks at the supply of bandwidth in finer detail than other studies, concluding that, while Internet backbone infrastructure can easily handle any volume likely to be thrown at it in the foreseeable future, broadband access capacity at the ISP level is likely to remain severely constrained, making the copious backbone capacity effectively moot.

FIRST, THE methodology: Rather than looking at the volume of traffic being generated by the current portfolio of applications being used by Internet users and simply extrapolating those loads into the future, Nemertes approached the problem of projecting future traffic from a perspective similar to that behind Moore’s Law.

In 1965, Intel co-founder Gordon Moore observed that the number of transistors that could be packed cost-effectively onto a single integrated circuit—a correlate for the processing power of the chip—roughly doubled every two years.

The key to Moore’s law was that it recognized the exponential nature of the growth curve for processing power, even if it couldn’t specify the particular innovations in chip design and manufacturing necessary to achieve those improvements.

All that was needed to confirm it was to verify that it was actually occurring, which Moore did by testing it against the best historical data then available. His predicted ratio has held ever since.

In estimating future Internet traffic, Nemertes examined the historical rate at which Internet users have utilized the total available Internet capacity using the devices and applications then available to them, rather than simply at the output of those applications. It then calculated the rate of change in that ratio over time.

Assuming a Moore’s Law-like pace of innovation in those applications and devices and applying it to projected utilization ratios yielded a theoretical growth curve for Internet traffic similar to the exponential increase in processing power observed by Moore. The result is projected traffic volume far higher than predicted by other models.

(The full methodology, including validation testing of the model, is detailed in the report and well worth reading.)

NOW THE crunch: Nemertes stresses that it has no expectation that its projected level of traffic will actually come about. Not because the Internet as a whole couldn’t handle it, but because the recent and expected growth in Internet access capacity (the “last-mile” bandwidth controlled by commercial ISPs) has been orders of magnitude lower than the growth in overall Internet capacity.

According to the report, “chronic under-investment” in last-mile infrastructure by ISPs means that, “Internet access infrastructure, particularly in North America, will likely cease to be adequate for supporting demand within the next three to five years.”

The problem is particularly severe, the report says, in cases where multiple users share a single circuit, as with cable broadband.

Closing the gap between access capacity and the total theoretical Internet capacity, the report estimates, would require an investment by ISPs over five years of between $42 billion and $55 billion, roughly 60% greater than service providers are currently planning.

If Nemertes is right, the implications for content owners are significant. Though the report does not predict that the Internet itself will collapse—there is plenty of spare capacity throughout the system except at the ISP level—it suggests that the theoretical demand for the kind of new innovative services that drive traffic growth will not be fulfilled because consumers will not be able to utilize the available bandwidth.

“The next Google, YouTube or Amazon might not arise, not because of a lack of demand, but due to an inability to fulfill that demand,” the report concludes. “Rather like osteoporosis, the underinvestment in infrastructure will painlessly and invisibly leach competitiveness out of the economy.”

Paul Sweeting is editor of Content Agenda. Get more of Sweeting's analysis here.