Rethinking the Internet

Many of us subscribe to the Internet religion, and rightfully so. If it’s not the most disruptive technology to date, it definitely has a place towards the top of the list. So it is with all due respect that I say it may be time for some changes.

For all intents and purposes, the Internet is a unicast (one to one) environment. Which brings me to my first grievance… it’s missing true multicast. Yes, some ISPs support multicast and the MBONE project is in existence, but neither really solve the problem. I’m certainly not the first one to touch on this, but it’s worth repeating because it is currently the most obvious and important deficiency of the Internet. It can be a serious drain on bandwidth and creates awkward architecture decisions for both large and small applications.

This has presented a particularly difficult problem for video distributors. It’s no secret that the major television networks are having a difficult time monetizing video distribution on the internet. Cost is at least partly to blame. Up until recently, television broadcasts were just that… broadcasts. As I understand it, cable or satellite companies send out one signal that that everyone taps into. Not one signal for every subscriber. Paying for an infrastructure for all subscribers once has a very different value proposition than having to pay for a scalable infrastructure with greater incremental costs for adding new subscribers.

My next grievance is a protocol related one. HTTP is a great technology, but it is intended to support polling for a single user. You open a TCP/IP connection, send in an HTTP request for specific information, receive the information, and close the connection. This was great when the web was a group of html documents, images, and other assorted downloads. Times have changed and we have more interaction on our web sites and web applications. It’s time to adopt new protocols and stop using existing ones inappropriately.

There are two good examples of this. I’ll start with Soap / Web Services. The idea of SOA (Service Oriented Architecture) on the Internet is great. To have access to a variety of free and commercial services on a public network presents many great possibilities and encourages innovation. Unfortunately the reality is less satisfying than the dream. SOAP, aside from being bulky as described here, suffers from HTTP limitations. This means that without really hacking it, you are stuck with a system that at it’s best can do remote method calls, but at it’s worst is required to poll for updates. For some applications this is ok, but for many applications realtime interchange of data would decrease latency for users and simultaneously decrease load on machines that previously had to handle polling.

RSS suffers from similar problems. In this case I feel XML is less of an issue due it’s purpose, but polling remains a problem. It’s time for a standard publish / subscribe protocol. RSS is not the only technology that could benefit from this type of protocol, but it makes for a compelling case. Let’s pretend for a moment this protocol exists and RSS has it’s basis in it. You immediately reduce all the issues of RSS readers polling often and downloading stale data. From a user perspective you see new entries more immediately. Let’s pretend one more time and say multicast is widely available. Feeds could then be broadcast which would also save bandwidth. So we’ve gone from every user downloading the same stale data over and over again to a system where one broadcast is made to active interested parties and offline parties can sync when they come online. In my eyes this seems like a huge improvement.

Ok, now for the coup de grâce. BitTorrent. If you were to tap any Internet backbone, it’s safe to say you’d see a lot of BitTorrent traffic. Here’s where it gets interesting. I think you can apply the changes above to increase BitTorrent performance. What if each BitTorrent client broadcast packets instead of sending them multiple times to different parties. I’m sure with some algorithm finagling, you could see some serious improvements in download times and bandwidth utilization. The hardest part would be bridging the gap between different sized connections, but this could surely be solved by partitioning data broadcasts in similar ways to what BitTorrent does now. With the standard publish / subscribe protocol there would be an obvious technology to manage the packet distribution and reduce the barrier for application developers to create new clients. Who knows, maybe with some commercial modifications this could even be a way to distribute videos in a cost effective way.

Technorati Tags: , , , , , , , , , ,