Calculating Total Network Capacity 48
New submitter slashbill writes "MIT's working on a way to measure network capacity. Seems no one really knows how much data their network can handle. Makes you wonder about how then do you calculate expense when building out capacity? From the article: 'Recently, one of the most intriguing developments in information theory has been a different kind of coding, called network coding, in which the question is how to encode information in order to maximize the capacity of a network as a whole. For information theorists, it was natural to ask how these two types of coding might be combined: If you want to both minimize error and maximize capacity, which kind of coding do you apply where, and when do you do the decoding?'"
This is a synopsis of the first of two papers on the topic.
Hard to reduce complexity (Score:5, Interesting)
Re: (Score:3)
Don't forget that the type of traffic that is passed over the net also is a factor involved.
It's possible to have 1000 users on a 10Mbps network if the only traffic they have is text terminal traffic but you can completely saturate a Gbps network with a few users doing processing of video streams.
I recall an experiment to do just that (Score:4, Funny)
It was a program by one Robert Tappan Morris, as I recall.
That didn't go over so well with everyone.
Re: (Score:3)
Morris only experimented over TCPIP on unix systems running finger. He bypassed/ignored X.25 and other non TCPIP networks.
Re: (Score:2)
It was a program by one Robert Tappan Morris, as I recall.
True, but he programmed it badly, and used active injection into the network to measure it, rather than programming each node to passively collect data and make decisions based on the results of that. In short, he was young and stupid.
Plot traffic, establish a norm, compare history (Score:1, Insightful)
Best way I've found to measure growth is to have a running history of traffic on each router. You don't need a $billion to do it. There are some decent enough FOSS tools out there to do it. MRTG [oetiker.ch] or Cacti [cacti.net] will work nicely and integrate with SNMP.
For a smaller network, you could run a span port and graph your own data with a shell script, or hook up NTOP [ntop.org]. which will give you real-time views of traffic but you would need to implement something to save those reports daily.
Re:Plot traffic, establish a norm, compare history (Score:5, Insightful)
I think they're trying to do something a bit more detailed and theoretical than seeing how much traffic is going through a given interface...
Re:Plot traffic, establish a norm, compare history (Score:5, Insightful)
Best way I've found to measure growth is to have a running history of traffic on each router. You don't need a $billion to do it. There are some decent enough FOSS tools out there to do it. MRTG [oetiker.ch] or Cacti [cacti.net] will work nicely and integrate with SNMP.
For a smaller network, you could run a span port and graph your own data with a shell script, or hook up NTOP [ntop.org]. which will give you real-time views of traffic but you would need to implement something to save those reports daily.
You suggest some good tools, but they primarily measure network utilization rather than capacity. The question isn't "how much data is my network handling now" but "how much data could my network handle at peak"?
Re:Plot traffic, establish a norm, compare history (Score:4, Funny)
Hook up a BitTorrent seedbox to the live Internet. You'll find out the maximum capacity pretty quickly.
Re: (Score:1)
Running a few seed boxes with torrents of "all-teh-Pr0n-ever-made", "Call-of-Duty-2-leak", Cracked-MS-Office", "Windows-8-preview", "Avengers" and "bullshit-singing-star-new-album" could get it done.
And one of every linux distro.
Re: (Score:2)
Just use the top 300+ most active torrents, and remove the clients internal limitations to make sure it does not choke itself. Could be fun.
Re: (Score:2)
I like that.
Re: (Score:2)
The question isn't "how much data is my network handling now" but" how much data could my network handle at peak"?
Just insult Anonymous [slashdot.org] and you should have your answer shortly.
Re: (Score:1)
Why read your own article? (Score:1)
From TFS: "Makes you wonder about how then do you calculate expense when building out capacity?"
They're not talking about "not knowing" the capacity of a given network like (e.g. what you buy in the pipe from the datacenter to your ISP. They're talking about the overall bandwidth between 2 points across all possible routings. It's the difference between knowing Ohm's law and computing the net resistance between 2 adjacent nodes on an infinite grid of 1 ohm resistors.
Re:Why read your own article? (Score:5, Funny)
... computing the net resistance between 2 adjacent nodes on an infinite grid of 1 ohm resistors.
You're just trying to get someone to post a link to the XKCD comic about nerdsniping [xkcd.com]. Well, it won't work.
Wait a minute... (Score:1)
Seems no one really knows how much data their network can handle
Doesn't that shoot a hole in the ISP's anti-bittorrent arguments?
Re: (Score:2)
While I think the "anti-bittorrent" argument is BS (In most cases)
To answer your question, no, ISP are claiming bittorrenting is "already" overloading the network. This article deals with predicting at what point that will happen.
Isn't this the same dilemmma? (Score:2)
You reduce network capacity, but now your routers need to be smarter, so they're taking longer to encode-decode or you're spending more on hardware to keep throughput the same.
Re: (Score:2)
Most of this work will end up in MIMO radios. It's not horribly applicable to wired networks, at least not with currently in-use technologies and routing protocols. (Almost all wired connections pass through a stateful firewall or two, and just even trying to load balance can cause issues with OOP processing.) Wired networks are more base theory fodder rather than serious proposals. (How these strategies and quantum optical cryptography might work together is interesting food for thought.)
Limiting factor (Score:1)
Re: (Score:3, Insightful)
By channel you mean "a network of noisy, independent, memoryless point-to-point channels"? The result in the paper says that
such channel can be seen as a network of error free channels. On such network it is already known that network coding delivers
a better performance than routing alone. (see the butterfly network example in https://en.wikipedia.org/wiki/Network_coding)
Re: (Score:1)
Re: (Score:2)
In the end, you want to eek out as much performance as possible, given the constraints of the law of diminishing returns verses increasing (or decreasing, depending) the limits. That last 0.0001% increase in performance may not be worth the effort needed to achieve it.
Theory is not the same as practice, but people often treat them the same.
antitrust (Score:1)
Applications.. (Score:3)
It sounds like they are studying the effect of having intelligent nodes in a network that not just forwards a packet, but also performs error correction, has some basic path intelligence, and sends the packet out multiple interfaces. The end node then receives these hybrid packets from different directions, some coming faster, some later, developing a map with the most efficient path.
One could argue that this could be used, for example, in a mesh MPLS cloud when a path through a specific hop (i.e. office) may be more efficient, because of network conditions, than going straight to the end node. However, this would require each node to have enough bandwidth to support the added traffic, over and above the normal location traffic. Which means requiring a larger budget for bandwidth that is only used in certain degraded conditions.
Basically, it's a study of the Internet and, in my opinion, would have little application in a corporate LAN. The reason why I say this is because a Corporate LAN is more deterministic in path selection and is limited by cost.
Re: (Score:2)
>
Basically, it's a study of the Internet and, in my opinion, would have little application in a corporate LAN. The reason why I say this is because a Corporate LAN is more deterministic in path selection and is limited by cost.
Meant to say WAN non LAN for the last sentence...
Re: (Score:3, Interesting)
It sounds like they are studying the effect of having intelligent nodes in a network that not just forwards a packet, but also performs error correction, has some basic path intelligence, and sends the packet out multiple interfaces. The end node then receives these hybrid packets from different directions, some coming faster, some later, developing a map with the most efficient path.
The eternal wheel of IT endlessly rotates old ideas into newness. Interpret that as either my mostly new source route bridged SDLC mainframe network in the early 90s or my decaying decrepit X.25 network in the late 90s. I played with some stuff like that using AX.25 as the phy layer around 1990. We had tools and papers and equations back then to analyze.
Did you know you can make networks like that oscillate if you're not careful? We also collapsed a few accidentally by packet flooding beyond a certain h
I did my master work in Network Capacity Planning (Score:5, Interesting)
and the answer is "It Depends". The traffic, the routing, the overall bandwidth (you never get 100% usage) all have factors. The easiest way is to look at your pipes (each segment is separate) and see the error rates, back pressure (QOS, Ethernet, etc.), average throughput breakdown (types of traffic), and usage percentage. This will give you a clear picture. Take those numbers and watch them over time, and you will get a clear picture of your network.
You cannot answer a question such as this truthfully if you take one sample size, and assume that is fact. Many sample sizes make the true picture, and then you can also see trends to determine if things are getting out of control.
10 types of people understand binary... (Score:1)
...how to encode information in order to maximize the capacity of a network as a whole...
I always send my data as a series of 0s and 1s. I tried using 2s, but they took up too much bandwidth.
Proofread summary next time please. (Score:2)
"MIT's working on a way to measure network capacity. Seems no one really knows how much data their network can handle. Makes you wonder about how then do you calculate expense when building out capacity? From the article: 'Recently, one of the most intriguing developments in information theory has been a different kind of coding,
Different from what? Compared to what, and on what context? The sentences preceeding that remark do not make any reference to any coding scheme whatsoever.
called network coding, in which the question is how to encode information in order to maximize the capacity of a network as a whole. For information theorists, it was natural to ask how these two types of coding might be combined: If you want to both minimize error and maximize capacity, which kind of coding do you apply where, and when do you do the decoding?'"
Two? Which is the other? There only mention (in the summary) of the newly proposed coding.
YES, I can infer that, for the most part (and then confirmed from reading the article) that the other coding the summary refers to is error-correcting coding. But it shouldn't be necessary to neither rely on prior knowled
Re: (Score:2)
Just remember to proof read your summaries.
This is, after all, extremelly important.
... which implies you did not comprehend the summary, nor the article. This is about testing capacity and is more along the lines of implementing something to expound upon dynamic routing. Which induces a clusterfuck of brainthink along the lines of "so we're buying bandwidth to supplement bandwidth that we should have put there, but maybe here" and so on and so forth. There's not a whole lot more to read into
Seriously though, it's not "error-correcting coding"
Re: (Score:2)
But I think my point still stands - the summary is sufficient for the purposes of summarizing the article.
Re: (Score:2)
Okay I'm just going ahead and apologize for not noticing your "summary refers to" bit in your paragraph. But I think my point still stands - the summary is sufficient for the purposes of summarizing the article.
How can it be sufficient when it refers to two different codes while only mentioning one? Just because you say your point stands does not magically makes it so. The only way to ensure that the summary is in tandem with the article is to read the article. And that defeats the purpose of a summary. If the summary refers to two different coding schemes, but only mentions one by name, then the question follows: is the presence of another coding scheme different from network coding relevant to the model presente
Re: (Score:2)
A long, well-written statement abstractly disagreeing with mine does not make a fact.
is the presence of another coding scheme different from network coding relevant to the model presented by the article?
No, it really isn't
Why would you want to know that ? (Score:1)
Here's a free hint... (Score:2)
That ideal encoding method isn't XML.
Re: (Score:2)
Actually, XML might be the ideal method of encoding. It depends on what you mean by "ideal" ;) XML makes it much easier for a human to decode than say a bit stream of binary coded data. Just saying, that without further context, your statement may not be entirely accurate, or it could be perfectly accurate.
Which in my estimation makes it not accurate at all.
So... (Score:1)
Trying to put OPNET out of business? (Score:1)
OPNET- great tools, horrific price...
Dave
Re: (Score:2)
OPNET? BAH! Way over priced for what it is. Try PathView by AppNeta. Better tool, lower cost.
Step one (Score:2)
Release Diablo III...