Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Calculating Total Network Capacity

Unknown Lamer posted more than 2 years ago | from the information-theory-at-work dept.

Network 48

New submitter slashbill writes "MIT's working on a way to measure network capacity. Seems no one really knows how much data their network can handle. Makes you wonder about how then do you calculate expense when building out capacity? From the article: 'Recently, one of the most intriguing developments in information theory has been a different kind of coding, called network coding, in which the question is how to encode information in order to maximize the capacity of a network as a whole. For information theorists, it was natural to ask how these two types of coding might be combined: If you want to both minimize error and maximize capacity, which kind of coding do you apply where, and when do you do the decoding?'" This is a synopsis of the first of two papers on the topic.

cancel ×

48 comments

Sorry! There are no comments related to the filter you selected.

Hard to reduce complexity (4, Interesting)

sideslash (1865434) | more than 2 years ago | (#40017175)

Didn't read the article, but I imagine that part of the difficulty is that network capacity isn't reducible to an individual scalar number, but rather looks like an N-dimensional graph. There are many points of failure and bottleneck depending on how each node behaves relative to other nodes.

Re:Hard to reduce complexity (2)

Z00L00K (682162) | more than 2 years ago | (#40017431)

Don't forget that the type of traffic that is passed over the net also is a factor involved.

It's possible to have 1000 users on a 10Mbps network if the only traffic they have is text terminal traffic but you can completely saturate a Gbps network with a few users doing processing of video streams.

Re:Hard to reduce complexity (0)

Anonymous Coward | more than 2 years ago | (#40020371)

Don't forget the other variables that do affect lines like hot and cold, humidity in the lines obtained by improperly sealed connectors, difference between one copper mined in Colorado and another mined in India. There is a huge amount of variables to consider if someone wanted a perfect precise model, but we are humans and okay is high grade quality anymore.

I recall an experiment to do just that (3, Funny)

i kan reed (749298) | more than 2 years ago | (#40017195)

It was a program by one Robert Tappan Morris, as I recall.

That didn't go over so well with everyone.

Re:I recall an experiment to do just that (2)

stiggle (649614) | more than 2 years ago | (#40017261)

Morris only experimented over TCPIP on unix systems running finger. He bypassed/ignored X.25 and other non TCPIP networks.

Re:I recall an experiment to do just that (1)

girlintraining (1395911) | more than 2 years ago | (#40018499)

It was a program by one Robert Tappan Morris, as I recall.

True, but he programmed it badly, and used active injection into the network to measure it, rather than programming each node to passively collect data and make decisions based on the results of that. In short, he was young and stupid.

Plot traffic, establish a norm, compare history (1, Insightful)

sl4shd0rk (755837) | more than 2 years ago | (#40017223)

Best way I've found to measure growth is to have a running history of traffic on each router. You don't need a $billion to do it. There are some decent enough FOSS tools out there to do it. MRTG [oetiker.ch] or Cacti [cacti.net] will work nicely and integrate with SNMP.

For a smaller network, you could run a span port and graph your own data with a shell script, or hook up NTOP [ntop.org] . which will give you real-time views of traffic but you would need to implement something to save those reports daily.

Re:Plot traffic, establish a norm, compare history (5, Insightful)

ender- (42944) | more than 2 years ago | (#40017257)

I think they're trying to do something a bit more detailed and theoretical than seeing how much traffic is going through a given interface...

Re:Plot traffic, establish a norm, compare history (5, Insightful)

bengoerz (581218) | more than 2 years ago | (#40017399)

Best way I've found to measure growth is to have a running history of traffic on each router. You don't need a $billion to do it. There are some decent enough FOSS tools out there to do it. MRTG [oetiker.ch] or Cacti [cacti.net] will work nicely and integrate with SNMP.

For a smaller network, you could run a span port and graph your own data with a shell script, or hook up NTOP [ntop.org] . which will give you real-time views of traffic but you would need to implement something to save those reports daily.

You suggest some good tools, but they primarily measure network utilization rather than capacity. The question isn't "how much data is my network handling now" but "how much data could my network handle at peak"?

Re:Plot traffic, establish a norm, compare history (3, Funny)

Ihmhi (1206036) | more than 2 years ago | (#40017711)

Hook up a BitTorrent seedbox to the live Internet. You'll find out the maximum capacity pretty quickly.

Re:Plot traffic, establish a norm, compare history (0)

Anonymous Coward | more than 2 years ago | (#40017947)

Max capacity of that node's slowest part. I'm fairly confident there are very few seedboxes capable of saturating a modern 10Ge or 40Ge network connection.

Re:Plot traffic, establish a norm, compare history (1)

Dishevel (1105119) | more than 2 years ago | (#40018191)

Running a few seed boxes with torrents of "all-teh-Pr0n-ever-made", "Call-of-Duty-2-leak", Cracked-MS-Office", "Windows-8-preview", "Avengers" and "bullshit-singing-star-new-album" could get it done.
And one of every linux distro.

Re:Plot traffic, establish a norm, compare history (1)

del_diablo (1747634) | more than 2 years ago | (#40018417)

Just use the top 300+ most active torrents, and remove the clients internal limitations to make sure it does not choke itself. Could be fun.

Re:Plot traffic, establish a norm, compare history (1)

daremonai (859175) | more than 2 years ago | (#40021037)

You have rather a sick definition of fun.

I like that.

Re:Plot traffic, establish a norm, compare history (1)

daremonai (859175) | more than 2 years ago | (#40020979)

The question isn't "how much data is my network handling now" but" how much data could my network handle at peak"?

Just insult Anonymous [slashdot.org] and you should have your answer shortly.

Re:Plot traffic, establish a norm, compare history (1)

Yokoshima (637029) | more than 2 years ago | (#40022601)

It's not "how much data could my network handle at peak" but rather "what is the maximum amount of information I can send through a network" where this 'maximum' is usually only attainable in ideal conditions. The summary is somewhat misleading since "capacity" relates to "information", in the information theory sense and "data" relates to the idea most posters seem to have (i.e. the kbps you download at). Also the arxiv.org paper is two years old but the work in Network Coding is very interesting and has been a very rapidly increasing field in the past years. I'm not from Caltech or M.I.T. but Medard is awesome and so is his work, just saying.

Why read your own article? (1)

Anonymous Coward | more than 2 years ago | (#40017311)

From TFS: "Makes you wonder about how then do you calculate expense when building out capacity?"

They're not talking about "not knowing" the capacity of a given network like (e.g. what you buy in the pipe from the datacenter to your ISP. They're talking about the overall bandwidth between 2 points across all possible routings. It's the difference between knowing Ohm's law and computing the net resistance between 2 adjacent nodes on an infinite grid of 1 ohm resistors.

Re:Why read your own article? (4, Funny)

camperdave (969942) | more than 2 years ago | (#40018039)

... computing the net resistance between 2 adjacent nodes on an infinite grid of 1 ohm resistors.

You're just trying to get someone to post a link to the XKCD comic about nerdsniping [xkcd.com] . Well, it won't work.

Wait a minute... (1)

koan (80826) | more than 2 years ago | (#40017471)

Seems no one really knows how much data their network can handle

Doesn't that shoot a hole in the ISP's anti-bittorrent arguments?

Re:Wait a minute... (1)

crashumbc (1221174) | more than 2 years ago | (#40017605)

While I think the "anti-bittorrent" argument is BS (In most cases)

To answer your question, no, ISP are claiming bittorrenting is "already" overloading the network. This article deals with predicting at what point that will happen.

Isn't this the same dilemmma? (1)

GodfatherofSoul (174979) | more than 2 years ago | (#40017535)

You reduce network capacity, but now your routers need to be smarter, so they're taking longer to encode-decode or you're spending more on hardware to keep throughput the same.

Re:Isn't this the same dilemmma? (1)

skids (119237) | more than 2 years ago | (#40020107)

Most of this work will end up in MIMO radios. It's not horribly applicable to wired networks, at least not with currently in-use technologies and routing protocols. (Almost all wired connections pass through a stateful firewall or two, and just even trying to load balance can cause issues with OOP processing.) Wired networks are more base theory fodder rather than serious proposals. (How these strategies and quantum optical cryptography might work together is interesting food for thought.)

Limiting factor (1)

tfcollins (2640835) | more than 2 years ago | (#40017651)

I dont really think coding is the real bottleneck breaker for getting to the Shannon limit of these channels. I believe a better routing mechanism would be a better approach.

Re:Limiting factor (3, Insightful)

Anonymous Coward | more than 2 years ago | (#40017849)

By channel you mean "a network of noisy, independent, memoryless point-to-point channels"? The result in the paper says that
such channel can be seen as a network of error free channels. On such network it is already known that network coding delivers
a better performance than routing alone. (see the butterfly network example in https://en.wikipedia.org/wiki/Network_coding)

Re:Limiting factor (1)

tfcollins (2640835) | more than 2 years ago | (#40018733)

This assumption is purely for academic, for in reality the networking overhead to make this assumption true is significant. The reduction of such overhead I feel is more interesting and practical. But in the end you want to eek out as much performance out of everything as you can.

Re:Limiting factor (1)

Archangel Michael (180766) | more than 2 years ago | (#40019437)

In the end, you want to eek out as much performance as possible, given the constraints of the law of diminishing returns verses increasing (or decreasing, depending) the limits. That last 0.0001% increase in performance may not be worth the effort needed to achieve it.

Theory is not the same as practice, but people often treat them the same.

antitrust (1)

Snipes420 (689240) | more than 2 years ago | (#40017673)

Its not in the box.... Its in the band.

Applications.. (2)

David_Hart (1184661) | more than 2 years ago | (#40017703)

It sounds like they are studying the effect of having intelligent nodes in a network that not just forwards a packet, but also performs error correction, has some basic path intelligence, and sends the packet out multiple interfaces. The end node then receives these hybrid packets from different directions, some coming faster, some later, developing a map with the most efficient path.

One could argue that this could be used, for example, in a mesh MPLS cloud when a path through a specific hop (i.e. office) may be more efficient, because of network conditions, than going straight to the end node. However, this would require each node to have enough bandwidth to support the added traffic, over and above the normal location traffic. Which means requiring a larger budget for bandwidth that is only used in certain degraded conditions.

Basically, it's a study of the Internet and, in my opinion, would have little application in a corporate LAN. The reason why I say this is because a Corporate LAN is more deterministic in path selection and is limited by cost.

Re:Applications.. (1)

David_Hart (1184661) | more than 2 years ago | (#40017723)

>

Basically, it's a study of the Internet and, in my opinion, would have little application in a corporate LAN. The reason why I say this is because a Corporate LAN is more deterministic in path selection and is limited by cost.

Meant to say WAN non LAN for the last sentence...

Re:Applications.. (0)

Anonymous Coward | more than 2 years ago | (#40018189)

Meant to say not not non for the last sentence. Posting anonymously to protect my karma from being modded down as offtopic or redundant. I just can't stop reading my own posts and replying to them. Reading my own post several times after posting it makes me feel very smart and proud of myself.

Re:Applications.. (0)

Anonymous Coward | more than 2 years ago | (#40019375)

When in reality you're a dumbass who doesn't take the 30 seconds to carefully review his post before submitting?

Re:Applications.. (2, Interesting)

vlm (69642) | more than 2 years ago | (#40018201)

It sounds like they are studying the effect of having intelligent nodes in a network that not just forwards a packet, but also performs error correction, has some basic path intelligence, and sends the packet out multiple interfaces. The end node then receives these hybrid packets from different directions, some coming faster, some later, developing a map with the most efficient path.

The eternal wheel of IT endlessly rotates old ideas into newness. Interpret that as either my mostly new source route bridged SDLC mainframe network in the early 90s or my decaying decrepit X.25 network in the late 90s. I played with some stuff like that using AX.25 as the phy layer around 1990. We had tools and papers and equations back then to analyze.

Did you know you can make networks like that oscillate if you're not careful? We also collapsed a few accidentally by packet flooding beyond a certain hard threshold. Also the behavior of an intelligent network of identical clones is easy to predict and damn near impossible if some of the clones are different in even the slightest way, not just different mfgr but even minor software revisions causing different bugs, but even improvements making some faster than others or removing bugs from some devices can cause collapse. Another fun one is trying to give a holographic picture of the routing to all nodes ends up in 90s era OSPF collapse when there's just too dang much in one area for it to ever converge. Another fun situation is when processing the routing data takes longer than sending the routing data (more or less similar to the previously mentioned OSPF collapse scenario)

Everything old, is new again... Personally I'm waiting for something like 3725 communication controllers to come back in vogue, kinda like akamai or anycast root DNS servers but did distributed back end also, not just front end caching.... a cool and weird tech.

I did my master work in Network Capacity Planning (5, Interesting)

mordred99 (895063) | more than 2 years ago | (#40017753)

and the answer is "It Depends". The traffic, the routing, the overall bandwidth (you never get 100% usage) all have factors. The easiest way is to look at your pipes (each segment is separate) and see the error rates, back pressure (QOS, Ethernet, etc.), average throughput breakdown (types of traffic), and usage percentage. This will give you a clear picture. Take those numbers and watch them over time, and you will get a clear picture of your network.

You cannot answer a question such as this truthfully if you take one sample size, and assume that is fact. Many sample sizes make the true picture, and then you can also see trends to determine if things are getting out of control.

10 types of people understand binary... (0)

fahrbot-bot (874524) | more than 2 years ago | (#40017975)

...how to encode information in order to maximize the capacity of a network as a whole...

I always send my data as a series of 0s and 1s. I tried using 2s, but they took up too much bandwidth.

Proofread summary next time please. (1)

luis_a_espinal (1810296) | more than 2 years ago | (#40018293)

People have to learn how to write a summary.

"MIT's working on a way to measure network capacity. Seems no one really knows how much data their network can handle. Makes you wonder about how then do you calculate expense when building out capacity? From the article: 'Recently, one of the most intriguing developments in information theory has been a different kind of coding,

Different from what? Compared to what, and on what context? The sentences preceeding that remark do not make any reference to any coding scheme whatsoever.

called network coding, in which the question is how to encode information in order to maximize the capacity of a network as a whole. For information theorists, it was natural to ask how these two types of coding might be combined: If you want to both minimize error and maximize capacity, which kind of coding do you apply where, and when do you do the decoding?'"

Two? Which is the other? There only mention (in the summary) of the newly proposed coding.

YES, I can infer that, for the most part (and then confirmed from reading the article) that the other coding the summary refers to is error-correcting coding. But it shouldn't be necessary to neither rely on prior knowledge (which varies from person to person) nor read TFA to find EXTREMELLY IMPORTANT information that is missing from a summary. A summary cannot leave crucial pieces out. Moreover, there are other types of coding related to information transmission (compression coding for instance) so at worst (and without having to rely on TFA), the summary is not just incomplete, but ambiguous.

I'm not trying to be mean or being a gramm3r nazi. Just remember to proof read your summaries.

Re:Proofread summary next time please. (1)

Caratted (806506) | more than 2 years ago | (#40021105)

Just remember to proof read your summaries.

This is, after all, extremelly important.

Seriously though, it's not "error-correcting coding" ... which implies you did not comprehend the summary, nor the article. This is about testing capacity and is more along the lines of implementing something to expound upon dynamic routing. Which induces a clusterfuck of brainthink along the lines of "so we're buying bandwidth to supplement bandwidth that we should have put there, but maybe here" and so on and so forth. There's not a whole lot more to read into except for hypotheticals (being revisited).

Re:Proofread summary next time please. (1)

Caratted (806506) | more than 2 years ago | (#40021245)

Okay I'm just going ahead and apologize for not noticing your "summary refers to" bit in your paragraph.

But I think my point still stands - the summary is sufficient for the purposes of summarizing the article.

Re:Proofread summary next time please. (1)

luis_a_espinal (1810296) | more than 2 years ago | (#40041617)

Okay I'm just going ahead and apologize for not noticing your "summary refers to" bit in your paragraph. But I think my point still stands - the summary is sufficient for the purposes of summarizing the article.

How can it be sufficient when it refers to two different codes while only mentioning one? Just because you say your point stands does not magically makes it so. The only way to ensure that the summary is in tandem with the article is to read the article. And that defeats the purpose of a summary. If the summary refers to two different coding schemes, but only mentions one by name, then the question follows: is the presence of another coding scheme different from network coding relevant to the model presented by the article? If so, why is the other coding's name ommitted? If it is not relevant, then it is superfluous information that has no place in a summary (but could have a place in the article in place, in a "prior research/history/intro" section, for instance.)

Summaries have very specific purposes, and they need to be written in a very specific way. And as a result, they need proofreading, perhaps more so than the article itself.

Re:Proofread summary next time please. (1)

Caratted (806506) | more than 2 years ago | (#40044231)

The article is really only about the one of them. The summary can refer to both of them and only explain the one it is about, while still being sufficient. It really is kind of pedentic, IMO. If you want to know more about the error-correcting code (that the article is not about), you will click the article and discover the one-liner dedicated to explaning why this article is not about it, regardless.

A long, well-written statement abstractly disagreeing with mine does not make a fact.

is the presence of another coding scheme different from network coding relevant to the model presented by the article?

No, it really isn't... it's just an introductive talking point to get you thinking about the intricacies of applying something like "network coding," which involves a lot more than a relationship with "error-correcting coding." There are plenty of other factors that preclude a successful application of "network coding," one of which happens to be error-correcting coding.

This isn't a question of the article, or the summary - thus my point stands that you did not comprehend what it was that was being stated. CSMA/CD is basically assumed to be present on a dynamically routed network, by any self-respecting geek that is going to be interested in this subject.

Re:Proofread summary next time please. (0)

Anonymous Coward | more than 2 years ago | (#40120961)

This isn't a question of the article, or the summary

Exactly. In a summary or elsewhere, you should never quote a sentence that contains references such as "different" (from something mentioned before) or "these two types" (where only one of them is part of the quoted text) without resolving the references by mentioning what they refer to. I don't understand how you can disagree with that. It's common sense. Every quotation is taken out of context, but when the context is (even grammatically) indispensable for understanding the quotation, it needs to be summarized.

Why would you want to know that ? (1)

freshlimesoda (2497490) | more than 2 years ago | (#40018611)

Just right size individual components right from storage to client level...check the average bandwidth used at all levels and allocate accordingly. Its like asking how many molecules of water in a glass..when you know you only need to drink one glass of water ... And wouldnt traffic/second increase or decrease with the choice of medium and communication protocol used over same distance.

Here's a free hint... (1)

jimmydigital (267697) | more than 2 years ago | (#40018643)

That ideal encoding method isn't XML.

Re:Here's a free hint... (1)

Archangel Michael (180766) | more than 2 years ago | (#40019537)

Actually, XML might be the ideal method of encoding. It depends on what you mean by "ideal" ;) XML makes it much easier for a human to decode than say a bit stream of binary coded data. Just saying, that without further context, your statement may not be entirely accurate, or it could be perfectly accurate.

Which in my estimation makes it not accurate at all.

So... (1)

rsmith84 (2540216) | more than 2 years ago | (#40019135)

Microsoft's Baseline Analyzer doesn't cut it anymore? ;-)

Trying to put OPNET out of business? (1)

BubbaDave (1352535) | more than 2 years ago | (#40019491)

OPNET- great tools, horrific price...

Dave

Re:Trying to put OPNET out of business? (1)

desertfool (21262) | more than 2 years ago | (#40020879)

OPNET? BAH! Way over priced for what it is. Try PathView by AppNeta. Better tool, lower cost.

Step one (1)

Fned (43219) | more than 2 years ago | (#40020211)

Release Diablo III...

Calculating Bankster Spreadsheets (0)

Anonymous Coward | more than 2 years ago | (#40022197)

Not one Nerd dare crack open the CAFR?
Not one Nerd dare wear a badge and go after the banks?
Not one Nerd dare restore the US Constitution?
What's your fucking excuse?

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>