Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Limiting Bandiwidth in a Shared DSL Environment?

Cliff posted more than 10 years ago | from the cutting-back-on-the-bandwidth-leeches dept.

The Internet 77

stylee asks: "We have a DSL connection that runs from a Cisco 675 DSL modem to a 24 port hub. Cat 5 cable has been run to the utility closet of each unit. The condo assoc. pays for the DSL from the monthly condo fees collected. The internet connection has been terribly slow the last few days, so I did a little snooping with ethereal and found that there is an individual who is using eDonkey 2000 to download and share movies. This user is eating up all the bandwidth. I want to set up a good router that can do load balancing so that an individual can't take up all the bandwidth and I was wondering what Slashdot would recommend. I would have to do it on the condo assoc. dime so it would have to be done on the cheap. Any suggestions?"

cancel ×

77 comments

Sorry! There are no comments related to the filter you selected.

Mmm... Linux (2, Interesting)

notamac (750472) | more than 10 years ago | (#8653019)

So a 2.4 (or a 2.6) kernel + iptables + some of the traffic shaping stuff (tc) sitting on a 486 that you can buy from the local second hand computer place for nothing, and things should be sweet.

Re:Mmm... Linux (5, Informative)

innosent (618233) | more than 10 years ago | (#8653323)

Or FreeBSD, we use our firewall box where I work, and use the traffic shaping portion of ipfw2 (man pages, ipfw at www.FreeBSD.org) to limit bandwidth to certain hosts. FreeBSD allows you to add a rule that passes all traffic through either a pipe or queue (pipe is what you want), set the bandwidth, size of the backlog queue, and monitor usage of the pipes. If you set it up as a transparent bridge (see the advanced network topics in the FreeBSD Handbook at www.FreeBSD.org), you won't even have to change host settings. This way, you can limit traffic on an individual (or group) basis, monitor usage, and just drop the box between the main switch and the dsl router, turn it on, and pretty much forget it (especially if you don't allow remote access to the firewall, except maybe ssh or a VPN).

The same can of course be done with Linux, but in my (though somewhat limited to my place of work) experience, FreeBSD's traffic shaper is a bit more reliable, and much easier to set up (it's all in the handbook). In our case, that box is a transparent bridge, accessible only via ssh or from the inside interfaces, with three NICs, one for the outside router, one for the inside public systems, and a third with private addresses, where natd (man natd, also integrated with ipfw via FreeBSD's divert sockets) translates the private addresses as they go out of one of the other two interfaces. We also run nagios (network monitor), etherape (looks cool when you see the traffic real-time on a GUI), and poptop (MSCHAPv2 capable VPN server), along with IDS logging via ipfw and tcpdump/ethereal, all on an old Duron we had laying around collecting dust.

In all, our Firewall/VPN/IDS/Traffic Shaper/Network monitor cost us about $250 in hardware, and two day's labor. I saw a similar product (though in a nice 1U rackmount case) listed for $6000 at CDW, so whatever you do, you can't go wrong with Linux or FreeBSD on cheaper hardware, unless your time is worth a few thousand dollars an hour.

Re:Mmm... Linux (4, Informative)

innosent (618233) | more than 10 years ago | (#8653442)

One other thing, if you don't want to limit on a host-by-host basis, you could do it by type of service. Say you allocate 80% of your available bandwidth to common web, instant messaging, mail, and DNS traffic, and the remaining 20% for everything else. Just watch your tcpdump/ethereal/etc. logs for about a week to see the normal behavior (and the abuses). This way, the normal, non-abusive services are quick, while the unknown/abusive services are limited, which has a side benefit of discouraging improper use. Hell, if you can lock down the most abused ports, set the pipe they go through to 2400bps, and see how many people still use them in a week.

Re:Mmm... Linux (1)

Wycliffe (116160) | more than 10 years ago | (#8656681)

Unfortunately this only works as long as people don't start masquerading their traffic as
something else.

Re:Mmm... Linux (0)

Anonymous Coward | more than 10 years ago | (#8665223)

You could dump the FreeBSD/Linux box out infront of your network and before you turn on any packetshaping you could use http://www.ntop.org (NTOP) to see what your traffic looks like.

As far as the guy who said "Until people start masquerading their traffic as other stuff", that's going to take someone very dedicated to their eDonkey to figure out and set that up.

Take a cue from my university... (5, Insightful)

xoran99 (745620) | more than 10 years ago | (#8653026)

What my university always did was, if a single user was using a lot of bandwidth on a constant basis, simply turn off their connectivity. When people learn to police themselves, stuff works better.

Re:Take a cue from my university... (1)

boaworm (180781) | more than 10 years ago | (#8653726)

Well that's one way to solve it. Still it doesnt work for all of us.

In my case, (I work as a sysadmin at a web-hosting company), we are considering implementing some sort of transparent bridge to limit traffic to WWW-servers in case we need to use our backup line. People can stand having slow WWW-traffic for a while, but mail/dns, especially smtp MUST work all the time.

In short, i cannot just pull the plug on our large web-servers :-), but i do want to delimit their traffic in case of a network problem.

Re:Take a cue from my university... (1)

Glonoinha (587375) | more than 10 years ago | (#8656896)

I was gonna mod you up but you were already pegged - so I will simply agree with you.

Unplug his ass. Unplug his wire from your switch and plug it into a cheapo Linksys cablemodem router (that isn't connected to anything else) so he gets a DHCP address - but no connectivity (that part is just for fun, makes it a real bitch for him to self-diagnose an outage when he can ping the router, get a DHCP address, and his neighbor still has Internet connectivity.) When he comes up wondering why he can't get out to the net show him the logs and explain that one guy using 95% of the bandwidth is not acceptable and he can either cut it out or go get his own dedicated pipe to the 'net.

Re:Take a cue from my university... (1)

jerde (23294) | more than 10 years ago | (#8657728)

when he comes up wondering why he can't get out to the net show him the logs and explain that one guy using 95% of the bandwidth is not acceptable and he can either cut it out or go get his own dedicated pipe to the 'net.

That would be the three-year-old's approach.

No, instead, just talk to the user first.

It is MUCH more complicated if that user is paying you money for the service (albeit not much money). If you didn't explicitly state in some sort of contract an acceptable use policy, YOU are the one who gets in trouble for discontinuing service.

Another example: the water bill is paid by my landlord, and so somehow built in to my rent. But there is a clause in the renter's agreement that states that "excessive" water use can result in a fine or charges. That protects the landlord.

Without that, I'm legally entitled to use something that is "included" in my rent -- fill all the neighbors swimming pools if I wanted to. The landlord would be the one getting in trouble for retaliating in any way.

Of course, this is only if people choose to be litigious bastards [thescogroup.com] . If people act like human beings, a nice friendly conversation can solve the problem.

But I can assure you: acting like a three-year-old and unplugging the user without warning WILL make them mad, and more likely to act like a litigious bastard [thescogroup.com] . :)

- Peter

Re:Take a cue from my university... (1)

Glonoinha (587375) | more than 10 years ago | (#8659115)

I agree, inasmuch as you will agree that the user seemed to adopt a three-year-old's approach to the shared resources in the first place (cue the 'squeal voice' : Mine - gimme - mine - me no share.) Either he knows he is eating ALL the bandwidth and doesn't care or he has no clue that he is flooding the pipe - one of those is true and both is equally childish.

But yes, I agree that the association needs to immediately draft a TOS (terms of service) letter with regards to the shared dataline. Unless he is on the association as a director it isn't a democracy - it is more like a monolithic theocracy or a democratic republic like the USA - the people get one vote per person to pick representation but at that point the representatives get to make the rules (and get voted out if they make unpopular rules.) Having owned condos in the past I can say that the rules can be fairly draconian and are not flexible - in fact many of the rules have 'fines' attached on a per-day basis (altering the exterior of your unit, for example, or parking where you are not supposed to park.)

Write up guidelines, be specific, have a dollar amount associated with not staying within the guidelines and everything pretty much sorts itself out.

Yea or just talk to the user first. I hate when stuff gets blown out of proportion - no need to start an act of Congress over an eDonkey - unless he wants to be an eAss.

FreeBSD (1)

Alcemenes (460409) | more than 10 years ago | (#8653032)

FreeBSD + ipfw should be able to handle this for you rather nicely. Find yourself an old P-200 or similar and put a couple NICs in it. That should be enough hardware to accomplish your goal.

traffic shaping (1)

nbdy (684290) | more than 10 years ago | (#8653035)

You can run traffic shaping using a linux box. It may be the reason that the eDonkey eat up most of the upload traffic so ack cannot be sent back and slow down the download too.

Re:traffic shaping (1)

darrelld2 (307106) | more than 10 years ago | (#8656129)

Why bother using another device? The router that he has is a Cisco IOS router. It supports the traffic shaping commands built into the IOS.

access-list 101 permit tcp any any eq

access-list 102 permit tcp any eq any

interface dsl 0
traffic-shape group 101 256000

interface ethernet 0
traffic-shape group 102 256000

simple and to the point. If you any more buckets go to Cisco's web site.

first off (3, Interesting)

glen604 (750214) | more than 10 years ago | (#8653037)

it sounds like your condo associaton needs an internet usage policy- considering this guy's actions could get everyone in trouble.

Re:first off (4, Funny)

ottawanker (597020) | more than 10 years ago | (#8653058)

it sounds like your condo associaton needs an internet usage policy- considering this guy's actions could get everyone in trouble.

No, actually as I see it, he's the whole building's alibi. Go ahead and download music, just blame him if anyone gets caught.

Re:first off (3, Interesting)

pete6677 (681676) | more than 10 years ago | (#8653419)

Whose name is on the bill?

Freshmeat (4, Informative)

wed128 (722152) | more than 10 years ago | (#8653040)

I always look on freshmeat.net for these solutions...here's a tip...

Linux Bandwidth Arbitrator [freshmeat.net] looks like it was designed for this sort of thing...

Re:Freshmeat (1)

Quixotic137 (26461) | more than 10 years ago | (#8655464)

Linux Bandwidth Arbitrator is a really good product. It's updated constantly, easy to set up and use (well, relatively), lots of options for buying it or downloading it, and it actually works.

If I knew what "bandiwidth" was... (0, Funny)

Anonymous Coward | more than 10 years ago | (#8653046)

I'd might be able to tell you how to limit it.

Re:If I knew what "bandiwidth" was... (0)

Anonymous Coward | more than 10 years ago | (#8653193)

If you knew the proper use of grammar, I'd might be able to ignore your silly remarks.
It's a typo. Get over it.

switch! (2, Informative)

moosesocks (264553) | more than 10 years ago | (#8653071)

What you need is a managed switch. They will allow you to limit bandwidth or completely disconnect a specific port. HP's switches are supposed to be particularly good

Be warned... a managed switch WILL cost several times more than a normal switch.

But apart from that, your only other choice is to use some sort of arbitrary setup to limit bandwidth to certain IP addresses and force each user to have one static IP (virtually impossible to enforce with your setup)

Re:switch! (0)

Anonymous Coward | more than 10 years ago | (#8653455)

Considering his MAC address won't change [unless he really wants to use eDonkey and goes out of his way to spoof it...], then you don't need to worry about the IP addressing side of things... layer 2 all the wayyyy. Use a Linux box rather than a managed switch. Why pay more than you need to?

Bandiwidth? (-1, Offtopic)

nosferatu-man (13652) | more than 10 years ago | (#8653072)

Is that like a bandersnitch?

'jfb

Not too hard, cut him loose (3, Insightful)

dnight (153296) | more than 10 years ago | (#8653077)

Refund the portion of his condo fees used for DSL, and tell him to get his own DSL line.

If he's illegally sharing files, he won't squawk too loudly.

OpenBSD or FreeBSD (5, Insightful)

plsuh (129598) | more than 10 years ago | (#8653080)

OpenBSD has support for limiting classes of bandwidth for quality of service as a part of the pf(4) firewall. See the part of the pf user's guide [openbsd.org] that covers how to do it.

FreeBSD also has built-in support via the altq facility that is a part of the ipfw firewall.

My druthers would be to use OpenBSD for this as it's not a CPU-bound problem and security on your router should be very high on your list of priorities.

--Paul

elcheapo box with linux (2, Informative)

womprat (154589) | more than 10 years ago | (#8653105)

At my house we have four guys and we all download pretty heavily (bittorrent, edonkey, gnutella, etc.) Unlimited this just chokes up the whole connection (a fragile cable modem that gets confused if it gets too many packets)
So I just run "tc qdisc add dev eth1 root tbf rate 250kbit latency 20ms burst 2kb". This keeps the network running at full speed with all the downloads going.
Checkout the Bandwidth Limiting HOWTO on tldp.org

Re:elcheapo box with linux (1)

0x0d0a (568518) | more than 10 years ago | (#8653381)

The problem is that this still has severe problems. It does avoid the awful latency problems once someone starts using the network and fills up the modem's outbound buffer. It does not evenly share out bandwidth.

It requires the hosts on the inside side of the shaper to regulate their own traffic, via TCP throttling from packet loss. Unfortunately, TCP only knows about the single stream that it's dealing with. Most P2P clients these days have *scads* of TCP connections open at any given time. Which means that if one guy is running eDonkey with 300 simultaneous downloads running on his computer, and another guy is just trying to browse the web, eDonkey is going to get 99.7% of the bandwidth, and the guy with the web browser (one pipelined HTTP connection) .3%, since his computer backs off just as quickly as the guy running the P2P client on each connection.

It's really necessary to use a system that shares out bandwidth evenly between hosts if your users are going to be opening many connections per host (and a P2P client almost always means that this is the case).

Also, if you want to keep the network usable for people that want to use the Internet comfortably *and* soak up any spare bandwidth on a host at the same time, you need to prioritize P2P traffic at the bottom, and things like ssh at the top.

Is he running a Windows box? (1)

NanoGator (522640) | more than 10 years ago | (#8653173)

Would it be too hard to change his hosts file so he thinks the internet's broken? Heh.

Bandiwidth (0, Offtopic)

bobo the hobo (302407) | more than 10 years ago | (#8653175)

The title of this post raises an interesting idea:
I think we should change the word "bandwidth" to "bandiwidth." It just sounds better!

DSL shaping system (5, Informative)

0x0d0a (568518) | more than 10 years ago | (#8653179)

I set up a DSL traffic shaper on Linux a bit ago. It's a bit of a pain in the ass to figure out the right things to do, and I don't have the script handy, but here are some pointers (given that this is from memory, some of this will probably be wrong).

Get a Linux box. Get two NICs (c'mon, NICs are cheap these days, and the DSL modem only needs a 10Mbit one).

Set up bridging on the Linux box.


ifconfig eth0 0.0.0.0
ifconfig eth1 0.0.0.0
brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1


If your boxes use DHCP, you might want to give your shaper an outside IP address (so that it can run ntp and the like, if nothing else). Use br0 as the interface -- this tripped me up at first.


dhclient br0


Add per-host rate limiting. There are two *excellent* solutions to do this automatically under Linux -- esfq and wrr. Both automatically detect IP addresses on one side and spread bandwidth out evenly. Neither is apparently actively maintained, unfortunately, so if you're using a 2.6 kernel, you're out of luck. Your best bet is probably HTB (which *is* included in 2.6 and I believe current 2.4 kernels). HTB requires you to manually create a child of the main HTB qdisc for each IP address, and filter based on source IP address (or source MAC address, which is probably more appropriate if you have a single Ethernet segment and dynamically assigned IP addresses) but lets you filter traffic differently for each host. For a small network, this may be feasible. I'd hang another qdisc off of the HTB that reduces the priority of P2P *within* each host's account, so that someone can use spare bandwidth for eDonkey or whatever, but still retains reasonably snappy SSH, even on their own box.

You must set the maximum flow of the HTB just below the DSL modem's data transfer rate, or else the modem's buffer will fill up when outbound traffic fills up its (big) buffer, making interactive use impossible. Keep reducing the limit and then ping flooding (ping -f) the outside world from an inside box. Keep a regular ping running in another terminal, and monitor it. When your system is working right *ping times should not climb above 150 or 200ms or so on a box*. No 1000ms latency. You should simply start seeing packet loss.

I must say that setting something like this up was a huge pain in the ass, and that if I had the script handy at the moment, I'd post it. The Linux networking/filtering/routing system is not as well documented as it should be, and is *not* always the most intuitive thing in the world. It is, apparently, pretty powerful, based on what I've read from folks that have used other systems, though. [shrug]

Speaking of which, I can't figure out why sfq is in mainstream Linux but esfq is not. SFQ is, to my mind, almost useless for most people. Who on earth wants to balance all their TCP flows evenly? Even per-host bandwidth allocation is a *far* more common problem, and one that vanilla Linux (and any 2.6 kernel) cannot handle well.

I did not find it necessary to use ebtables or ipchains to produce an effective traffic shaper. YMMV.

Re:DSL shaping system (2, Informative)

0x0d0a (568518) | more than 10 years ago | (#8653327)

I did not find it necessary to use ebtables or ipchains to produce an effective traffic shaper.

And by this I mean that all the commands that you'll have to use that I didn't already list should start with tc.

Re:DSL shaping system (1)

0x0d0a (568518) | more than 10 years ago | (#8653349)

Oh, yes. If you're just giving br0 a static IP, don't forget to bring the interface up.


ifconfig br0 1.2.3.4 [or whatever your IP is]
ifconfig br0 up

Options on Freshmeat (1)

0x0d0a (568518) | more than 10 years ago | (#8653403)

As another followup, I looked on Freshmeat, and couldn't find anything that did what I wanted. It seems that it's all the rage to have traffic shapers also do NAT, which I *really* did not want.

I don't think Freshmeat is currently a good place to go when looking for a traffic shaping system to do this sort of thing.

pump/dhclient followup (2, Informative)

0x0d0a (568518) | more than 10 years ago | (#8653438)

As another aside, some distros bundle pump as the DHCP client, rather than dhclient.

Oh, speaking of DHCP, big tip for Red Hat/Fedora users. Absolutely do not use the vanilla ifup scripts that Red Hat provides. They *suck*. If you are on any kind of a consumer DSL connection, every now and then (perhaps rare, perhaps common) you will lose your connection, for whatever reason. For some reason, Red Hat sets up their copy of dhclient to *give up* if it fails to get a dhcp lease, which means I frequently endured having a power outage at my house when I was away (killing the line) and then being unable to reach my computer remotely because it never acquired a DHCP lease.

Look in /etc/sysconfig/network-scripts/ifup, and search for a line that looks like the following:

DHCLIENTARGS="${DHCLIENTARGS} -1 -q -lf /var/lib/dhcp/dhclient-${DEVICE}.lea
ses -pf /var/run/dhclient-${DEVICE}.pid -cf /etc/dhclient-${DEVICE}.conf"


Change the -1 (telling dhclient to die if it can't get a lease immediately) to a -w (telling it to keep trying) in that line, and you won't have to endure your Linux box randomly becoming unreachable and losing the IP address on its interface.

HTB matching (2, Informative)

0x0d0a (568518) | more than 10 years ago | (#8653561)

Don't forget to add a default class to the HTB to match any MAC that all your previous matching work didn't match. That way, anyone that you *haven't* added a MAC entry for (adding a child to the HTB tree) will at least go into a general class and get connectivity...they just have to share it with all the other people in the "general" class.

You may want to toy with the idea of having a perl script or something look at unmatched packets or maybe scrape the ARP cache (arp -a) to automatically add new entries to the HTB tree.

You will want to be sure that this box is set to autorestart on power failure.

You will want to include instructions (probably on the face of the box, as well as on file with whoever owns the property) on how to remove the box from the loop. That hard drive will fail someday.

For some reason, when I insert my shaper in between my DSL modem and the local network and start using it, I seem to see a delay of a minute or two before requests from clients on the inside start hitting the ouside). This confuses me immensely, since a major benefit of using a bridge over a pseudo-bridge is that the ARP entries, the MAC-IP mappings, stay the same. No idea what the cause is.

IPCop (3, Informative)

Anonymous Coward | more than 10 years ago | (#8653242)

IPCop [sourceforge.net] v1.3 w/ Wondershaper [surestorm.com] or wait a couple more weeks for 1.4 which will have bandwidth shaping built in. It's a linux distro just for firewall/routers, runs on anything from a 486 up.

Simple solution... (4, Funny)

meta-monkey (321000) | more than 10 years ago | (#8653260)

Several posters have already mentioned managed switches, linux routers with iptables, etc, but I've got a much simpler solution for you. It's a wonderful product manufactured by the Louisville Slugger [slugger.com] corporation called a "baseball bat." With this fine product in hand, march over to the offending user's apartment, and smartly inform him that he is using too much bandwidth. If he refuses to self-throttle his bandwidth, offer to throttle him and his computer with the genuine wood Louisville Slugger baseball bat. Problem solved. Thank me later.

Re:Simple solution... (1)

innosent (618233) | more than 10 years ago | (#8653406)

Here's the problem though, have you seen a P2P client that has a "no, don't use my entire bandwidth, I want to download at 2400bps" option? If people download anything, their system will attempt to move packets at the fastest speed possible, and one heavy user can affect all others. It doesn't matter who the user is today, the original poster wants a solution to the problem tomorrow. A DSL line is not that fast, so chances are pretty good that if one user downloads something large, they could max out the bandwidth, and if 20 all download at the same time (Windows Updates, hopefully), it's pretty much a coin flip for any of them to work. Windows is especially bad at limiting itself to available bandwidth, since it retries packets too often, and 20 machines waiting on bandwidth just means 20 machines retrying packets over and over again. Multiple Windows machines on limited bandwidth are actually quite capable of DoSing themselves, especially at lower (10Mbps) network speeds, or through hubs (you will have more collisions than actual traffic). This is why limiting traffic as it leaves the last switch before the router works well, machines are limited as they go out, and can be told by the QoS engine of the limiting hardware to slow down.

Re:Simple solution... (0)

Lord Bitman (95493) | more than 10 years ago | (#8653698)

Here's the problem though, have you seen a P2P client that has a "no, don't use my entire bandwidth, I want to download at 2400bps" option?


yes.

Re:Simple solution... (1)

Genom (3868) | more than 10 years ago | (#8654915)

"Here's the problem though, have you seen a P2P client that has a "no, don't use my entire bandwidth, I want to download at 2400bps" option?"

The better bittorrent clients let you do exactly that, specifying a limit to how much bandwidth it can consume. ^^

Re:Simple solution... (1)

Wycliffe (116160) | more than 10 years ago | (#8656586)

>
>The better bittorrent clients let you do exactly that, specifying a limit to how much bandwidth it can consume.
>

The better bittorrent clients let you limit the UPLOAD speed, I have yet to see one that lets you limit the DOWNLOAD speed, which is what the original poster was asking. If you know of one, please share, because I have been looking for one.

Re:Simple solution... (1)

DRue (152413) | more than 10 years ago | (#8657526)

The better bittorrent clients let you limit the UPLOAD speed, I have yet to see one that lets you limit the DOWNLOAD speed, which is what the original poster was asking. If you know of one, please share, because I have been looking for one

The way bittorrent works is that your upload speed sets your download speed. If you don't upload at all, you won't be able to download. That's why it works!

Re:Simple solution... (1)

Wycliffe (116160) | more than 10 years ago | (#8658050)

I have my bittorrent upload capped at 5K, but my bittorrent download still hits 50K+ which all but kills my ability to do anything else while I'm using bittorrent.
If there is a download/upload ratio, presumably based on my numbers it is greater than 10/1.
I would like my download to be capped at 20K, but I haven't found a bittorrent client that lets me do that yet.

Re:Simple solution... (1)

DRue (152413) | more than 10 years ago | (#8667388)

Well.. All I know is what I read from bt's page:

Q: I don't want you stealing my bandwidth! How can I stop it from uploading?

A: You could hack the source to not upload, but then your download rate would suck. BitTorrent downloaders engage in tit-for-tat with their peers, so leeches have very little success downloading.

Re:Simple solution... (1)

darksoulz (700833) | more than 10 years ago | (#8659442)

Shareaza [shareaza.com] will let you limit up and down speed. Supports Gnutella1, Gnutella2, eDonkey, and Bittorrent

Re:Simple solution... (1)

Wycliffe (116160) | more than 10 years ago | (#8694401)

Here is the creator of bittorrent's opinion of shareaza, which I found kindof interesting:

http://groups.yahoo.com/group/BitTorrent/message/4 912 [yahoo.com]
Suffice it to say that shareaza is not only written incompetently but makes every attempt it can to squeeze whatever it can out of the network, regardless of how much damage to the network as a whole that results in. In BitTorrent the amount of damage you can do is fairly limited. In edonkey it took the whole search system down. Come to think of it, I'm feeling actively hostile to shareaza, and have no inclination to help make it support BitTorrent better, lest more people start use it.

Re:Simple solution... (1)

AuraSeer (409950) | more than 10 years ago | (#8665368)

Here's the problem though, have you seen a P2P client that has a "no, don't use my entire bandwidth, I want to download at 2400bps" option?

Of course. The problem user has one already, assuming his version of eDonkey is reasonably current.

Re:Simple solution... (1)

0x0d0a (568518) | more than 10 years ago | (#8653409)

Policy solutions are generally less scalable, harder to enforce, and cause more social issues in enforcement than technical solutions. If one can manage a technical solution, I'd prefer it.

More to the point, a traffic shaper knows about the current demands that all the computers are putting on the network. Each individual computer with throttling capabilities does not. Sure, Bob can throttle his traffic down to 3KBps, but that means that when nobody else is using the network, he isn't taking advantage of it. It also means that if someone's doing something that *does* require low latency (ssh, perhaps web browsing), his P2P client bandwidth usage isn't scaled back to provide bandwidth for it.

M0n0Wall (5, Informative)

mcowger (456754) | more than 10 years ago | (#8653265)

Monowall (www.m0n0.ch/wall) is a greaqt application for this. Can run from CDROM, CF or on a Soekris board - can do per IP bandwidth limiting/shaping, and totally free, based on BSD. It was trivial for me to set it up here.

Didja try asking him? (4, Insightful)

NanoGator (522640) | more than 10 years ago | (#8653329)

I'd recommend politely approaching the guy and asking him to throttle it down a bit. If he agrees, problem solved. If he refuses, cut his connection. Why spend more money to solve the problem of one abuser?

Re:Didja try asking him? (1)

innosent (618233) | more than 10 years ago | (#8653461)

Because after you've told the tenth person, your time tracking them down, talking to them, and making sure they comply has been worth more than the traffic shaper. One person may cause the problem today, but three more may start next week. Plus, you have to police them if you don't put a shaper in, which also costs you time/money.

Linux Advanced Routing and Traffic Control. (1)

Michael Spencer Jr. (39538) | more than 10 years ago | (#8653394)

http://lartc.org [lartc.org]

It's difficult to understand, much less set up, but essentially the stuff from this site can solve your problem by tightly controlling outbound traffic (since it is possible to have perfect control over what packets you release to the network) and by loosely attempting to control inbound traffic (since it isn't really possible to perfectly control what packets other people send you).

For example, my home setup has four priority classes:

Class 0:10 is for high priority traffic: ping replies, TCP ACK packets, and online gaming.

Class 0:20 is for everything not otherwise classified.

Class 0:30 is for BitTorrent traffic -- lower than normal, but higher than all the other p2p stuff. I do this because BitTorrent traffic is very likely to be directly related to a file I'm personally interested in.

Class 0:40 is for lowprio.mspencer.net and other misc filesharing programs. If the rest of the Internet connection is busy, class 0:40 ends up with around 24 kbit/sec out of my total 640 kbit/sec upstream.

I guarantee you can adapt this as needed, so each user has a fair slice of upstream available, but if someone's not using their slice then everybody else can split it. (So at 4 AM one user can still get the whole line speed, but at peak usage everybody gets the same bandwidth.)

The other side of the coin is ingress policing. I don't have a lot of experience with this, but you'll almost definitely need it. Basically the policing module tries to slow inbound packets by throttling the outbound acknowledgements. It's not perfect but it can help.

Some filesharing programs incorrectly state they are "firewalled" when you use a setup like this. Instruct the user to just tell his client to retest so it can confirm he's not firewalled.

My final paper for my 4000/8000 level networking class was regarding my traffic shaper. Maybe it'll help.

http://mspencer.net/traffic-shaper.doc [mspencer.net] in Word 2000 format.
http://mspencer.net/traffic-shaper.txt [mspencer.net] in plain text.

--Michael Spencer

Re:Linux Advanced Routing and Traffic Control. (1)

innosent (618233) | more than 10 years ago | (#8653589)

Not a bad solution, but the ingress traffic from P2P software will mostly circumvent this, unless the problem is outbound traffic from the offending user. This is where FreeBSD's pipes and integration with ipfw come in handy. IPFW is stateful, so for each outbound connection that should be limited, the response can be forced through the same limits (though the ipfw man pages suggest using separate pipes with larger queues, a single pipe with a small queue size works better in my opinion). If the rule that caught the outbound connection is something like:
ipfw pipe 7 bw 300Kbit/s
ipfw add 2100 pipe 7 all from any to any dst-port 80 setup keep-state

and one of the first rules is something like:
ipfw add 100 check-state

then the responses to the packets must flow through the same pipe, limiting traffic in both directions. If QoS for all packets is important, and you don't want to deal with set bandwidth, you could use FreeBSD's queues, which are worst-case fair-weighted fair queues, or you can use a queue for a specific pipe (for queueing traffic to a limited-bandwidth pipe). Read through the man page for ipfw, specifically the traffic shaper portion, basically anything needed to solve this sort of problem is already available there.

Re:Linux Advanced Routing and Traffic Control. (1)

Michael Spencer Jr. (39538) | more than 10 years ago | (#8655770)

I know what you mean, but I also wanted to let everyone know with my original post that ingress policing under Linux solves this problem.

http://lartc.org/howto/lartc.adv-filter.policing.h tml [lartc.org]

(If that isn't easy to understand, keep in mind it's section 12 of a long HOWTO with lots of conceptual material. If you start reading from the beginning, and skip sections that don't involve your problem, everything should start making sense.)

For example:

tc filter add dev $DEV parent ffff: \
protocol ip prio 20 \
police rate 640kbit buffer 10k drop \
flowid :1

This adds a simple filter rule that limits inbound traffic to 640 kbit/sec and drops matching *outbound* traffic to slow down inbound traffic. You don't have to do the whole line at once: this is class-based and the above example assumes only one class, so you could add several classes, one for each user, and make them borrow from each other when others' classes aren't maxed out. (Just don't make them 'bounded' or 'isolated' and you get this borrowing for free.)

What would the end result of this be? If you set up seven queues (using u32 filters on each rule to match each individual user) all underneath one parent queue representing the entire downstream pipe then you get some interesting and fair behavior:

Suppose you have a max line speed of around 70 KB/sec, and seven active users. Six are well-behaved, doing one HTTP file download each, and the seventh is running as many filesharing clients as he possibly can. Each user would notice about 10 KB/sec download speed on their download. (If one user's download wasn't capable of going any faster, then maybe the other guys would borrow his spare KB/sec and go a bit faster, until he downloaded something else that used all of his allocation.) The badly-behaved filesharing user, though, might be attempting 30 downloads at once. He'd still be getting 10 KB/sec across all of the connections though. The other endpoints of his connections may be trying to send him faster than 10 KB/sec, but traffic policing will notice this is over his limit and will cut off his acks to match. So this one user may notice an 80% frame loss rate and almost useless web browsing, but everyone else will have pretty much normal service.

And the best part is: when things aren't busy any more, this "partitioning" of bandwidth just neatly gets out of your way and shares any unused classes with other users. So at 4 AM this filesharing user might get 70 KB/sec across all of his downloads. But if someone pops on to check their mail, this user's downloads will get pushed down to make room for the new user.

Another thing to keep in mind: for these filters to work well, you need to give them some overhead. If your actual linespeed is 640 kbit/sec, set the filters to a max of 620 kbit/sec. This way it can detect and act upon overlimit conditions before inbound and outbound queues start filling up. If you set the ingress filter to a max which is the same as your line speed, you won't be able to detect when people are sending packets faster than you can receive them: your ISP will be helpfully buffering your packets in an inbound queue and adding tons of latency.

So to recap: this ingress policing will work for you too. You'll have to learn the weird way these filters work -- but they're very powerful. As with most learning in Linux, half of the documentation-reading work is bringing you up to speed with the universal concepts needed to understand what you want to do, and the other half is understanding how those concepts translate into this specific bit of software.

So if the person who posed the original question is more familiar with Linux than with your OS, it sounds like they can accomplish the same thing with Linux. No need to force someone to switch just yet. :-)

--Michael Spencer

Re:Linux Advanced Routing and Traffic Control. (0)

Michael Spencer Jr. (39538) | more than 10 years ago | (#8653662)

OK, here's the text of my paper, for those of you who don't want to click a link. :)


Traffic Conditioning For Inexpensive Installations
Business-Class Performance From Free Software and Commodity Hardware

By Michael Spencer



Broadband internet connections don't handle heavy server loads very well. When many
connections are in contention for the same limited upstream bandwidth, problems occur
that degrade overall link performance. I have found a solution that can be implemented
with inexpensive software on existing hardware, which sustains reasonable performance
ever under extremely heavy load. I will describe the problems that occur when dozens of
connections all compete for bandwidth, offer some possible theoretical solutions, and
then describe an implementation in detail that solves these problems.

Before I talk about the problem, you might want to know if the solution is reasonable.
My proposed solution only works well if your upstream bandwidth is constant. If you
have a home DSL connection with a fixed upstream rate, this solution is ideal. If you
share a university or company internet connection, and you don't actually administer the
university or company connection, this solution won't work well for you. Cable modem
users might or might not work, depending on whether or not their upstream is fixed in
hardware, or simply shares whatever is left over after all other users are done with it.

My proposed solution uses advanced networking features in the 2.4 series Linux kernel.
You will need to have a Linux machine responsible for routing all Internet-bound traffic
to your border router (cable modem, DSL modem, etc.). The ideal way to do this is to
put two network interfaces in a dedicated machine. One interface is on the local network
segment, and one interface leads directly to the border router.

Not everyone has this kind of hardware just laying around, so it may be possible to
implement this solution even if your linux machine and border router share the same
network segment with the rest of your network, or even if you only have one machine
connected to your border router (or your border router is a card installed inside your
machine).

If you only have one network segment but you already have a dedicated Linux machine
on the network, it may be possible to reconfigure your network so all traffic must pass
through the Linux machine. That is the configuration I use at home, and my sample
configuration detailed below assumes this.

If you only have one computer connected via ethernet to a border router, or if your border
router is a card installed inside your computer, you may still be able to use this technique.
VMware Incorporated sells a virtual machine monitor called VMware ($100 for a
personal or educational license, $300 for a commercial license). You can create a limited
Linux virtual machine with VMware and bind that to the network interface your Internet
connection is on. Then configure the VMware machine to communicate directly with the
border gateway, and configure your desktop computer to use the VMware machine as its
default gateway. You will need to leave VMware running at all times when you need to
use the Internet, but VMware can be configured to use as little as 16 MB of memory and
very little CPU time.

Once you have a Linux machine between your border router and your network, you will
need to add support for several advanced options to your Linux machine. You will
probably need to recompile your kernel with support for the experimental Shaper device,
as well as several items in QoS and/or fair queuing: the CBQ and SFQ packet
schedulers, the U32 classifier, and Traffic Policing. These options can be found under
Networking Options in the kernel configuration program.

You will also need two userspace tools: tc and shapecfg. Tc can be found in the iproute2
package in your distribution, or source is available from ftp://ftp.inr.ac.ru/ip-
routing/iproute2-2.2.4-now-ss??????.tar.gz . Shapecfg is available with most
Linux distributions already.

Before we talk about implementation details, it helps to understand why this solution is
needed in the first place.

The biggest problem is that most inexpensive border gateway devices (cable modems,
DSL routers) are configured with rather large packet queues. If you send them data faster
than they can transmit it, they will happily queue up to nearly two full seconds worth of
data. When that queue is almost completely full, any new packet you send will either
have to wait up to 2000 ms before being transmitted, or it will be dropped entirely.

We must understand the nature of the problem, and the kinds of things any possible
solution can hope to achieve. We have complete, perfect control over our upstream, but
not much control at all over our downstream.

This causes problems with TCP's error recovery mechanisms. If one stream is sending
huge amounts of data, filling up the border router's queue, but a second stream is sending
small amounts of data to an unreliable host, the unreliable host's requests for
retransmission will have to wait up to an additional 2000 ms before they see the data they
requested.

Time-sensitive traffic is also hurt by a long send queue. If you have one stream that's
time-insensitive but is moving a lot of data, it will assume it gets the best transfer rate
when the router's buffer is full. If you have another stream running at the same time
that's time-sensitive but not moving much data, it's going to have to endure the high
latency induced by the full buffer in the router.

Last, TCP doesn't actually work with other connections to ensure a fair division of
bandwidth. If you have several streams, each to different hosts, each stream is going to
try to use as much of the link as possible without flooding. Foreign hosts that can
acknowledge packets quickly may receive a disproportionately large amount of the
available bandwidth.

The Linux kernel offers a variety of tools for solving these problems. The first is the
shaper device. This is a virtual ethernet device that's bound to an existing ethernet
device, and configured with a maximum speed. For example, if the physical ethernet
interface you're shaping is eth0, the shaped interface is shaper0. All packets sent via
interface shaper0 actually exit via interface eth0.

The other tools are all part of the advanced routing and traffic control subsystem.

The second tool is the queuing discipline, called a qdisc. When packets are enqueued for
transmission but can't be transmitted immediately, the system has a choice of several
behaviors to follow when dequeuing and transmitting packets. If no other behavior is
specified, Linux uses the pfifo (plain first-in first-out) qdisc. This qdisc simply
transmits packets in the order they were received.

One alternative qdisc is the sfq (Stochastic Fair Queuing) qdisc. This qdisc tries to
track the individual flows present in the queue, and tries to allocate a virtual queue for
each. It then dequeues packets with a round-robin scheduler. The end result is that all
flows have a chance to send data at an equal rate.

The third tool is the traffic-control class and the classifying filter. Each class represents a
specific kind of traffic, as determined by filter rules. Individual classes may contain other
classes, and can have maximum transmission rates or priorities assigned to them. Each
class also has one qdisc assigned to it. Packets enqueue in a class, and dequeue using the
qdisc assigned to the class.

I will now present an example implementation of these techniques, with observed
performance results and a discussion.

I designed my traffic conditioning solution with the following goals and restraints in
mind:

? My internet connection is a home-office DSL line provided by Qwest, with a solid
640 kbit/sec upstream data rate. This rate isn't shared with any other subscribers.
? My border router is a Cisco 678. It supports arbitrary routing rules, but doesn't
have any provisions for fair queuing or traffic shaping.
? I have three computers on my network: a linux server called mspencer.net, a
Windows 2000 desktop machine called michael.mspencer.net, and a Windows XP
desktop machine called luann.mspencer.net.
? The users of luann.mspencer.net are end-users, and their perceived performance is
important. They don't run any applications that send a large amount of data, so
their outbound traffic should have an extremely high priority.
? The desktop machine michael.mspencer.net runs three main types of traffic:
time-critical traffic like multiplayer gaming; normal interactive traffic like web
browsing; and low-priority high-throughput traffic like peer-to-peer file sharing.
? The server machine mspencer.net runs five main types of traffic: interactive
sessions with users; ftp sessions for identified users with valid usernames; public
http traffic with public Internet users; bulk low-priority http traffic with public
Internet users requesting extremely large objects (over 50 MB); and low-priority
peer-to-peer file sharing traffic.

My first task was to instruct my machines to route all traffic to and from the public
Internet through my Linux server, mspencer.net. Before making any changes, my
network was as follows:
? router.mspencer.net (209.180.104.206) was the Cisco 678 DSL router. Its routing
table said to forward all traffic destined for 209.180.104.200/29 directly onto the
ethernet interface.
? michael.mspencer.net (209.180.104.204), luann.mspencer.net (209.180.104.205),
and mspencer.net (209.180.104.202) all routed outbound Internet traffic directly
through router.mspencer.net (209.180.104.206).

I changed the Cisco router's routing table such that it would pass packets to mspencer.net
(209.180.104.202) directly onto the ethernet segment, but would route any traffic
destined for michael.mspencer.net (209.180.104.204) or luann.mspencer.net
(209.180.104.205) through mspencer.net (209.180.104.202).

Next I turned on ip packet forwarding on mspencer.net (209.180.104.202). I didn't
modify its routing table. mspencer.net still knows it's directly connected via ethernet to
both desktop machines and the router.

Finally I updated the network settings for both desktop machines, telling them to use
mspencer.net (209.180.104.202) instead of router.mspencer.net (209.180.104.206) for
their default gateway.

My network then looked like this:
? router.mspencer.net (209.180.104.206) had its routing table changed to:

cbos#show route<br>
[TARGET] [MASK] [GATEWAY] [M][P] [TYPE] [IF] [AGE]<br>
0.0.0.0 0.0.0.0 0.0.0.0 1 SA WAN0-0 0<br>
209.180.104.201 255.255.255.255 209.180.104.202 1 SHAR ETH0 0<br>
209.180.104.203 255.255.255.255 209.180.104.202 1 SHAR ETH0 0<br>
209.180.104.204 255.255.255.255 209.180.104.202 1 SHAR ETH0 0<br>
209.180.104.205 255.255.255.255 209.180.104.202 1 SHAR ETH0 0<br>
209.180.104.200 255.255.255.248 0.0.0.0 1 LA ETH0 0<br>

216.161.72.0 255.255.255.0 0.0.0.0 1 A WAN0-0 0<br>
? mspencer.net (209.180.104.202) had ip forwarding enabled
? Both michael.mspencer.net (209.180.104.204) and luann.mspencer.net
(209.180.104.205) had their default gateway changed:


I have not changed the physical topology of my network at all. I still only have one
ethernet switch with four machines plugged in, and none of my machines are multi-
homed. However, my Linux machine is now able to control all of my Internet-bound
traffic.

My first task was to attach a class-based queue to only my Internet-bound traffic. I want
my desktop machines and my Linux server to be able to communicate at full ethernet
speed without causing the Linux server to believe Internet bandwidth is being consumed.

I compiled support for traffic shaping and class-based queuing into my kernel and
rebooted. I also downloaded binary packages for the shapecfg and tc tools.

I created a shaper device with the command shapecfg attach shaper0 eth0 and
shapecfg speed shaper0 10000000. This created a virtual ethernet device called
shaper0 with a transmission limit of 10 mbit/sec. Please note that while the original
intended use of the traffic shaper device is to actually limit your transmission rate, I used
it merely to create a second ethernet device that transmits on the same physical link.

I brought the shaper interface up and assigned it the same IP address as my ethernet
interface. This won't cause a problem inbound packets will always arrive via eth0, but
outbound packets may be sent via eth0 or shaper0.

Next I configured a second IP address for certain Apache virtual hosts. I created an
ethernet alias eth0:0 with IP 209.180.104.203, and also a matching shaper alias shaper0:0
with the same IP.

Next I adjusted my routing table so outbound traffic destined for a host on the local
ethernet segment was sent through interface eth0 like normal, but outbound traffic
destined for the public Internet was sent through interface shaper0. While this has no
immediate effect (both interfaces send data to the same wire) this lets me single out
traffic on the shaper0 interface.

[root@mspencer /root]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
209.180.104.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 209.180.104.206 0.0.0.0 UG 0 0 0 shaper0

I then started implementing the Ultimate Traffic Conditioner script found at
http://lartc.org.

I edited the root class, called 1:, changing its queuing discipline from the default pfifo
(plain first-in first-out) qdisc to the cbq (class-based queuing) qdisc.
tc qdisc add dev shaper0 root handle 1: cbq avpkt 1000 bandwidth 10mbit
This class doesn't perform any shaping or prioritizing of traffic on its own. Its subclasses
will do that.

Next I created a main class, called 1:1.
tc qdisc add dev shaper0 parent 1: classid 1:1 cbq rate 530kbit allot 1600 prio
5 bounded isolated
This class has a maximum transmission rate of 530 kbit/sec. It is also marked bounded
and isolated. A bounded class is not allowed to borrow unused bandwidth from sibling
classes, and an isolated class is not allowed to lend unused bandwidth to other siblings.

I then created two child classes, called 1:10 and 1:20.
tc class add dev shaper0 parent 1:1 classid 1:10 cbq rate 53kbit allot 1600
prio 1 avpkt 1528
tc class add dev shaper0 parent 1:1 classid 1:20 cbq rate 477kbit allot 1600
prio 2 avpkt 1528
Because these classes are siblings, and neither is bounded nor isolated, they will borrow
unused bandwidth from each-other. However, if both classes are being sent enough traffic
to fill their queues, they will neither lend nor borrow from each other. That is, if class
1:10 is being asked to send at 400 kbit/sec and class 1:20 is being asked to send at 600
kbit/sec, then neither class will borrow from the other. They won't share bandwidth at a
rate proportionate to their inputs (40% for 1:10 and 60% for 1:20). They will stick with
their assigned rates (10% for 1:10 and 90% for 1:20).

I changed the default queuing discipline (qdisc) on both 1:10 and 1:20, so packets are
dequeued fairly.
tc qdisc add dev shaper0 parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev shaper0 parent 1:20 handle 20: sfq perturb 10
The Stochastic Fair Queue queuing discipline (sfq) was explained above. It will try to
identify individual flows in the traffic that passes through its queue, will create a virtual
queue for each flow, and will try to dequeue packets from each virtual queue with a
round-robin scheduler. The result of this is that if you have twenty connections all
flooding the interface at their maximum speeds, each connection will receive roughly one
twentieth of the available bandwidth. The perturb setting tells sfq how often to
recompute the hash buckets it uses to form virtual queues with.

I then added filters, to determine which packets will be handled by which classes.
tc filter add dev shaper0 parent 1: protocol ip prio 11 u32 match ip protocol 1
0xff flowid 1:10
The u32 classifying filter works by comparing individual bits in a packet against a match
criteria. The protocol keyword is an alias for the address of the protocol number in the
IP header. The next number (1) specifies what value to compare with, and the number
after that (0xff) is a bitmap which specifies which bits in the specified location should be
compared. 0xff corresponds to 11111111, which means that all eight bits must match.
0x01 corresponds to 00000001, which means that only the rightmost bit would have to
match. Traffic matching this filter is handled by class 1:10.

tc filter add dev shaper0 parent 1: protocol ip prio 10 u32 match ip tos 0x10
0xff flowid 1:10
This filter checks the value of the type-of-service flag in outbound packets. Public
Internet routers generally don't honor the requested type of service, but some applications
(especially ssh) set the type of service on outgoing packets. This filter adds interactive
ssh session traffic to the high priority class 1:10.

tc filter add dev shaper0 parent 1: protocol ip prio 12 u32 match ip protocol 6
0xff match u8 0x05 0x0f at 0 match u16 0x0000 0xffc0 at 2 match u8 0x10 0xff at
33 flowid 1:10
This filter checks several bits in the IP header, to match only TCP acknowledgements.
We want to give acknowledgements high priority, so downloads go faster.

tc filter add dev shaper0 parent 1: protocol ip prio 40 u32 match ip dst
0.0.0.0/0 flowid 1:20
This filter is a catch-all that should send all other traffic to 1:20.

The net result of this is that certain kinds of traffic are selected as high-priority, and are
transmitted almost immediately. All other traffic is grouped together and forwarded
according to sfq's round-robin scheduler.

This had a dramatic affect on my link's performance. My average round trip (as
measured with ICMP echo requests to and from my ISP's nameserver) went from 1800-
2000 ms to a striking 60-90 ms. The reason for this was that ICMP packets were not
required to wait with the rest of the outbound traffic. It was put at the head of the line
immediately. Note that this would not have been possible if the Cisco DSL router was
allowed to maintain the outbound queue.

Before applying this sfq-based conditioner, my per-task bandwidth usage looked like this:
? Desktop michael.mspencer.net: around 93 bytes/sec
? Bulk http downloads: 40.5 KB/sec
? FTP user pseudonym, retrieving one file: 9.5 KB/sec
? FTP user dev, retrieving one file with a download accelerator (four simultaneous
transfers): 4.6 KB/sec
? All other http traffic: 16.1 KB/sec
Grand total: 70.7 KB/sec. Because my upstream link can't actually send that quickly
some of that was being dropped, and the DSL router's queue was kept completely full.

After applying the sfq-based conditioner, per-task bandwidth usage looked like this:
? Desktop michael.mspencer.net: around 8 bytes/sec (demand was lower)
? Bulk http downloads: 52.2 KB/sec
? FTP user pseudonym, retrieving one file: 8.4 KB/sec
? FTP user dev, retrieving one file with four simultaneous transfers: 3.7 KB/sec
? All other http traffic: 868 bytes/sec (demand must have died down)
Grand total: 65.2 KB/sec. This is very close to, but not beyond, my upstream link's
capacity. No outbound packets were dropped by my DSL router, and the router's queue
stayed empty.

These results were mostly satisfactory, but with some problems. FTP user pseudonym is
capable of much higher transfer rates, and his transfer is more valuable to me than the
bulk http downloads consuming more than 80% of my available bandwidth. Upon
further examination, two http users were downloading eight to ten different sections of
the same file simultaneously, using a download accelerator (user-agent DA 5.0).

This reveals another weakness of the sfq queuing discipline: it weighs each connection
equally, so if someone uses many simultaneous transfers they will be allocated a
disproportionately high amount of bandwidth.

I then implemented additional classes and classifier rules for the kinds of traffic I wanted
to give lower priority to.

tc class add dev shaper0 parent 1:1 classid 1:30 cbq rate 69kbit allot 1600
prio 3 avpkt 1528
tc class add dev shaper0 parent 1:1 classid 1:40 cbq rate 69kbit allot 1600
prio 4 avpkt 1528
tc qdisc add dev shaper0 parent 1:30 handle 30: sfq perturb 10
tc qdisc add dev shaper0 parent 1:40 handle 40: sfq perturb 10

I also modified class 1:20 (normal traffic), decreasing its bandwidth allocation.
tc class add dev shaper0 parent 1:1 classid 1:20 cbq rate 371kbit allot 1600
prio 2 avpkt 1528

I then added filter rules for low-priority traffic, so those kinds of traffic would be handled
by classes 1:30 and 1:40. For example:

tc filter add dev shaper0 parent 1: protocol ip prio 13 u32 match ip src
209.180.104.203 flowid 1:30
This assigned all bulk-download http traffic to class 1:30.
tc filter add dev shaper0 parent 1: protocol ip prio 26 u32 match ip sport 3072
0xfc00 flowid 1:40
This assigns all traffic with source ports in a range common to a specific peer-to-peer file
sharing program to class 1:40.

Note that because these two new classes are not isolated, they will donate any bandwidth
they don't use to their siblings. Since the normal-priority class 1:20 is not bounded, it
will borrow bandwidth from siblings if it has more demand than it can handle alone.

After applying these changes my link's performance was mostly the same from my point
of view. My users, however, noticed a big change.

? Desktop michael.mspencer.net: around 8 bytes/sec
? Bulk http downloads: 12.7 KB/sec
? FTP user pseudonym, retrieving one file: 44.5 KB/sec
? FTP user dev, retrieving one file with a download accelerator (four simultaneous
transfers): 4.4 KB/sec
? All other http traffic: 1.0 KB/sec
Grand total: 62.6 KB/sec. My allocations didn't add up precisely to what they did
before, so less data was allowed to transmit. Overall performance was still excellent, and
bandwidth was allocated to more appropriate tasks.

One future improvement might be to create a new queuing discipline based mostly off of
the sfq source code. Even though my bulk http downloads were getting a suitably low
amount of bandwidth, that total allocation was divided up unfairly between two
download accelerator users and three single-transfer users. The modification I have in
mind would be to simply reimplement sfq with a new name, but make the new version
blind to port numbers. That way, if a download accelerator user tries to increase their
transfer speed by opening more connections they don't actually receive more data, and
don't gain any unfair advantage over the other users. For example, they might go from
one connection (6 KB/sec) to two connections (3 KB/sec each) to six connections (1
KB/sec each) but not actually receive any more data. Meanwhile other users with only
one connection are unaffected.

Many consumer and home-office broadband internet connections have more than enough
bandwidth for heavy-duty server tasks, but lack a sophisticated traffic management
system that enables reasonable performance under heavy load. Business DS-1 (T1)
connections already come with Fair Queuing enabled, but most administrators aren't even
aware of the difference. They do notice, however, that a home DSL line with one third of
the upstream bandwidth of a DS-1 tends to perform much worse under only one third of a
T1's normal workload. With the traffic management techniques discussed here, home
server owners can handle the same kinds of heavy workload as more expensive lines.


Michael Spencer Jr. Page 9 5/5/2003

throttled (3, Informative)

megabulk3000 (305530) | more than 10 years ago | (#8653503)

If the offending user's on OS X (which they probably ain't, but) they should install Throttled [intrarts.com] on their machine. That's what I use to keep my roommates from getting too pissed about pokey net connections when I'm "riding the donkey."

I used to use CarraFix, but Throttled whips the shit out of it.

I had to play around with the startup file for a few hours to get it working right. Here's my relevant modifications, if anyone's interested:
/usr/local/sbin/throttled -s $MAXSPEED -d 17777 -p 1 -d 17778 -p 2
#added another socket for mldonkey
/usr/local/sbin/throttled -s 5120 -d 5555 -p 3
# all rules below are for ipfw, there is many ways you can set this up.
# we have simplified this for new users by removing ip specific ipfw rules.
# this fixes isses for dynamic ip users, but if you want rules bound to
# a single ip you can use either of the examples below.
#
# the line below finds your ip automatically
# IP=$(/sbin/ifconfig $INTERFACE inet | /usr/bin/sed -n 's/^.*inet\ \(\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}\).*/\1/p' | tail -n 1)
#
# you can also specify the ip address by doing
# IP=192.168.1.7
# IP = any
IP=192.168.1.151
# default prioritized configuration (wincent.org style config)
# these rules allow http/https/ssh/telnet/smtp/aim/pop/irc/sirc
# to be prioritized by the throttle.
#
# Setting up the configuration this way catches more file transfer types
# and minimizes lag in response driven services.
# prioritize http/https
/sbin/ipfw add divert 17778 tcp from $IP to any 80 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 443 out xmit $INTERFACE
# prioritize ssh and telnet
/sbin/ipfw add divert 17778 tcp from $IP to any 22 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 23 out xmit $INTERFACE
# prioritize imap and smtp
/sbin/ipfw add divert 17778 tcp from $IP to any 143 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 25 out xmit $INTERFACE
#carrafix style
/sbin/ipfw add divert 17778 tcp from $IP to any 25 via $INTERFACE
# prioritize ftp directory listings
/sbin/ipfw add divert 17778 tcp from $IP to any 21 out xmit $INTERFACE
# prioritize aim or iChat
/sbin/ipfw add divert 17778 tcp from $IP to any 5190 out xmit $INTERFACE
# prioritize msn
# /sbin/ipfw add divert 17778 tcp from $IP to any 1863 out xmit $INTERFACE
# prioritize pop3
/sbin/ipfw add divert 17778 tcp from $IP to any 110 out xmit $INTERFACE
# prioritize irc and sirc
/sbin/ipfw add divert 17778 tcp from $IP to any 6667 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 6668 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 9999 out xmit $INTERFACE
# prioritize hotline and carracho "listing" ports (client end)
/sbin/ipfw add divert 17778 tcp from $IP to any 5500 out xmit $INTERFACE
/sbin/ipfw add divert 17778 tcp from $IP to any 6700 out xmit $INTERFACE
# prioritize hotline and carracho "listing" ports (server end)
# /sbin/ipfw add divert 17778 tcp from $IP 5500 to any out xmit $INTERFACE
# /sbin/ipfw add divert 17778 tcp from $IP 6700 to any out xmit $INTERFACE
#throttling mldonkey
/sbin/ipfw add divert 5555 tcp from $IP 4662 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 udp from $IP 4666 to any out xmit $INTERFACE
#carrafix style
#/sbin/ipfw add divert 5555 tcp from $IP to any 4662 via $INTERFACE
#/sbin/ipfw add divert 5555 udp from $IP to any 4666 via $INTERFACE
#throttling overnet (are all these necessary?)
/sbin/ipfw add divert 5555 tcp from $IP 4391 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 tcp from $IP to any 4391 out xmit $INTERFACE
/sbin/ipfw add divert 5555 udp from $IP 4391 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 udp from $IP to any 4391 out xmit $INTERFACE
#throttling news
/sbin/ipfw add divert 5555 tcp from $IP 119 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 tcp from $IP to any 119 out xmit $INTERFACE
#throttling bittorrent
/sbin/ipfw add divert 5555 tcp from $IP 6881-6999 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 tcp from $IP to any 6881-6999 out xmit $INTERFACE
#throttling fasttrack
/sbin/ipfw add divert 5555 tcp from $IP 1214 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 tcp from $IP to any 1214 out xmit $INTERFACE
/sbin/ipfw add divert 5555 udp from $IP 1214 to any out xmit $INTERFACE
/sbin/ipfw add divert 5555 udp from $IP to any 1214 out xmit $INTERFACE
#throttling gnutella
#/sbin/ipfw add divert 5555 tcp from $IP 6346 to any out xmit $INTERFACE
# bind to throttle low priority services.
/sbin/ipfw add divert 17777 tcp from $IP to any out xmit $INTERFACE

Cisco has declared the 675 router dead. (1)

Futurepower(R) (558542) | more than 10 years ago | (#8653761)


Cisco has declared the 675 router dead, and stopped supporting it. Before they declared it dead, there were frequent security upgrades, giving the impression that it might not be secure now. Cisco had bought the 675 technology from another company; it was not designed as a Cisco product.

So, maybe it would be sensible to buy a new router, and maybe that router would have load balancing. SMC [smc.com] seems to be a reputable company, but I don't see any SMC routers with balancing.

Re:Cisco has declared the 675 router dead. (1)

darksoulz (700833) | more than 10 years ago | (#8659521)

I know someone that's had numerous pieces of SMC networking equipment and hasn't had any good luck with any of it. He warns everyone to stay as far away as possible.

You didn't say how many units (0)

Anonymous Coward | more than 10 years ago | (#8653777)

but it's less than 24. Evev if it's only 12 units, that's still a lot for 1 DSL line. Get your own line and call it a day.

Or get one of these. [overclockersclub.com]

Give the RIAA a call ? (3, Interesting)

Alex (342) | more than 10 years ago | (#8653913)


Alex

Re:Give the RIAA a call ? (0)

wtlssndlssfthlss (747938) | more than 10 years ago | (#8654075)

Don't you mean the MPAA? I'm sure the RIAA would love to get their hands on him, if he were eating that much bandwidth with music, alone...

SHADDDDUP !!! (0)

Anonymous Coward | more than 10 years ago | (#8739371)

Someone should slap you for that comment.... The RIAA is a bunch of theives.... You are ignorant for helping them to rip off your favorite artists... Well and you as well.

Have the Windows user run Netlimiter (1)

Hobart (32767) | more than 10 years ago | (#8653945)

If it's one Windows user and you don't have the time/resources to set up a free-Unix bandwidth shaper, you can ask the offender to run NetLimiter ... it costs money, but works great, and even improves transfer performance (If you cap your upload and download a few percent below the actual maximum capacity on the line, it doesn't back off and have to retransmit dropped packets from bandwidth overage). Google for it, I think it's at http://netlimiter.com

Re:Have the Windows user run Netlimiter (2, Informative)

Nogami_Saeko (466595) | more than 10 years ago | (#8654417)

Netlimiter is good for running on an individual machine (I run it myself to prevent my mailserver and HTTPD from eating all my upstream), however there are better windows solutions for gateways.

http://bandwidthcontroller.com/

Is a fairly decent gateway traffic shaper - not quite as configurable as linux solutions, but fairly easy to set up and you can limit by a number of options, port, protocol, etc.

Free trial version to so you can see if it works for you. $50 to buy.

N.

Some problems with traffic shaping (1)

kompiluj (677438) | more than 10 years ago | (#8653952)

I have done traffic shaping with FreeBSD/ipfw2 and found out the hard way that some viruses that initiate a lot of connections can take up unproportional share of bandwith. For instance on of the users has had a virus that was making roughly about 700 thousand outbound connections daily, but not causing much traffic, since all connections were single UDP packets. After we have disconnected it from the hub the overall response time and transfer speeds for other computers have increased.
Apart from such queer incidents, which I think are inherent to all bandwith controlling schemes, including HW firewalls, I would definitely go for a Linux/FreeBSD/OpenBSD solution. The HW requirements are not high - we are running 100 MBps connection for about 500 users with a "legacy" double pentium II 400 computer.

Easy solution, provided you plan & spare some (-1)

ReluctantBadger (550830) | more than 10 years ago | (#8654297)


1. Go here [soekris.com] . Buy a net4801, a case, flash card and power supply for a few hundred dollars. Alternatively, an old 486 with some quality Intel/3COM NICs from your local 2nd-hand shop.

2. READ THIS [openbsd.org] .

2. Buy and install this [openbsd.org] .

3. Read this [openbsd.org] , this [openbsd.org] and this [openbsd.org] .

Trolling aside, Linux has got its places but if you want to do things right in a scenario such as the one you describe, OpenBSD is the only smart choice.

Dummynet is the way to go! (1)

lth (145996) | more than 10 years ago | (#8654697)

Dummynet [unipi.it]

Quote from the above linked page:
Unlike other traffic shaping packages which run in userland, dummynet has a very little overhead, as all processing is done within the kernel. There is no data copying involved to move packets through pipes, just a bit of pointer shuffling, and the implementation is able to handle thousands of pipes with O(log N) cost, where N is the number of active pipes.

All you need is an old PC, two NICs. You can boot Dummynet (running on PicoBSD) from a floppy..

Linksys 54G wireless router (1)

AMystery (725537) | more than 10 years ago | (#8654995)

Remember how it is based on linux? there are several wonderful replacement firmwares for it that give you some filtering options, you probably don't need the wireless part, but its what I've been deploying lately. check out the simandhi firmware and look at sveasoft.com. WRT54G as I recall, very wonderful little box and now you can make a hotspot and sell access to it also. the router isn't expensive, $80US I think. If you know enough, you might can do the filtering in straight iptables instead of the web interface, but that is a bit beyond me. good luck!

www.Freesco.org to the rescue... (1)

vertical_98 (463483) | more than 10 years ago | (#8655325)

get a 486/66, slap in 2 Nics, d/l freesco [freesco.org] and install it. Search the forums for bandwidth limiting. [freesco.org]

The biggest issue I have had with freesco was a) bad floppies and b) finding supported nics. 3Com 3C509s and 3C905s both work great. On the ISA ones make sure you turn off PnP.

I've used this product for over 2 years without an issue. I'd reboot it once a month just because, but I can't think of a time I had to.

Good Luck
Vertical

MIkrotik? (1)

Oliver Wendell Jones (158103) | more than 10 years ago | (#8655448)

You can download a series of floppy disk images and turn just about any old PC with two NICs into a router with all sorts of limits, including P2P Filtering! www.mikrotik.com [mikrotik.com]

Shorewall has traffic shaping built in (1)

harryk (17509) | more than 10 years ago | (#8657276)

Shorewall has traffic shaping built in, but what it sounds like you might prefer to do is put in a Quality of Service system. Just reduce the priority of the outbound traffic, or block it all together, depending on how strict you want to be.

I've successfully down this to allow bittorrent transfers to take a lower priority than my VoIP traffice from my phone. It seems so far to have worked quite well. I had some trouble getting the qos-htb and tc qdisc stuff to work. Possibly because of the versions, but didn't bother trying to figure it out. Shorewall worked quite well for my purposes.

Check out the LEAF project over at leaf.sourceforge.net, its a very active project with great user and developer support.

I use FreeBSD 5.2.1 + pf/altq (1)

var1ety (662348) | more than 10 years ago | (#8657342)

For those that don't want to migrate to OpenBSD FreeBSD 5.2.1 has support for OpenBSD's pf and altq via a port, although you need to patch your source tree by hand. FreeBSD 5-current has fully integrated support for pf and altq, although I would wait for FreeBSD 5.3-RELEASE, rather than trying to use current. I personally found ipfw and the queueing subsystem extremely hard to use. That said, FreeBSD 5.2.1 on my k6-233 works great for our LAN. I use it to prioritize dns/www/smtp/pop3/imap, put leechers into their own bandwith limited queue, and set aside ~4kb/s upload for priority stuff. It has had an amazing effect - lag is completely gone, and leechers don't affect the latency of the LAN's connection anymore.

Re:I use FreeBSD 5.2.1 + pf/altq (1)

thrruss (766294) | more than 10 years ago | (#8701339)

Hi! How do you manage your traffic priority? I have the same problems with P2P. Do you know a good doc? Thanks very much

don't limit when quiet; the PRIO chain (1)

jago25_98 (566531) | more than 10 years ago | (#8657616)

All this is excellent for corporate scale infrastructure,

but it's a lot of work for the everyday DSL people who have a brother who runs eDonkey 24/7.

Really it would be nice is something was available to balance all ports equally so that:

-=WHEN THE BANDWIDTH IS FREE IT IS NOT LIMITED=-

I guess the "prio" chain may be help with this. It isn't as well documented as htb. If anyone can figure out how to balance everything in as little lines as possible using something like prio, please share it as that would save us all some time grappling with htb and cbq.

Maybe a little late here (1)

UserChrisCanter4 (464072) | more than 10 years ago | (#8657789)

Get a Linksys 802.11g Wireless router. Because the firmware is just a customized linux kernel, and Linksys finally GPLed out their code, there's a fairly active community that's into hacking the firmware code to add all sorts of functionality that Linksys never considered, including QoS and Packet Shaping.

Just lock the ports for all of the popular P2P apps that have fixed ports down to 50kbps up and down, and call it a day. If I was on a shared DSL, I'd completely understand this, and even appreciate that a way to stop stepping on my neighbors toes had been implemented FOR me. If this guy's a jack-ass and he starts playing games futzing with ports on the apps that allow it, kick his ass to the curb and tell him that he can get his own service if he needs that sort of bandwidth.

A mix of hardware and software (1)

natefanaro (304646) | more than 10 years ago | (#8659220)

The hardware being your boot and the software being his ass. Not only is his downloading slowing you down but with all the RIAA crap going around it could be a legal liability. You have to think that if something happens legally, will this 15 year old behind a router going to get sued? Or the registered name/owner of the DSL service?

nevermind the bandwidth... (1)

fist_187 (556448) | more than 10 years ago | (#8685415)

it sounds to me like your first concern should be that 24 households are on the same hub.... i for one wouldn't feel comfortable knowing that anyone else in the area could just open up ethereal and check me out. look into buying a 24 port switch for privacy's sake; i'm sure your neighbors will be more than willing to chip in for one if you explain how they differ from hubs.

as for the bandwidth issue, 24 households sharing a single DSL line is a bit of a stretch, especially if some houses have several computers in them. it seems strange to me that people don't want to pay for their very own DSL line but at the same time expect good bandwidth!

bottom line? you will probably not be able to do this cheaper (in the long run) than everyone getting their own DSL connection. of course, since you already have the closet with the hub and the cables run, you could set it up so that small groups of neighbors can agree to split a single DSL connection -- presumably those with low-bandwith habits. just buy a small hub/switch for each group (or a managed switch that does VLANs). that way, the bandwidth hogs can pay for their own line and the rest of the users can get a lower per-month cost by collaborating.
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?