Bad Behavior on the 'Net - Who Pays the Bandwidth Bill? 654
rakolam asks: "I am involved with network management in the hosting department of a fairly large ISP. Constantly we have customers who dispute inbound bandwidth spikes and demand service credits on their burstable connections. Events such as the Slammer Virus literally have everyone knocking on their salesperson's door at the end of the billing cycle. My position is that the internet is a public space, and by placing themselves in that space, one has to realize the consequences (and the implications of burstable billing). I'd like Slashdot's perspective on this. Should ISP's ultimately eat the costs of malicious behavior? Is the customer ultimately responsible for the bandwidth they've generated, regardless if it's desired or not? Is this a new frontier for insurance companies?"
analogous to water/electric company IMHO (Score:5, Insightful)
Re:analogous to water/electric company IMHO (Score:5, Funny)
-prator
Re:analogous to water/electric company IMHO (Score:5, Insightful)
Do you then go ask for a credit from the utility because of the excessive/unexpected use?
Re:analogous to water/electric company IMHO (Score:5, Interesting)
Of course my small scale situation may not translate to a large business account.
Re:analogous to water/electric company IMHO (Score:5, Informative)
The original question though is what should the ISP have done. IMO they should have firewalled access to the affected ports and then split the cost.
Re:analogous to water/electric company IMHO (Score:5, Insightful)
If you hold the customer responsible, then people angry with that person can just drive up that person's cost by choosing to flood him.
Re:analogous to water/electric company IMHO (Score:3, Interesting)
If you flood, you pay
If you get hacked and your machine used for flooding, you pay (afterall its your own fault your machine was insecure)
If you GET flooded, then you take it up with your isp and take action against the culprit.
Re:analogous to water/electric company IMHO (Score:4, Interesting)
Re:analogous to water/electric company IMHO (Score:3, Insightful)
Exacly. Not even a large account. If you shut me off for the rest of the month, I've got a problem. I need to have my site accessible. I just want to pick and choose which access (legitimate) I want to pay for.
Someone else said the ISP should firewall off the "bad" traffic. Does the ISP then complain to its upstream provider about that bandwidth? Someone has to either pass on the cost of that bandwidth or eat it.
Where do you draw the line? You could argue that your ISP has no business charging you for inbound UDP packets to SQL server port (1443 was it?) since you expect to only provide http on port 80. Next month there is another virus/worm that causes another spike, but this time by flooding the net with bogus TCP traffic on port 80. Now do you try to get your ISP to take that off your bill because it was from a virus/worm?
Re:analogous to water/electric company IMHO (Score:5, Informative)
Most colo providers I'm familiar with bill on 95th percentile bandwidth, which means that they drop the top 5% of samples (typically 5-minute average) and bill you for the bandwidth of the highest remaining sample. This means that you can absorb short-term heavy bandwidth spikes without being charged, up to about a day and a half worth of time per month.
In any case, the ISP should have no way of knowing WHAT traffic creates the bandwidth spike, unless I specifically request that they monitor my port. Of course, smart ISPs will exploit these incidents by offering firewalling services as a value-add, even if it's just stateless filtering at the router, as a way for customers to "insure against unexpected traffic spikes from virus/worm activity".
Of course, if I was paying for virtual web service, rather than a server colo and bandwidth fee, I should not be charged for non-web traffic, and I doubt any ISP would have the balls to do so.
Re:analogous to water/electric company IMHO (Score:3, Informative)
Re: maximum exposure (Score:3, Informative)
A telephone company would build a system for anticipated peak service and would add some room for expansion. As a result, the telephone company would build an expensive system with excess capacity.
Although costs were fixed, telecom companies would bill customers for time used. To do this, they would set a rate for normal usage that would be high enough to cover the costs of the peak usage network.
I imagine that the Internet is somewhat the same way. Internet companies build for peak usage and set a rate for normal usage that will cover the cost of the peak usage network.
The thing that happens in a DOS attack is that the DOS attack pushes the services used from the normal level to peak usage levels for a prolonged period.
Since most of the network's costs are fixed, the DOS attack really doesn't cost the network that much more. A DOS attack doesn't spontaneously generate more routers and fiber optic connections.
The end effect of the attack is that it screws up billing. Remember the normal usage rates are set high enough to cover the cost of peak capacity. The DOS attack creates a situation where the end user is suddenly being charged the rate calculated for normal usage at the volume of peak usage.
Now, I realize the Internet has an extremely layers of service provides. Many ISPs are just a middlemen paying metered rates. The ISP is caught in the same trap of screwed up billing. The cost of the ISP providers didn't go up during the attack.
The big bills for both the ISP and end user are the result of flaws in the billing and metering processes and not actual higher network costs. The challenge is to keep the charges from the DOS attack from screwing up the billing systems.
BTW, I do not mean to imply in this thread that DOS attacks are cost free. Just that the bandwidth consumed during the attack is really not costing the network that much more. The machines, cables and wires have more stuff going through them. The DOS attacks cost the the support people in the ISP time, and have a cost in lost opportunity, they also create billing nightmares. The DOS attack does not actually cost the real dollar amounts that suddenly appear on bills.
Re:analogous to water/electric company IMHO (Score:3, Insightful)
For unexpected use, of course you can't demand a freebie, since it is understood that the fountain is for public use. However, suppose someone presses the button on the fountain and holds it for several hours without drinking anything. This seems like theft, to me.
Any service offered to the public has certain bounds within which it is expected to be used. People should have the authority to prevent others from abusing their services.
If someone is DOSing me, and I have no authority or technical capacity to stop their attack, then why should I pay for someone else's criminal behavior? If I immediately pull the plug on my network, call up the ISP to inform them, yet the packets still come cascading in... I have acted in good faith to do everything possible.
The current situation is like being able to watch the guy pressing the button on the fountain, and paying for the water, yet not being able to do anything to stop it. How can that be *my* fault?
Re:analogous to water/electric company IMHO (Score:5, Informative)
Re:analogous to water/electric company IMHO (Score:4, Funny)
Holy cow, that circus next door, it's not free?!
Re:analogous to water/electric company IMHO (Score:5, Insightful)
I lean towards the consumer not having to pay, considering they didn't request the traffic and are therefore not resonsible for it.
Re:analogous to water/electric company IMHO (Score:5, Insightful)
We don't live in an (entirely) communist world. We don't get to pass out resources indiscriminately. We have a fixed amount of resources, and as with any case of supply and demand, the person holding the supply can (and should) charge for using the resource. In the case of network bandwidth, the resource is not obvious, but it is still tangible: It is network equipment and opportunity costs.
Re:analogous to water/electric company IMHO (Score:3, Insightful)
You can also use the analogy of junk faxes. Your machine is set up and the number is available for anyone to call, but people can be prohibited from using your resources by sending you junk faxes.
Though with out specific laws it probably comes down to contract and at that point it is probably buyer beware, whether you agree with it or not.
Re:analogous to water/electric company IMHO (Score:3, Insightful)
What you are advocating is like claiming that you should pay the phone compnany for every time someone calls your phone, even if you don't answer it, even if you leave it off the hook, even if you leave it unplugged.
Re:analogous to water/electric company IMHO (Score:3, Insightful)
A virus or other Internet contaigon could come from somewhere waaaay outside your jurisdiction. If some server in China is constantly bombarding your incoming pipe with virus activity, bogus web requests, port scanning, etc. then you're stuck footing the bill.
With all of this said, I think ISPs should provide some sort of insurance to their burstable customers. You could get so much bandwidth per billing cycle but leave room for error in the event your customer can verify that they received "hacker traffic" or somesuch. Perhaps even build in clauses that say the end-user is required to notify the ISP of problematic access within a certain timeframe, that way they can take action further up the pipe to block said packets.
If a user, however, comes up at the end of the month and complains about lots of unwanted traffic, well, hire an admin to look after your connection and come see us next month.
Re:analogous to water/electric company IMHO (Score:5, Insightful)
This would be like dealing with stolen credit cards. When a credit card is stolen the owner gets 24 hours to report it and is only liable for $50. If they wait up to 72 hours, they are only liable for $500. I'm not sure what happens after that. This system protects both the credit card company and the credit card user by insuring prompt reporting of stolen credit cards and fraudulent activity (and can hopefully catch the crook). This system has worked fairly well.
The implications for ISPs and their customers for a similar system would be pretty interesting. The customers who actively monitor their network traffic and help to head off problems would be rewarded by being less liable for damage, while ISPs would be free to give the full bill to those who ignore their bandwidth usage. This system should lead to lower costs for the better customers and discurage neglegance possibly leading to better service for all.
Re:analogous to water/electric company IMHO (Score:3, Insightful)
The only way to really take care of this is to put a firewall in front of the box doing the metering. If the firewall rules are written properly, things like the MSSQL bug won't make it past the firewall.
Re:I say charge the customer (Score:3, Insightful)
If they're being charged for incoming bandwidth (especially incoming UDP bandwidth like the slammer worm) then shutting off their server will not help.
As long as the router continues to send those packets to that IP, they'll keep getting those packets. It doesn't matter if the packets just fall off the end of an unplugged cable -- incoming bandwidth is incoming bandwidth is incoming bandwidth.
If I sent a huge SYN attack to your home DSL connection, and your machine crashes, are you responsible for the bandwidth before your machine goes down? Are you responsible for the bandwidth after your machine has crashed, but before the ISP's realized you're not on the other end anymore?
Re:I say charge the customer (Score:5, Interesting)
For one thing, the packets go down the wire wether the service is running or not. Thousands of requests per second to a box that isn't running the service still has to respond and say "sorry, not running here". Even if it's a few bytes per, it adds up quickly.
Should a customer be charged for requests coming in for a service they don't offer? No, that's the point of the firewall (or packet filter really).
ISPs could have a new revenue stream by looking at this problem differently.
They can offer a firewall for a per-month fee and waive any bandwith increases as a result of DDOS attack or other work-checking that could be blocked by the firewall. An active firewall could proxy HTTP requests, also filtering out common IIS exploits.
User doesn't want the firewall? Fine, you're responsible for all charges.
This would at least give end users an option instead of what will border on collusion when all the AUP/TOSs change to read the same thing.
Re:I say charge the customer (Score:3)
Where's the incentive for the ISP?
Whats to stop e-embezzelment? (Score:3, Interesting)
No law against this. It like me providing you with a doorbell service. If I want more money, I just keep pushing the button. If you were dumb enough to sign up for this then you'd better trust me.
Bandwith insurance, like health care? (Score:4, Insightful)
In health care, you have a pool of people, really sick, regular, and extra healthy (hold the fries)
As long as there are not too many sick people, the cost can be spread over everyone in the pool.
But when there are too many sick people, it does not work, and someone is left to pay the bill.
But as rdewald draws a comparison to utlities, I agree that bandwidth should be more like a utility.
But frankly, it is at least an order of magnitude easier for someone to maliciously use your bandwitdh than use your water or electricity, or even your POTS line. You have to be physically present there. Obviously in cases of bandwidth theft or malicious consumption, that is not true.
Lastly, to go out on a limb, IMHO, personal computer and network technology is still not ready for home use. We would really like to think do, but it seams that we are still at the point where autos were in the hand crank era. You gotta be or be related to a mechanic to own one. They are still really complicated machines that we geeks love. Now that is improving in some areas, such as open source operating system integrity and useability, but worse in others as there are few end users who really understand security issues. Can we draw an analogy to health safety with health care/ health insurance? You tell me.
Well, thats my 2 bits.
Re:analogous to water/electric company IMHO (Score:3, Interesting)
1. If someone floods my house with water or punctures my pipe, they pay not me.
2. If the gas company has a leak and blows my house up they dont get to bill me for the gas (although famously gas companies have tried to do this to people!)
If you bill people for incoming traffic you have a problem, and its going to make a nasty mess when it hits, be it by losing all your customers, whatever.
If you bill people for outgoing traffic with bursting do your customers a favour, you've got traffic shaping so let them set maximum billing costs. The customer can relax a lot more if they know the "worst case bill" for each month and suffer nothing more than loss of burst when its exceeded.
You can even have sales phone them and try and sell them more burst bandwidth. All of a sudden your caring ISP wants to offer you some extra options instead of customers phoning the evil bastards at the ISP who scammed them, two perceptions for the same thing. The difference is the customer has the control so feels happy
This is the same whole reason that an ISP who shapes customers who exceed a bandwidth cap right down does better than one who goes around cutting people off. Given then 1/2 speed at 75% usage, and 28.8 at 100% and they are normally happier than getting the boot.
Charge on sent traffic. (Score:5, Interesting)
Alas, unless every ISP participated, this model wouldn't work well.
Re:Charge on sent traffic. (Score:3, Interesting)
When I thought of getting a burstable line from Digex, their billing process was to bill my incoming/outgoing data rate based on my peak usage EXCLUDING the top 10% of our usage time. That way if there's a usage spike (or a SQL Slammer spike), then it would be considered an anomaly and wouldn't be billed for. That seems like a rather fair system for me, since there's no real way to distinguish wanted traffic from unwanted traffic and bill based on that.
Fairer - sent or solicited - a modest proposal: (Score:5, Interesting)
Good idea but it doesn't quite go far enough.
You should be billed for the traffic you CAUSE or SOLICIT, and thus have control over. Much of internet traffic is things like web browsing, which invovles a small request soliciting a large reply. If you suck down 60 megabytes of web porn, MP3s, or ftp downloads, it's your bill. Similarly if you host a server, which accepts little requests and pours out data, it's your bill.
But if somebody starts sending you unsolicited packets, that's like somebody making nuisance calls or pages. (You will notice that pagers, at least, are generally NOT billed by the page. They tried that, and the customers rebelled because they had no way to block idiots with autodialers.)
So something with a little deeper visibility is in order. Here's a fair approach:
TCP: You get billed if you make, attempt to make, or accept, a connection. You don't get billed for attempted connections you refuse or that don't get completed (i.e. SYN and other DOS attacks).
UDP: You get billed for outgoing UDP packets. If the billing machine is sufficiently stateful, you might also be billed for incoming UDP packets that ARE replies to a recent outgoing UDP request using a well-known UDP request/reply protocol. (This would prevent cheating but still protect you against getting billed for both DOS attacks and forged-reply billing attacks.)
ICMP: All are free except outgoing EHCO REQUEST (ping), because they're a mandated part of the network overhead. (You don't want to bill inbound ECHO REPLIES to prevent billing for forged reply attacks. But you might bill ECHO REQUEST as if it went both inbound and outbound, to cover the expected ECHO REPLY without making the billing machine stateful about ping "connections".)
That should pretty much cover it. Customers would:
- be fairly billed for the bandwidth they used, caused to be used, or allowed to be used,
- not be billed for unsolicited "phone calls", DoS attacks, or mandated network overhed, and
- have a strong financial incentive to keep their system secured against crackers and malware (such as viruses and worms).
And installing a get-around-the-billing hack (like PPP-over-ECHOREPLY) would be a violation of terms-of-service and cause for disconnection - or changing the billing of that customer back to "all bandwidth co$t$" B-)
A Blend of the two? (Score:3, Insightful)
The real question becomes where do you set the line? But that could be determined by the average user usage, perhaps a study could be done over the course of a few months to see where people fall on this whole thing.
RonB
Re:Charge on sent traffic. (Score:2)
Huh? Since when [digitalroutes.co.uk] do web servers have to be on port 80? Same with FTP.
Re:Charge on sent traffic. (Score:3, Informative)
And you post this from hotmail? Are you just trying to supply a counterexample in the same breath?
When I worked at AOL, OC48 installations were a regular occurrence.
Users just won't pay (Score:5, Insightful)
Re:Users just won't pay (Score:5, Informative)
The CC company doesn't eat that. The vendor does for accepting the stolen card
The customer always pays (Score:4, Insightful)
Re:The customer always pays (Score:4, Insightful)
But it's not true. If McDonalds loses $80 in a lawsuit to somebody burned with hot coffee, they *can't* just raise their prices to recoup; their prices were already set to maximize profit before. So what gives? Profit. McDonald's shareholders lose, not the public at large.
Simple policy (Score:5, Interesting)
Any massive bandwidth they log after that, is their responsibility. You notified them, and they did not listen.
After a few incidents like that, they will start to listen to your warning messages.
Re:Simple policy (Score:5, Interesting)
This is a thorny issue. The real answer is that the twit whose server got owned and is spewing garbage out on the net should be responsible for paying. But enforcing that is going to be a problem.
Re: Simple policy (Score:4, Insightful)
Re:Simple policy (Score:5, Insightful)
I don't know what the solution to the problem is exactly. As it stands now I pay for any bandwidth used regardless of how or why it was used. It would be much better if those charges could be passed along to the person responsible for abusing your bandwidth, but how that could be enforced is beyond me.
One thing I have to note here is that the person posing the question is talking about INBOUND spikes not outbound. So your points are even less relevant.
Re:Simple policy (Score:5, Interesting)
If you are hosting business internet lines give the customers 2 options.
1. Wide open internet. Nothing is filtered on the ISP end, as it stands today, and the customer is 100% liable for ANY traffic circulating between the internet and the customer, solicited or not.
2. Abuse Managed Internet. Charge a fee to the customer per month, which get the customer:
- Any abuse, aka DOS attempts removed from the monthly bandwidth
- The ISP will filter abuse attempts before they occur, so if there is a code red floating around, allow a transparent proxy / firewall throw the packets away before it causes your customers harm.
The trade off for the customer is more assured price, and quality of service for the price of flexability and a nominal charge.
It's not the ISPs responsibility (Score:4, Insightful)
That depends on what service he has with you (Score:2, Insightful)
Communication (Score:2)
On one hand, if the ISP says that it is not accountable for attacks and internet slowdowns that it has no control over, then the people shouldn't expect anything when they happen. On the other hand, if the ISP uses this communication as an excuse not to protect itself properly against such attacks, then the customer should take his buisness elsewhere or be properly reimbursed for their losses.
It's in the contract (Score:5, Insightful)
In other words (Score:5, Insightful)
-Peace
Re:In other words (Score:5, Interesting)
*/. did NOT warn the page
*The page in question NEVER receives the amount of traffic necessary to bring it down.
*Let's assume it happened on a Saturday, when they had minimal support
*The company can PROVE they lost revenue.
Re:In other words (Score:3, Funny)
proof of malicious intent (Score:4, Insightful)
While
The day anybody becomes liable for linking to a page on the internet will be the end of the world wide web...that's the whole premise of the thing...
The only thing I can think of is something similar to the robots.txt file...have your webserver have a slashdot.txt file that says something like NoSlashdotLinkage = true in it or something, anything similar to the thing for preventing search engines.
contract... (Score:3, Interesting)
Since DiffServ and other standards based solutions are ready to be implemented, perhaps you should consider talking to your most whiney clients about it?
Yes I know it doesn't apply to all clients, and not every provider has the extra router/switch cpu power to implement them on all links...
But wouldn't such a solution be a good way to keep the more demanding clients(increasing the value they get: bandwidth for the right traffic) and decreasing the tax hackers and Distributed DOS and misconfigured systems make them pay (for undesirable traffic). Maybe you should suggest this as a customer retention measure, for those clients where it makes business sense.
Look at your audience! (Score:2)
Your asking this of slashdot? The literal definition of the slashdot effect?
Heads up. (Score:2)
I propose that ISPs who wish to charge by the byte need to develop a systemtray icon (or equivalent) that allows the user to see the accumulated traffic. Then there won't be any (or as many) surprises.
The place where I colo.... (Score:2, Interesting)
I was happy they cared and they where happy to have me care enough about them and me not to run M$.
To eat or not to eat (Score:4, Interesting)
simple (Score:3, Insightful)
In fact, I'd prefer a pricing model that is fixed for inbound and metered on the outbound. It puts a financial burden on spammers, copyright violators and the tragic/stupid victims viruses. On the other hand, if you've got something to sell, you should be more than happy to pay for bandwidth used to move that merchandise.
This is a good question! (Score:2)
As for slammer, the idiots running the servers with open ports to the databases should pay for their bandwidth - serves them right. Hell, they're already wasting money licensing the World's least secure web server, so why not throw a little more into the trashcan?
Balanced response. (Score:5, Insightful)
Inform them that if they ignore those suggestions, and future problems end up costing them money, then they'll have to foot the bill.
This way, the customer walks away happy and informed, and if they're really willing to be a good net citizen, they won't come back crying.
If they're not willing to do what's required of them, they'll get stuck paying for it.
Re:Balanced response. (Score:3, Insightful)
it depends... (Score:2)
As far as I know, which is very little, there is no such thing. You get 2gbps and that's the end of it.. there's no such thing as "it's burstable to 10gbps..yada yada yada".. but why is the poor guy who can barely afford the T-1 getting penalized?
Just my opinion.. everyone has one.. I got more than most..
---
You can't judge a book by the way it wears its hair.
OT: What makes up bandwidth costs? (Score:3, Interesting)
Can someone give me an idea of where the price for bandwidth ultimately comes from?
Different cost model (Score:2)
I would propose that content providers be given free bandwidth provided by the telcos since, after all, they are the reasons people like me pay for broadband. In effect, the consumers will subsidize the cost of the content providers. After all, that's what you really pay that $20-50/mo for... The content!
Liability = Incentive to be vigillant (Score:2, Insightful)
It also would cause Individuals to generate greater pressure on Distributors to get patches out and visible to the general public. If the general public took more of an interest in internet security, there'd potentially be much fewer DDos Zombies out there.
There's nothing quite as eye-opening as a huge bill sitting on the table staring back at you.
And that's my 2 cents.
Throttle (Score:2, Interesting)
It Depends (Score:3, Interesting)
If you want to keep that customer, you do what it takes to keep the customer. Remember the golden rule, 1 bad customer experience gets passed onto 20 people. If you think that this customer is going to put with this, fine go ahead and charge them. If you don't you should suck it up. If they leave, not only will the money that you get from them goes to zero, but they will bad mouth you to enough other people that it does have a negative impact on you attempting to acquire more customers.
In other words, be a good guy, suck it up and the customer will trust you more the next time you attempt to raise their bill. Blow them off and the only that you might get from them is the finger.
Re:It Depends (Score:3, Insightful)
If they're part of an ISP, they probably have already got FINGERD.
Monitoring and Opting Out (Score:5, Interesting)
We adjusted our monitoring process to detect these spikes early and contact our ISP to deny traffic from the offending subnets. Luckily, our ISP was willing to do this, even though they still incurred traffic from inbound packets. Luckily, these attacks originated from a few subnets that could be isolated.
As a further kludge, we eventually disabled ICMP altogether on our routers, and lived without ping and traceroute.
Having a host on the net is a risky proposition. You pay for inbound and outbound traffic, regardless of the source, packet type, or quantity. DDoS attacks can not only prevent your server from being accessable, they could literally bankrupt you if you become a target and don't take preventative measures.
Hmm... One click bankruptcy. I wonder if anyone has tried to patent this yet...
Our ISP was technically capable of detecting and thwarting various attacks. Ultimately, the policy of monitoring and contacting an ISP when traffic exceeds a certain threshold seems like a workable solution for average co-locaters.
Given the architecture of the Internet, it's difficult to see how we could shift the burden to pay away from the server to the client. It seems like a problem remarkably similar to the problem of spam.
Bad business (Score:5, Insightful)
I liked the analogy someone else came up with, such as someone running an extension cord from your house to theirs. Who is responsible here?
If I had hosting with your company, and the slammer bug hit servers that your sys admins failed to update, then you better eat that burstable bandwidth bill or a lawsuit couldn't be far behind (depending on the amount, of course). If the servers were my responsibility, including keeping them updated, etc, then I could understand your reasoning.
If a DDoS attack cripples my site, and you expect me to pay for that, you're sorely mistaken.
The simple fact is if they caused it, they paid for it. This includes patches/fixes the customer should've implemented. If you run and maintain that server for them, then no bill increase should be applied.
If someone out in the world caused it, a random malicious event that they just so happened to be on the brunt end of, just throw away that burstable bandwidth bill and make sure your customer knows you did them a favor.
It may not be your place as to pay for that second scenario, but you'll keep your customers longer, keep them happier and keep word of mouth on your company going strong.
It's just good business. Were this my company, I would never even think of treating customers this way.
How badly do you want to keep the customer? (Score:3, Insightful)
You need to ask yourself- how much did the excess bandwidth really cost, and how much is this customer worth to me in the long run? Probably, keeping that customer will make far more impact on your company in the long term than if you charged them, pissed them off, and inspired them to switch to another ISP.
Look at it historically (Score:5, Interesting)
Here's the problem Jerky... (Score:3, Informative)
You can have the most sophisticated firewall on the planet, but due the immutable laws of IPv4 you can NOT drop a packet until you see the packet. At which point you've already used the bandwidth (and incurred the cost) required to transport the packet that you're just going to drop.
This has nothing to do with patching your server. If you don't patch your server, and you get hit with a worm, and your box starts consuming huge amounts of bandwidth to attack other hosts, then it's your fault, and its OUTBOUND traffic, and you absolutely should pay for it. But having your server patched does not stop you from receiving inbound packets. They may not harm your server when they get to it, but you already paid for the transit.
BTW, This is why it's illegal for a telemarketer to call you on your cell phone. Because in theory you had to answer the call (and incur expense) BEFORE you knew who was on the other end.
This is a similar issue, except that we're not talking about telemarketers... which are businesses that more or less follow the rules. We're talking about script kiddies that don't care about the rules. Or in a worse case, we're talking about a competitor, or enemy, or rival that just wants to DOS you for a month until you go out of business because of all the excess bandwidth charges you're paying!
The technology limits the liability of the consumer. The ISP must take some responsibility here and put systems in place that protect the consumer.
-JE
Re:Here's the problem Jerky... (Score:3, Informative)
Of course this is for leased lines, not metered bandwidth in most cases, but the concept remains the same. We watch our own backyard, when something happens we react and get the problem resolved. If one of our cable modems is spamming or spewing slammer all over the Earth, we notice and shut off the offender. If we didn't care to look, we would get negatively impacted, just like the guy that doesn't notice his machine spewing out slammers, or nimda, or getting slashdotted.
Take an active role in your internet usage and you are largely immune to this sort of billing. You are responsible for your own stuff, if you aren't taking care of your stuff, I sure as hell shouldn't be expected to eat the cost.
It is YOUR FAULT if you get four hundred and eighty million hits. You put up the site. If you get slammer, you should have patched. Quit crying about your bill and administer your system.
Ounce of prevention, blah blah blah.
Utility Billing.. (Score:3, Interesting)
95th percentile model anyone? (Score:3, Insightful)
Our upstreams bill us this way, and all of our burstable downstream customers are billed this way. It works well that way.
To play the game, let's force ISPs to a few rules (Score:3, Insightful)
Treat it like they treat Phreaking... (Score:3, Insightful)
The answer should equate to who should eat the cost of a DoS trojon.
Legal Liability (Score:3, Insightful)
In the case where the theft occured (mutually) from both a commercial and private victim, the commercial victim is generally assigned the majority of the loss because they are considered to have superior knowledge and been in a better position to have prevented the theft from taking place.
Since the theft was allowed by two enteties (the target Computer and the ISP servers that allowed the theft to take place), both entities would probably be apportioned a percentage of the cost.
Since this has never gone to court, there is no case material to set some form of guidelines.
My guess is that apportioning the entire blame to the customer (and billing them) would not hold up if the customer filed against you.
Depending on what measures your ISP has taken to prevent this type of abuse (filters, scanning, etc.) you could probably get away with some form of apportionment where the customer is billed for part of the cost.
Tom
ITU rule on charging (Score:4, Interesting)
Big attacks should be reported to Homeland Security. [nipc.gov] (Really. Effective March 1, Homeland Security runs the National Infrastructure Protection Center. ISPs are going to be dealing with them on a regular basis.)
I get What i Pay for (Score:3, Insightful)
Because YOUR (isp) system of delivering bandwidth is faulty or doesnt account for abuse potentials is NOT my (consumer) fault.
If you decide to enforce a D/L cap, i myself will not be your customer....
If i was the average joe who opted to take on that bandwidth cost then i would blame YOU the ISP for allowing malicous data to be replicated at obvious expense.... as in if a port is responsible for great amounts of malicous (repetitive, near obvious redundant packet exchanges indicitive of an attack, worm, or virus).
The whole thing is, as an isp... the service you provide should be a fully enclosed package... no hidden/additional costs. And bandwidth capping should not incur automatic additonal costs to the consumer after a limit is reached, it should result in a great limiting of bandwidth (after a certain amount is reached) or in a blocked connection (allow only the company's IP until the customer buys more bandwidth).
My personal opinion, we are getting dicked by the tele-comunications industry from the top down... everything from home phones, cable, cell phones, broadband, T1's and more are greviously over-priced at a near basement cost to the mother companies. By the time a consumer recieves their data the fixed price of hardware and the cost of ELECTRICTY has been multiplied ten-fold. Mid-Range ISP's are being squeezed by the big players, and in turn are having to offer misleadingly high "bandwidth" speeds with BullShit Capping.
Downloading megabytes into your cell-phone doesnt cost sprint shit, but youll have to pay 1.00 per DL.
Of course the tel-co's are screaming bloody murder about their losses, but it isn't from data rates.
As a last note.... when we were all using 56kbps modems you could DL for days on end... you could call your local BBS and be charged a phone call while DLing full-speed for hours.... No extra cost... didn't cost them a thing since we payed for the phone-call.... Now that High-Speed is in the home.... and the tel-co's found they could save even more money by offering bandwidth speeds based on diluted averages of many users, they think it's fair to make more money by punishing those who ACTUALY USE THEIR bandwidth. Bandwidth which is only ELECTRICTY. Do you honestly think Time warner can offer 500 channels of digital cable, with "on demand" channels (where you can choose a movie and play it immedietly) for 60$ bucks a month and not provide that same (nearly continuous) data rate to internet connections?
luckily.... with the advent of online movies, music and application servers and such, soon even joe email will be needing a constant high-speed connection.
Just my two cents.... VISION
--Enter The Sig--
Re:I get What i Pay for (Score:4, Insightful)
Good for low-useage servers with very short spikes of popularity.
You've just said that the ISP should eat the cost of the extra bandwidth...why? You agreed to burstable charges...they gave you more in advance, on condition you would pay for it with your next bill.
"Because YOUR (isp) system of delivering bandwidth is faulty or doesnt account for abuse potentials is NOT my (consumer) fault."
"If you decide to enforce a D/L cap, i myself will not be your customer...."
With that type of an attitude, you're saying you are entitled to unlimited bandwidth. The datacenter has an OC-48 into it...does that mean you're entitled to that? Not unless you paid for it...
The network has the capability to deliver high speeds, but if you didn't pay for that speed you're not entitled to it any more than someone who doesn't have the service at all is.
Just like in real life (Score:5, Interesting)
Suppose you live on a crosspoint of several countries. Your house happens to be located in a dangerous curve on the road. Also for some reason your house looks to some kiddies like it asks to be vandalized.
For these reasons you get a lot of breakin attempts, occasionally a truck crashes through your walls. All this is not only by people from your own country, but from neighbouring countries as well.
You install warning lights and other measures so cars and trucks don't come in crashing. You call the police when kiddies vandalize your home, but they says they can't do anything.
All this costs you a lot of money and headaches.
In real life there are several ways to defend yourself:
Now apply these principles to your hosting server.
Suppose your house is rented. Is the person renting you the house responsible for every breach? Did he warn you before you signed the contract? Is it his responsability to call you every time some vandals are passing on the road? Or some truck may crash into your home?
Of course your ISP can warn you for every threat that may be coming, but what if there's no warning time? Or he misses a small thing that happens to affect your server bigtime? Is the ISP really responsible?
Be careful out there...
Hrm (Score:5, Interesting)
ISP A has customer X. ISP B has malicious user Y. Malicious user Y sends huge quantities of packets to user X.
The question seems to be, should ISP A eat the cost, or should customer X eat it? Why the hell are those the only two options?! It seems to me like ISP *B* should eat the cost, since the malicious packets were sent through their network in the first place. ISP B can attempt to recover their loss directly from malicious user Y.
The ISP *and* the customer are both victims in a DOS attack. Whoever runs the network which *initiated* the attack should be responsible.
Partial solution with IPv6 (Score:3)
Under criminal and most other law, the criminal becomes liable for both direct and indirect damages. As an example, if a gang robs a bank and a gang member gets shot by a clerk, the gang leader is charged with homicide/murder/manslaughter, as appropriate. In this case, the spammer, worm originator, or other attacker should similarly be held liable for direct and indirect damages -- meaning everything from bandwidth to cleanup.
IPv6 allows many security features, including authentication and nonrepudiation. An ISP (or anyone for that matter) can easily use their logs to verify that packets are from a particular source. By rejecting all packets unless traceable, and then keeping the traces around, the responsible party can be easily found by talking to everyone along the chain until someone either has no logs or originated the attack.
Once you've found the person, simply either eat the cost as is done now (if they are a little person infected with a worm/virus but don't have logs), OR try to get money from them and blacklist from future systems (if they are a real criminal).
Something I would LOVE to see is a system that holds everyone responsible. An Internet where to get an address block you sign away certain rights. You would assert that you will either keep logs of all activities or pay for any damages [see above]. When any software is released for use on this new network, the software company would be held liable for damage done by their software [see Outlook worms]. Any software using the network would have to properly record all network transactions thorugh cryptographicly secure undeniable means. Lastly, all commercial communication, unless specific one-to-one talking or client/server requests like the web, would be strictly forbidden, again with damages paid [no spam]. That is my Dream Internet.
frob.
If I get slashdotted... (Score:4, Funny)
One way or another...
Oh yes, he will pay.
Solvable through bandwidth throttling (Score:3, Insightful)
Such a setup would allow for full utilitization of the network bandwidth and avoid all the hassle of pissing people off by sending them extra bills or suspending their account.
Perhaps the backbone should eat the 'cost' (Score:3, Interesting)
However, if you think about it - the ISP wont be having to pay its provider more if it does "Above 1Mb/s on *this* pipe.. above
What if the ISP doesnt hit the utilisation required for it to be charged extra, but individual systems within its network get hit hard by a particular virus? (Slammer for example didn't pick IPs properly at random, so some IPs would be hit, others wouldn't)
In this situation, I think the ISP should let them off the fee. The ISP hasn't been charged any extra for the slammer traffic, so it should let the customer off the charge. It'll do wonders for loyalty if you can see your provider is fair and reasonable about things.
The other situation to consider is when an ISP does get billed by its backbone provider heavily for extreme and unsual utilisation.
Alright, hold that thought. Right at the top levels of backbone providers, there is no direct cost associated with using 80% or 10% of a backbone line. It simply is. It's at this stage I think, that they should possibly relieve their clients of bills that are easily attributed to big viruses that are doing the rounds. Granted, then what do you do about spam? Where do you draw the line as to what is 'unsolicted/extreme/garbage' traffic?
Another solution I've just thought of is to extend the period that an average is worked out over, so that over the year if you're under 1Mb/s, you don't get charged extra. It should even out massive, but short lived spikes from worms such as Slammer.
Yes, I know contracts are normally clear about traffic levels and bills that you will receive if you break them, but I do think it's unfair for a small site that has just gone colo to suddenly get a bill 10x its normal bill since the latest worm has been targetting its machine, primarily since there is no direct cost to the ISP, or the ISPs provider, that can be attributed to this extra traffic (as long as there is spare capacity!).
People should be accountable (Score:5, Interesting)
In addition, Making the people responsible for their personal worm/virus traffic would make folks would be more proactive about virus prevention and more cautious of which sites they visit. This IMHO is a Good Thing.
Another potential positive would be that people might start wondering "Why does my friend/relative who runs Linux never complain about viruses?" and "Gee with all these viruses that only affect microsoft products, maybe I should look elsewhere for my software needs."
At least in my state, you are responsible for your car's emissions. If your car is polluting above the state limit, regardless of the reason, it is your responsibility to fix it. They don't care what the reason is for your excessive emissions, whether it was rust, hungry chipmunks, incompetant redneck mechanics, or just a poorly built ford suv. And they have a system of mandatory repairs and/or fines in place to enforce this. This is a Good Thing.
Real world not like posts on /. (Score:3, Insightful)
What many posts in this thread do not seem to take into account is the greater reality that is the web. With a completely patched server and firewalling that drops packets not desired to hit said server, incoming bandwidth is changed none-whatsoever. You have zero control over traffic until that traffic hits a device under your direct control. With most ISP's, that device can only be placed well past their traffic monitoring point. Ergo, you pay for bandwidth whether you want it or not.
You do have the ability to reduce the total amount of bandwith consumed by dropping unwanted return connections but that may be irrelevant if your site is subjected to a DDoS attack.
The largest problem lies in determining whether traffic is "legitimate" traffic BEFORE it passes through the ISP's network to the client. That said, there are a great many possible ways to accomplish this, such as:
The above are merely ideas or concepts, I will leave implementation to those that require the features. But it gives a good idea of the directions that an ISP can go to mitigate the costs of unwanted bandwidth. Just like Credit Card companies will call a customer to verify that they really do want to purchase that Tiffany diamond in a State they've never visited before, maybe ISP's should be monitoring traffic for irregular patterns and contacting customers to verify that the traffic is legitimate.
ISP's can't merely turn a blind eye when the entire netblock they serve starts sending or receiving traffic generated by the latest worm, virus, etc. They should do their best to mitigate their losses and losses of their customers.
I'm not saying that customers are without blame, just that the people running ISP's may have more technical knowledge that that of their customers and should be proactive in protecting those customers from further harm. If you want a real-world, non-technical example, think Firestone and Ford. A problem created outside of Ford that could have been eliminated before reaching the customer if only greater due dilligence had been used. By ignoring or overlooking the problem (I don't know the exact details) both Ford and its customers were negatively impacted. Was it Ford's fault that the tires were faulty? No. Could they have done something about the tires earlier? Possibly. Could the customer do something about the tires? Yes, but only after they knew of the problem by experiencing the negative consequences.
The scenario doesn't differ much when applied to unwanted bandwidth. If ISP's fail to do their part, unwitting customers will always suffer.
Make it work like credit card liability. (Score:3, Insightful)
Same thing for bandwidth. If the customer notices a problem and notifies the ISP so they can take steps to block / track the attack then they shouldn't have to pay. However, if they are too lazy to monitor their own gear, and/or call the ISP they deserve every dollar they get charged. The customer needs to be a partner with the ISP in fighting these sorts of things, otherwise the ISP never has a chance to catch the real criminals.
Of course, all this is for medium size and up ISP customers. Smaller businesses and/or individuals may just want a "turn it off if it goes above x" until I call model, which is completely reasonable.
The joys of running a web server over DSL (Score:3, Interesting)
Two days ago I put my personal web-site up. It's sitting on a linux box (Apache) behind my firewall, which only lets incoming connections initiated on port 80 through.
In two days I have had maybe 100 hack attempts. All using variations on "GET
But, WTF... they're using up MY bandwidth. Why can't ISPs take some responsibility for detecting script kiddies. There can be exactly no un-patched useless WinNT boxen out there. Why shouldn't Mr ScriptKiddy be asked to pay for the bandwidth?
In telephones (in the UK, at least), calling party pays. If someone is hammering my bandwidth malicously (or at least dumbly) why should they pay?
And why can't get an ISP that "traps" stupid requests, and reports them to the users ISP. Too many issues and that ISP is blocked.
Why not?
(I'm thinking about setting up a DDOS system on anybody that tries to 'hack' my server. Just for a laugh, obviously.)
How it works here (Score:5, Informative)
A few notes about charging for bandwidth:
These are some of the steps we use to protect ourselves and our customers. Your milage may vary.
(We use packeteer for rate limiting, but I keep eyeballing OpenBSD/AltQ/PF for both rate limiting and firewalling for our customers).
Not vandalism, wireless spam (Score:3, Informative)
Compare this to someone constantly text-messaging spam to your wireless phone. You could quickly run up an insane bill that way, and there's really nothing you could do about it. The wireless company is contractually in its rights to charge you.
But it won't.
That's how they work. Someone screws with you, typically the provider eats it, especially if there was nothing you could do about it. That puts the incentive back onto the one entity who can actually do something about it: the providers. True for wireless. True for credit cards. True for just about anything where the end user can't do anything to stop the abuse.
The ISPs can do something about it. They have chosen not to because of how we (the geeks) developed the internet. It's too trusting. But at the end of the day, your ISP does know who you are, because they send you a bill. And they could apply uniform terms of service if they chose to, and only talk to other ISPs who have similar terms.
The RBLs are the future. They just don't go far enough. When they're willing to not just cut off SMTP but entire connectivity to other ISPs who aren't willing to play by uniform rules, then we'll start to see some changes. What kinds of rules? Here's some for starters:
- Authenticated mail only. Yep, this looks like banks' "know your customer" rules. You can be anonymous all you like up to the point that you connect to the mailer. But the guy who forwards mail for you is going to be held responsible for your behavior. Yes, that will radically change the free-service providers (yahoo, hotmail, etc). They're free to come up with solutions that don't require them to know exactly who you are, but if they host spammers, we're not going to talk to them. This is just the logical extension of RBLs.
- Same deal for acting as a DDoS zombie. The owner of the unpatched box is responsible, but it's the responsibility of the ISP to be able to identify that person for legal action. If they can't or won't, then we don't talk to them.
None of this says that you can't be anonymous most of the time. It just says that if you're disrupting service and causing real losses due to your actions or lack of actions, your ISP is going to have to hand you over, or they're going to be held responsible. The right to privacy has to be balance with responsibility for your actions.The old-world networks (phones) have worked this way for years. I can block my out-bound caller-id. I can have an unlisted phone number. I can be very anonymous on the phone. But if I'm named in a law suit or criminal complaint, the phone company will hand me over in a heart beat. The only way around this is pay phones with cash. It's hard to run a large-scale scam that way.
And no, this doesn't mean that an ISP's logs are free game to the RIAA. But it does mean that if the RIAA wants to name a specific "unknown party" in a lawsuit, the ISP is obligated to identify them. Before you get excited, that's exactly the current situation. The RIAA just wants to get the info without actually suing you (which is wrong, and luckily some ISPs have resisted). ISPs need to be willing to say they will only interconnect with other ISPs who play by the same rules.
Yes, this will fragment the internet for a short period of time. So do the RBLs. But economics will fix it fast enough, especially if entire connectivity is cut off.
ISPs aren't 'the internet' (Score:3, Informative)
ISPs don't have infinite bandwidth.
I know, its quite a strange idea. But think of this.
If you're a ISP in a single location, chances are you're buying a few (hundred?) megabits off your upstreams. Unless your upstreams are happy to filter traffic they send to you (and unless its a very large DDoS, most of them will take a while to implement any access control), the ISP will still be charged for traffic sent to a customer even if the customer chooses to reject it.
Similarly, if the ISP provides filtering support for their customers, they still receieve the traffic and bite the usage.
Now, if you're a large ISP and have links to other peering exchanges. Even, say, you peer enough to not really need transit. These inter-state links still cost money. And they're fixed. So if a customer is hit with a DDoS they'll still be carrying it _somewhere_.
Even if this mythical tier-${LOWNUM} ISP with lots of fat peering links has some magical scripts to filter out DDoS traffic to a given customer range, it still will hit their border routers. So their peering cross connects have already been filled. The only way around this is to deal with their peers..
But they don't really have the incentive to spend all their time dealing with smaller networks being attacked. They'd be worried with keeping their network from melting under a few larger ones.
The flipside. If you're an ISP with enough bandwidth (and not high-profile sites like irc servers or pr0n) you might be willing to bite the costs of various attacks as part of a marketing point. Customers may come to you because you have a reputation of being lenient under attacks. Perhaps. But thats a delicate line.
Me, I dig flatrate pipes. Usage based pipes is just asking to be owned by excess traffic. If I buy a megabit then all I really have to worry about is service degradation due to DoS. ISPs, in my experience, will help you with that. But if you're on a usage based pipe which then gets owned by a DDoS you're struggling after the fact to get a rebate. Good luck.
(Although, that said, perhaps you guys should consider asking for usage based pipes that _have_ a bandwidth cap. Figure out what your maximum spend amount is, say 5mbit, and then ask for a usage-based pipe based on that. That way you limit your liability _AND_ getting the cheaper transit. Most of the time.)
Allow customers to set an "incoming" quota (Score:3, Interesting)
If the users don't set a quota, then they are liable. If they do, then you are the insurance carrier. (I guess that it has to be an extra cost service.)
It is important to customers that they be able to predict the size of their connection bill. If they can't, this can cause a lot of trouble. But you could offer an insurance policy that basically says "You won't have to pay more than X amt. I'll bounce the excess if a spike happens." You might want to think carefully, though, about what your cost exposure would be, before you decide on the cost of the policy. (Even having an expensive policy, though, should be a reasonable answer to the current customer complaints.)
Problem is... (Score:3, Informative)
Ideally you'd be able to roll over bandwidth for exactly one month as in subtracting the previous month's rollover at the end of the month. Your bandwith would be continously throttled to the rate at which you'd expend all of your bandwdth at the end of the month. Without rollover, the ISPs would have a huge sawtooth pattern in monthly load and one of the sides of the teeth being nearly vertical. The rollover is more for the benefit ofthe ISPs than anything, so is upstream port blocking, allowing ISPs to blockunwanted traffic at its boarders.
Re:Insurance (Score:2)
Re:Blame the ISPs (Score:3, Interesting)
"Half the problem here is that we bill for bandwidth in the wrong way. By billing on traffic, we open ourselves to exactly this sort of problem - it would be like billing for water consumption based on pressure (rather than volume)."
This doesn't make sense to me. Pressure is like access... nothing flows until you make it flow. It is just the potential for flow. Volume of water flowing (think of it as molecules=packets) is analogous to packets flowing and is a much fairer way of charging for bandwidth since the person pays for what they used (exactly like they pay for the water they use).
"The reason ISPs bill per megabyte is so they can bill multiple customers for the same piece of infrastructure... and at the same time, over-subscribe that piece of infrastructure."
I think you have this backwards. When you charge for a connection ("access") then you can bill multiple customers because you can safely assume that not all of them will be utilizing their access fully. We had an upstream provider that had 19 PVCs on one T1 connection upstream... and was charging every one of its downstream customers for a T1! This is what is meant by "oversubscription". How, exactly, would you double bill for a measured amount of packets?
According to your theory the grocery store should only charge the first customer because then his "infrastructure" costs would be met.
"Strangely enough, paying a fixed fee based on the size of your connection is where the whole thing started. Paying per byte is a relatively recent (several years, but still recent) concept, thought up by greedy providers who realised they can charge many customers for something that is essentially free."
Bandwidth measurement was (and still is) more expensive to count and to bill than simple access. A simple connection is simple; you just provision the PVC and start billing. That's why everyone started out that way. Once the technology was in place (cheaply enough) to allow ISPs to measure bandwidth, then - and only then - could they charge for it.
I don't know how you can think that it's "free". Is your transportation free even though you've paid off your car? ISPs have to charge enough to pay their engineers, their billing people, their sales people, plus have enough to cover capital expenses for new equipment (which the customers will demand because their needs increase). Plus the ISP has to pay its own uplink charges for bandwidth (usually metered). And then, of course, there's the interest payments on the loans taken out to buy the original equipment. No, you're dead wrong. Bandwidth is not "essentially free".
"Take a look at the profit levels of some of the bigger providers in your country. Here in Australia, Telstra, Optus and Connect all report multi-million (and in many cases billion) dollar profits. Nobody can tell me that the core connectivity of the Internet isn't currently a profitable business."
I don't suppose the plethora of bankrupt US providers would convince you otherwise, either. The profit margin for an ISP is razor thin and getting thinner as providers drop prices in an attempt to gain customer base (and profitability). Even AOL is struggling. No ISP in the US is making billion dollar net profits.
I think your understanding of economics is as weak as your understanding of pressure and volume.
Re:Don't understand bandwith charge (Score:3, Informative)
You have your T1. So do 3000 other people. The ISP has calculated that on average, only 15% of your T1, alone with everyone else's, is used in any given month.
That T1 has to connect to something, don't it? It's not a point to point connection to every single site you go to. Your T1 will drop into a DS3, ATM, POS connection. The ISP has calculated what they need to run in the back end, and what they need at the various peering points with other providers.
Let's say the ISP only has 3000 T1 customers. That's a total available bandwidth of 4632 Mb/s for all T1s combined. But since on average only 30% of that is used, that falls to 694. They play it safe and decide that on the backbone they triple that amount (which is not the case. Usually it's less than double). That's still only 2084 Mb/s (or 13 DS3s). Your price for a T1 has been calculated using these numbers. Suddenly everyone uses their T1 at full capacity 24/7. The ISP has to put in more pipes to accomadate this. This means their bill to the backbone have skyrocketed. Since your original price was based on 15% utilisation, and now it's 100% utilisation all the time, what do you think will happen? Your bill will go up significantly. The ISP is in business to make money. If it has to put in another 16 DS3s that will run at 100%, they've more than doubled their operating costs. Why should they take a loss? They are totally justified in raising their prices.
This is how the real world operates.
Re:Internet = Public space? LOL! (Score:3, Interesting)
The internet is not a public space all the time (irc and message boards would be public spaces), but if you allow yourself to be on the internet, you are allowing others to access your space. If you put a computer directly on the internet, it is not your ISPs job to secure that for you. It is YOUR job to maintain the integrity of your own machine. if someone hacks your machine because you failed to close a port, that is your fault. trying to blame the ISP is not going to get you anywhere.
My AT&T (now comcast) cable modem specifically has a clause in the terms of service that say something like, "your connection is your responsibility. if you allow others to use it and they do something illegal, since it is your connection, that means you did something illegal."