×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Corporate Data Centers As Ethernet's Next Frontier

timothy posted more than 4 years ago | from the build-me-a-beowulf-cluster dept.

Networking 152

alphadogg writes with a story that's about the possibilities for the next generation(s) of Ethernet, stuff far beyond 10base-T: "Ethernet has conquered much of the network world and is now headed deep into the data center to handle everything from storage to LAN to high-performance computing applications. Cisco, IBM and other big names are behind standards efforts, and while there is some dispute over exactly what to call this technology, vendors seem to be moving ahead with it, and it's already showing up in pre-standard products. 'I don't see any show-stoppers here — it's just time,' says one network equipment-maker rep. 'This is just another evolutionary step. Ethernet worked great for mundane or typical applications — now we're getting to time-sensitive applications and we need to have a little bit more congestion control in there.'"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

152 comments

Hmm... (1)

Enki X (1315689) | more than 4 years ago | (#25454305)

Are there any foreseeable applications for the consumer world?

Re:Hmm... (1)

Enki X (1315689) | more than 4 years ago | (#25454415)

Oh, nevermind.... I misunderstood...I was thinking: Who besides data centers needs this? But a lossless network would be nice...

Re:Hmm... (3, Interesting)

pla (258480) | more than 4 years ago | (#25454929)

Are there any foreseeable applications for the consumer world?

Connect your new keyboard and mouse via ethernet.
Connect your new HDD* via ethernet.
Connect your new video card via ethernet.
Connect your new scanner via ethernet.
Connect your new CD/DVD/BR via ethernet.
Connect your new printer* via ethernet.
Connect your new webcam* via ethernet.

No more USB cables with a million different connector types. No more PATA or SATA cables. No more serial or parallel cables. No more trying to figure out where to plug a given device in on a motherboard or looking for spare PCI/whatever slots - Just one type of cable and they all plug into a switch-like section of the motherboard.

Now, some devices (video cards as the most obvious) will still require extra power, but most devices could probably manage with a variant on PoE, meaning the inside of your case goes from rats-nets of assorted cable types, to a half-dozen or so tidy round cables.

* Yes, you can already get network enabled versions of these, but they count as a real full-fledged network endpoint, not as a slave device "local" to a particular computer.

Re:Hmm... (5, Insightful)

Yetihehe (971185) | more than 4 years ago | (#25455127)

No more USB cables with a million different connector types.

You realise "no more different connector types" was the reasoning with USB?

Re:Hmm... (4, Insightful)

Grey_14 (570901) | more than 4 years ago | (#25455411)

And sadly, you'd see the same issues it with this standard too, because an ethernet RJ-45 plug isn't appropriate to plug into a cell phone, digital camera or mp3 player, but a 5-pin mini-connector isn't appropriate to run 25 feet to a switch/router either.

Re:Hmm... (2, Informative)

Shatrat (855151) | more than 4 years ago | (#25455607)

I think the biggest reason that there are hundreds of different USB connectors is that standardized plugs don't help sell 30 dollar Apple or Sony branded AC/DC adapters.
I really loved my old motorola phone with the mini-usb connector, now my LG phone doesn't share it's connector with any other device I own.

Re:Hmm... (1)

killmofasta (460565) | more than 4 years ago | (#25456217)

I disagree completely. All you ever need is power and signal. How did PC-Card makers make the connection? A slide out connector. Digital camera and mp3 player, they can use it too.

The 5 pin-mini connector was designed to make it easier to get the orientation correct without bending the pins, ( its called a DIN connector, and its a german design/standard ). I think its appropiate to run 100 feet to a router, but the RJ Physical standard is so much more reliable.

Why use mini-din? If the connector is at the back of the computer, then all you need to do is twist, and you can get the orentation right. If its RJ, then just put the tab on the bottom. ( Note: I have returned ethernet cards for their jack orientation. I dont think that Motherboard mafactures will EVER GET A CLUE. ( I have never seen a motherboard that has the connections in the top, and the tab on the bottom )

The engineer who designed the motherboard connector for Ethernet should be SHOT.

Re:Hmm... (1)

mockidol (1031242) | more than 4 years ago | (#25455847)

Yes there are mini and micro usb plugs for devices themselves but every computer has a standard USB A pot now that they all can plug into.

I'd say it worked pretty well. You can't fault them for pulling micro USB on your cell phone, unless you want it to be twice a thick.

Re:Hmm... (2, Informative)

TheRaven64 (641858) | more than 4 years ago | (#25455639)

* Yes, you can already get network enabled versions of these, but they count as a real full-fledged network endpoint, not as a slave device "local" to a particular computer.

You can with protocols like iSCSI or ATAoE. A lot of enterprise gear uses iSCSI, which makes a remote device appear like a local SCSI device, and some consumer-grade NAS devices running Linux can act as ATAoE devices, which does the same thing but with ATA instead of SCSI and over raw Ethernet frames rather than over IP.

Re:Hmm... (1)

killmofasta (460565) | more than 4 years ago | (#25456077)

Mod parent BRILLIANT.

I actually called MaBell, and talked to some design engineers for the RJ connectors, when I was thinking about wiring my first Network. Phone jacks and Ethernet cables are ULTRA reliable when implemented properly. ( Umm.. Did I tell you my first network, is still in use and the cables are still working? )

The idea that every component uses them is extrodinary. If all my devices except my Video card used Ethernet/RJ11, and was self configuring? It sure would make everything a WHOLE LOT EASIER!

Make your motherboard a Ethernet switch! BRILLIANT!

Umm (1)

jav1231 (539129) | more than 4 years ago | (#25454307)

Yeah, I can't wait until they rip out all of the Stallion COM port boards from our data center and put in those new-fangled switches and routers! Boy, we's gonna be uppity!

Re:Umm (2, Insightful)

marcosdumay (620877) | more than 4 years ago | (#25456755)

You probably don't have a storage area network, running over some proprietary fiber protocol, or some hight performance proprietary cluster, or a supercomputer around, do you? All those things are fading out as Ethernet evolves to do those kinds of jobs, but they didn't disapear yet.

I'm already on ... (1, Funny)

Anonymous Coward | more than 4 years ago | (#25454317)

20baseU. Some hotshot near me is trying 30baseV - show off!

what am I missing with this article? (2, Insightful)

poetmatt (793785) | more than 4 years ago | (#25454359)

FTA: "But in its current state, Ethernet is not optimized to provide the service required for storage and high-performance computing traffic -- speed alone won't cut it, vendors say. Ethernet, which drops packets when traffic congestion occurs, needs to evolve into a low latency, "lossless" transport technology with congestion management and flow control features, CEE and DCE backers say."

If I understand right, they're trying to change Ethernet because of TCP/IP? Isn't that kinda, backwards as a concept?

We could add a "token" and make it a "ring"! (4, Interesting)

khasim (1285) | more than 4 years ago | (#25454457)

And to make it easy we could call it "TokenRing".

Or maybe a token passing bus! Maybe call it "ARCnet".

Seriously, if there are problems with Ethernet ... for the usage you envision ... don't try to change Ethernet.

You take the parts you want from Ethernet and you create a NEW standard with the other features you want.

But you leave Ethernet as Ethernet. That way there is no confusion.

Re:We could add a "token" and make it a "ring"! (4, Funny)

Legion_SB (1300215) | more than 4 years ago | (#25454757)

I agree. I don't want any of this "doesn't drop packets" Ethernet either. Packet loss is critical to a number of my in-house applications.

Re:We could add a "token" and make it a "ring"! (2, Insightful)

Bill, Shooter of Bul (629286) | more than 4 years ago | (#25455395)

Something new won't sell. People wont adopt revolutionary products as easily as they will adopt incremental upgrades with a known and trusted brand. So calling it "Uber-fiber hyper gylde" won't sell as well as "Ethernet v10".

People will deal with confusion. They deal with it all the time. Its the only way they know to deal with the walrus.

Re:We could add a "token" and make it a "ring"! (4, Funny)

R2.0 (532027) | more than 4 years ago | (#25456133)

"Uber-fiber hyper gylde"

Combination personal lubricant and laxative?

Re:We could add a "token" and make it a "ring"! (1)

besalope (1186101) | more than 4 years ago | (#25456653)

Something new won't sell. People wont adopt revolutionary products as easily as they will adopt incremental upgrades with a known and trusted brand. So calling it "Uber-fiber hyper gylde" won't sell as well as "Ethernet v10". People will deal with confusion. They deal with it all the time. Its the only way they know to deal with the walrus.

Yup, Brand recognition is a bitch.

In actuality (1)

earnest murderer (888716) | more than 4 years ago | (#25455619)

Since nobody buys anything not labeled ethernet it's going to be called ethernet anyway. Maybe ethernet+ or ethernet ring or some BS marketing term.

The technology changes have been substantial since Xerox was pumping 3Mbit/s through a coaxial cable but we continue to call it ethernet because PHB's don't want to bother implementing a new networking standard.

Re:what am I missing with this article? (5, Informative)

LWATCDR (28044) | more than 4 years ago | (#25454479)

No.
Ethernet uses collision detection and resending to to manage packets.
Well it used to anyway. I am not sure about Giga-E
The way Ethernet used to work is that a sender would listen to see if the line was clear and then send a packet and listen at the same time. If the packet was damaged by a collision the sender would wait a random amount of time and then try to resend.
This system really bugged a lot of people but it was cheap and it worked.
This is the actually physical layer and not TCP/IP.

Re:what am I missing with this article? (1)

poetmatt (793785) | more than 4 years ago | (#25454585)

Ahh thank you. I don't know that much with regards to networking, but I would imagine well before this standards issue haven't there been many forms of this already implemented, say as routing ethernet connections to switches? I mean only one true "Sender" is on each ethernet line, right?

Re:what am I missing with this article? (1)

vux984 (928602) | more than 4 years ago | (#25454921)

Ahh thank you. I don't know that much with regards to networking, but I would imagine well before this standards issue haven't there been many forms of this already implemented, say as routing ethernet connections to switches? I mean only one true "Sender" is on each ethernet line, right?

Wrong.

Collisions still occur when multiple computers try to talk to a single computer at once. Of course this is an extremely common scenario in a typical client-server network. However, with a switch at least one packet always gets through.

So technically you could argue that there was not really a "collision" but the computer that didn't get its packet through is still told there was one so that it retransmits.

And to further complicate things, switches generally have buffers and can hold multiple packets bound for the same port, and transmit them in sequence rather than reporting 'collisions'. But the buffers can fill (especially if a lot of clients are hitting the same server at once), and then the switch has to start reporting 'collisions' again.

Re:what am I missing with this article? (2, Informative)

amorsen (7485) | more than 4 years ago | (#25455585)

So technically you could argue that there was not really a "collision" but the computer that didn't get its packet through is still told there was one so that it retransmits.

No. Switches don't notify the source that the packet was dropped. TCP's retransmit works without explicit notification.

Collisions are a thing of the past. Dropped packets aren't.

Re:what am I missing with this article? (1)

vux984 (928602) | more than 4 years ago | (#25456043)

No. Switches don't notify the source that the packet was dropped. TCP's retransmit works without explicit notification.

Some switches report port buffer overflow as a collision.

Consider dropping a packet destined for a remote system with high latency. The RTT time used to tune TCP retransmits might be several thousand milliseconds. Where if the switch reports that it dropped the packet the CSMA/CD retransmit time would be MUCH shorter.

Clearly if the congestion issues are transient and temporary and the latency is high you can achieve much better overall performance by signaling for retransmits at the ethernet level, rather than relying on TCP retransmits.

It also can boost the reliability of connectionless protocols like UDP.

Re:what am I missing with this article? (3, Informative)

erc (38443) | more than 4 years ago | (#25455609)

However, with a switch at least one packet always gets through.

Wrong. There is no "collision detection", the only way to tell that you had a collision is if the packet didn't get there. If two devices transmit at the same time, you get a mangled packet that won't pass CRC and gets dropped.

What they're really looking for is token ring - which was (and still is) a superior protocol - for Ethernet, as you increase the bandwidth utilization beyond 10%, you get so many collisions that your throughput goes through the floor, while for token ring, the throughput degradation is much more gradual. For bandwidth utilization above 10%, token ring is far superior to Ethernet.

Why Ethernet was adopted over token ring has more to do with religious issues and who can scream the loudest and bully their way through technical issues with emotion than it has to do with technical superiority.

Re:what am I missing with this article? (2, Informative)

LWATCDR (28044) | more than 4 years ago | (#25455771)

I think it had a lot more to do with cost.
Ethernet was available first and had more hardware suppliers so the cost went down.
Token ring was really popular with IBM. It was almost a standard for IBM systems. I have a few microchanel Token Ring adapters if you need them :)
FDDI is Token Ring on fiber.

Re:what am I missing with this article? (2, Informative)

erc (38443) | more than 4 years ago | (#25456231)

Haha, thanks for the offer but I don't have anything that will take MCA boards ... that was an interesting attempt by IBM to retake the PC market they lost. Oh, well.

As to Ethernet being first, I thought it was the other way around?

Re:what am I missing with this article? (2, Informative)

afidel (530433) | more than 4 years ago | (#25456407)

Huh, in Ethernet which is CSMA/CD you listen to the wire before starting to transmit, this doesn't avoid all possible garbled packets but it does avoid most if things are working to spec. Also because VTT goes higher than normal transmit levels during a collision there IS detection. The reason that ethernet won over tokenring is that IBM charged a hefty fee per port for tokenring. There was also real world reliability problems with tokenring as early designs used a physical ring or string layout instead of the star topology of ethernet (later tokenring switches did allow for physical star topology but this was well past the point where ethernet had won the mass market).

Re:what am I missing with this article? (1)

vux984 (928602) | more than 4 years ago | (#25456815)


What they're really looking for is token ring - which was (and still is) a superior protocol - for Ethernet, as you increase the bandwidth utilization beyond 10%, you get so many collisions that your throughput goes through the floor, while for token ring, the throughput degradation is much more gradual. For bandwidth utilization above 10%, token ring is far superior to Ethernet.

That was true in hub based networks, but hasn't been true for a long time in switch based networks.

Why Ethernet was adopted over token ring has more to do with religious issues and who can scream the loudest and bully their way through technical issues with emotion than it has to do with technical superiority.

Ethernet was adopted because it was significantly cheaper, far simpler to set up, and most networks are low utilization. And the arrival of full duplex switches has made ethernet nearly as good as token ring.

Re:what am I missing with this article? (1)

Paralizer (792155) | more than 4 years ago | (#25455761)

Collisions still occur when multiple computers try to talk to a single computer at once.

Collisions occur when there are more than one sender on a collision domain, they don't have to be sending to the same host. Imagine you have four computers on a hub. Computer A sends a message to B while C simultaneously sends a message to D -- this is a collision.

When a packet is sent to a hub the hub immediately sends it out all ports -- it's like a set of spliced together wires. A switch switches, it tries to figure out what port it should send it to based on the destination MAC address, then sends it just to that port. This way multiple packets could be sent to different hosts on the same switch at the same time without causing a collision.

And yes, switches do have outbound buffers for each port so that if two sources try to send to the same host they can be done in sequence rather than causing an outbound collision on the destination port's collision domain. I am not sure what happens if this buffer becomes full, I had always assumed the switch would just begin dropping the packets (as indicated by this Cisco document [cisco.com]). I'd be interested to read any sources you might have that talks about generating collision messages though.

Re:what am I missing with this article? (3, Informative)

vux984 (928602) | more than 4 years ago | (#25456673)

Collisions occur when there are more than one sender on a collision domain, they don't have to be sending to the same host. Imagine you have four computers on a hub. Computer A sends a message to B while C simultaneously sends a message to D -- this is a collision.

We are really just talking about how collisions occur on a switch. Technically, they CAN'T occur on a full duplex switched network. The collision domain is the switch port and the PC port, and both can talk at once (full duplex).

Hypothetically though, if you set aside buffering, a 'collision like' conflict occurs when multiple PCs try to talk to a single port, except that one gets through and the rest are 'blocked' which is what I was trying to say. Of course, due to buffering, this is 'handled' and the conflict is actually pushed back to when the buffer overflows instead.

And yes, switches do have outbound buffers for each port so that if two sources try to send to the same host they can be done in sequence rather than causing an outbound collision on the destination port's collision domain. I am not sure what happens if this buffer becomes full, I had always assumed the switch would just begin dropping the packets (as indicated by this Cisco document).

http://www.webopedia.com/TERM/B/backpressure.html [webopedia.com]

Dropping packets is one option. The other is to use 'back pressure' to signal to the PC to back off a bit. This can be done by sending 'fake collisions' or via 802.3x Flow Control 'pause' signals. Many switches support these modes including those from intel and cisco.

Its often better to just dropping the packets and let tcp deal with it, but in some cases you can get better performance by using flow control/back pressure features.

Re:what am I missing with this article? (2, Informative)

Paralizer (792155) | more than 4 years ago | (#25454741)

Yes, that is the case strictly at layer 1 of the OSI model. However at layer 2 we have the switch. By segmenting the collision domain up and creating one for each port rather than the entire unit we no longer have collisions and CSMA/CD is no longer needed. Unfortunately wireless still uses CSMA/CA which operates similar to what you described, except it requests silence of the 'wire' first before trying to send rather than retransmitting when a collision occurs. Switches are still part of ethernet since they operate at layer 2. TCP/IP doesn't come into play until layer 3 when we get TCP/IP IP addresses.

Ethernet itself is not reliable, which is why we use TCP in TCP/IP as the transport protocol so we know if we need to retransmit due to undelivered packets. I can't imagine how they would go about 'fixing' ethernet though, as the GP pointed out. If you pass something along between a series of switches/routers/nodes there must be the possibility something fubars and you lose that information in transit - be it a noise on the wire or maybe the node runs out of memory and can not process it.

Re:what am I missing with this article? (1)

squizzar (1031726) | more than 4 years ago | (#25455737)

Doesn't it also send a big blast down the line to make damn sure that everyone knows the packet has been mangled? Not particularly important, but I always thought it was kind of like screaming profanities when something goes wrong.

Re:what am I missing with this article? (0)

Anonymous Coward | more than 4 years ago | (#25455933)

No, full duplex Ethernet does not use collision avoidance or detection. Only half duplex Ethernet uses CSMACD (carrier sense multiple access collision detect). With full duplex there are dedicated send and receive 'wire' pairs.

Re:what am I missing with this article? (1)

marcosdumay (620877) | more than 4 years ago | (#25456711)

Fast Ethernet (10/100Mbps) already didn't work that way. Current Ethernet do not work at a bus topology anymore, only star. That means there are no more colisions.

Re:what am I missing with this article? (1)

betterunixthanunix (980855) | more than 4 years ago | (#25454497)

No, ethernet itself is the reason that those packets are dropped. It is possible to have IP on some other network, like token ring or FDDI, bother of which actually achieves higher throughput than ethernet for a given bandwidth. IP is known to be "unreliable" because there is nothing in IP that corrects for dropped packets, but those packets are dropped because of the network type that is used (or because of physical considerations on that networking, like a disconnected cable or radio interference).

Re:what am I missing with this article? (3, Interesting)

amorsen (7485) | more than 4 years ago | (#25455657)

t is possible to have IP on some other network, like token ring or FDDI, bother of which actually achieves higher throughput than ethernet for a given bandwidth.

Nope, both of which have higher overhead than full-duplex ethernet. They have better throughput than half-duplex ethernet, which is about as useful as being better than avian carriers. Half-duplex ethernet should just be banned entirely. Maybe that would make Linksys wake up.

Re:what am I missing with this article? (4, Interesting)

afidel (530433) | more than 4 years ago | (#25454573)

No, they want Ethernet as a transport to contain a lot of the features of TCP so that they can lay their own protocols on top of it while being able to assume it's a reliable transport. That will increase the cost of ethernet by including that intelligence down the stack. Basically the cost of ethernet ports is plummeting compared to things like fiberchannel due to economies of scale and so cash strapped datacenters are trying to use it for everything, but it's not ideally designed to handle those other protocols so the other technology areas are trying to mold ethernet to meet their needs. Basically the way I see it if the industry does what is right there will be no 100Gbit Fiberchannel, there will be 100Gbit FCOE adapters.

Re:what am I missing with this article? (1)

sam0737 (648914) | more than 4 years ago | (#25454917)

How this can be done?

Another post mentioned the collision detection and backoff property of the Ethernet, but that's all about within the same broadcast domain.

the TCP retransmission is there in case the router in the middle drop it (congestion), or in case where we are not using Ethernet in the bottom layer (hey we use Wifi and Edge!)

There is no way for Ethernet to guarantee the reliability given the simple fact that the Internet is not made up by Ethernet Think about the OC-xxx / ATM / Optical link and Wifi part, not even mentioning the cross broadcast domain boundary.

And that, neither TCP or upper protocol can drop the reliability responsibility because of no lower layer protocols guarantee that.

Re:what am I missing with this article? (1)

amorsen (7485) | more than 4 years ago | (#25455679)

Another post mentioned the collision detection and backoff property of the Ethernet, but that's all about within the same broadcast domain.

Collision domain, not broadcast domain.

Re:what am I missing with this article? (2, Interesting)

afidel (530433) | more than 4 years ago | (#25456177)

Since 10Gbit Ethernet has the collision domain defined as the two endpoints there IS no longer a collision domain on the wire, just a virtual one in an oversubscribed switch. This isn't about guaranteeing transmission over the internet, it's about guaranteeing reliability in a LAN/MAN/WAN Ethernet network. The idea is you will have one set of wires, one physical protocol with several personalities sitting on top. The biggies are TCP/IP and FCOE but there are other things like remote DMA that can greatly benefit from a reliable network layer transport.

Re:what am I missing with this article? (1)

jgaynor (205453) | more than 4 years ago | (#25454673)

This article sucks donkey nuts.

"Ethernet, which drops packets"

Ethernet switches Frames. It does not route packets. That's like saying a railroad track can drop a car because it doesn't like the passengers on it.

"they're trying to change Ethernet because of TCP/IP"

Your question just confuses things more because TCP segments are l4, as opposed to packets (l3) and frames (l2).

Re:what am I missing with this article? (1)

poetmatt (793785) | more than 4 years ago | (#25454763)

I guess the basic answer is:

I got it wrong, and so is the concept?

The core concept is ... problematic. (1)

khasim (1285) | more than 4 years ago | (#25455163)

They want a system that SHARES the available resource (bandwidth in this case) ... but that allows each machine on it to behave as if it had EXCLUSIVE access to that resource.

And they want to base that system off of a technology that evolved from a concept based upon COLLISIONS.

And the reason they want to do that is ... because that old technology is ubiquitous.

Not because it is well suited to their needs. Not because it is inexpensive to modify. Not for any good technological reason.

But because everyone already has it.

So they want to make networks more expensive for EVERYONE so that they THEY can sell their products for less.

Fuck that.

Re:The core concept is ... problematic. (2, Insightful)

TheRaven64 (641858) | more than 4 years ago | (#25455727)

It sounds like they just want bandwidth reservation and isochronous transfers on Ethernet. Something that would establish a virtual circuit and then not drop frames. Something with an Asynchronous Transfer Mode, perhaps?

Re:The core concept is ... problematic. (1)

amorsen (7485) | more than 4 years ago | (#25455807)

So they want to make networks more expensive for EVERYONE so that they THEY can sell their products for less.

You don't have to buy switches which support the new features. Ethernet is just a low-overhead way to serialize frames onto 4 pairs of wire (or one pair of fiber).

The concept is problematic for a different reason: They believe that advanced features are needed because ethernet doesn't have enough bandwidth. That will only be true for a few years, then everyone will be doing multiple 10Gbps links to switches with 500+ Gbps bandwidth on the backplane -- you can even buy that today, it's just a bit pricey.

Re:The core concept is ... problematic. (1)

afidel (530433) | more than 4 years ago | (#25456521)

No they want it because a LOT of money gets sunk into the development of new standards and PHY chips for other protocols that could be more easily implemented as a personality on top of a reliable ethernet transport. A good example is Fiberchannel, most of the physical equipment and low level signal encoding is the same between say 8Gbit FC and 10Gbit ethernet so if you had a reliable ethernet layer with reliable switches you would only need one type of switch, one type of wire, and one type of admin. This represents a real cost savings to both the industry and the industries largest consumers, the enterprise.

Re:what am I missing with this article? (2, Interesting)

mikael (484) | more than 4 years ago | (#25455261)

For a high-performance system with a large number of nodes, the cost of the actual network to connect everything together can cost more than the CPU's and servers themselves. To get high performance from this network, everything has to be tied so tightly together, that is is considered a component in itself, the network fabric. Also, the actual communication through the network cables is the slowest part of the system. So this price/performance ratio is what customers will be considering when buying a system.

The vendors want to keep the cheap network hardware (cables, connectors, switches) because the consumer market has driven down costs down to commodity prices. But Ethernet uses the cheapest method of shared communication - packet collision detection ("keep shouting until someone hears you"). I've read some research papers which say they can get up to 90% efficiency now.

High performance network architectures (FDDI, token ring) are a bit more civilized - they had a token that was passed around - only the machine with the token could send any data.
So there was never any lost packets. Other methods give each pair of devices a unique time-slot on a multi-slice basis. Or there are crossbar switch architectures like telephone exchanges that allow multiple connections to exist at the same time.

So the vendors want it both ways - the cheap commodity prices of Ethernet hardware, combined with the high efficiency of existing high-end network hardware.

The changes that they want only really affects the Link layer of TCP/IP, where collision detection is currently being performed instead of token passing or sychnronised time-slicing.

Re:what am I missing with this article? (1)

LilGuy (150110) | more than 4 years ago | (#25455841)

Ethernet drops packets without tcp/ip. It's called a collision.

Re:what am I missing with this article? (1)

LilGuy (150110) | more than 4 years ago | (#25455889)

er.. frames :/ .. but i hadn't realized someone already responded to you down below.

-over 9000 redundant

Re:what am I missing with this article? (1)

poetmatt (793785) | more than 4 years ago | (#25457317)

I got like 10 responses, about half were a flamewar with some people. I thought switches and such already make up for this, and the comment above yours makes me concerned as well, the whole decommoditize aspect (if it's accurate)

Re:what am I missing with this article? (0)

Anonymous Coward | more than 4 years ago | (#25456627)

What I think a lot of people are missing is the new standard what Cisco likes to call "Data Center Ethernet" is a blend of Fiber Channel and Ethernet. Yes, Ethernet has all sorts of flow control goodness, but not at the same time and order tolerances required for SCSI controller commands required for a SAN.

Ethernet.. (1, Funny)

onion2k (203094) | more than 4 years ago | (#25454369)

The derivation of "ethernet" is quite interesting.

Ether - from the chemical compound Diethyl Ether, an anaesthetic.
Net - meaning 'full of holes'.

I think that's right.

Welcome to 1980! (4, Funny)

snarfies (115214) | more than 4 years ago | (#25454417)

"Ethernet has conquered much of the network world and is now headed deep into the data center to handle everything from storage to LAN to high-performance computing applications."

Ethernet? Used for LAN? Jeepers, who'd ever have though of using Ethernet for THAT! I bet it'll be much faster than my 300-baud modem! And we can even connect storage devices to it!

Re:Welcome to 1980! (1)

rrohbeck (944847) | more than 4 years ago | (#25457279)

Ethernet in the datacenter? Never heard of that - must be a novel concept!
My first thought was "What? A dupe from the eighties?"

So Ethernet's Next Frontier isn't Audio? (1)

Walpurgiss (723989) | more than 4 years ago | (#25454445)

I could have sworn that ethernet's next big step was going to be home audio. Doh.

Why did I buy this http://www.usa.denon.com/ProductDetails/3429.asp [denon.com] then? >

Re:So Ethernet's Next Frontier isn't Audio? (1)

Enki X (1315689) | more than 4 years ago | (#25454511)

Good question...

Re:So Ethernet's Next Frontier isn't Audio? (1)

earnest murderer (888716) | more than 4 years ago | (#25455699)

So his one's will be straighter and his zero's rounder.

Less jitter and a fuller more spacious digital experience. Everything is analog don't you know. Digital is just analog with corners!

Re:So Ethernet's Next Frontier isn't Audio? (3, Funny)

jsalbre (663115) | more than 4 years ago | (#25455455)

Ohh.... That hurts my brain on so many levels. "...directional markings are provided for optimum signal transfer." Wouldn't want the electrons to forget which way they were supposed to go, eh?

Re:So Ethernet's Next Frontier isn't Audio? (0)

Anonymous Coward | more than 4 years ago | (#25456611)

How about this one? "high quality insulation and woven jacketing to reduce vibration"

You don't want those electons vibrating your cable. That inhibits flow.

Network Vendors (3, Interesting)

maz2331 (1104901) | more than 4 years ago | (#25454481)

This seems like a total kludge being put together by networking equipment vendors to find a way to differentiate their products and return to the days where a 10 Base-T hub was $1000.

Network gear is now mostly a commodity, except at the super high end.

The vendors hate that - so they are trying to push the host's functionality into the LAN gear instead. They don't want to provide "dumb pipes" any longer, they want to monkey around with the traffic and protocols, and provide virtual servers and such in their boxes.

Really, it's just an attempt to make the servers the commodity and their gear the expensive part.

Except... you already can implement this yourself with existing equipment and software, with much better control and no vendor lock-in. For low-end solutions, a Linux cluster works great behind an UltraMonkey front end. For higher-end ones, well, that's what IBM z-series mainframes are for.

What a great solution in search of a problem.

Re:Network Vendors (1)

jd (1658) | more than 4 years ago | (#25455481)

Well, most of the good traffic control algorithms are already provided as standard by most GOOD server OS' (Linux, OpenBSD, NetBSD, and the like), most routers and router/switches have also provided those same algorithms, leaving the fast-n-basic switches as the only "dumb" devices (well, other than the CEOs, who are the dumbest devices in any system).

There is a drawback in making Ethernet too expensive by adding too much smarts to the system that I think people are missing. Infiniband isn't THAT much more expensive, but can run at speeds of 24 Gb/s and already has some congestion control. That's 2.4x the bandwidth of the fastest ethernet currently available already, reducing the need for congestion control in the first place, and it does already supply QoS facilities that are quite useful. Once you push Ethernet in the datacentre above the cost of Infiniband in the datacentre for the same performance, Ethernet is going to lose. Price was the only real reason people stuck with it.

QoS is a powerful tool and I like QoS a lot, but QoS can be done cheaply, efficiently and effectively without impacting price or performance. That is not what is being done, and that is why I have a problem with it.

Re:Network Vendors (1)

amorsen (7485) | more than 4 years ago | (#25455845)

Price was the only real reason people stuck with it.

Cabling convenience and range count too.

Re:Network Vendors (1)

ToasterMonkey (467067) | more than 4 years ago | (#25455935)

What? This isn't between Infiniband and ETHERNET! This is the convergence of Fibre Channel with Ethernet. This LOWERS the price of Fibre Channel, which is already king of storage networking in the enterprise. There is no way in hell this will HELP Infiniband in the corporate world. Infiniband will continue to live in HPC land.

Re:Network Vendors (1)

R2.0 (532027) | more than 4 years ago | (#25456025)

"For low-end solutions, a Linux cluster works great behind an UltraMonkey front end"

Demonstrating very succinctly why such technologies won't be adopted by corporate America.

UltraMonkey. Who thinks this shit up?

but what about industrial applications? (1)

saveonweb (939227) | more than 4 years ago | (#25454605)

I think Ethernet is going to take over the data traffic for large corporates and client connections. However, there still exists applications worth multi billion $ that can absolutely not use such a horrible non-reliable protocol and will continue to use serial/profibus communications.

Quiz Time (2, Funny)

inaneframe (971456) | more than 4 years ago | (#25454627)

From the article:
"Ethernet is not optimized to provide the service required for storage and high-performance computing traffic -- speed alone won't cut it, vendors say. Ethernet, which drops packets when traffic congestion occurs, needs to evolve into a low latency, "lossless" transport technology with congestion management and flow control"

Q: Packet loss and traffic congestion are to Ethernet as:
A) blue screens are to Windows
B) registers are to assembly
C) mustard is to sausages

FDDI is laughing from the grave (1)

viridari (1138635) | more than 4 years ago | (#25454629)

The IT world went the cheap route and embraced ethernet. FDDI was technically superior, but required at least two brain cells to set up properly. Brain cells are optional with ethernet.

Perhaps instead of building an inverted house of cards, this consortium should re-examine where FDDI left off and pick up from there.

Combo Firewire/Ethernet port (5, Interesting)

pecosdave (536896) | more than 4 years ago | (#25454641)

There's a draft of Firewire that hasn't hit yet that uses standard Ethernet cables. The port is supposed to be able to use either Firewire OR Ethernet and be able to switch between them depending on what it's plugged into. This sounds ideal to me.

Re:Combo Firewire/Ethernet port (0)

Anonymous Coward | more than 4 years ago | (#25454985)

And Steve Jobs still won't equip it on the new MacBooks.

Re:Combo Firewire/Ethernet port (1)

phillymjs (234426) | more than 4 years ago | (#25455021)

Would have been nice if Apple put that into the new MacBook, since they were so tight on space for ports. Though it wouldn't surprise me if it did have that capability in hardware but it hasn't been worked out in software yet and will appear later as an update. Believe me, I'd happily cough up the $2 for that enabler.

~Philly

More like $30 for the cable that is needed. (1)

Joe The Dragon (967727) | more than 4 years ago | (#25456779)

More like $30 for the cable that is needed.

Re:More like $30 for the cable that is needed. (1)

phillymjs (234426) | more than 4 years ago | (#25457049)

Doubt it. RJ45 and 6-pin Firewire connectors are hardly proprietary. Cheap cables would be available in no time from 3rd-party vendors. Apple probably wouldn't even bother making one.

~Philly

Think of "Ethernet" as "Soup" (4, Informative)

Ancient_Hacker (751168) | more than 4 years ago | (#25454649)

Ethernet is more of a generic name than a specific thing. It's more like "soup" than it is like "VHS".

Ethernet started as a daisy-chained garden-hose-size coax cable with vampire taps. Then RG-58 with BNC connectors, then twisted pairs to a hub, then switching hubs, then wireless... Not much stayed the same, not speed, media, topology,... except maybe carrier-sense. It's basically a comforting name, with the Ethernet-of-the-day varying at the chef's whim.

Keeping the name while tossing out the last remaining bit of commonality is a bit bizarre.

Nope, Carrier Sense is gone too... (3, Informative)

sirwired (27582) | more than 4 years ago | (#25454923)

Nope, the CSMA/CA part of Ethernet is gone also. Specs for a GigE hub exist in the standards, but nobody ever implemented them. (Switching got to be too cheap for anybody to bother.)

Obviously it didn't even get specc'd out with 10Gb Ethernet.

Oh, the frame format is still more-or-less the same from Classic Ethernet. Not identical, but still pretty close.

SirWired

Re:Think of "Ethernet" as "Soup" (1)

sjames (1099) | more than 4 years ago | (#25456655)

Ethernet started as a daisy-chained garden-hose-size coax cable with vampire taps. Then RG-58 with BNC connectors, then twisted pairs to a hub, then switching hubs, then wireless... Not much stayed the same, not speed, media, topology,... except maybe carrier-sense. It's basically a comforting name, with the Ethernet-of-the-day varying at the chef's whim.

Even better, all of those are merely the physical layer. Ethernet itself is the formatting of the frame (destination, source, protocol, data (payload) CRC). However, that has also evolved with the addition of vlan tags, MPLS, etc such that ethernet is more a colloquialism for a whole alphabet soup of standards that mostly work together.

As long as everyone keeps that in mind and has the good taste to add any heavy extra features like reliable transport as layer 3, things will be fine. Otherwise, it will probably fail for the same reason they want to replace FC in the first place.

Ethernet has had its day (1)

Viol8 (599362) | more than 4 years ago | (#25454785)

Why do we still use ethernet? Ethernet was designed to work with multiple access cables in 10B2 and 10B5 layouts with backoff algorithms and all the other stuff that goes with detecting and avoiding collisions. With 10BT all that stuff is irrelevant now - what 10 base T is effectively is a highly complex serial cable carrying just one machines data to a router or switch. All the overhead of frame encoding and decoding and the collision system should be ditched and something more appropriate to a 1 -> 1 connection used instead.

Re:Ethernet has had its day (1)

amorsen (7485) | more than 4 years ago | (#25455945)

Ethernet encoding is simple and cheap. CSMA/CD is gone with 10G and I haven't seen a 1Gbps half duplex connection.

Yes, half duplex should just be banned entirely, but if you can implement it in a $0.10 10Mbps ethernet chip, you can probably survive the added $0.01 in your 1Gbps adapter, even if it never gets used.

SAN over Ethernet has real promise, but... (4, Interesting)

sirwired (27582) | more than 4 years ago | (#25454815)

Fibre Channel over Ethernet has real promise, but these new requirements are a real stumbling block.

What many of the posters here may not realize is that storage traffic (and the "standard" SCSI it uses) is extremely intolerant of dropped frames. A link that in the TCP/IP world would be specatacular is wholly unsuited for block-level storage, which is too latency sensitive to have time to recover from dropped data.

Since the most common cause of dropped frames within a data center is congestion, FCoE requires your Ethernet to implement frame-based flow control, which prevents the congestion from occuring, along with the resulting frame loss.

SirWired

Re:SAN over Ethernet has real promise, but... (1)

stevebyan (806118) | more than 4 years ago | (#25455567)

So why the heck didn't they base FCoE on SCTP, instead of trying to require the darn _network_hardware_ to implement TCP-like congestion control?

Re:SAN over Ethernet has real promise, but... (2, Interesting)

ToasterMonkey (467067) | more than 4 years ago | (#25455617)

Fibre Channel over Ethernet has real promise, but these new requirements are a real stumbling block.

Something to note is that the Ethernet in FCoE is really not the same Ethernet we use today. The acronym really confuses things. The article offers some better names for the new Ethernet standard, "Converged Enhanced Ethernet (CEE)", "Data Center Ethernet (DCE)." It really is the convergence of Fibre Channel and Ethernet, NOT Fibre Channel glued to the back of Ether. Think of it more like a gigantic leap for Ethernet (and IP/TCP eventually, as functionality is pushed down a few layers), not so much a downgrade of Fibre. Also, this mostly applies to 10Gig Ether, which is already pretty damned different from previous forms of Ethernet.

These [ieee802.org] are the new Ethernet standards.

I think it's necessary to explain all this because while most posters don't know dick about storage and think existing Ethernet is good enough for everything, a good number of them might also be SAN admins that shrug it off without knowing that the "Ethernet" in the acronym has changed. HBA's aren't going anywhere, they will just be running more IP traffic now =) Also, if iSCSI is still around (gag me with a spoon if it is) it will at least have a better foundation to stand on. Damn I really hope FCoE ends its misery though.

ATM.. hello.. is anyone there.. (0)

Anonymous Coward | more than 4 years ago | (#25454833)

Lemme see here.. ethernet.. collision detection... vs ATM collision avoidance..

collision avoidance.. imagine that no collisions..

multiple levels of QOS.. more addressable space than IPV6.. AND intelligent, on the fly, reconfigurable routing that supports redundant paths natively.

naw.. ATM.. it's too hard..

we're gonna slap more standards on ethernet to make it do things it can't do inheirently and then slap more stuff on top like mpls and 802.1Q and stuff to make it like atm.. but easy. ya..

laymen are morons.

This is FUD... (4, Interesting)

volxdragon (1297215) | more than 4 years ago | (#25454907)

Ethernet already has flow control at the link-level - they're called stop frames (and since all modern switches give you dedicated links to end workstations and have some amount of hardware buffering, collisions/overrun aren't an issue). Now, since the world really runs on IP (doing raw ethernet would only ever work in the most local of LAN applications which is rather pointless in most deployments), and IP has TOS bits (which every real modern router can classify, queue, and throttle per-queue all in the hardware fast-path with no additional latency), I'm failing to see what they're proposing to solve since the problem is already solved. 1G/10G switches are used all over data centers and in HPC situations today (and have been for years)...

Re:This is FUD... (1)

stevebyan (806118) | more than 4 years ago | (#25455827)

The problems they are trying to solve are two-fold:

1) there are a number of legacy protocols (UDP-based stuff, reliable multicast, etc) that have no congestion control. Typically in the past the datacenter would provision a separate network for these if low loss and/or low latency were important. What with virtualization and cost-pressure, everyone now wants a single converged network fabric. To handle these legacy protocols in a converged network, the network hardware itself (switches and NICs) now has to provide congestion control.

2) TCP doesn't provide application-level framing, hence hardware cannot parse upper-level protocol headers in TCP segments. Therefor it isn't possible to build cheap NICs/HBAs that can handle 10 GbE wire-rate iSCSI or iWARP RDMA or iFCP. Rather than fix this by moving iSCSI, iWARP, etc to SCTP, which does provide application-level framing over IP (along with TCP-compatible congestion control), the industry has decided to instead use raw Ethernet frames for application-level framing and push the congestion control into the NICs and switches (i.e. FCoE).

Meanwhile, the IETF can say "I told you so", since the storage industry ignored the network weenie's advice to base iSCSI on SCTP back in 2002 or so.

This all seems brain-dead to me, but I'm sure Cisco will make a mint out of owning all the datacenter switching infrastructure.

Re:This is FUD... (1)

voidptr (609) | more than 4 years ago | (#25456433)

doing raw ethernet would only ever work in the most local of LAN applications which is rather pointless in most deployments

Which is exactly the deployment FCoE and several upcoming ethernet uses are aimed at.

A handful of SAN boxes serving FCoE on the same segment as the servers they're serving. Basically the same way you provision FC today. The storage and servers are extremely local, and there's no reason to stuff IP in the middle when it will never be routed.

If you want less performance but the ability to route it over an IP network, use iSCSI.

Re:This is FUD... (1)

rnxrx (813533) | more than 4 years ago | (#25457291)

They're generally called pause frames and they've not been universally implemented across the products of various switch and NIC vendors, let alone OS drivers.

The technique discussed (PFC) is actually an extension of this mechanism. Whereas the current model drops *all* traffic in the case of a buffer congestion event, PFC allows for certain types of traffic (e.g. best effort) to be paused/queued to insure bandwidth is available for other types of traffic (lossless services - like FCoE).

Overall, the intent is to provide the same sorts of delivery guarantees currently available on Fibre Channel via buffer credits to a converged Ethernet network.

"The next generation, far beynod 10Base-T" (0)

Anonymous Coward | more than 4 years ago | (#25455077)

Oh golly - the "next generation(tm)", far beyond 10BASE-T!!!

Who knows, maybe they will reach something so fast they could call it 100BASE-T, or even 1000BASE-T - only the future will tell, and the future clearly belongs to this upcoming "next generation technology".

As for how to differentiate between Ethernets... (1, Funny)

Anonymous Coward | more than 4 years ago | (#25455235)

You can have Ethernet Classic and New Ethernet!

AND...

If New Ethernet is a dud, you can say it was a marketing gimmick to get people to drink... err, switch back to Ethernet Classic!!!

USB (3, Informative)

psbrogna (611644) | more than 4 years ago | (#25455279)

Given the direction SATA & USB is going, the rate at which its bandwidth has increased relative to traditional CATx ethernet, and the relatively lower cost of interconnection devices, is Ethernet really the best? If we're going to making significant wiring changes in server rooms I'd prefer to just do it once and standardize on the cheapest, fastest "2-wire" solution that makes sense.

Re:USB (1)

ToasterMonkey (467067) | more than 4 years ago | (#25456109)

Bzzzzzzzt!

Those are not network protocols. You know, the N in SAN and LAN? Also, SATA nor USB can operate at 10Gbps.

Re:USB (1)

psbrogna (611644) | more than 4 years ago | (#25456937)

I do get the diff between the layers. What I'm trying to suggest is that if there's going to be some rewiring in the fishbowl, why not use the fastest, cheapest solution that has the potential to support all the scenarios? I don't have a 10 Gb/s server room, I have a little 1 Gb/s server room; the SATA & USB specs are either already > than MY twisted pair bandwidth and for a lower cost or will be there in the near future and by trends I've observed will continue to out pace twisted pair bandwidth/$.

My Point.... (1)

maz2331 (1104901) | more than 4 years ago | (#25455985)

My point wasn't based on Infiniband or Myrinet, or any other interconnect. It just seems to me that all of this is a solution desperately searching for a problem.

Different technologies exist for a reason: they do what they are designed for as optimally as possible. Trying to make one network technology that does "Everything" is doomed to epic failure. It will become a compromise that does lots of things, and few of them well.

Not going to displace SAN's anytime soon (1)

tdp252 (519328) | more than 4 years ago | (#25456071)

I work as a Storage Architect at a large telco. For moving massive quantities of data from point-a to point-b SAN's still rule the day for us and I don't see that changing anytime soon.

Ethernet has several major deficiencies that make it less attractive for being the dump truck of data movement. I'm writing this while thinking of the large enterprise Backup and Recovery environments out there but there are other applications that involve moving massive (think 100+ terabytes nightly) amounts of data around that SAN's are still better suited for.

First of all SAN's by inherent design have the ability to aggregate data across multiple ISL's (trunks) in real-time. If you have 2 pipes between switches your I/O's will be evenly distributed across the links adjusting in real time as needed to fully utilize both links. Need more bandwith? Simply plug in another ISL, done.

Ethernet routing isn't quite as intelligent. Being that data transfers are session based, you can have a completely flooded trunk with the other sitting there idle for endless hours.

While true the next session that starts may choose the 2nd under-utilized path, your pretty much SOL on performance with the first one if it becomes saturated.

This isn't quite as painful if your data transfers involve a vast number of relatively "quick" transactions such as FTP's but with NFS/CIFS mounts these may not be re-evaluated for days, weeks or months. So once a route has been picked you are essentially stuck with that routing decision across your trunk until the session has been re-established.

SAN's were designed for large scale data movement and high IOPs from the beginning. Ethernet on the other hand needs quite a bit of tweaking and still comes up short for some large enterprise applications.

Re:Not going to displace SAN's anytime soon (1)

voidptr (609) | more than 4 years ago | (#25456609)

First of all SAN's by inherent design have the ability to aggregate data across multiple ISL's (trunks) in real-time. If you have 2 pipes between switches your I/O's will be evenly distributed across the links adjusting in real time as needed to fully utilize both links. Need more bandwith? Simply plug in another ISL, done.

So does Ethernet: EtherChannel [wikipedia.org].

An FCoE deployment is going to be as local as an FC SAN. You aren't going to do IP routing, you're just doing local ethernet frame switching. There's no IP involved.

Imagine your current FC network today, and replace all the switches and cables with Ethernet.

Re:Not going to displace SAN's anytime soon (1)

tdp252 (519328) | more than 4 years ago | (#25456835)

Ether-channel may provide additional bandwith on each pipe that's being used to connect two switches, but it doesn't solve the issue of each session deciding only once which of the multiple pipes to use.

Kinda obvious when you think about it. (1)

m.dillon (147925) | more than 4 years ago | (#25456831)

What are ethernet's two biggest strengths? Think about it a moment.

It is high speed at a very low cost, and the ubiquitousness of the technology.

It doesn't really matter that the technology is inferior to the numerous very fast low latency solutions available today because all of those solutions are also very high-cost, low-volume solutions relative to ethernet, and have no ability to be incrementally upgraded. It doesn't matter that there are congestion issues when pushing a packet-switched technology to its limit, because anyone who has lived with this stuff for the last few decades knows that you simply overbuild it for your needs and all those problems disappear. It doesn't matter that the price point is GiGE because the technology has proven over and over again that the price point moves very quickly with adoption. The price point will soon be 10GiGE, and after that 100GiGE. It is unarguable. It doesn't matter that ethernet traditionally has higher latencies then currently existing higher-cost solutions. Latency has never been a show stopper for anyone except the super-computing dweebs.

The price of hardware has dropped around the ears of the higher cost solutions to the point where only a very small application space can actually gain an advantage using them. It will only get worse over time.

So even though, GiGE isn't quite fast enough to match platter rates on the best disks, let alone transfer rates available from SSDs, it doesn't matter. The low cost and clear future trends for the technology will trump everything. And the people making the decisions have the choice: They can be early adopters of 100GiGE, or choose to spend more of a premium on 10GiGE, but with the flexibility of NOT having to rip up their entire infrastructure. The ability to piecemeal-replace infrastructure is its own trump card.

-Matt

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...