×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Multi-Threaded SSH/SCP

kdawson posted more than 6 years ago | from the recovering-wasted-bandwidth dept.

Networking 228

neo writes "Chris Rapier has presented a paper describing how to dramatically increase the speed of SCP networks. It appears that because SCP relies on a single thread in SSH, the crypto can sometimes be the bottleneck instead of the wire speed. Their new implementation (HPN-SSH) takes advantage of multi-threaded capable systems dramatically increasing the speed of securely copying files. They are currently looking for potential users with very high bandwidth to test the upper limits of the system."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

228 comments

A likely story (5, Funny)

sqldr (838964) | more than 6 years ago | (#22404034)

Hi, I've invented a new way of downloading pron^H^H^H^H^H^Hcopying files across a network. If you have uber bandwidth, please contact me urgently!

Re:A likely story (5, Insightful)

slyn (1111419) | more than 6 years ago | (#22404072)

Makes you wonder how many innovations can either be directly attributed to or partially attributed to the distribution of porn (not (necessarily) that this is).

VHS v Betamax comes to mind.

Re:A likely story (-1, Redundant)

miffo.swe (547642) | more than 6 years ago | (#22404170)

I often wonder how internet would look like without pr0n. Where would things like online video, webshops, webcams, bandwidth and browsers be standing without it?

Without the pretty grlz i could live with a text only internet any day.

Re:A likely story (2, Insightful)

harry666t (1062422) | more than 6 years ago | (#22404386)

Get a /real/ girl.

Re:A likely story (2, Funny)

neumayr (819083) | more than 6 years ago | (#22404570)

They're overrated..

Re:A likely story (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#22404592)

CmdrTaco gave a presentation last night at the Holland, Michigan Taco Bell bathroom. He discovered a new way to have gay sex with 2 guys at once.

Re:A likely story (2, Funny)

empaler (130732) | more than 6 years ago | (#22404634)

AND they get pregnant.

Re:A likely story (1)

Sczi (1030288) | more than 6 years ago | (#22405110)

And then you're just going to want to post their nudy pics on the net anyway, so we're back to square one.

FUNNY?! That's not funny, try for TRUE (1)

empaler (130732) | more than 6 years ago | (#22405450)

Srsly. I'm gonna be a father this year. DO NOT WANT!

Re:FUNNY?! That's not funny, try for TRUE (1)

Tolkien (664315) | more than 6 years ago | (#22405496)

After someone says something like that, I can only feel bad for the baby. :|

Re:FUNNY?! That's not funny, try for TRUE (5, Insightful)

Anonymous Coward | more than 6 years ago | (#22405972)

I won't lie to you -- being a parent is no laughing matter. It is a ton of work. It can be amazingly stressful and expensive. I've been through periods that I look back on now and wonder how the hell I managed to pull through without going completely insane. But if you ask me, the rewards outweigh the difficulties ten to one.

When your child first looks up at your face and you see actual recognition in her eyes... when you see all the blocks fall into place as she figures out how to do something for the first time... look, I know it sounds really sappy and smarmy, but seriously (srsly) it is absolutely indescribable. This thing started out as a bit of genetic code from two people, and now it is actually self-aware and sentient. How cool is that? What geek can't be astonished at these emergent properties, derived from a program more complicated than you can possibly imagine -- a program that has spontaneously evolved over time?

And you get to see her mental map evolve. You watch branches get added to her decision tree. You observe as she learns how to acquire information, process it, and decide how to act upon it. And all the while, you mold her view of the world based on your interactions with her. I don't know about you, but I find that not only fascinating, but incredibly rewarding.

Before my daughter was born, I was terrified too, and somebody had said these things to me, I would've said, "Yeah, okay, I'm sure it's great and all, but I'm sure you're exaggerating somewhat." That's because there is something that happens to you when it's your kid. There's some very ancient, very basic code that gets turned on in your brain that says "this life is your responsibility, and you must do everything you can to ensure its safety, survival, and growth". I can't explain it because I honestly believe it's something buried deep beneath the conscious mind.

Whatever the case, if you honestly don't want the baby, for it's sake, put it up for adoption. Don't make it live a life with a father who doesn't care for it. I'm being absolutely serious here. Find a loving couple who are unable to have kids of their own.

(Posting AC because this is way offtopic, and because there are a lot of single, selfish, bitter child-haters out there with mod points to burn... but I had to say something.)

Re:A likely story (2, Interesting)

Anonymous Coward | more than 6 years ago | (#22405048)

...what's wrong with looking at porn with a real girl?

Re:A likely story (5, Insightful)

shenanigans (742403) | more than 6 years ago | (#22404426)

I think this story is interesting because it shows a general trend: increased focus on multi-threading. We will see much more of this in the future as multi-core and multi-processor systems become more common. This trend is driven not by porn though, but by that other big driving force behind the computer industry, gaming.

Re:A likely story (5, Insightful)

rapier1 (855437) | more than 6 years ago | (#22405990)

Actually, this is one of the main reasons why we did this. If you look at where processor architectures are going they aren't necessarily increasing the pure computational power of each core as much as they are using more and more cores. If you are in a situation where you have a single threaded process that is CPU bound - like SSH can be - you'll find that throughput rates (assuming that you aren't network bound) will remain flat. In order to make full use of the available network capacity we'll have to start doing things like this. There is still a lot of work to be done but we're pleased by our progress so far.

Re:A likely story (5, Insightful)

mikael (484) | more than 6 years ago | (#22404810)

People are always willing to pay more to be entertained that to be educated.

Which explains why football players and movie stars will get paid more than the innovators that carried out the research to develop the broadcast technology that helped to make those stars famous.

Re:A likely story (1, Insightful)

Anonymous Coward | more than 6 years ago | (#22405104)

People are always willing to pay more to be entertained that to be educated.

Which explains why football players and movie stars will get paid more than the innovators that carried out the research to develop the broadcast technology that helped to make those stars famous.
Another big issue is that there are fewer stars than researchers. As such, it is more difficult to replace any given star than to replace any given researcher. The star does a greater portion of the work than the researcher (even if all the stars do a smaller portion than all the researchers).

Replace a star NFL quarterback with his backup, big drop in performance of the team. Replace a top researcher with a new hire, a big drop in performance of her immediate team but the overall group of teams is impacted only slightly.

Oblig applicable Dilbert (4, Funny)

bigmouth_strikes (224629) | more than 6 years ago | (#22404234)

Wally: "My proposed work plan for the year is to stress-test our product under severe network conditions. I will accomplish this by downloading large image files from the busiest servers on the net."

(PHB rejects suggestion)
(later)

Wally: "I was this close to making it my job to download naughty pictures."
Dilbert : "It's just as well; I would have had to kill you."

( http://books.google.com/books?id=dCeVfKrZ-3MC&pg=PA77&source=gbs_selected_pages&cad=0_1&sig=xD5tmMhG1RcspLch8gCIJu8ro2U#PPA79,M1 [google.com] )

this is just what i've been looking for (2, Funny)

Anonymous Coward | more than 6 years ago | (#22404052)

not that scp as-is isn'--stalled--

Must be why rsync over ssh is much faster (4, Insightful)

MichaelSmith (789609) | more than 6 years ago | (#22404060)

I get a lot of use out of ssh for moving files around and rsync is definitely the best way to do heavy lifting in this area. Improving scp would be good to. I can't wait to hear what Theo thinks about this. I don't see him as a fan of adding complexity to improve performance.

Big scp copies through my wifi router used to cause kernel panics under netbsd current of about a year ago. I never had that problem running rsync inside ssh.

Re:Must be why rsync over ssh is much faster (1)

Anonymous Coward | more than 6 years ago | (#22404136)

Eh? I've not even read the article but the summary clearly states that crypto is a bottleneck for some networks. Multithread that and everything tunneled over SSH should see improvements on modern multiproc boxes -- at least where it's the crypto overhead and not bandwidth that's the current bottleneck.

Re:Must be why rsync over ssh is much faster (4, Interesting)

MichaelSmith (789609) | more than 6 years ago | (#22404152)

There is definitely something funny(strange) about the way scp does bulk copies. It stops and starts. Other applications happily stream through encrypted ssh connections.

And in my experience rsync is faster.

Re:Must be why rsync over ssh is much faster (5, Interesting)

drinkypoo (153816) | more than 6 years ago | (#22406010)

If you just want to copy some files from system to system in an encrypted fashion, then the BEST option by far is to use tar, and pipe it through ssh like so:

tar cvfpz - * | ssh user@host '( cd /destination ; tar xvfpz - )

This example will compress and encrypt your data before sending it; on the other end, the file is streamed to tar. This example requires GNU rar or a close facsimile.

Now, if you want to UPDATE a directory, use rsync:

rsync -av -e ssh * user@host:/destination/

Because rsync will do partial checksums and send parts even of BINARY files if the whole file has not changed, and doesn't re-send unchanged files, rsync makes sense when updating a directory. But it provides no speedup benefit over using tar, and in fact the directory scans it does before the sync mean that it may actually be slower.

Use scp only for copying single files, because you're right, scp chokes between each file.

Re:Must be why rsync over ssh is much faster (1, Insightful)

cheater512 (783349) | more than 6 years ago | (#22404416)

Rsync doesnt encrypt. SSH/SCP does.

Rsync is only really useful as a synchronizing method between a source and a out of date copy.
Then its real benefits get shown.

Re:Must be why rsync over ssh is much faster (3, Informative)

AnyoneEB (574727) | more than 6 years ago | (#22404520)

Unless your servers are running rsh, rsync is probably going to get routed through ssh, in which case it gets encrypted just like scp. ref [everythinglinux.org] :

Secure Shell - The security concious of you out there would like this, and you should all be using it. The stream from rsync is passed through the ssh protocol to encrypt your session instead of rsh, which is also an option (and required if you don't use ssh - enable it in your /etc/inet.d and restart your inet daemon if you disabled it for security).

Re:Must be why rsync over ssh is much faster (1)

Tony Hoyle (11698) | more than 6 years ago | (#22404876)

That's just flat out wrong. rsyncd has its own, unenctypted, protocol.

You can run it from inetd or as a daemon, but it's unrelated to rsh.

That connection may or may not be encrypted depending on the route it takes.. VPNs tend to be encrypted for example, but LAN connections not.

Re:Must be why rsync over ssh is much faster (1)

blueg3 (192743) | more than 6 years ago | (#22405832)

From "man rsync":

There are eight different ways of using rsync. They are: ...
* for copying from the local machine to a remote machine using a remote shell program as the transport (such as ssh or rsh). This is invoked when the destination path contains a single : separa-tor.
* (same as the above, but copy from remote to local machine)

Note that the remote machine could alternately have an rsync server. However, this is not required -- if the remote machine does not have an rsync server, transport is done via a shell. In my experience, this often how people use rsync, and they often use ssh as the transport shell.

Can't we all just get along? (2, Interesting)

piojo (995934) | more than 6 years ago | (#22405836)

Okay, it's very simple.

Encrypted and tunneled over SSH, rsync is spawned by a login shell at the other side:
rsync /some/path myhost.com:my/directory

Not encrypted, rsyncs daemon must be running at other end:
rsync /some/path rsync://myhost.com/my/directory OR
rsync /some/path myhost.com::my/directory

Re:Must be why rsync over ssh is much faster (1)

rasputin465 (1032646) | more than 6 years ago | (#22405998)

And if you have both available, it becomes a little ambiguous. However, this excerpt from the rsync man page indicates it uses ssh as default (unless the system is old):

Once installed, you can use rsync to any machine that you can access
via a remote shell (as well as some that you can access using the rsync
daemon-mode protocol). For remote transfers, a modern rsync uses ssh
for its communications, but it may have been configured to use a dif-
ferent remote shell by default, such as rsh or remsh.

You can also specify any remote shell you like, either by using the -e
command line option, or by setting the RSYNC_RSH environment variable.


Alternative solution for a trusted LAN (5, Interesting)

Diomidis Spinellis (661697) | more than 6 years ago | (#22404096)

If you want to speed up transfers and you're working on a LAN you trust (i.e. you don't worry about the integrity and confidentiality of the data passing through it), you can dramatically increase throughput using socketpipe [spinellis.gr] . Although the initial socketpipe communication setup is performed through client-server intermediaries such as ssh(1), the communication channel that socketpipe establishes is a direct socket connection between the local and the remote commands. This eliminates not only the encryption/description overhead, but also the copying between your processes and ssh or rsh.

Re:Alternative solution for a trusted LAN (4, Informative)

upside (574799) | more than 6 years ago | (#22404176)

You can also use a cheaper cipher. From the ssh manpage:

-c blowfish|3des|des
                          Selects the cipher to use for encrypting the session. 3des is
                          used by default. It is believed to be secure. 3des (triple-des)
                          is an encrypt-decrypt-encrypt triple with three different keys.
                          blowfish is a fast block cipher, it appears very secure and is
                          much faster than 3des. des is only supported in the ssh client
                          for interoperability with legacy protocol 1 implementations that
                          do not support the 3des cipher. Its use is strongly discouraged
                          due to cryptographic weaknesses.

Re:Alternative solution for a trusted LAN (1, Informative)

Anonymous Coward | more than 6 years ago | (#22404258)

Or just compile from source and enable the 'none' "cipher".

I surely missed having that option when copying files between hosts on my LAN. I don't need to hide data from myself. If someone else connects and encrypting data is a concern, I'll simply not use the 'none' "cipher".

Re:Alternative solution for a trusted LAN (1)

dissy (172727) | more than 6 years ago | (#22405008)

Or just compile from source and enable the 'none' "cipher".

I surely missed having that option when copying files between hosts on my LAN. I don't need to hide data from myself. If someone else connects and encrypting data is a concern, I'll simply not use the 'none' "cipher".
-1 redundant, -1 fail at slashdot

from the linked article:

Dynamic Windows and None Cipher
This is a basis of the HPN-SSH patch set. It provides dynamic window in SSH and the ability to switch to a NONE cipher post authentication. Based on the HPN12 v20 patch.

Re:Alternative solution for a trusted LAN (1)

Chris Pimlott (16212) | more than 6 years ago | (#22404404)

And if you really trust your network, you can recompile SSH with the 'none' cipher enabled, which (as the name implied) uses no encryption on the datastream whatsoever.

Re:Alternative solution for a trusted LAN (2, Interesting)

HAKdragon (193605) | more than 6 years ago | (#22405564)

I may be speaking out of ignorance, but doesn't that defeat the point of SSH?

Re:Alternative solution for a trusted LAN (5, Informative)

KiloByte (825081) | more than 6 years ago | (#22404508)

Actually, it appears that (at least on Debian) AES is already the default. Selecting 3des gives tremendous slowdown; blowfish is somewhat slower than AES.

Copying 100MB of data over 100mbit ethernet to a P2 350Mhz box (the slowest I got) gives:
* 3des 1.9MB/s
* AES 4.8MB/s
* blowfish 4.4MB/s

Re:Alternative solution for a trusted LAN (2, Informative)

Neil Watson (60859) | more than 6 years ago | (#22405206)

Actually, it depends upon the SSH protocol. Both Debian and Cygwin have this to say:

        -c cipher_spec
                          Selects the cipher specification for encrypting the session.

                          Protocol version 1 allows specification of a single cipher. The
                          supported values are "3des", "blowfish", and
                          "des". 3des (triple-des) is an encrypt-decrypt-encrypt
                          triple with three different keys. It is believed to be secure.
                          blowfish is a fast block cipher; it appears very secure and is
                          much faster than 3des. des is only supported in the ssh client
                          for interoperability with legacy protocol 1 implementations that
                          do not support the 3des cipher. Its use is strongly
                          discouraged due to cryptographic weaknesses. The default
                          is "3des".

                          For protocol version 2, cipher_spec is a comma-separated list of
                          ciphers listed in order of preference. The supported ciphers are:
                          3des-cbc, aes128-cbc, aes192-cbc, aes256-cbc, aes128-ctr,
                          aes192-ctr, aes256-ctr, arc
                          four128, arcfour256, arcfour, blowfish-cbc, and cast128-cbc. The default is:

                                      aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,
                                      arcfour256,arcfour,aes192-cbc,aes256-cbc,aes128-ctr,
                                      aes192-ctr,aes256-ctr

Re:Alternative solution for a trusted LAN (1)

MoogMan (442253) | more than 6 years ago | (#22404348)

Or use netcat/nc (installed by default on most Linux distros). Server cats it's output directly to a file (or to tar -x). Client grabs it's input from a file.

Server: nc -l 1234 | tar -x
Client: tar -c file_list_here | nc localhost 1234

Re:Alternative solution for a trusted LAN (2, Informative)

Diomidis Spinellis (661697) | more than 6 years ago | (#22404636)

Nc is useful, but it still involves the overhead of copying the data through it (once at the client and once at the server). Nowadays, in most settings this overhead can be ignored. But, given the fact that a well-behaved application will work with a socket exactly as well as with a pipe or a file descriptor, I thought it would be very elegant to be able to connect two instances of (say) tar through a socket. Hence the implementation of socketpipe. Socketpipe sets up the plumbing and then just waits for the programs to finish.

This is the setup using nc:

tar --pipe--> nc --socket--> nc --pipe--> tar

and this is the setup that socketpipe arranges:

tar --socket--> tar

Re:Alternative solution for a trusted LAN (0)

Anonymous Coward | more than 6 years ago | (#22405058)

It can also be done in another way, using standard ssh (so it can be used on a lan you don't trust):

tar zcf - * | ssh remotehost tar zxvf - -C /path/to/remotedir

You also get compression on the fly, which helps a bit; replace ssh with rsh if you wish.
It also works out of the box across different *nix variants: no problem if you don't have socketpipe on the receiving end (i'm assuming it's less widely available tha ssh).

Regards,
Alberto.

Re:Alternative solution for a trusted LAN (1)

houghi (78078) | more than 6 years ago | (#22404824)

The situation where this can be usefull will be almost non-existing. The places that could benefit from the increase will most likely have several people working.

That means the better way is to encrypt BY DEFAULT. Security is not so much a technical issue. It is also a social one. Encrypting by default will benefit you in the long run.

This means also sending gpg signed messages by default. Much shorter, beter and more usefull then the legal sh|t they attach now.

Re:Alternative solution for a trusted LAN (1)

angus_rg (1063280) | more than 6 years ago | (#22405616)

There is also no security in socketpipes, so it isn't much different from using FTP from a security standpoint.

I think the point of this article is to improve a "secure" method of data transfer by making the way it deals with encryption better. Even using weaker encryption methods, you still have this problem.

I think the idea makes perfect sense and probably could improve file transfer dramatically. Realistically, it is just a software encryption accelerator. If they could implement it in the NIC card, that would be a better solution, but probably a headache from a OS programming/encryption update standpoint.

How will this affect application deployment (1)

Zombie Ryushu (803103) | more than 6 years ago | (#22404116)

How will this affect the operation of applications wich use SSH to standardie applications accross Linux like urpmi's --parallel parameter?

This is one of the most useful aspects of Mandriva, but as the number of nodes I have to manage increases, I find RPMS being SCPed to other nodes taking longer and longer. I think this is because even though with Kerberos Authentication is much faster, urpmi is waiting until one node finishes copying the files to start copying to the next node in the Domain.

Thoughts?

Sweet! (5, Insightful)

miffo.swe (547642) | more than 6 years ago | (#22404142)

I really hope this will make it into OpenSSH after some security auditing. The performance gains was pretty impressive. It will make ssh much more fun for rsync, backups and other times when i transfer large files. I also wonder if one cant get similar performance gains with normal ssh and for example forwarded X-windows. That would be very interesting indeed.

Re:Sweet! (1)

Jah-Wren Ryel (80510) | more than 6 years ago | (#22404952)

I also wonder if one cant get similar performance gains with normal ssh and for example forwarded X-windows.
Probably not. The X11 protocol is very latency sensitive, so the bottleneck tends to be round-trip times rather than raw throughput.

I haven't read the article, so I don't know what it says about per-packet set-up times, but I wouldn't be surprised if latency was actually increased due to the overhead of having to at least decide to distribute encryption work across multiple CPUs.

Re:Sweet! (3, Informative)

Per Wigren (5315) | more than 6 years ago | (#22405124)

Use NX [nomachine.com] instead of plain old remote DISPLAY or ssh's X11 forwarding or even VNC! It's silly fast! You get a perfectly usable desktop even on slow, high latency connections. The free edition is free as in GPL.

Re:Sweet! (0)

Anonymous Coward | more than 6 years ago | (#22405398)

Gentoo's ebuild for openssh [gentoo-portage.com] includes a hpn USE flag for it. Go forth and build the system & tools as you need/want them. Portage is best package manager out there.

To *have* such problems... (5, Interesting)

pla (258480) | more than 6 years ago | (#22404198)

the crypto can sometimes be the bottleneck instead of the wire speed.

Between two devices on my gigabit home LAN, the CPU barely even registers while SCP'ing a large file (and that with every CPU-expensive protocol option turned on, including compression). What sort of connection do these guys have, that the CPU overhead of en/decryption throttles the transfer???


Coming next week: SSH compromised via a thread injection attack, thanks to a "feature" that only benefits those of us running our own undersea fiber.

Re:To *have* such problems... (5, Informative)

dm(Hannu) (649585) | more than 6 years ago | (#22404252)

They claim that the first bottleneck is actually flow control of buffers, which prevents utilizing full network bandwidth in normal gigabit connections. The threads will help only after this first bottleneck has been cleared. They have patches to fix both problems. The slashdot summary was therefore a bit inaccurate, and reading TFA certainly helps.

Re:To *have* such problems... (5, Informative)

egede (266062) | more than 6 years ago | (#22404748)

The limitations of transfer rates for scp is often the round trip time that consumes time for confirmation of received packages. This is a serious issue for transfers from the Europe to the US West Coast (around 200 ms) or to Australia (around 400 ms). Having several parallel TCP streams can solve this problem and has been in use for many years for transfer of data in High Energy Physics. An example of such a solution is GridFTP http://www.globus.org/toolkit/docs/4.0/data/gridftp/ [globus.org] .

Re:To *have* such problems... (1)

Vellmont (569020) | more than 6 years ago | (#22405730)


The limitations of transfer rates for scp is often the round trip time that consumes time for confirmation of received packages. This is a serious issue for transfers from the Europe to the US West Coast (around 200 ms) or to Australia (around 400 ms).

Huh. I'm surprised TCP sliding window protocol doesn't take care of that. Shouldn't it account for filling up the pipeline between sender and receiver?

Re:To *have* such problems... (2)

Andy Dodd (701) | more than 6 years ago | (#22406002)

Older implementations of TCP only allow for a 64 KB window size. (Older meaning "really old" - nearly any implementation in the last decade implements extensions that allow for much larger transmit/receive windows.)

Many apps set fixed window sizes (incl. apparently standard SSH - the webpage implies 64K.)

Linux can "autotune" window sizes, but most OSes don't, hence the need for an app to be able to specify a larger window.

Even with larger window sizes, TCP congestion control starts breaking on networks with high speed and large BDP. For example, even with a BER of something like 1*10^-12, a gigabit connection will drop something like one packet per hour - on a gigabit connection with 400 ms latency, 1 hour is apparently about how long it will take for the congestion control algorithm to even ramp up to full wire speed. One packet drop and all of a sudden TCP throttles way back - hence the large amount of work on advanced congestion control algorithms that handle high BDP networks better. (Such as the current default algo in Linux, CUBIC.)

Using multiple TCP streams works around this problem in a more cross-platform manner as it does not depend on the OS's TCP implementation getting tweaked.

Re:To *have* such problems... (0)

Anonymous Coward | more than 6 years ago | (#22404318)

those of us running our own undersea fiber.

What, don't you? Pfft, luddite.

Re:To *have* such problems... (1)

sydbarrett74 (74307) | more than 6 years ago | (#22405054)

those of us running our own undersea fiber.
What, don't you? Pfft, luddite.
Well I had mine but some NSA spy^H^H^H^Hidiot seaman with a wayward anchor took them out.

Re:To *have* such problems... (5, Interesting)

totally bogus dude (1040246) | more than 6 years ago | (#22404364)

Have you measured your actual throughput on the file transfer? It tends to take a crapload of tuning to get anywhere near saturating gigabit, even if you're not using encrypted transfers.

I wrote the bit below which I'll keep because it might be interesting to someone, but dm(Hannu) already mentioned the claw flaw in the logic behind the PP and article summary: if the CPU is the bottleneck, how could adding more threads possibly help?

Just for a laugh I used scp to copy a 512 MB file from my file server to itself, an Athlon 3700+ running at 2.2ghz. I got about 18 megabytes / second out of it. I took a snapshot of top's output right at the end (97% complete) and the CPU usage was as follows:

ssh: 48.6%
sshd: 44.9%
scp: 3.7%
scp: 1.3%
pdflush: 0.7%

So this system was pretty much pegged by this copy operation, and it achieved less than a fifth the capacity of a gigabit network link. Obviously the system is capable of transferring data much faster than this; the source was a RAID-5 set of 5 new 500 GB drives, and the destination was a stripe across two old 40 GB drives. I'd also repeated the experiment a few times (and this was the fastest transfer I got) so it's likely the source file was cached, too.

I do agree that there's probably more interesting and useful things to optimise (and make easy to optimise) than scp's speed, but I know for sure that scp'ing iso images to our ESX servers in a crapload slower than using Veeam's copy utility or the upload facility in the new version of Infrastructure Client (at least I think it's new, never noticed it before).

Re:To *have* such problems... (4, Interesting)

mikael_j (106439) | more than 6 years ago | (#22404414)

A possible problem source here is that you're also doing disk I/O, when transferring data on my home network I've noticed that rsyncing things for redundancy purposes I end up with a lot more CPU usage (even when reading from a RAID5 via a hardware controller) than if I just pump random data from one machine to another. I reommend you try just transferring random data and piping it directly to /dev/null on the receiving machine to see if there's any difference in CPU usage.

/Mikael

Re:To *have* such problems... (1)

Kyojin (672334) | more than 6 years ago | (#22404478)

That's mostly a good idea except for the randomly generated bit. Random number generation can be cpu intensive. A better option might be to randomly generate 500Mb of data in memory and send that to dev/null over whatever link, using the same data each time.

Re:To *have* such problems... (2, Interesting)

totally bogus dude (1040246) | more than 6 years ago | (#22404674)

True enough, but my main point was that getting to actual gigabit speeds in the first place is actually pretty difficult. Plus, I couldn't find an easy way to copy only X amount of "random" data via scp which was the point of the article. Regardless, copying data is rarely if ever a useful thing to do with scp, anyway.

Re:To *have* such problems... (1)

sammydee (930754) | more than 6 years ago | (#22404384)

This could be very useful for older computers or embedded devices such as routers.

Re:To *have* such problems... (1)

alexhs (877055) | more than 6 years ago | (#22404842)

Hello !

My home server/internet gateway is a Pentium MMX at 200MHz, with a 100 Mb/s NIC.
With SCP (default options, server sending), I can transfer at 8Mb/s.
With RCP, at 25 Mb/s

Re:To *have* such problems... (1)

BitZtream (692029) | more than 6 years ago | (#22404954)

This was my first thought as well. I've never had SSH use more than a tiny amount of CPU. As you said, it will be great to have some race conditions in there to make it all that much more secure. $10 says OpenSSH won't be in a big rush to put these patchs in place.

Re:To *have* such problems... (1)

Danathar (267989) | more than 6 years ago | (#22405410)

PSC = Pittsburgh Supercomputing Center (At Carnegie Mellon).

They are a node on the Teragrid which has throughput over some segments of around 100Gb/s

I want this yesterday (literally)! (1)

Immerial (1093103) | more than 6 years ago | (#22404200)

Aw man, if only I had this yesterday! Just had someone download ~6GB worth of video files from one of my machines and just as it shows in the article, throughput was pegged at ~2MB/sec. It took most of a day to send it. This would also kick some serious bootae for uploads to YouTube if they were to ever implement it... for those of us uploading GBs of GBs of stuff.

Re:I want this yesterday (literally)! (1, Funny)

Anonymous Coward | more than 6 years ago | (#22404446)

According to file size and type, it must be Island.Fever.4.PROPER.avi

Re:I want this yesterday (literally)! (1)

Immerial (1093103) | more than 6 years ago | (#22405828)

Wow... I know Walter Lewin's lectures are exciting but I think that pushing it ;-)

Doesn't only affect Linux, but Cisco routers also. (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#22404250)

On the lower end Cisco routers (8xx-series), a router-to-router SCP is definitely horribly CPU-bound. I occasionally need to do WAN speed testing to satisfy client concerns about "teh Interwebs are slow", and normally I'd like to do just a computer-to-computer copy of a big file and graph the SNMP stats for a "flat peak" which indicates traffic policing, but every so often I don't have access to the client's computers so I have to do a router-to-router file copy and now that Cisco has removed their FTP server from their IOS firmware (due to a yet-unfixed "Multiple Vulnerabilities in the IOS FTP Server" bug that was almost a year ago) , I'm really only left with SCP and it's not up to the task of bandwidth testing because it runs dog slow.

TDz.

Poor implementation then (-1, Troll)

NotZed (19455) | more than 6 years ago | (#22404314)

Pretty sad it takes someone to 'discover' a fix for such an issue.

This sort of thing should've been coded in from the start (is it in rsync?).

i/o / cpu decoupling is known issue with known and simple solutions (although linux doesn't make it as easy as it could).

Linux kernel version 2.6.17 to 2.6.24.1 (3, Funny)

arkarumba (763047) | more than 6 years ago | (#22404420)

Preferably, we would like to test it any very high bandwidth systems running Linux kernels version 2.6.17 to 2.6.24.1.

Re:Linux kernel version 2.6.17 to 2.6.24.1 (0)

Anonymous Coward | more than 6 years ago | (#22405624)

You are forgetting that the latest revisions of 2.6.22 and 2.6.23 have the vmsplice fix applied. So you'll need to limit yourself to 2.6.17 to 2.6.21. And not vendor kernels either; Debian has patched 2.6.18 and Ubuntu has patched one (I think it's 2.6.18) and no doubt many other vendors too.

bbftp (0)

Anonymous Coward | more than 6 years ago | (#22404424)

There exists already a multi-threaded & multi-stream file transfer tool, bbftp [freebsdsoftware.org] . It is not a drop-in replacement for scp, as it uses a protocol optimized for bulk files, but doubtless some of its ideas could be backported.

Pretty much totaly incorrect summary (5, Informative)

Eunuchswear (210685) | more than 6 years ago | (#22404438)

Almost all the improvements they talk about come from optimising TCP buffer usage. The summary of the fixes:

HPN-13 A la Carte
  • Dynamic Windows and None Cipher
    This is a basis of the HPN-SSH patch set. It provides dynamic window in SSH and the ability to switch to a NONE cipher post authentication. Based on the HPN12 v20 patch.
  • Threaded CTR cipher mode
    This patch adds threading to the CTR block mode for AES and other supported ciphers. This may allow SSH to make use of multiple cores/cpus during transfers and significantly increase throughput. This patch should be considered experimental at this time.
  • Peak Throughput
    This patch modifes the progress bar to display the 1 second throughput average. On completion of the transfer it will display the peak throughput through the life of the connection.
  • Server Logging
    This patch adds additional logging to the SSHD server including encryption used...
So the main part of the patch set is "It provides dynamic window in SSH and the ability to switch to a NONE cipher post authentication" and the only part that has to do with threading is marked "This patch should be considered experimental at this time".

By the way, does anybody else think "the ability to switch to a NONE cipher post authentication" is pretty dodgy?

Re:Pretty much totaly incorrect summary (2, Informative)

Arimus (198136) | more than 6 years ago | (#22404874)

By the way, does anybody else think "the ability to switch to a NONE cipher post authentication" is pretty dodgy?


Not really, for some of the stuff I do via SSH: eg logging into my webhost to untar a patch and apply it the only part of the transaction I want to be secure is my initial password/key-exchange post authentication I really don't give a stuff who sees me type

cd ~/www
tar xvfz ~/patch.tar.gz
or any of the other commands I type in. However it should be down to the admin of the system in the first place to decided whether to allow NONE down-grade (Either on system wide or per user/session basis) and then down to me as a user to decide whether to take advantage.

Re:Pretty much totaly incorrect summary (0)

Anonymous Coward | more than 6 years ago | (#22404998)

By the way, does anybody else think "the ability to switch to a NONE cipher post authentication" is pretty dodgy?


Not really, for some of the stuff I do via SSH: eg logging into my webhost to untar a patch and apply it the only part of the transaction I want to be secure is my initial password/key-exchange post authentication I really don't give a stuff who sees me type

cd ~/www
tar xvfz ~/patch.tar.gz
or any of the other commands I type in. However it should be down to the admin of the system in the first place to decided whether to allow NONE down-grade (Either on system wide or per user/session basis) and then down to me as a user to decide whether to take advantage.

Everyone who sees you type, can inject package that says that you said rm -rf.

Re:Pretty much totaly incorrect summary (1)

dissy (172727) | more than 6 years ago | (#22405070)

By the way, does anybody else think "the ability to switch to a NONE cipher post authentication" is pretty dodgy?
Yes, you almost might as well just use telnet or rlogin.

The only advantage ssh with no cipher is that an attacker will not see your authentication details (password or key) to login to the remote machine.

Unfortunatly just like telnet, using ssh with the none cipher opens the connection up to tcp hijacking and injection of packets, so the attacker doesnt really need your password anymore, they can just execute commands as you on the server once you are authenticated.

My guess is with the dynamic tcp window size patch they use, this might be harder than normal to pull off, but I would still feel more comfortable having that tested by someone more knowledgable before I start using it.

Fortunatly CPU has never been my bottleneck with ssh/scp, as alot of my machines on the lan still use 100mbit, so this isnt exactly targeted at me.

Re:Pretty much totaly incorrect summary (1)

Atzanteol (99067) | more than 6 years ago | (#22405592)

Yes, you almost might as well just use telnet or rlogin.

Sorta. If you're on a private network sending a 4Gig ISO (or other large file/files) why do you need the data to be encrypted? Encrypting credentials is sufficient.

Re:Pretty much totaly incorrect summary (1)

dissy (172727) | more than 6 years ago | (#22405794)

Sorta. If you're on a private network sending a 4Gig ISO (or other large file/files) why do you need the data to be encrypted? Encrypting credentials is sufficient.
Exactly. As long as you can trust (or at worse, assume) your LAN is secure.
Much easier to do on a home network where it is either just you, or you and family.
A rather safe assumption to make if your client machines and 'servers' are secured from eachother, limiting the potential damage from an infected wi^H^H^H client machine.

THe main use I see is that the scp command is damn handy compared to most any other command line method of transfering files. Especially so with RSA/DSA keys.
As far as people that use GUIs go, generally samba (or your native file sharing protocol of choice) is plenty good enough, and it doesnt get easier than drag&drop in a GUI.

Now that would be an interesting benchmark. SCP using the NONE cipher, compared to things like NFS, samba, appletalk (or whatever they use these days as native in OSX) etc, and then speeds for a direct netcat thrown in.
I'd venture a guess that samba would still have more overhead, but I wonder about NFS, and would like to see this compared to netcat speeds.

Re:Pretty much totaly incorrect summary (2, Insightful)

tbuskey (135499) | more than 6 years ago | (#22405738)

By the way, does anybody else think "the ability to switch to a NONE cipher post authentication" is pretty dodgy?

I'd like it when I tunnel a new SSH or scp through another SSH tunnel. We call it a sleeve. I've had to sleeve within a sleeve's sleeve before to get through multiple SSH gateways and firewalls to an inner system. You can tell ssh to use XOR but I'm not sure you can in scp.

Of course, if speed is paramount, you can use netcat inside the sleeve(s) to copy files. No encryption of the netcat, but it's inside an encrypted sleeve so the stream is encrypted.

Hardware acceleration (4, Interesting)

AceJohnny (253840) | more than 6 years ago | (#22404502)

I've been wondering, does there exist hardware accelerators usable by OpenSSL or GnuTLS? I work in embedded systems, and our chip includes a crypto and hash processor. I'm surprised nothing equivalent exists on modern PCs, or have I just not been looking in the right places?

Re:Hardware acceleration (0)

Anonymous Coward | more than 6 years ago | (#22404558)

hoho

Re:Hardware acceleration (2, Informative)

neumayr (819083) | more than 6 years ago | (#22404646)

There were crypto acceleration cards, but I think the market was fairly small. They made sense for sites with lots of https traffic, but nowadays general purpose cpus are blazingly fast compared to back then.
So I guess they disappeared..

Re:Hardware acceleration (1)

xehonk (930376) | more than 6 years ago | (#22405016)

There are still accelerators in the server market.

But for a typical desktop they're pretty pointless. My amd x2 3800 can scp with about 45MB/s to an equally fast machine. Though netcat is still faster with approx. 57MB/s, those 45MB/s are enough to make me happy.

Re:Hardware acceleration (1)

bzzzt (313005) | more than 6 years ago | (#22404684)

Those embedded systems usually have slow CPU's which are outperformed by hardware-accelerated encryption boards. Your average desktop CPU running openssl will be a lot faster than most cheap accelerator cards. Most of the time it's cheaper to invest in a faster generic CPU than a piece of custom hardware.

Re:Hardware acceleration (1)

TeknoHog (164938) | more than 6 years ago | (#22404730)

I've been wondering, does there exist hardware accelerators usable by OpenSSL or GnuTLS? I work in embedded systems, and our chip includes a crypto and hash processor. I'm surprised nothing equivalent exists on modern PCs, or have I just not been looking in the right places?

The VIA C7 processor has hardware crypto acceleration (AES and some helper functions) that's supported by OpenSSL out of the box. Applications still require some patching, for example OpenSSH. The reason seems to be that the application has to choose the encryption engine used by OpenSSL.

http://bugs.gentoo.org/show_bug.cgi?id=162967 [gentoo.org]

Re:Hardware acceleration (2, Interesting)

htd2 (854946) | more than 6 years ago | (#22404802)

There are a number of vendors who supply PCI/e/x/etc crypto accelerators. However these are mostly targeted at servers where the overhead of serving up numerous encrypted streams of data is much higher than on a client PC.

The Sun T1 and T2 processors in the T2000 and T5000 also have onchip crypto units 1 on the T2000 and 8 on the T5000 which accelerate OpenSSL traffic by offloading DES, AES, MD5 etc.

Re:Hardware acceleration (1)

pixr99 (560799) | more than 6 years ago | (#22404870)

I've been wondering, does there exist hardware accelerators usable by OpenSSL or GnuTLS? I work in embedded systems, and our chip includes a crypto and hash processor. I'm surprised nothing equivalent exists on modern PCs, or have I just not been looking in the right places?
Similar features exist, though not on all "modern PCs." VIA's C3 chips have a random number generator. C3 chips with the step 8 or later Nehemiah core actually have a hardware AES implementation. I think the C7 family has these features as well. And then, of course, there's a bunch of Hifn-based hardware that can be attached to a PC. Soekris Engineering [soekris.com] has PCI and mini-PCI versions of their Hifn-based crypto boards.

Re:Hardware acceleration (1)

Jah-Wren Ryel (80510) | more than 6 years ago | (#22405006)

Typically the overhead of the memory copy across the PCI bus to the crypto card and back into main memory is higher than just letting the host cpu do the crypto work itself (AES was designed to be very efficient even on slow cpus, far better than DES/3DES).

As others have mentioned, the Via cpus have built-in accelerators which avoid those memory copies

Re:Hardware acceleration (0)

Anonymous Coward | more than 6 years ago | (#22405030)

In most desktop scenarios [today] ciphers/hashes are certainly going to be faster than your IDE or PCI bus can cope with [realistically]. So the only thing a cipher could help with is latency. Unless you put it in your flow, e.g. between you and your MAC. Farming data off from PCI [e.g. your harddisk], to a crypto chip, then back over PCI to your NIC isn't going to be any faster than just using the main cpu. At best it lowers the load, but it's really not worth the cost.

For starters, adding crypto makes it harder to export devices. Which adds cost.

sftp and ASCII mode transfers (0)

Anonymous Coward | more than 6 years ago | (#22404526)

sftp still not have an ASCII transfer mode, unlike the ftp it replaces. What is the deal with this? How much extra code would it take to do this? Of course you can workaround it, but it wastes times and causes confusion and complaints (google sftp and ascii).

Re:sftp and ASCII mode transfers (2, Insightful)

raynet (51803) | more than 6 years ago | (#22404828)

The lack of ASCII transfer mode is a good thing. A LATIN15 transfer mode might be handy. Less time wasted and less confusion and complaints amongst my clients.

Re:sftp and ASCII mode transfers (1)

colfer (619105) | more than 6 years ago | (#22405424)

WinSCP implements SFTP with ASCII/text transfer. WinSCP uses SFTP by default, and falls back to SCP if SFTP is not available. But even in SFTP-only mode, it does the line ending change for ASCII transfers. Some SFTP clients, such as the version of Filezilla I tried a few years ago, do not support ASCII transfer, do not change the line endings.

News just in! (5, Funny)

j.a.mcguire (551738) | more than 6 years ago | (#22404642)

Crowds were shocked to discover that multi-threaded software runs faster on a multi-core system, it was hard to contain the excitement over the discovery which could revolutionise software development!

Why not loopback? (1)

neumayr (819083) | more than 6 years ago | (#22404662)

I don't get it, if they need lots of bandwidth to test their scp acceleration code, why can't they do it over the loopback interface?
Is there some limit to that I'm not aware of?

Re:Why not loopback? (1)

Shakrai (717556) | more than 6 years ago | (#22405872)

That wouldn't tell them very much about their TCP changes though. Much better to do the test on a real live network.

why look for high bandwidth only ? (1)

richlv (778496) | more than 6 years ago | (#22404666)

it's weird that they look for high bandwidth users only. they could also use low-powered systems to test the approach :)
of course, if they want to test how well it scales, that's a different matter.

Comcast (1, Funny)

OglinTatas (710589) | more than 6 years ago | (#22404678)

"They are currently looking for potential users with very high bandwidth to test the upper limits of the system."

You comcast users can forget about it, then.

Old news? (2, Interesting)

Damocles the Elder (1133333) | more than 6 years ago | (#22404932)

Digging around for the best way to apply the patch without screwing up my portage updates, I came across a request for this to be merged into the portage back in 2005, and is apparently usable with the HPN useflag.

Not that I'm that surprised to see this is old news, since they're apparently on major revision 13...

Multithreading waaay to go... (2, Insightful)

Crass Spektakel (4597) | more than 6 years ago | (#22405268)

Today you can buy a machine with eight cores, 8gb memory and 1tb harddrive for less than 2000. And most software will only use one core and a maximum of 2gb memory.

WE NEED MULTITHREADING NOW BIG AND EVERYWHERE.

Multithreading is maybe the biggest change in software development. In contrast to advanced command sets like MMX, SSE and so on it is not about some peep hole optimization, about replacing a bunch of x86_32 commands with some SSE commands, it is about changing the whole approach, finding new algorithms and redevelop much if not all software we are used to work with.

And we are lightyears away from using appropiate software on on todays systems.

See compressors: Most can't multithread at all. Those who can have more issues than the SCO buisness plan, eg reduced efficency, scale very bad, can't accept input from stdio or can't output on stdout. Basically compression should be a good starting point but even here todays solutions are incredible far behind the hardware.

Also consider network communication, graphical interfaces, games, printing, non-enterprise realtime presentation and so on and so forth.

The revolution is NOT here. People aren't even talking about it.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...