Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Sending Files w/o Sending Clear Passwords?

Cliff posted about 11 years ago | from the between-ftp-and-scp dept.

Security 151

Ambush_Bug asks: "I've done some googling around, but to no avail. I'm wondering if the Slashdot knows of a remote login protocol which exists in security space somewhere between ssh & rsh/ftp/telnet. Basically, the point is that I don't care if my data are encrypted, but I'd rather not send my password around as plaintext. I'm sending extremely large astronomical images which don't compress very well (noisy backgrounds...) and sftp is just too slow, but our sysadmin isn't fond of rcp, ftp and the like. Is there something in between?"

Sorry! There are no comments related to the filter you selected.

Push 'em. (2, Informative)

TheSHAD0W (258774) | about 11 years ago | (#7215661)

Put the data up in an obscure or passworded http or ftp server directory, then log in to your remote host via ssh and grab them remotely.

Re:Push 'em. (1)

Bob.Smart (168245) | about 11 years ago | (#7216354)

To be more explicit:
(1) mkdir --mode=og+x-r ~/public_html/private/
Apache can't read this directory but can access specific files in it.
(2) mkdir ~/public_html/private/xyzzy/
where xyzzy is any random password. Put the files you want to transfer in that directory and retrieve them with wget.

Re:Push 'em. (1)

WoTG (610710) | about 11 years ago | (#7216682)

Just a quick note, if you use a passworded http directory, your password is NOT sent in clear text on most modern web server configurations. They are usually hashed with MD5 somehow - there's a standard, but I can't seem to Google it right now. (This doesn't encrypt the data, it just secures your password)

Re:Push 'em. (1)

MSG (12810) | about 11 years ago | (#7216787)

Actually the password is base64 encoded, to be safe over 7bit media. It's still very much "plain text".

Re:Push 'em. (1)

WoTG (610710) | about 11 years ago | (#7216905)

OK, you had me worried for a bit (no pun intended)!

I had to Google for "HTTP Authentication" to come up with this: Apache reference [apache.org] page. It DOES use MD5, but I don't really know how popular this digest authentication is... I know my web host supports it, but I don't remember how I ended up figuring that out.

Re:Push 'em. (1)

Jeremiah Cornelius (137) | about 11 years ago | (#7216838)

1) Digest Auth HTTP. (http://www.ietf.org/rfc/rfc2617.txt) For IIS it's a check-button, Apache it's a mod: mod_auth_digest (Requires capable browser: IE or Moz).

2) One-time Passwords - think S/Key. I think Apache needs mod_auth_pam, and the PAM module for S/Key needs to be configured. I havn't done this, but the buzz is opiepasswd is the most flexible PAM module.

KERBEROS. A whole bunch of work to kerberize one service, but if you start here, you can move to K5 for most of your infrastructure. I like the approach. K5 and LDAP - this was the Windows 2000 approach, and it works great on Unix. GSSAPI software, like NFS v4, will support K5 extensions. Samba 3 is now a final release, and gets you there too.

DVD (2, Interesting)

Dancin_Santa (265275) | about 11 years ago | (#7215662)

Snail mail doesn't require a password.

Re:DVD (1)

larry bagina (561269) | about 11 years ago | (#7215958)

snail mail does require encoding binary data into text (base64 or uuencode) which adds an extra 15% overhead or so.

Re:DVD (1)

kernelistic (160323) | about 11 years ago | (#7216427)

Snail mail requires no such thing. You either use iso9660 or UDF to encode the disc.

Re:DVD (1)

DjReagan (143826) | about 11 years ago | (#7217066)

Eh? Snail-mail - you know.. like the post office, and the letterbox out front of the house? People use it to send pieces of paper in envelopes to each other. Sometimes they send parcels. In this case, the guy would use it to send a CD with his data burnt on it.

I'm not quite sure why you think that requires Base64

Perhaps you're thinking of E-Mail?

Re:DVD (1)

PerlGuru (115222) | about 11 years ago | (#7218251)

I'm sorry to see these other poor saps didn't get the joke, I for one did. I quite enjoyed it I must say.

Re:DVD (1)

91degrees (207121) | about 11 years ago | (#7217415)

Bandwidth is pretty good as well if you're sending a lot of data.

Re:DVD (1)

xsbellx (94649) | about 11 years ago | (#7217798)

"Never underestimate the bandwidth of a station wagon loaded with mag tapes". -- Unknown

SCP (2, Informative)

CyberVenom (697959) | about 11 years ago | (#7215664)

Try scp. It is included with ssh.

Re:SCP (with -c none) (3, Informative)

Anonymous Coward | about 11 years ago | (#7215704)

specifically "scp -c none", which will send the data in the clear, but still do the secure authentication (ie. no cleartext passwords)

It isn't always enabled; your admin may have to set it up. Google around for the details.

Re:SCP (with -c none) (2, Informative)

CyberVenom (697959) | about 11 years ago | (#7215772)

Here's a link to the manpage. http://www.die.net/doc/linux/man/man1/scp.1.html

Re:SCP (1, Informative)

Anonymous Coward | about 11 years ago | (#7216583)

And if you're working with a win box, WinSCP3 is a pretty good app.

use an IM client (0)

Anonymous Coward | about 11 years ago | (#7215675)

What about running an IM client and using its file transfer feature? Most IM clients offer plaintext file transfers, and most have secure authentication processes (e.g., MSN Messenger uses an MD5 challenge-response system).

You can probably use something like Jabber.

The answer is... (0)

Anonymous Coward | about 11 years ago | (#7215676)

CDRW+UPS.

Innovative protocol that requires a stack of blank CDs and a business account with UPS (unless you want to drive to a drop-off).

CDRW+Fedex is a similar fork of this protocol.

ummm... (1)

Tr0mBoNe- (708581) | about 11 years ago | (#7215735)

Basically if you are sending something... no matter what, it is broken down into packets and sent. Your computer does not care what it is, just where it is going. It is the job of the receiver to understand it. The reason yer SFTP is slow is because yer internet connection is throttled for non HTTP packets. This is common in large networks and schools.

What I would do is burn it onto a CD and snail mail it to the person. If the file is to large to put onto one cd, then probabally it is too large to send in one sitting.

If all your traffic is local though, like still on the same LAN or WAN, or (taking a large leap) Metropolitan Area Network, the transfer rates should be no problem.

One other thing. If your institution is using extremley large images or files, generally your sys admins are nice enough to allow you to tansfer them at generally astronomical speeds. If yours are being little trolls holing their bandwidth and crushing the boned of all who cross their path is seek of the precious BW, might i suggest a Knight in Shining Armor, or their boss.

SFTP slowness (3, Informative)

lpontiac (173839) | about 11 years ago | (#7215942)

The reason yer SFTP is slow is because yer internet connection is throttled for non HTTP packets. This is common in large networks and schools.

There are a couple of other things that can slow SFTP and SCP down:

  • You're encrypting. Not a problem on a fast machine with a slow link, but on a slow machine with a fast link, it's noticable. Another poster has already pointed out you can configure ssh to not compress.
  • The SSH2 protocol implements its own flow control, over and above what TCP is already doing. A really simple implementation of the protocol that won't allow multiple packets to go out without (yet) being acknowledged will slow down heaps - when PuTTY improved it's packet handling I saw scp over 802.11 go from 20KiB/s to 450KiB/s.

    Apparantly, when using OpenSSH, you'll want to use the -B option to bump up the internal buffer size way beyond the 32768 byte default.

one time passwords (1)

FiDooDa (23111) | about 11 years ago | (#7215738)

it seems that one time passwords could help you out here.

It doesn't encrypt passwords for cleartext protocols but if the password is used only once it's not a great risk.

I used it on OpenBSD (ftp server) and it worked great.

OpenBSD S/Key FAQ section [openbsd.org]

Re:one time passwords (1)

FiDooDa (23111) | about 11 years ago | (#7215790)

*sigh* I should of wrote: because the password is used only once it's not a great risk

Re:one time passwords (0)

Anonymous Coward | about 11 years ago | (#7217354)

You mean "I should have written".

Re:one time passwords (0)

Anonymous Coward | about 11 years ago | (#7218122)

You mean he meant "Ah shoulda writin".

Re:one time passwords (1)

lavv17 (613412) | about 11 years ago | (#7217339)

if your ftp server supports s/key or opie, then you can send the one-time password automatically without s/key calculator using lftp [lftp.yar.ru] .

sftp? (0, Offtopic)

PhlegmMaster (596165) | about 11 years ago | (#7215742)

I always thought sftp was secure and allowed file transfer.

Re:sftp? (1, Insightful)

Anonymous Coward | about 11 years ago | (#7216099)

Congratulations on not even reading the submission text, you unbelievably stupid piece of shit.

Re:sftp? (0)

Anonymous Coward | about 11 years ago | (#7218524)

rofl...that got modded "insightful." I mean, it was insightful, but it's just hilarous to see that comment with an "insightful" tag.

PPP? (1)

CyberVenom (697959) | about 11 years ago | (#7215744)

Not sure exactly how you would set it up, but PPP supports unencrypted data streams with hashed passwords when using authentication like CHAP. or... Maybe you could write a Perl (or Python, or shell) script that issies a challenge and spits out whatever files you want if the challenge succeeds. Hell, maybe I'll write one right now - here I go! (check this thread in a few minutes to see the finished product...)

Custom Perl Server! (1)

CyberVenom (697959) | about 11 years ago | (#7216325)

Here You go!
(Darn, /. doesn't seem to like code. Well here's a link to the code then!)

server.pl [sablesoftware.com]
client.pl [sablesoftware.com]

Let me know what you guys think! :)

This code should provide all of the mentioned features: an MD5-based challenge authentication, and unencrypted transfer of files.

Re:Custom Perl Server! (0)

Anonymous Coward | about 11 years ago | (#7217316)

This ROCKS! Nice and simple authentication, sweet. The other solutions (rsync, ssh -c none, etc.) also look good, but it's great to have something in perl.

Re:PPP? (1)

pyite (140350) | about 11 years ago | (#7216342)

What the? How would you manage to run PPP over IP? It's a WAN protocol. SFTP, on the other hand, uses the SSH Transport Layer protocol, and as such, can run on top of TCP/IP. Have fun writing your "script."

Re:PPP? (1)

DA-MAN (17442) | about 11 years ago | (#7216364)

> How would you manage to run PPP over IP?

Microsoft managed to do it. The protocol is called pptp and does just that. I know this is off topic, just wanted to point out that ppp over ip is totally doable.

http://www.poptop.org [poptop.org]

Re:PPP? (1)

aminorex (141494) | about 11 years ago | (#7216855)

There's a project on sf.net for ppp over
telnet over tcp over ip.

Useful for getting a back-door into your
company's network.

Re:PPP? (1)

CyberVenom (697959) | about 11 years ago | (#7217168)

That sounds like exactly what I meant when I mentioned PPP.

Re:PPP? (1)

CyberVenom (697959) | about 11 years ago | (#7216371)

I think the whole point is that encrypting the entire datastream is too processor intensive in this case, so SFTP (or even SCP) would be out of the question. PPP could potentially be run over a TCP/IP socket (like VPN/PPTP). Not that PPP itself is really the point, I just thought that there might be a way to use existing authentication methods. The script I wrote does not use PPP, but implements a challenge handshake based on some of the same concepts as CHAP.

Re:PPP? (1)

pyite (140350) | about 11 years ago | (#7216564)

Well, the reason I raised an eyebrow is because you seemed (IMHO) to point to PPP being the core of this whole thing. PPTP is something different entirely (a something I hate with a passion but that's another discussion all together ;)). Really, PPP, or PPTP, not CHAP are critical to this. All you really need is a challenge response. Avoiding this all together, we can just use a one time password. There are numerous (good) ways for implementing those.

One-time passwords (3, Informative)

jhealy1024 (234388) | about 11 years ago | (#7215758)

If you use one-time passwords, you can use a totally insecure connection because the password is invalid immediately after you use it. Thus, even if it gets sniffed, it doesn't give an attacker anything they can use to get into your system. Thus, the connection is totally insecure, but your password remains safe. Sounds to me like just what you would want.

Look into libpam-opie on linux or s/key on the *BSDs for more info. Some good background is available from the FreeBSD manual:

http://www.freebsd.org/doc/en_US.ISO8859-1/books /h andbook/skey.html

It integrates well with most of the "basic" services on those OSes, so you shouldn't have much trouble getting it off the ground.

The one pain is that you have to look up a new password off of a card or piece of paper every time you log in. Alternately, some programs allow you to compute the OTP challenge/response on the fly (you could even write a script to help you out if you do this often enough).

Definitely worth a look...

OTP Calculators (1)

Cadre (11051) | about 11 years ago | (#7216153)

For those on the go an OTP calculator for the Palm OS: http://palmkey.sourceforge.net/ [sourceforge.net] .

Don't authenticate (1)

duffbeer703 (177751) | about 11 years ago | (#7215810)

If you're on an internal network, setup an anonymous FTP server...

I've seen setups where anyone can upload a file to a certain directory, then some script routine runs every so often and moves the file to the actual place where you want the file to go.

FTP and SSH (1)

ccarr.com (262540) | about 11 years ago | (#7215851)

I've never tried this, but I once heard that it's possible to create an SSH tunnel for port 21, FTP's control port. The data is actually transmitted in the clear over other ports, but the protocol-related transmissions take place over the encrypted port. I'm not sure how this would work; since the tunneling would mask the client's orgin to the server, I would expect problems negotiating the data ports.

very simple - tunnel ftp over SSH (3, Informative)

stonebeat.org (562495) | about 11 years ago | (#7215882)

FTP uses 2 ports: port 21 for control connection(passwd/authentication) and port 20 for data transfer.
You just need to tunnel 21 through SSH, and leave 20 unecrypted.
Very simple technique, but very powerful. I use SSH tunneling everyday.
openssh supports tunneling and the windows downloadable form http://www.ssh.com also supports it. takes 3 mins to setup the tunnel.

Re:very simple - tunnel ftp over SSH (1)

user555 (145309) | about 11 years ago | (#7216655)

FTP is a strange protocol and there are about a million gotcha's with trying to do ftp on SSH.

To start with you sysadmin must allow you to ssh in to the machine running ftp. (Mine doesn't allow it because he does want people slowing down the server by for example compiling on it...)

Even then this will probably only work for passive ftp.

Read the Orielly SSH book if you're curious. They discuss how to do this for about 20 pages.
BTW, getting both data and commands encrypted is next to impossible.

Bottom line. Go ahead try it. If you have the right setup it might be easy. But don't be surprised if you have trouble.

Re:very simple - tunnel ftp over SSH (1)

gl4ss (559668) | about 11 years ago | (#7217581)

btw ftp allows quite funky configurations too.

for example, if you limit the clients that can use it to the active mode ones, you can actually portforward the control connection from any computer you like on internet(so the computer that actually serves the files, and brings up the data transfer connections can sit behind a firewall that allows no incoming connections)

Anonymous FTP (1)

WolfWithoutAClause (162946) | about 11 years ago | (#7215905)

I think you should use anonymous ftp (your operator should be fairly happy with that, it's the one remaining, legitimate use of ftp there is), then log in securely using ssh and move it somewhere safe and run an md5 checksum on it to check that it made the trip without any modifications or corruption.

I don't see that you have much advantage from using a secured ftp in this case- 99.999% of the time you won't get hacked or anything and in this case the data you are hauling isn't sensitive anyway. Just don't use the anonymous ftp account for anything that needs the security.

Try a faster cipher (4, Informative)

semanticgap (468158) | about 11 years ago | (#7215926)

Sftp uses ssh as the transport. Chances are your ssh configuration defaults to 3des which is painfully slow, you might do better by specifying blowfish as your cipher, or if you are really lucky, your sysadmin has compiled ssh with "none" cipher enabled (but my guess is you are not so lucky, even though ssh with none as cipher addresses your problem precisely - passwords are encrypted, and the rest isn't).

To tell sftp to tell ssh to use blowfish I believe you need "sftp -oCipher=blowfish"

Re:Try a faster cipher (3, Informative)

kwerle (39371) | about 11 years ago | (#7216349)

This is exactly the right kind of thing to do.

I use scp, and so the command I issue is

scp -c blowfish SomeFile me@TargetHost:/somepath

On my 11Mb/s 802.11 network I am capped by bandwidth, not by CPU.

WebDAV or HTTP? (1)

YellowElectricRat (637662) | about 11 years ago | (#7215927)

You could use WebDAV (works on IIS and Apache) or, as a slightly more tricky alternative, plain HTTP uploads (you'd need an upload handling script).

As long as you enable authentication, and make sure it's not basic authentication (use digest authentication, or if it's a windows box, NTLM), you're set - your password is encrypted, but your data isn't.

Similarly, you could use either WebDAV or HTTP uploads over an HTTPS connection WITH basic authentication, which gives you overall encryption on the lot, but that's not really what you were after...

Re:WebDAV or HTTP? (1)

utopiabound (199550) | about 11 years ago | (#7216443)

You don't need a script. There's a standard PUT command in the HTTP spec. If you use windows you can drag and drop files into a netscape window to transfer them up. On a *nix you can use curl (with the --upload-file option). Just make sure it's password protected and you're ready to go.

tunnel FTP through SSH (1)

stonebeat.org (562495) | about 11 years ago | (#7215929)

oh btw if you dont want to setup tunneling manually, you can also purchase appgate's mindterm, which a a webbased tunneling java program, that is very easy to setup.

-c None (1, Redundant)

KagatoLNX (141673) | about 11 years ago | (#7215934)

It was mentioned above but not modded up, use scp with -c none.

That should use scp with no cipher. You can, however, still use a key pair for authetication. Thus, auth is secure, transmission is plain.

Re:-c None (1)

Fished (574624) | about 11 years ago | (#7216294)

Newer versions of OpenSSH don't support -c none. (The usual security zealot nonsense from the OpenBSD people from what I can tell. There *is* a place for -c none.)

Re:-c None (1)

shepd (155729) | about 11 years ago | (#7217617)

>Newer versions of OpenSSH don't support -c none. (The usual security zealot nonsense from the OpenBSD people from what I can tell. There *is* a place for -c none.)

Then try lsh [lysator.liu.se] ! It's GNU software instead. Has fewer bugs [lysator.liu.se] , it seems, too. :-)

Re:-c None (1)

swdunlop (103066) | about 11 years ago | (#7217749)

And fewer eyes watching the source code.. Fewer bugs in a project of similar complexity with fewer users is not necessary a clean bill of health.

Re:-c None (1)

shepd (155729) | about 11 years ago | (#7217772)

>And fewer eyes watching the source code.. Fewer bugs in a project of similar complexity with fewer users is not necessary a clean bill of health.

Normally I'd agree, but the OpenSSH project has led me to believe that there are times this analogy breaks down, badly.

Worse than that is BIND. There's likely even more people poring over that project, but for years it was plagued with the worst kind of bugs. I wouldn't be surprised if there's a lot more for people to find.

There's many more examples, some worse, some not.

I really do like lsh. It's fast, quick to compile, and has a REALLY streamlined set of libraries and installation procedure. Plus its author seems to read slashdot. :o)

Re:-c None (0)

Anonymous Coward | about 11 years ago | (#7218276)

Shep, this is really getting silly now. Why haven't you responded to this post [slashdot.org] yet? You went out of your way to answer the guy who asked you to configure a bare-bones desktop without any features or software, but you haven't responded to the guy who wanted to hear your answer to the G5. Why not?

You said, and I quote here, "Hardware to hardware, then, I challenge you to bring up a Mac system (non-laptop) which I can't beat with similar or better PC hardware, of similar or better quality." Those are your words, Shep. You really ought to stand by them.

Post your best configuration for a machine that approximates the G5's capabilities and feature set, and the price. Let us see the truth for ourselves.

Because, you see, if you don't either prove your point or admit you were wrong, you're just a goddamned motherfucking Mac-hating troll. And we can't have that, Shep. We really can't.

What's it going to be, Shep?

Re:-c None (0)

Anonymous Coward | about 11 years ago | (#7218383)

>What's it going to be, Shep?

Dude, are you *ever* going to check out my site [soiledbinaries.com] about you? Bookmark (Ctrl-D) it for future reference!

Re:-c None (0)

Anonymous Coward | about 11 years ago | (#7218643)

Dude, are you *ever* going to respond to this post [slashdot.org] ?

Re:-c None (0)

Anonymous Coward | about 11 years ago | (#7217947)

Then try lsh! It's GNU software instead.

This is not an endorsement.

Remember what you've learned from the FSF, kids. Use GNU software, go to jail!

Demand only genuine BSD software. Accept no substitutes.

How about kerberos? (3, Informative)

DeathBunny (24311) | about 11 years ago | (#7215949)

Sounds like a good application for Kerberos. From the RedHat Kerberos docs:

"Kerberos is a network authentication protocol created by MIT which uses symmetric key cryptography to authenticate users to network services -- eliminating the need to send passwords over the network. When users authenticate to network services using Kerberos, unauthorized users attempting to gather passwords by monitoring network traffic are effectively thwarted."

Just find yourself an FTP client and server that both support Kerberos. Here's a few links to get you started:

Kerberos section of the RedHat 9 manual:
http://www.redhat.com/docs/manuals/linux/ RHL-9-Man ual/ref-guide/ch-kerberos.html

Kerberos FAQ:
http://www.cmf.nrl.navy.mil/CCS/people/kenh/ kerber os-faq.html

MIT Kerberos page:
http://web.mit.edu/kerberos/www/

Re:How about kerberos? (1)

inburito (89603) | about 11 years ago | (#7216064)

Yes. Mod parent up. Kerberos is exactly what this person is looking for.

sftp too slow - WHY? (1)

wowbagger (69688) | about 11 years ago | (#7215952)

OK, the question I would have is, exactly WHY is sftp too slow?

The raw throughput of sftp isn't much less than ftp, given that you have enough CPU on both ends of the link for the encryption/decryption.

You speak as though the slowdown of sftp is very large compared to ftp - not the few percent the protocol itself would add. This would lead me to beleive that you are running slow due to the encryption itself.

So, first of all I would check the CPUs of the machines involved - unless you are running an old junque P75 you should not have a big problem filling most pipes.

Secondly, check what encryption algorithm you are using for the link once it is up. Some of the algorithms require less CPU than others.

It really sounds like SSH/SFTP would be the solution you want, but you just need to to a bit of tuning.

Re:sftp too slow - WHY? (1)

epine (68316) | about 11 years ago | (#7216290)

Soekris will soon have a new PCI crypto accelerator, the VPNL401 [soekris.com] Encryption at 400Mbs. That ought to be enough encrypted bandwidth to map every prospective Starbuck's franchise in the Virgo Galactic Cluster.

Even without hardware crypto, any modern 1GHz CPU can saturate a fat pipe using AES or Blowfish as the cypher algorithm. Quit blaming sftp and find a way to make sftp work properly.

Blowfish tops out at 10Mb/s? (2)

Nonesuch (90847) | about 11 years ago | (#7216319)

I haven't done the investigation as to why, but SSG (and by extension, scp/sftp) doesn't approach the throughput of FTP.

Most likely culprit is the general protocol overhead of SSH, even when compression and strong encryption are both disabled.

For example, across a 100Mbps switch between two machines, transferring large (700MB ISO images) files with FTP or even NFS gives an average throughput of 70Mbps, while using 'scp' (blowfish, no compression) tops out at 10Mbps for the exact same file.

Oddly, even using 'scp' on loopback on a 1.4Ghz FreeBSD machine hits the same hard limit of ten megabits?

Re:sftp too slow - WHY? (1)

MarkusQ (450076) | about 11 years ago | (#7216357)


The raw throughput of sftp isn't much less than ftp, given that you have enough CPU on both ends of the link for the encryption/decryption.

You speak as though the slowdown of sftp is very large compared to ftp - not the few percent the protocol itself would add. This would lead me to beleive that you are running slow due to the encryption itself.

So, first of all I would check the CPUs of the machines involved - unless you are running an old junque P75 you should not have a big problem filling most pipes.

I didn't dig into it at the time (had other things to worry about), but about a month ago I had to transfer data between a half dozen identically configured 1.7 GHz RH9 boxes. In this environment sftp was about 1/8th the speed of ftp on large files over an issolated 100Mbs network.

I'd thought as you do until I saw what was happening, at which point I switched to ftp for the rest of the transfers.

-- MarkusQ

Re:sftp too slow - WHY? (0)

Anonymous Coward | about 11 years ago | (#7216738)

cause 20% or more of your pipe is being used by encryption.

Re:sftp too slow - WHY? because: (1)

Splork (13498) | about 11 years ago | (#7217057)

ssh sucks for bulk data transfer. it is *not at all optimized* for such a use. its intended primarily for interactive traffic or low bandwidth (read: internet connection speed) traffic.

for bulk data transfer over high speed networks it blows chunks not only due to the crypto speed but also due to data buffering issues, loads upon loads of data copies (scp and sftp send all data through at least one local pipe with tiny buffers if not more), etc. the crypto dominates, but even with "-c none" its efficiency blows chunks.

and don't advocate tunneling over ssh as a solution either. TCP over TCP is a bad idea.

and if the target is a supercomputer or data center machine it probably -does not- have the CPU cycles to deal with the encryption overhead of such a large data transfers...

Re:sftp too slow - WHY? (1)

DjReagan (143826) | about 11 years ago | (#7217101)

The implementation of ssh you are using can have a huge impact on the transfer speeds. I was recently using ( an admittedly slightly old version of) SSH Communications's SSH software, and getting around 250Kb/sec with the blowfish cipher - getting rid of that and installing OpenSSH bumped the throughput up to 4.5Mb/sec on blowfish.

netcat? (1)

Atzanteol (99067) | about 11 years ago | (#7215955)

Occasionally I want to make a disk-image to a remote machine (typically sending 6 gig). I use netcat to just send the bits as fast as possible.

Machine a:
# nc -l -p 1234 > file

Machine b (via ssh session):
# nc machinea 1234 file_to_send

Re:netcat? (1)

man1ed (659888) | about 11 years ago | (#7217189)

Watch for botched HTML tags.
That should have read:

Machine b (via ssh session):
# nc machinea 1234 < file_to_send

E-Mail (1)

afabbro (33948) | about 11 years ago | (#7215959)

Mail it and have a procmail-driven script that puts it in the right directory. Use perl's MIME and other mail modules to make it easy (both for sending and procmail). No, this isn't 100% secure, but the potential damage is not much worse than mail-bombing - the worst someone could do is intentionally fill up your storage...which they can probably already do. And there's no password at all this way.

Or just use scp, which is easier still.

Re:E-Mail (1)

steppin_razor_LA (236684) | about 11 years ago | (#7217114)

Thats got to be one of the worst suggestions I've ever heard.

Email is a 7 bit medium. If you take a file and encode it for email, you are basically getting 8 bit content to work in a 7 bit medium. This means that the overall file size is increased. Since the author was complaining about speed, adding that overhead isn't likely to help.

That + the fact that email really isn't designed for transferring large files around. You'd probably break the hell out of whatever mail system you were using anyways.

There are plenty of other suggestions here (i.e. a HTTP based application.. or better yet just encrypting the control channel)..

Few quick options (1)

psychosis (2579) | about 11 years ago | (#7215981)

in bashrc (or whateverrc):
alias fastscp="scp -prC -c blowfish"

the -C (compression) won't do much for your images, but is great for most content - think inline zlib compression. blowfish is a reasonably fast cipher as well.

also, if you're hell-bent on not sending encrypted data, you could set up ssh to not use encryption (type "none"), then use a non-password authentication method - either hostbased or publickey.

note, though, that the default for scp/ssh is to NOT use compression. why the insistence on not being a little more secure? in my experience, the encryption overhead is not too big a burden on file transfer over a 100Mbps network. Anything less than that and the bandwidth is the bottleneck, not the cpus on either end.

you may also be able to use an ssl-enabled web page to authenticate, then POST the files over a non-ssl connection... not sure if that's do-able, though.

you can still use scp (1)

Polo (30659) | about 11 years ago | (#7215991)

Actually, scp will work fine, just use 'none' as the cipher.

You can even automate things using public key authentication so there aren't passwords.

a simplified recipe for setting up public key authentication is:

ssh-keygen -t rsa

it will ask for a file name, and then generate two files: file and file.pub

copy file.pub to user@remote in $HOME/.ssh/authorized_keys

(If the file already exists, append the contents of the file to the end)

copy file to your local machine's $HOME/.ssh/identity

If everything is set up ok, you should be able to do:

ssh user@remote

and you should be logged in without a password.

Then try:

scp -c none localfile user@machine:remotefile

no passwords, secure login, fast transfers.

Use SCP with the 'none' block cypher (1)

LunaticLeo (3949) | about 11 years ago | (#7216022)

You need to compile openssh with 'none' cypher. This is not copiled by default.

The 'none' block cypher will transfer you data in the clear. This gives a near-ftp speed transfer of your data. However, the good thing is that you get the full SSH authentication with passwords encrypted.

If you can't convince your Sysadmin to compile and install a SSH with 'none' cypher. The next best thing is to use the 'blowfish' cypher. It impacts cpu usage and transfer speed less than any either cypher I have tested.

BTW, the usage is as follows:

scp -c none file remote.ip:/dest/

or

scp -c blowfish file remote.ip:/dest/

Good luck.

Kerberos FTP (1)

jdurham (530204) | about 11 years ago | (#7216031)

I've never used it, or even looked at it, but perhaps something like a Kerberized FTP. You may want to try different encryption protocols for scp/ssh. SSH defaults to using IDEA for encryption, which is 64% of non-encryption speed. You can switch to Blowfish, which runs at 88% non-encryption speed. Just use the -c option with ssh/scp. And finally you can look at an unsecured file transfer, such as stock FTP, over IPSec. Check out FreeS/WAN if you are using Linux.

Kerberized FTP (1)

0x0d0a (568518) | about 11 years ago | (#7216062)

(a) Use kerberized FTP. Kerberos is a bitch to set up, but if your sysadmin is paranoid about security, he should be using it. Kerberos just deals with authentication, so it's possible to use non-encrypted systems that still use Kerberos authentication.

(b) Why is it "too slow"? A modern system running AES can saturate a 100Mbits/sec network.

Re:Kerberized FTP (1)

ComputerSlicer23 (516509) | about 11 years ago | (#7217388)

I've always used rsync, and just sucked the files down. I've found that SSH is considerable slower then FTP or RSYNC on 100Mbit/sec. Granted that was on a dual processor PIII-700 Xeon (both to and from). I used that stock redhat cipher, and then switched to blowfish, while that helped, it still sucked. There are a few interesting tidbits mentioned in the threading about it could possibly be OpenSSH buffering, and flow control that is causing issues.

I pegged the box about about 60-80% CPU utilization also, which was more then a minor nusince when I was using it to transfer backup copies of my 120GB production database.

Kirby

FTPS (1)

reynaert (264437) | about 11 years ago | (#7216167)

You can use ftps (ftp with ssl). It does encrypted authentication, and encrypted data transfer is optional. However, few ftp servers and clients support it, as there's really no good reason to use it when you've got ssh.

Kerberos (1)

Fished (574624) | about 11 years ago | (#7216271)

Kerberos will do this, although you will have to put a bit of effort into setting up a domain. Essentially, it allows you to do fully authenticated rsh, rcp, ftp, etc. while never transmitting your password. Instead, you get a cryptographic key.

Another option would be to chose a faster encryption algorithm for ssh than the default. In particular, I've seen the arcfour cipher recommended for speed (although I've not used it.) Check out the man page. Older versions of ssh - which you could presumably still find somewhere - support "none" as an encryption option. Using this, you could setup an RSA key and avoid entering a password altogether. One of these options would be easier than setting up kerberos, but not nearly as cool or useful otherwise.

Finally, you could setup an anonymous ftp server, accessible only from your remote IP address with a world writable "drop box" in it (which would be, however, readable only by you.) This might be the easiest thus far, but I misdoubt that your sysadmin will care for it if he's security paranoid. It is, however, very secure since it is only vulnerable to a Denial of Service attack which could be avoided using a quota, and even without the quota vulnerable only from you IP address. You could do something similar with an http form and/or a webdav server.

Or, instead of uploading the files, could you download them? That is, could you put up a webserver on the box where the files are captured, then use wget or something to pull them down without a password at all? That might be the easiest of all.

And, lest we forget, where there's a perl, there's a way. :) You could hack something up using XML::RPC or something in about ten minutes, no doubt. Authenticate by transmitting crypted passwords or something.

Hope this helps.

Rsync over ssh (1)

Leknor (224175) | about 11 years ago | (#7216350)

I use rsync over ssh to make up for many of scp/sftp weaknesses. eg:

rsync -e ssh /local/file user@host:/remote/file

(you should read the rsync man page as that is over simplified, I tend to use a more complex command line that meet my needs.)

By using ssh2 keys and the ssh-agent I don't get prompted for passwords and I get the benefit of rsync's ability to do partial transfers and other cleverness. It rocks.

Use rsync direct over tcp (2, Informative)

DDumitru (692803) | about 11 years ago | (#7216514)

Most *nix distributions have a copy of rsync loaded.

In this case you are using rsync directly over tcp/ip connections, sometimes called "daemon mode".

This mode features:

o high-strenght crytpo on passwords, but no encryption of data.
o passwords that are 100% independent of the system passwords.
o 100% streaming, even with large numbers of small files.
o restart of failed transfers where they left off.
o delta transfers for files where only parts change.
o optional gzip style compression.
o plus a lot more neat stuff.

Info on rsync is at:

http://rsync.samba.org

If you have a Linux system with xinetd or equivilent, there is a good change that you already have an /etc/xinetd.d/rsync control record for the daemon. You then need an /etc/rsyncd.conf file plus a "secrets" file to hold the passwords.

rsync directly on top of tcp/ip is how most "mirror sites" sync to their masters. It is about the only "practical" way to send gigabytes over the internet.

A couple of caveats. If stuff is twitchy, try to use the latest version of rsync (2.5.6) on both ends. Also, if you use the --compress option with already compressed (or encrypted data), there is a gziplib boundary bug. There are patches for this, but if you are sending uncompressable data, just leave off the --compress option.

Re:Use rsync direct over tcp (0)

Anonymous Coward | about 11 years ago | (#7216760)

somebody mod this guy up.

rsync rocks.

Try rsync over ssh (1)

OneFix (18661) | about 11 years ago | (#7216520)

This method can also be used as a push or pull.

rsync -azve 'ssh' /localpath remote.site:/remotepath

to do a push (reverse the order of the last 2 options for a pull).

rsync should be faster than any other protocol (lower overhead), the -z option will compress data, and you can always set up an authorized key via ssh on the remote system to allow for password-less access...

scp -c blowfish (1)

bertvl (66173) | about 11 years ago | (#7216590)

How about using scp with the blowfish cipher?

scp -c blowfish ...

It has very low overhead, so shouldn't be too much slower than using ftp...

correcting mis-information, and a solution (3, Informative)

menscher (597856) | about 11 years ago | (#7216605)

First off, do NOT follow the advice of all the idiots saying to use scp -c none. That will not encrypt your password, despite all the uninformed claims to the contrary. The encryption type is determined during the initiation of the connection. The password is simply part of the data, sent later. (If anyone chooses to dispute this, please provide evidence to back up your claims.)

Second, if you can afford some slowdown, use -c blowfish. The default is usually 3DES, which is incredibly slow. Blowfish is 11 times faster.

Finally, if you have some control over what applications are installed at each end, look into SafeTP [berkeley.edu] . It encrypts the password, but not the data. Exactly what you asked for.

man scp. (1)

Eivind (15695) | about 11 years ago | (#7216653)

ssh, trough scp can do what you want. It never sends passwords in the clear, and it'll happily use whatever cipher you specify for encrypting the data itself. If you want no cipher, you need only specify the command-line switch "-c none"

Make sure you understand the security requirements (1)

user555 (145309) | about 11 years ago | (#7216762)

There's a reason scp/sftp encrypt everything: sniffing passwords is the simplest and most obvious attack against ftp but it's not the only one.

There are numerous other attacks. And they only require control of a machine on the same lan as the server or client.

You don't care if an attacker sees your data fine.
Do you care if they corrupt the in transit? (FTP has no way to authenticate the data.)

More seriously, someone could also hijack your FTP session. There are even script kiddie tools to do this. Once someone has taken over your FTP session they can control you account. For example, they replace .ssh/authorized_keys with something different and then have shell access.

Don't care about what happens on that machine, well once someone controls your account they can probably get your password. Just stick a few trojans in your path. You'd never notice and they'd get your password and happily break into your other accounts.

Re:Make sure you understand the security requireme (1)

aminorex (141494) | about 11 years ago | (#7216868)

All of this is true, but it's far beyond the
pale of the threat model that the original
question implies.

It's ALWAYS possible to describe an attack
on the security of ANY system. No news there.

kerberos or srp (1)

perlchild (582235) | about 11 years ago | (#7216772)

kerberos might be a bit admin-intensive(and you mentioned you were trying to convince your admin), therefore I recommend you look at http://srp.stanford.edu/ [stanford.edu] which might do what you require It even has windows-based binary client versions of ftp for those that require them.

HTTP Digest Auth or S/Key (1)

aminorex (141494) | about 11 years ago | (#7216835)

you could just wget the files from the far
end, if you enabled digest auth on your httpd.
alternatively, there is s/key auth for ftp.

SSH (1)

XiChimos (652495) | about 11 years ago | (#7216929)

Go into you SSH options, pick a nice MACing algo and then just put none for data encryption. In the key-exchange you get to pick your functions, or none if need be.

QED.

Anon ftp with IP range restrictions (1)

gad_zuki! (70830) | about 11 years ago | (#7216943)

Either that or dummy accounts/throw away passwords.

Considering you're going to be doing this more than once you might as well have your own ftpd set to only accept connections from your IP using anonymous login.

Why isn't there ssh with no encryption? (1)

Whip (4737) | about 11 years ago | (#7216947)

What I want to know is... why doesn't ssh allow you to do protected logins (encrypted passwords, public-key authenticated logsin, whatever), but then do actual data transfer without encryption? When I'm at work, I want to be able to ssh everywhere (out of convenience: ssh-agent + X11 forwarding + autentication forwarding rock), but I sure as hell don't need my gigabyte-size files encrypted to go 50 feet across the LAN!

I seem to remember that (very) old ssh versions actually had this feature, but best I can tell, it's gone in newer (e.g. openssh) versions. Anyone have a clue why this is?

FTP with TLS support (1)

baffle (144921) | about 11 years ago | (#7216973)

If memory serves me right, you can use an FTP-server with TLS support. It encrypts the control-connection and leaves the data-connection unencrypted.

I think this feature is supported by ProFTPD.

ftps (1)

lavv17 (613412) | about 11 years ago | (#7217318)

There is secure extension of ftp protocol (ftps), which is usually turned on by AUTH TLS command in ftp protocol.

In this ftps protocol, control connection where the password is transmitted is encrypted, and the data connection is encrypted optionally.

lftp [lftp.yar.ru] supports AUTH TLS and turns on this secure extension by default if it is supported by server. In Red Hat 8 this extension is supported by default in wu-ftpd.Unfortunately, in Red Hat 9 it is not supported, since wu-ftpd was replaced with vsftpd.

ftp over ssl (1)

Phacka (144710) | about 11 years ago | (#7217324)

check out pure-ftpd, it can encrypt ftp control channel, you can find a list of compatible clients inside the archive

SAMBA (1)

theridersofrohan (241712) | about 11 years ago | (#7217653)

Use samba with the "encrypt passwords" option.

sockets (1)

fok (449027) | about 11 years ago | (#7217954)

use connection with netpipes [purplefrog.com] .
Like this:

server$ faucet 3000 -out tar cf - .
client$ hose server 3000 -in tar xvf -

You can secure the connection with iptables
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?