×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Skein Hash... In Bash

Unknown Lamer posted more than 2 years ago | from the sixty-four-bits-required dept.

GNU is Not Unix 90

First time accepted submitter Matt16060936 writes "...Last night (err.. 3am this morning) I finished an implementation of the Skein 512-512 hash algorithm (version 1.3). I'm a fan of Skein and hope it wins the SHA-3 competition next year. One of the nice things about Skein is how quickly it's been adopted by many platforms and implemented in many languages. To that end, I present Skein 512-512 implemented in Bash."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

90 comments

Ah, Bash (4, Insightful)

jeffmeden (135043) | more than 2 years ago | (#37391032)

The cause of, and solution to, all of life's problems. Here's to Bash! If it weren't for you, I would be a terrible sysadmin. Thanks to you, I am a terrible sysadmin!

Re:Ah, Bash (4, Funny)

Anonymous Coward | more than 2 years ago | (#37391242)

Don't bash bash.

Re:Ah, Bash (-1, Flamebait)

Anonymous Coward | more than 2 years ago | (#37391448)

Why not? It's "mostly POSIX-compliant" garbage, whose code sometimes works in Korn shell, used by would-be sysadmins who understand neither Linux nor UNIX.

The previous poster put it so crisply: "If it weren't for you, I would be a terrible sysadmin. Thanks to you, I am a terrible sysadmin!"

Is that not the truth!

Re:Ah, Bash (0)

Anonymous Coward | more than 2 years ago | (#37396540)

Why is the parent rated "Flamebait"?
Because there is no rating "Stupid"

bash? (0, Informative)

Anonymous Coward | more than 2 years ago | (#37391050)

wtf is wrong with you

Re:bash? (1)

idontgno (624372) | more than 2 years ago | (#37392538)

Agreed. Why wasn't this done as a Perl golf contest?

Re:bash? (1)

inode_buddha (576844) | more than 2 years ago | (#37393568)

fp.pl? (Score:5, Funny)

by CptChipJew (301983) Alter Relationship on Friday January 23 2004, @12:24AM (#8062958) Homepage Journal

open(heart_to_perl);
content-type: haiku/firstpost;
or die "i fail it";

Yes, I actually saved that post all these years. It's funny.

Optimized for 64-bit processors (1)

tepples (727027) | more than 2 years ago | (#37391052)

The Wikipedia article states: "The function is optimized for 64-bit processors". On a 32-bit computer, the Bash program will die with an error message "This program is written for 64-bit architectures." So if Skein is adopted as SHA-3, it might run unacceptably slowly on as legacy PCs with pre-x86-64 CPUs and on handheld devices with an ARM CPU.

Re:Optimized for 64-bit processors (0)

Anonymous Coward | more than 2 years ago | (#37391110)

I just checked his 64-bitedness checker... and it doesn't work at all, so don't worry. Now, the rest of the program, well I don't even know how to check that.

Re:Optimized for 64-bit processors (1)

jomama717 (779243) | more than 2 years ago | (#37391492)

I'll agree with your assessment of the 64-bitness checker, and with the GP's contention that it might run unacceptably slow on legacy machines:

~/p4/scripts/skein $ uname -a
Linux mjm 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:50 UTC 2011 i686 i686 i386 GNU/Linux
~/p4/scripts/skein $ time ./skein.sh < ./skein.sh
70d0...7e

real 4m30.224s
user 3m19.416s
sys 0m28.850s

Re:Optimized for 64-bit processors (1)

butlerm (3112) | more than 2 years ago | (#37391520)

You have got to be kidding. Implementing a hash algorithm in a scripting language is an inside joke. The overhead of the language in this case is easily ten thousand times the overhead of the actual calculations, and more likely ten times that.

Re:Optimized for 64-bit processors (2)

butlerm (3112) | more than 2 years ago | (#37391490)

A slightly slower than usual hash algorithm is not likely to be noticeable on a modern client, even a handheld device.

All else equal, a 64 bit algorithm should get roughly twice the amount of work done per operation as a comparable 32 bit algorithm, so the performance overhead on 32 bit architectures isn't nearly as bad as it looks. Ten to twenty percent maybe.

And if you are really concerned about performance, you write an implementation in assembly language.

Re:Optimized for 64-bit processors (1)

tepples (727027) | more than 2 years ago | (#37392000)

A slightly slower than usual hash algorithm is not likely to be noticeable on a modern client

Even if the hash is run every time a process is started, in order to check that the executable has not been altered since it was installed?

Re:Optimized for 64-bit processors (1)

LordLimecat (1103839) | more than 2 years ago | (#37392094)

Yes. You could probably run several thousand rounds of Skein on an input each second, and thats being conservative.

Re:Optimized for 64-bit processors (3, Informative)

butlerm (3112) | more than 2 years ago | (#37392256)

On the slowest 32 bit processor tested (32 bit, ARM v4, 75 Mhz), NIST benchmarked Skein using portable C code at just under 1 megabyte / sec. That is about twice as slow as SHA-2 on the same processor, and certainly slow enough that you might notice in the case you mention. On modern 64 bit processor (Intel Q6600, x86_64, 2.4 Ghz), more like 286 MB / sec for Skein, about twice as fast as SHA-2. See here [nist.gov] (pdf).

The striking difference between 32 bit and 64 bit implementations is much more than I would have guessed, but that may be merely a matter of optimization. For now it looks like a good excuse to use SHA-1 or SHA-2 when doing the sort of thing you describe on slow processors. For something like SSL or IPSEC, you aren't likely to notice the difference, because the bandwidth to a typical mobile device just isn't that high.

Re:Optimized for 64-bit processors (1)

drinkypoo (153816) | more than 2 years ago | (#37392992)

All else equal, a 64 bit algorithm should get roughly twice the amount of work done per operation as a comparable 32 bit algorithm, so the performance overhead on 32 bit architectures isn't nearly as bad as it looks. Ten to twenty percent maybe.

It depends on which processor you're using. x86-64 not only has a crapload more registers but it actually has general purpose registers. x86 instructions often (maybe even usually) expect your operands to be in certain registers and put the results in certain registers, to the extent that x86 really has zero general purpose registers. This is mitigated by register renaming on modern platforms. On a modern machine the cost might not be immense. On an older machine, it will be. Also, longer words mean less loads and stores...

Re:Optimized for 64-bit processors (0)

Anonymous Coward | more than 2 years ago | (#37392016)

Yeah, on a 32-bit processor you might have to suffice with merely outperforming the alternatives rather than utterly and totally crushing them.

Too bad:(

Not an issue (1)

pavon (30274) | more than 2 years ago | (#37393110)

Look at page 25 of the Skien Whitepaper [schneier.com]. Using C implementations, Skien-512 outperforms SHA-512 and Skien-256 is only about 75% slower than SHA-256 on a 32 bit CPU. That isn't unacceptably slow.

first post (1)

somersault (912633) | more than 2 years ago | (#37391076)

Damn. Slashcode won't let me post the Skein encrypted version of first post (Filter error: That's an awful long string of letters there.)

Re:first post (1)

Anonymous Coward | more than 2 years ago | (#37391482)

Thank goodness, since hashes aren't encryption.

Re:first post (1)

somersault (912633) | more than 2 years ago | (#37391958)

That depends on how well the hash function distributes its hashes, the size of the original message, and how good your rainbow tables are :p

Re:first post (0)

Anonymous Coward | more than 2 years ago | (#37395060)

No, it doesn't.

Hmmm (2, Insightful)

Anonymous Coward | more than 2 years ago | (#37391104)

Amazing how code is always produced in the middle of the night...

Re:Hmmm (0)

Anonymous Coward | more than 2 years ago | (#37391454)

Children are often produced in the middle of the night ( well, conceived )

Sourceforge? (1)

mhh91 (1784516) | more than 2 years ago | (#37391142)

Why didn't you post this on Github?

At least it has syntax highlighting, I'm no expert at Bash, but I don't think anyone would appreciate reading it with no syntax highlighting.

Re:Sourceforge? (1, Interesting)

gstoddart (321705) | more than 2 years ago | (#37391300)

At least it has syntax highlighting, I'm no expert at Bash, but I don't think anyone would appreciate reading it with no syntax highlighting.

People have been doing clever things in bash long before there was syntax highlighting. In fact, it took me years to finally accept that as anything other than clutter. Though, I still can't stand colored output of the "ls" command.

Having cut my teeth using line editors over 300 baud modems (or terminals wired into the mainframe via serial cables), I am sometimes amused to watch people who have only ever had these modern fancy tools when they're in a real environment like a client site. Because suddenly they can't do anything because they're so dependent on these things.

Now get off my lawn. :-P

Re:Sourceforge? (1)

catmistake (814204) | more than 2 years ago | (#37394292)

I can totally relate to what you are saying because I am one of the worst offenders... yet I'm a painter. I have all these cool brushes and rollers and pneumatic sprayers and scrapers and paint in all manner of colors and shades. But when I get to the client site, I look like an idiot because I have to resort to scraping off old paint with my fingernails and painting several coats of paint with my fingers. It takes forever and I gotta tell you it gets aggravating. But what am I gonna do? I got so used to all the great tools I use at home that I really suck and despise finger painting, and I am constantly having to redo work to correct mistakes. I guess that's what I get by getting so used to working easier with the tools I have collected over my career.

/I'm really a sarcastic sysadmin

Now get off my lawn. :-P

Now... why don't you just c'mon over across the street... go on and head around back... there's a cold keg tapped and the girls are smoking in the hot tub... careful of the dog poo... I don't know who's dogs those are...

Re:Sourceforge? (1)

gstoddart (321705) | more than 2 years ago | (#37397518)

LOL ... funny.

No, seriously ... when I first saw syntax highlighting I was literally like "WTF is this crap". It was just a jumble of colors on the screen, and I found it quite distracting.

I was used to working on monochrome VT52s and VT100s ... so the first time I saw syntax highlighting, I turned it off. To this day, I find the color output of "ls" actually conveys less information that knowing that an "@" is a symlink.

Now, of course, I'll take syntax highlighting any day of the week. But it really did take me several years to get there, because I was used to working in vi, and vi didn't do such things.

Re:Sourceforge? (1)

catmistake (814204) | more than 2 years ago | (#37405804)

I get where you're coming from... but control is an illusion. That's default in linux. I only first noticed that stuff when I originally jailbroke my iPhone... saurik turned Darwin (BSD) into linux... at least that's what it looked like. After that, I noticed it was like that in linux... just didn't notice it before. You can customize your command line till you're blue in the face... add paths, colors... etc.. but kind of a waste of time. Or you could just drag your .bashrc or .bash_login around with you, and then feel at home whereever you go. Or... idk, maybe you're right... leave it alone ya damn kids!

Self-encrypting (1)

Lexx Greatrex (1160847) | more than 2 years ago | (#37391202)

Sadly the hash of the bash script is only marginally less readable to me than the source.

Re:Self-encrypting (1)

gknoy (899301) | more than 2 years ago | (#37392070)

A skim of the functions looks like it is a clever implementation of bitshifting left and right (in varying amounts), as well as a block portion. May I suggest Applied Cryptography (http://www.schneier.com/book-applied.html) ? While it may not cover this particular hash algorithm (perhaps recent versions do?), a lot of the actions used here are covered there. The first half of the book (third?) is non-code, and VERY informative to anyone interested in how encryption works.

Dreadfully slow (1)

vadim_t (324782) | more than 2 years ago | (#37391276)

I've had times when I'd have found it useful to have something like base64 or md5 in shell script form to require less dependencies on ancient installations, assuming it could be made work with anything approaching acceptable performance.

Unfortunately this isn't it. A 194 bytes file took 3 seconds. The skein script (10K) takes 2 minutes and half, and it's ridiculously memory intensive too. The process grew to 150 MB in size.

And the "Useless use of cat" Award goes to (1)

kwark (512736) | more than 2 years ago | (#37391316)

Matt Tomasello for:
echo 'Usage: cat FILE | skein [ARGS]'

Re:And the "Useless use of cat" Award goes to (0)

Anonymous Coward | more than 2 years ago | (#37391566)

Heh yeah I think it was IBM that posted all the misuses of the shell and misusing cat particularly with grep was one of them.

It could easily be:
skein [args] FILE

Re:And the "Useless use of cat" Award goes to (0)

Anonymous Coward | more than 2 years ago | (#37391648)

Bah typed that from a phone so there was no HTML typed slashdot removed the less than symbol between args and FILE

Re:And the "Useless use of cat" Award goes to (1)

LordLimecat (1103839) | more than 2 years ago | (#37392364)

If the Skein bash function was written to take in text input and spit out a hash, using cat would allow using a filename as the source of the text you want SKEIN'd. If you tried doing "Skein [args] FILE", you would just get the hash of the filename.

How do you propose the SKEIN function determine whether the second argument is a filename, or the text to be hashed? Should he add that funciton as a further switch, and also implement the function of reading the contents of a file (which would presumably use "cat")?

Re:And the "Useless use of cat" Award goes to (1)

Hatta (162192) | more than 2 years ago | (#37392616)

/. ate his "less than" sign. What he means to say is:

skein [args] < FILE

Where bash handles opening, reading, and piping the file into STDIN. No need to start another process.

Re:And the "Useless use of cat" Award goes to (1)

El_Oscuro (1022477) | more than 2 years ago | (#37392682)

I would recommend the standard Unix conventions for stuff like this:

$ skein [args] [file] [--text 'some text']

  1. Loop through parameters, processing any that are valid arguments (including --help to display usage).
  2. First parameter that is not a valid argument is considered the file (--text means it is text).
  3. If no parameter for file/text, read from STDIN
  4. Write all error messages to STDERR (echo xxxx 1>&2). Exit with non-zero code
  5. Echo hash to STDOUT. Scripts can process it any way they want

Re:And the "Useless use of cat" Award goes to (1)

illtud (115152) | more than 2 years ago | (#37392986)

Am I being trolled?

If the Skein bash function was written to take in text input and spit out a hash, using cat would allow using a filename as the source of the text you want SKEIN'd. If you tried doing "Skein [args] FILE", you would just get the hash of the filename.

skein < FILE

surely?

Re:And the "Useless use of cat" Award goes to (1)

LordLimecat (1103839) | more than 2 years ago | (#37398166)

No, I didnt think of that as I deal with Linux CLI for about 10% of my job and I usually dont deal with reading input from files. It has been a long, long time since I have used that file pipe. I used to deal with it in Windows, and I got tired of using it when it didnt work properly with various commands (like set), so Ive always used alternatives, and that habit sticks with me in Linux as well.

Really, I dont get the big complaint about using cat to handle it, though, several people here seem to be advocating reinventing the wheel, and in bash script to boot.

Re:And the "Useless use of cat" Award goes to (1)

oursland (1898514) | more than 2 years ago | (#37393206)

Although, as Hatta pointed out, slashcode ate his less than symbol, you're still incorrect. "skein [args] FILE" would only make argv[argc-1]="FILE" not the zeroth file descriptor (stdin) equal to "FILE", which is necessary if "echo FILE | skein [args]" and "skein [args] FILE" are to agree. If such a program was made to hash the last argument you'd have to go through a heck of a lot of work escaping your file contents and trying to avoid the maximum argument length.

Re:And the "Useless use of cat" Award goes to (1)

LordLimecat (1103839) | more than 2 years ago | (#37398180)

It didnt eat my less than symbol, I just didnt think of using a file input pipe.

Re:And the "Useless use of cat" Award goes to (1)

WuphonsReach (684551) | more than 2 years ago | (#37397098)

How do you propose the SKEIN function determine whether the second argument is a filename, or the text to be hashed? Should he add that funciton as a further switch, and also implement the function of reading the contents of a file (which would presumably use "cat")?

Simply put, you follow the conventions established by md5sum and other command line hashing tools that have existed for many years/decades prior.

Re:And the "Useless use of cat" Award goes to (1)

LordLimecat (1103839) | more than 2 years ago | (#37398132)

Would that involve adding more bash code to his already long script that he probably doesnt ever want to have to re-work on?

Also, as others have pointed out, you could use unix file pipes to handle reading the file; adding more arguments to it really would be retarded.

Re:And the "Useless use of cat" Award goes to (1)

snookums (48954) | more than 2 years ago | (#37395284)

Matt Tomasello for:
echo 'Usage: cat FILE | skein [ARGS]'

I find this kind of pedantry a bit dull. When I'm hacking around on things at the command line I almost always use that form. If you're building up a pipeline of various parts to get something done, and you need to insert at the beginning, or reorder components, it's much easier if the input to the pipeline is a clean, separate term.

Trivial example:
cat FILE | foo is easily edited to head FILE | foo in a way that foo < FILE is not.

Re:And the "Useless use of cat" Award goes to (1)

Rysc (136391) | more than 2 years ago | (#37397188)

Hacking around on the command line is one thing. Writing examples is another. When you save your code for long-term re-use or reference purposes it should be well written.

Re:And the "Useless use of cat" Award goes to (1)

maxwell demon (590494) | more than 2 years ago | (#37397602)

But what's the problem with using cat here? It's not necessary, but it doesn't harm either, does it?
Also, well written code means maintainable code. Maintainable code means code that is easy to change. Which speaks for cat.

BTW, I've even found uses for the apparently useless use of cat in cat | command or command | cat (note: no filename on the cat!). That's when using programs which test whether standard input or standard output is a terminal, and I want the non-terminal behaviour even though I'm typing in a terminal.

Re:And the "Useless use of cat" Award goes to (1)

Rysc (136391) | more than 2 years ago | (#37401996)

But what's the problem with using cat here? It's not necessary, but it doesn't harm either, does it?

In fact it does do harm. Not even counting tangentials like "it encourages people to do this reflexively so they will get bitten when the difference really matters", it execs one more program and uses a good chunk more memory.

Maintainable code means code that is easy to change. Which speaks for cat.

I dispute that "cat foo | grep bar" is easier to change than "grep bar BTW, I've even found uses for the apparently useless use of cat in cat | command or command | cat (note: no filename on the cat!). That's when using programs which test whether standard input or standard output is a terminal, and I want the non-terminal behaviour even though I'm typing in a terminal.

In such a case it's both "not useless" and "being used interactively," where it probably matters less.

Re:And the "Useless use of cat" Award goes to (0)

Anonymous Coward | more than 2 years ago | (#37400180)

I use that style all the time, because:

cat file|grep something

allows you to change "something" very quickly - up arrow, alt-backspace and it's gone. By comparison:

grep something file

requires you to left-arrow to "something" before you can change it.

Re:And the "Useless use of cat" Award goes to (1)

kwark (512736) | more than 2 years ago | (#37401392)

Using cat effectively doubles the required IO operations.
A solution to your "left arrow to something":
arrow up
ctrl-arrow left
ctrl-w

Re:And the "Useless use of cat" Award goes to (0)

Anonymous Coward | more than 2 years ago | (#37400956)

rubbish! modern compilers will optimi... oh.

Why is this on the front page? (0)

Anonymous Coward | more than 2 years ago | (#37391406)

Don't get me wrong... as a coder I'm all up for stuff being done on with an inappropriate language choice just for shits and giggles or to see if it can be done... but why the hell is some guy's bash script making the front page? Is this really 'newsworthy'? (Yes yes, I must be new here.)

Bash can be a great language and although it can sometimes be a stretch, its generally possible to massage anything out of it that you would do in any other sane procedural language (as long as you don't care about runtime). As someone else said, something done in a turing tarpit such as intercal would be much more impressive!

Re:Why is this on the front page? (1)

blair1q (305137) | more than 2 years ago | (#37391470)

>Is this really 'newsworthy'?

inasmuchas anything newsworthy for nerds can be said to be non-newsworthy for nerds, and the line between nerd and non-nerd moves with the subject, then yes, this could be newsworthy for bash or skein nerds

on the other hand, it's the first i've heard that they were cooking up a SHA-3. so maybe there are multiple layers of nerdiness involved.

tomorrow i expect to see your posting of how you've implemented it in brainfuck.

Re:Why is this on the front page? (1)

pjt33 (739471) | more than 2 years ago | (#37392186)

it's the first i've heard that they were cooking up a SHA-3. so maybe there are multiple layers of nerdiness involved.

It's hardly [slashdot.org] the [slashdot.org] first [slashdot.org] time [slashdot.org] it's been mentioned [slashdot.org] on Slashdot.

Turing-complete (0)

KiloByte (825081) | more than 2 years ago | (#37391576)

Newsflash: a known algorithm implemented in a Turing-complete language. Uhm...

Re:Turing-complete (1)

danlip (737336) | more than 2 years ago | (#37392660)

Newsflash: a known algorithm implemented in a Turing-complete language poorly suited for that algorithm

fixed that for you.

I implemented it last night using my abacus and slide rule

Re:Turing-complete (1)

GrandTeddyBearOfDoom (1483117) | more than 2 years ago | (#37397122)

I always like to challenge people to try it in Brainfuck or with a single instruction computer. Turing completeness is more of a theoretical thing as it ignores difficulty, and polynomial time Turing machine stuff only partly captures that.

Re:Turing-complete (1)

maxwell demon (590494) | more than 2 years ago | (#37397900)

I've done it on my (purely theoretical) single instruction computer.
Now, it wasn't exactly difficult because the single instruction of that computer has been well chosen.

Here's the implementation:

DWIM

Validation? (1)

hickmott (122356) | more than 2 years ago | (#37391846)

Now do the validation suite for it. That's really the hard part.

Re:Validation? (1)

dfsmith (960400) | more than 2 years ago | (#37393486)

Well, given the hex<—>string routines should probably be printf '%02x' instead of '%x', I'd give it a slim chance of validating.

Mod article up (0)

Anonymous Coward | more than 2 years ago | (#37392164)

Kudos, submitter!

This is the kind of stuff Slashdot needs more of.

I'm sick of Slashvertisements and Idle garbage I can find on reddit.

mod Up (-1)

Anonymous Coward | more than 2 years ago | (#37392320)

it transforms into of various BSD as ffitingly

mod 3Own (-1)

Anonymous Coward | more than 2 years ago | (#37392510)

I'll h4ve offended disCussions on

How Hard...? (0)

Anonymous Coward | more than 2 years ago | (#37392652)

Would it be to port this BASH script to other shell's scripting languages; that is something I'd like to see?

Re:How Hard...? (1)

Rysc (136391) | more than 2 years ago | (#37397268)

There's nothing in it that fundamentally requires bash, you just need a shell which supports arrays. It might not perform as well, however, without some of the bashisms. I'd say porting this will be of minimal effort.

As Dr. Seuss said (0)

Anonymous Coward | more than 2 years ago | (#37393632)

Skein Hash in Bash gives me a rash.
That's why I go skiing at Attitash...

Power fail (0)

Anonymous Coward | more than 2 years ago | (#37397804)

Seems the 64bit check fails under Power (bash 2.05b/ppc64).

Seems I have a Zero bit system! :-)

Error in script? (1)

amirulbahr (1216502) | more than 2 years ago | (#37405200)

On one of my systems:

me@here$ uname -m
i686
me@here$ getconf LONG_BIT
32
me@here$ echo $(( 1 << 32 ))
4294967296
me@here$ echo $(( 1 << 64 ))
1

Your test would consider my ARCH as 64 when it is clearly not. But then why does left-shifting 64 times not overflow?
Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...