Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ask Slashdot: Are Timed Coding Tests Valuable?

samzenpus posted about a year and a half ago | from the beat-the-clock dept.

Businesses 776

First time accepted submitter xkrebstarx writes "A buddy of mine recently applied to a large tech company. Before setting up a phone interview with him, the unnamed company issued a timed coding test to gauge his coding prowess. He was allotted 45 minutes to complete an undergraduate level coding assignment. I would like to ask the Slashdotters of the world if they find value in these speed-programming tests. Does coding quickly really indicate a better programmer? A better employee?"

cancel ×

776 comments

Sorry! There are no comments related to the filter you selected.

I dunno... (5, Insightful)

Joce640k (829181) | about a year and a half ago | (#42544543)

I dunno...but it will weed out the pretenders/bullshitters.

Re:I dunno... (1)

JWSmythe (446288) | about a year and a half ago | (#42544573)

    I guess it would prune out those who would take 2 hours to print "Hello World".

Re:I dunno... (5, Insightful)

hackula (2596247) | about a year and a half ago | (#42544733)

I never believed the whole "95% of interviewees fail the FizzBuzz test" until I started interviewing candidates. People with 15 years of "experience" on their resume would regularly fail or give up. I also encouraged googling, including just searching for the exact problem, and I encouraged questions and told them that both behaviors were seen as a good thing. IDK how someone could possibly get through a CS program and still fail this test, but it happened regularly.

Re:I dunno... (3, Funny)

Anonymous Coward | about a year and a half ago | (#42544771)

wth is a fizzbuzz test?

Re:I dunno... (4, Informative)

beelsebob (529313) | about a year and a half ago | (#42544793)

Well, you just failed for not googling...

But, print all the numbers from 1 to 100, except rather than printing multiples of 3, print fizz, rather than printing multiples of 5, print buzz, and for multiples of both, print fizzbuzz.

Re:I dunno... (2)

Gr8Apes (679165) | about a year and a half ago | (#42544851)

so this should take you what, about 3 minutes, given a normal typing speed, compile and run time?

Re:I dunno... (4, Informative)

beelsebob (529313) | about a year and a half ago | (#42544873)

And yet, as stated above, surprisingly, 95% of job applicants fail, just like the AC above.

Re:I dunno... (2)

hackula (2596247) | about a year and a half ago | (#42544929)

Should take 3, I give 30. Still fail. Also, any language they want no matter what the job requires.

Re:I dunno... (1)

tibit (1762298) | about a year and a half ago | (#42544967)

LOL, in 45 minutes you should do it in assembly for any architecture out there, even ones you never used before. Might take a bit longer in brainfuck, though :)

Re:I dunno... (4, Interesting)

Cryacin (657549) | about a year and a half ago | (#42544779)

These tests are vital when interviewing. We recently put out an ad for a senior programmer, and got about 100 or so responses. After 25 responses hit the circular filing cabinet, as they were obvious resume spam with no cover letters, we declined around a further 50. Of the remainder, we asked a simple question. In JavaScript, without using the reverse() method, reverse an array of numbers containing 1,2,3,4,5 in the most efficient way possible.

We got 5 responses back, which we interviewed. The other 20 were out. These tests are meant to weed out the crap that would waste our time, and honestly, the 5 guys that responded, responded in under 5 minutes.

Re:I dunno... (2)

arth1 (260657) | about a year and a half ago | (#42545009)

To be fair, there are a lot of senior programmers who don't speak javascript. They may have experience in it to the point of setting up the algorithms that the regular programmers implement, or assisting with eyeballing for glaring errors, but may not be javascript programmers themselves.

An efficient solution would, depending on the language, be to either change the pointers of the array, or create a new index to replace the existing one, and thus avoid copying each element, but I can fully understand seniors not knowing the best way to do this in an arbitrary language without looking it up.

But the most efficient way possible (and that's what you asked for), given that it only has five elements, all known, would likely be something like (not that I speak JavaScript):

a[0]=5;a[3]=2;
a[a[1]=4]=1;

Re:I dunno... (1)

Talderas (1212466) | about a year and a half ago | (#42544807)

I had to look up what the test is, because I'm not familiar with the term or game...

Now, I don't by any stretch of the imagination call myself a coder. It's not a primary function of my job but isn't it just a basic modulus operation?

Re:I dunno... (0)

Anonymous Coward | about a year and a half ago | (#42544971)

I believe the idea is 1) can they do it and 2) do they brute-force it with an unoptimized algorithm or come up with a nice solution. 1) is a start and 2) is what you want.

Re:I dunno... (1)

deniable (76198) | about a year and a half ago | (#42544987)

It's also a loop, some print statements and a couple of comparisons with booleans. A nice little package and one of the test pieces I do when I'm learning a new language.

Re:I dunno... (1)

buddyglass (925859) | about a year and a half ago | (#42544763)

Presumably the exercise is more complicated than "print Hello World".

Re:I dunno... (5, Insightful)

beelsebob (529313) | about a year and a half ago | (#42544593)

The other thing it'll do is near guarantee that they get some bugs from all applicants, they'll then have something to discuss in the phone interview, and gage the applicant's responses and proposed fixes.

Re:I dunno... (3, Interesting)

SJHillman (1966756) | about a year and a half ago | (#42544767)

Mod parent up

I took a couple of programming classes in college and we had some timed tests. However, failing to get the desired results within the allotted time was only enough to knock off a partial letter grade (IE: go from a B+ to a B). The professors mostly wanted to see how you approached a problem, whether you commented your code, if any horrible coding practices jumped out, etc. In the follow-up, they would also want to know how you would fix any issues you came across if given more time.

How much you got done was still somewhat important - there was a basic expectation that you would have some functionality working and other pieces at least partially completed. After all, if you have to spend 45 minutes looking up how to do a basic loop, then you're probably not experienced enough for anything beyond entry level (if that), so it helps weed out the people who are outright lying about skills.

Re:I dunno... (1)

Gr8Apes (679165) | about a year and a half ago | (#42544879)

...After all, if you have to spend 45 minutes looking up how to do a basic loop, then you're probably not experienced enough for anything beyond entry level (if that), so it helps weed out the people who are outright lying about skills.

I'd say if you need to spend time looking up how to do a basic loop in a language you're supposed to know, perhaps coding isn't a profession you should be in. I hear McDonald's is hiring.

Re:I dunno... (2)

beelsebob (529313) | about a year and a half ago | (#42544911)

Which is why this test is useful ;)

Re:I dunno... (2, Insightful)

Anonymous Coward | about a year and a half ago | (#42544613)

Exactly. I don't believe that a speed programming will always identify the "better" programmers or employees as the OP appears to be making a correlation between.

I would suspect that the company is just trying to weed out those who actually have no experience, not those who are "better". It is just a method of screening which can be very effective when dealing with large numbers of applicants.

Re:I dunno... (1)

tibit (1762298) | about a year and a half ago | (#42544995)

The problem is: in most cases, 90%+ of applicants are useless and will fail such a test. It's crazy how many people apply to programming jobs without knowing anything about programming. Apparently, they themselves are OK with such a waste of time. Idiots.

Re:I dunno... (2)

deniable (76198) | about a year and a half ago | (#42545007)

It sounds like less of a speed comparison than an upper limit to stop people wasting both sides' time.

Re:I dunno... (0)

Anonymous Coward | about a year and a half ago | (#42544655)

Even 20 or 30 lines of code will show how tidy you are and how you do or don't break down a particular issue into functions and subroutines (although obviously, everyone will try to be neat)

Re:I dunno... (1)

Anrego (830717) | about a year and a half ago | (#42544737)

Pretty much this.

Typically prior work is used for this purpose, but I could see this being valuable as a "you must be this competant before we will bother talking to you seriously" type test.

Also without knowing the scale of the coding assignment, we can't know if 45 minutes is barely enough time or a very reasonable amount of time.

NOPE. (2)

TheBlackMan (1458563) | about a year and a half ago | (#42544557)

If asked for simple answer, here it is.

The answer is... (1, Insightful)

oPless (63249) | about a year and a half ago | (#42544559)

Emphatically NO

Re:The answer is... (2)

0xdeadbeef (28836) | about a year and a half ago | (#42544669)

Paula Bean, is that you?

Of course (5, Insightful)

Bogtha (906264) | about a year and a half ago | (#42544565)

They are extremely valuable - they let you know which potential employers don't have a clue about programmer productivity / expertise.

Re:Of course (0)

mykepredko (40154) | about a year and a half ago | (#42544717)

Very insightful.

Somebody with points please mod the parent up.

myke

Re:Of course (0)

Anonymous Coward | about a year and a half ago | (#42544983)

I agree they are extremely valuable, but I actually like them.

I find the tests are beneficial to both parties.

A) The employer can gauge the skill of the candidate.
B) The test can tell the candidate quite a lot about the job on offer and the interviewer.

The time allotted is often much less than required. This will tell the employer if you panic easily, or if you are confident enough to say the task demands more time. Also look for questions that demand more information. They will want to see if you will try to provide an incorrect solution to a question that requires more info before beginning. Also impossible questions: what does error code #non-existent signify. Those that answer fail. Pepper many of those on a questionnaire and you can easily cut the wasters

For the candidate, the questions will give an indication as to how well thought-out the selection process is - the smarter the questions the better. Simple questions like what is the purpose of an 'abstract class' is a give-away the question were simply copied without much consideration. And in my view demonstrates incompetence on the part of the interviewer.

For me, maximum 15 questions, some trick, and write code. Something deceptively simple. eg create a person class, use interfaces/inheritance as you wish to create two specialist classes, employee, contrator. This will demostrate who can use OOP, design patterns, preference for inheritance/composition, coding style. Interpretation of spec under pressure.

Also, allows for folks who have not done computer degree, or have degree in other field. If I'm dropped from the selection process because I don't have a degree, despite impressive resume, I don't want to work for them, as they don't have the skill to recognise talent, and depend on tick-box selection process.

Re:Of course (1)

tibit (1762298) | about a year and a half ago | (#42545015)

Have you ever done a lot of "programmer" hiring? Such a test is valuable to both the employer and the employee. The employer won't waste time talking to clueless idiots. The employee will know that the employer is serious about weeding out idiots and thus having resources left to talk to people who know what the fuck they are doing. Such tests have not much to do with what most of your job is. Software engineering is a lot about design, testing, etc., but if you can't code your way out of a fizzbuzz, it doesn't matter what else you pretend to know.

Re:Of course (5, Informative)

buddyglass (925859) | about a year and a half ago | (#42545017)

I took one of these during an interview. They put me in a room w/ a computer my IDE of choice and had me do a project. The general gist was to build a program that read instructions from a file, parsed them, carried them out, then output the result at the end. The instructions all had to do with string manipulation. After I finished, they brought my code up on a projector and had four or five developers do a sort of code review, asking me why I chose to do various things in certain ways. So, not only did they test whether I could finish the project successfully, they got to see whether I can speak intelligently about design decisions and handle constructive criticism from other developers. Seemed like an extremely valuable exercise.

No undergraduate level stuff for me (2, Insightful)

loufoque (1400831) | about a year and a half ago | (#42544567)

I personally discard any company that tests undergraduate-level stuff.
This kind of thing is completely irrelevant to the skills of a senior engineer.

Re:No undergraduate level stuff for me (5, Insightful)

beelsebob (529313) | about a year and a half ago | (#42544601)

I personally discard any applicant that thinks like this, it means they can't actually think about algorithms, data structures, and how to design something well, and in doing so will produce sloppy assed code.

Re:No undergraduate level stuff for me (2, Informative)

nedlohs (1335013) | about a year and a half ago | (#42544625)

You really think that would be for a senior engineer position?

Unless it's an ungraduate assignment from a very specialized course it's going to be simple to do in less than 45 minutes. The only assignment I have done or set at an undergraduate level that wouldn't have been so was the "write a ray tracer" from a computer graphics course - and only because I don't remember the math off the top my head (it almost writes itself once you have that), I'd need to reread some stuff first.

Is it going to "indicate a better programmer", No. Is it going to indicate a "better employee", No. It probably will weed a large number of the really bad programmers though. And sure it'll cull a few potentially good employees , that might be worth culling those really bad ones though.

Re:No undergraduate level stuff for me (5, Insightful)

buddyglass (925859) | about a year and a half ago | (#42544751)

You would be surprised at the (low) quality of some candidates whose resumes suggest they're qualified for a senior engineer position.

just like speed writing (2)

jehan60188 (2535020) | about a year and a half ago | (#42544577)

they are as valuable as speed writing tests
Took the GRE a year ago, and got a 0 on the writing portion since I decided to write an essay about how useless speed writing is.
Real writing, is a very useful skill- that's when you have access to a piece of writing software that can do simple things like spell check!
Similarly, knowing how to spit out code is not nearly as useful of a skill as knowing how to break a problem down, and decide what tools would be best for it.

Re:just like speed writing (1)

clickety6 (141178) | about a year and a half ago | (#42544685)

they are as valuable as speed writing tests Took the GRE a year ago, and got a 0 on the writing portion since I decided to write an essay about how useless speed writing is. Real writing, is a very useful skill- that's when you have access to a piece of writing software that can do simple things like spell check! Similarly, knowing how to spit out code is not nearly as useful of a skill as knowing how to break a problem down, and decide what tools would be best for it.

Gammar Nazi recommends that you upgrade to "a piece of writing software" that can also perform simple grammar checks.

Re:just like speed writing (3, Funny)

SurfMan (969573) | about a year and a half ago | (#42544811)

Funny how you misspelled "Gammar Nazi"

Re:just like speed writing (0)

Anonymous Coward | about a year and a half ago | (#42544785)

they are as valuable as speed writing tests

Speed writing tests can be very valuable when looking for a secretary. They are absolutely useless when looking for a plumber.

For a programming job, I'd consider them useful only to weed out those who need a minute to find a key: They obviously haven't used the keyboard much, and thus can't have much programming experience. Other than that, if your programming speed is limited by your typing speed, you are either a genius or not thinking enough about what you write.

Re:just like speed writing (1)

realityimpaired (1668397) | about a year and a half ago | (#42544875)

The difference is that spelling and grammar are inherently important skills to be able to write clearly and effectively. You do *not* want to rely on spell check, and most good writers will turn spell check and grammar check off, at least in my experience. They'll catch mistakes on proofreading, and the more experienced writers tend to make fewer mistakes to begin with. It's specifically because of that that "speed writing" tests (as you put it) are actually useful at measuring how effective somebody is going to be. If they are able to communicate clearly, and effectively, and do it with a short period of time allotted (such that they can't proofread), then you have a pretty good measure of how efficient they will be at communicating when they do have the time given.

The same logic applies to speed coding, I expect. Though as other posters have pointed out, it could also be used specifically with the expectation that you're going to make mistakes, so that they can ask you how you would go about correcting any bugs that arise during the interview.

Useful for weeding out non-programmers (5, Insightful)

Ckwop (707653) | about a year and a half ago | (#42544595)

We use Fizzbuzz and a short SQL test that take a total of 30 minutes for the first part of the test. If they fail this, we can them and don't give them an interview.

A surprising number of people fail this test.

We then have a larger problem with much more time allotted. Here we're looking for style and quality of construction.

That said, even with this longer test, the people we hire tend to get the same distance through the test. They're at least within the same half of an order of magnitude.

At the end of the day, in a paid position you can and do have a deadline to work to. You can't take forever building something. You have to produce the goods!

Re:Useful for weeding out non-programmers (0)

Anonymous Coward | about a year and a half ago | (#42544753)

Out of interest, what do you ask for the SQL test and what would you typically ask for the larger problem?

Re:Useful for weeding out non-programmers (0)

Anonymous Coward | about a year and a half ago | (#42544847)

At the end of the day, in a paid position you can and do have a deadline to work to.

Just do it like Microsoft: When it's time to release, first move the release date a few times, then only take the parts which are ready (don't mind if you already announced the other parts, just silently omit them; make up the lack of new code by a new shiny colour scheme and UI tweaks), and only fix the bugs after you released the code. :-)

Re:Useful for weeding out non-programmers (1)

Hardhead_7 (987030) | about a year and a half ago | (#42544981)

I'm working through a "Teach Yourself to Program" book right now. Figured I'd give it a try. What do you think, want to hire me? :D

But no, seriously, I'm interested in feedback.

https://gist.github.com/4502183 [github.com]

Re:Useful for weeding out non-programmers (1)

Ja'Achan (827610) | about a year and a half ago | (#42545045)

Look into the "modulo" operator (Wikipedia has a somewhat cryptic page about it [wikipedia.org] ). Basically, your multipleof3 function can be replaced by (0 == (i % 3)). Your main loop contents is somewhat messy and redundant, too.
But it look like it'd work.

Re:Useful for weeding out non-programmers (0)

Anonymous Coward | about a year and a half ago | (#42545019)

At the end of the day, in a paid position you can and do have a deadline to work to. You can't take forever building something. You have to produce the goods!

What company is this? I want to avoid idiots who produce bad software by thinking like this.

Re:Useful for weeding out non-programmers (3, Insightful)

davidbrit2 (775091) | about a year and a half ago | (#42545025)

Yeah, give them a simple task. Something that any reasonable programmer should be able to do in 15 minutes. Then give them a solid hour to work on it. If they can't produce something working in that time, that's a pretty informative result. The time limit isn't a speed challenge; it's meant to be very generous, and act as, "Look, we need to move on..."

Personally, I like to give a few different options from which they can choose freely. Something procedural, something OO, and maybe something in SQL or a functional language. Perhaps a couple different choices for each - around 6 to 8 total. That way you don't run the risk of excluding a worthy candidate because you happened to design some problem they aren't really specialized in, and if they can't handle any of them, that's a nice big red flag.

Sometimes you need to think on your feet (2)

fatphil (181876) | about a year and a half ago | (#42544597)

So yes, they are valid.

I heard of one respected hi-tech company giving a 3 hour C programming task. I scribbled my solution on the back of a beermat in about 3 minutes. Were I to have been able to do it in a functional language, it would have been a 1-liner.

I can only imagine that a 45 minute test would be "write a program which says 'hello' to the world".

Facebook? (0)

Anonymous Coward | about a year and a half ago | (#42544599)

Let me guess, is the company in question Facebook?

I think the coding test + phone interview + on-site interviews are solid combo. They ensure you hire coders that are smart and can complete assignments quickly without too much room for error.

Is the test indicative of the actual job? Well it depends on the project and I think that for quick iterating projects it's a good fit.
That being said, I wouldn't use the same approach to hiring people to build a rocket...

Does coding quickly really indicate a better progr (0)

Stumbles (602007) | about a year and a half ago | (#42544607)

No. If you need an explanation then your a crappy ass programmer, manager, etc.

Re:Does coding quickly really indicate a better pr (1)

Anrego (830717) | about a year and a half ago | (#42544827)

Personally I doubt this test is measuring coding speed.

45 minutes is probably plenty of time. They've probably structured it so anyone competant for the position will probably be able to do it with time to spare. The goal then I suspect is to weed out non or bad programmers prior to doing the proper interview where actual skills will be assessed. This feels like an initial "you must be this good to enter" test more than a "how many SLOCS an hour are you going to give us" test.

Aren't all tests timed to some degree? (5, Interesting)

khakipuce (625944) | about a year and a half ago | (#42544609)

A lot of companies use coding tests as part of the interview process and pretty much there will be some time limit, whether stated or not. They are not going to let you sit there for two days to answer 20 questions or complete a 10 line routine.

As to the value of rigid timing, then that is a bit dubious.Do you want fast and sloppy or slow and accurate? Does this tell you something about the organisation and whether or not you want to work there? I feel it really depends on how they treat the results WRT the timing.

not really (0)

Anonymous Coward | about a year and a half ago | (#42544639)

coding quickly does NOT indicate a better programmer - for an undergraduate level coding assignment, it only indicates that you've been programming for a while and that it does not take you long to complete simple logic tasks.

Re:not really (2)

fuzzyfuzzyfungus (1223518) | about a year and a half ago | (#42544761)

coding quickly does NOT indicate a better programmer - for an undergraduate level coding assignment, it only indicates that you've been programming for a while and that it does not take you long to complete simple logic tasks.

It certainly isn't the only skill or(if the applicant pool is any good, even a distinguishing skill); but demonstrating familiarity and facility with your tools is a good thing...

Yes (5, Insightful)

Splab (574204) | about a year and a half ago | (#42544643)

If the people performing the test are any good.

First of all it will weed out the anti authority programmers. (There is a lot of people who will refuse to do this - the door is right there...)

Next, it will test if you can handle stress - quality of the code (should be) is irrelevant in these kind of tests. But you learn a lot about how people act under stress.

Personal experience, during an interview I was asked to implement a hash map on a whiteboard. What they where looking for was not an actual shiny working example (hands up, those who don't need to go look in a book to find a proper hashing function (or the interwebs)) - they wanted to see how I handled myself in a stressful situation.

Re:Yes (0)

Anonymous Coward | about a year and a half ago | (#42545037)

That's sick! Stress is not something I look for in employment. Well, not if I want to last longer than 5 years.

Re:Yes (4, Informative)

SirGarlon (845873) | about a year and a half ago | (#42545051)

If the people performing the test are any good.

This is a key point. I "failed" one of these interviews because my implementation updated the left-hand side of a loop condition, and the kid who was interviewing me didn't have the listening skills or mastery of basic algebra to understand that can be equivalent to updating the right-hand side.

The take-away for me was, thank God I'm not working there with him as a colleague or, worse, a supervisor.

This was at Google, where everyone thinks they're hot shit.

Depends (1)

hcs_$reboot (1536101) | about a year and a half ago | (#42544649)

on the problem difficulty. In order _not_ to eliminate potentially good candidates, the time allocated to the interview problem should be - say - twice what it usually is for that kind of problem. The problem has to be timed, in order for the candidate to feel really involved in its solving ; it's an interview, not a school.

Useful (0)

Anonymous Coward | about a year and a half ago | (#42544665)

It is very useful, as it show a programmers real programming style under pressure. For instance, if 40 mins you produce a solid working solution, that has things like clear variable, function and object names and with good comments, then your likely to be a very productive member of the team. If you produces a working solution but the way you have written means no one else can figure out how it works, then they can not bother interviewing and move on.

45 minutes is plenty of time! (2, Funny)

Anonymous Coward | about a year and a half ago | (#42544691)

To find completed code for that assignment online.

Yes. (1)

drolli (522659) | about a year and a half ago | (#42544701)

Somebody with a complete education must be able to solve an undergrad problem without thinking much, just sytematically applying knowledfge and understanding.

In the same way in which anybody can expect that I (Phd in physics) will solve a decay equation correctly without trying aroung for hours or set up a equation of motion for a dynamical system without looking up for hours in textbooks (i am at least supposed to know where to find the solution), i expect that a self-announced programmer is able to write down somthing which applies a simple algorithmic principle to a small problem.

Sure it does (3, Insightful)

beinsvein (2752465) | about a year and a half ago | (#42544703)

If a job requires a skill that is easy to test, it should be obvious that you want to test it. Programming is such a skill. Sure there are tasks within programming that can't be tested in 45 minutes, but there are also tasks that can. I'd feel I knew more about a programmer's skills having seen a couple dozen lines of code she's written than for instance hearing her last employer's opinion, which may be biased by all sorts of interests, or reading the list of projects she'd worked on, where you don't know how she contributed. College grades in programming courses might provide the same kind of information, but courses may not be standardized and the candidate might have developed her skills since college.

Re:Sure it does (1)

elucido (870205) | about a year and a half ago | (#42544775)

If a job requires a skill that is easy to test, it should be obvious that you want to test it. Programming is such a skill. Sure there are tasks within programming that can't be tested in 45 minutes, but there are also tasks that can. I'd feel I knew more about a programmer's skills having seen a couple dozen lines of code she's written than for instance hearing her last employer's opinion, which may be biased by all sorts of interests, or reading the list of projects she'd worked on, where you don't know how she contributed. College grades in programming courses might provide the same kind of information, but courses may not be standardized and the candidate might have developed her skills since college.

Wow, showing is better than telling.

Heck yes (5, Informative)

dkleinsc (563838) | about a year and a half ago | (#42544709)

The basic rule of programming interviews is that you should demand that they actually program. It doesn't necessarily have to be a difficult problem: I've handed somebody a standard Fizzbuzz in an interview, and the competent candidates will solve it in 2-5 minutes, while the incompetent candidates won't solve it in 15 minutes.

The reason this is necessary is that on paper, the incompetent candidates can look identical to the competent candidates.

We found that broken code was a better test (4, Interesting)

mykepredko (40154) | about a year and a half ago | (#42544715)

When I was at RIM, we used a broken quick sort method that the candidate was asked to fix. We didn't time how long it took the candidate to implement the fix, but it generally required the candidate to do some research as to what was wrong. One of the team leads created a simple app that tested the performance (ie speed) of the fix.

What was really interesting to me was the number of candidates who refused to do the test (50-60%) because they said it was "beneath them". The big problem was, RIM's HR (OD) that insisted we interview the candidatest that refused to do the test because we were losing potentially half the candidates that were responding to the job applications (this was when RIM was The Place To Be).

The best candidates were the ones that did the test and asked if we had any more. These candidates also tended to produce code that ran sort the fastest.

myke

Re:We found that broken code was a better test (1)

SJHillman (1966756) | about a year and a half ago | (#42544865)

In my experience working with helpdesk/systems people, those they say something is 'beneath them' are usually the ones who can't figure it out anyway. As a sysadmin, there are a lot of tasks that are 'beneath me' by way of organizational structure and job description, but I happily do them anyway. It can be fun to throw some simple stuff in there to break up the bigger stuff.

having experienced something like this (1)

buddyglass (925859) | about a year and a half ago | (#42544721)

I would say a highly time-constrained exercise is not so useful since it will disqualify some candidates who might otherwise turn out to be valuable developers. That said, I think a non-highly-time-constrained coding exercise could be a very valuable tool. If you give a guy four hours to code up something that should take about two and it still has terrible bugs or the code is complete spaghetti (or he doesn't finish in the first place) then you can be pretty sure he's not someone you want to hire. This strikes me as infinitely more useful than "write binary sort on the whiteboard while I stare at your back".

They're very good... (2)

clickety6 (141178) | about a year and a half ago | (#42544725)

But they should shorten the limit to 10 minutes or less.

.

It shouldn't take any competent programmer more than a few minutes to google some fitting example code, rename a few variables and update the comments :)

Re:They're very good... (1)

vlm (69642) | about a year and a half ago | (#42544819)

Exactly. Its an honesty test. FizzBuzz is a stereotypical "do you know what a modulus is, the mod operator in your language of choice, some manner of "for loop and or incremental counter, and some basic logic and precedence rules in your language?" Since everyone thinks its brilliant after you do it a couple times it should be pretty quick, but something "similar level" thats actually new rather than what everyone else uses, should take 2 to 10 minutes. So give them one minute. Anyone who gets it perfect just googled and copy-pasted.

Debugging tests are much more fun. Redesign a system is much more fun. FizzBuzz is at least fast.

Re:They're very good... (1)

vlm (69642) | about a year and a half ago | (#42544845)

some manner of "for loop and

F look at that syntax error. Extra quote typo. See, you can tell I squirted that out in only a minute instead of giving it the 2 to 10 minutes it deserved. If it was perfect prose in that limited time, you'd know I cut and pasted that shit right outta Knuth.

Yes and no (0)

Anonymous Coward | about a year and a half ago | (#42544729)

It does not in and off itself measure how good a programmer is, but then again; what is a good programmer?

In here we tend to think a good programmer is measured in technical excellence, but to people running companies, a good programmer is just as much a person that can get something working as fast as possible. In the latter view, a speed-coding test is definately valuable.

I lean towards thinking that any test is both meaningfull in the sense that it will give you a glimpse of the person being tested, and meaningless in it's ability to give an accurate picture of that persons skills and ability to apply them under the circumstances prevailing in your working environment. You can prune out some candidates with tests of various sorts, but at the end of the day, hiring someone is still a leap of faith that may or may not end up showing that your gut-feeling about the person at the interview(s) was right.

No (0)

Malephex (906619) | about a year and a half ago | (#42544731)

Read subject.

Eh, it tests for something... (1)

fuzzyfuzzyfungus (1223518) | about a year and a half ago | (#42544749)

Unless the problem is so easy, or the time limit so relaxed as to be irrelevant, testing speed definitely tests some sort of skill(it sure as hell isn't joe codemonkey who wins those speed-coding competitions); but whether it tests a relevant skill is somewhat less clear and more job-dependent.

If you are hiring people to program something of any significant size, you will have time/deadline issues; but they will be less of the "Bob can't get a bubble sort written in less than 40 minutes!" and more of the 'as our morass of legacy complexity grows, the time needed to make any significant change approaches infinity because changing anything potentially changes almost anything else in unpredictable ways' type problem. There are people who are more and less skilled in the art of designing complex systems such that they don't(to the degree possible) turn into spaghetti, and such that you can change just one thing without discovering surprise changes everywhere; but that is a very different kind of 'speed', and probably a more valuable one.

I would do poorly (1)

elucido (870205) | about a year and a half ago | (#42544765)

Timed tests would only stress me out to the point where I would probably code poorly due to worrying about the clock. I suppose it depends on the difficulty of the project of course. If it's say a few hundred lines of code I could do that maybe an hour or so. If it's less than that then I could do it even quicker.

How many lines of code? How complicated are we talking about? What language is it in? If it's something like Python I could for sure write a web browser or some kind of bot in 100 lines of code in an hour. It will be sloppy, it will have bugs, it wont have a graphical user interface, but it would work.

If I'm judged on quality too then I'd fail. If I'm judged on bugs then I'd fail. In an hour you'd get whatever I could pull together into working code.

Quick is not best. (2)

GhigoRenzulli (1687590) | about a year and a half ago | (#42544769)

Ask my wife.

Like standardized testing (1)

ironjaw33 (1645357) | about a year and a half ago | (#42544781)

I recently interviewed at a few companies that administered programming tests like this. I thought I would have performed better on the test as a sophomore or junior undergraduate than I would now with a graduate CS degree and several years of experience. Implementing breadth first search in Java (and all the required tree data structures) with minimal syntax errors in under 10 minutes is a lot like the BJ scene in Swordfish. It's no more than a puzzle with some added drama.

Companies that use tests like this will wind up hiring only people who can solve them, usually to the detriment of being able to solve large scale system design problems as well as being able to work as part of a team. Being able to solve problems that can't be figured out in an hour is one of the major differences between school and the real world. A CV of academic publications or a project portfolio will go a lot farther in my book when judging capability.

A different way of phrasing TFH (0)

Anonymous Coward | about a year and a half ago | (#42544787)

Ask Slashdot: Are You Confident You Could Ace a Timed Coding Test?

and suggest reading comments with that in mind.

I have an experience (0)

Anonymous Coward | about a year and a half ago | (#42544795)

I once applied for a PHP web developer position and was asked to WRITE ON PAPER a piece of code that would count how many 1 there are in a made up 32 bit binary.

Why would you ever do that sort of thing with PHP and especially in a website code?

I've worked with PHP developing web apps and such for over 10 years and I never had to do such a thing. Obviously I failed to write the "proper" code!

Thou the interview was so mindnumbingly stupid, that I would of probably declined the job anyway.

Possible? (0)

Anonymous Coward | about a year and a half ago | (#42544799)

What, do you mean Swordfish *wasn't* accurate?
http://www.youtube.com/watch?v=rUY8HysBzsE

Better than the alternatives (1)

raymorris (2726007) | about a year and a half ago | (#42544801)

If you have maybe an hour to find out whether or not someone is a decent programmer, I can't think of too many things better then asking them to program.
The text of the question asked about programing FAST. Generally, programing quickly is programming poorly, so hopefully they gave him 45 minutes to do a 30 minute task and they are mostly interested in looking at the code, not at the time. Hopefully, they just didn't want to give him three days to post the assignment on one of the help boards and get other people to do most of the code for him.

Emergency patch (1)

Liamecaps (2428636) | about a year and a half ago | (#42544813)

I often have to perform emergency tasks in less than 45 minutes, so it's fine, we give timed tests to candidates too. It just gives you a rough idea of their skills and how they cope with stress. It's just one part of the interview process.

yes, any test can be valuable (0)

Anonymous Coward | about a year and a half ago | (#42544817)

People lie. I have interviewed people for System Engineer positions that claimed they were developers... they couldn't tell me what an array was or code hello world on a white board in any language.

And if you say you know C++ you better know cin and cout.

If a web based test is timed, it keeps them from looking everything up online or in a book. Or tries too.

Depends on what you are looking for (0)

Anonymous Coward | about a year and a half ago | (#42544835)

Like all tests like this in the interview process, it isn't that the candidate gets the work done per se, but the insights you get into his work style, problem solving skills, and how he functions under pressure.

If you are saying "complete this or you fail the interview", then you are definitely missing out. However if you are simply using it as a tool to get access to the above information about the candidate, it can be quite useful.

Depends (0)

Anonymous Coward | about a year and a half ago | (#42544839)

While on one hand I see it from the big company point of view not wanting to waste their time and money on someone who sucks, but then you do risk the chance of missing out on someone who actually is good, just not in a timed no prep situation.

I've been on both sides of the table, giving the tests and taking the tests. I know I sucked on a couple curve balls being thrown my way, just wasn't in the right mind set for those questions, ended up not getting any of those jobs.

On the other side, I've given some curve balls questions earlier in my career to potential employees, but I've since switched going line by line on their resume. Seeing how how low level they can explain what they did, asking key questions along the way. For instance, a programmer says they built a ASP.Net Web Application as their most recent project. A good question I'd ask, explain the architecture. If they say they called out to their Entity Framework from the code-behind, no master pages, no caching etc. I know the level of programmer I'm dealing with.

Another thing is passion, if they aren't passionate about their job and are just looking for a paycheck, that's not the type of programmer I'd look for. Do they read up and better their craft in their free time with little programming projects on the side? For me, that's a key element in the industry today. There is a huge gap between those that are devoted to our craft and those who aren't, you can see it in their work.

To me all of those elements far out way one's ability to handle a curve ball timed programming test in evaluating someone.

It Doesn't Matter If Tests Help Employers (2)

assertation (1255714) | about a year and a half ago | (#42544855)

I've been programming for about 14 years. I have seen this conversation come up plenty of times. Many people come up with good reasons why such programming tests and other programming job interview techniques unfairly weed people out. I've been a victim of it myself at some interviews and have gone on to do STELLAR work at OTHER places. The bottom line is that employers feel that they need SOMETHING, even something crappy, so... these interview tests are a fact of life.

However misguided, if you are a programmer and you want a new job, you have to deal with this reality.

Fortunately, in 2013 there are a plethora of programming interview prep books out there so at least you have a chance if somebody throws a pop quiz at you about the tower of hanoi, asks you to write a script to print out only prime numbers or asks you to write a script to reverse a string using no speciality functions though you have not thought about such mishigas in years.

If it helps HR separate the chaff (0)

Anonymous Coward | about a year and a half ago | (#42544857)

In 2009, i.e. the depths of the Great Recession, we tried to hire a couple C++ developers. HR must have sent us 100 resumes, we winnowed it down to about 10. Despite all claiming to be senior level C++ developers, not a single one could explain simple subclassing concepts, and I wager none of them could have even managed to compile with gcc if we'd handed them an already written program.

So I'd say at an easyish timed write-and-compile test would have some value.

Insulting but necessary (1)

EmperorOfCanada (1332175) | about a year and a half ago | (#42544887)

Insulting but necessary; recently I a friend who is an awesome coder walked out of an interview when they handed him an final exam like coding quiz. He felt that it was too impersonal a way to assess for a position as the lead senior developer for a company. But at the same time I would say that the vast bulk of people I have encountered in the programming world were shockingly short of programming skills. A few years ago I developed a system for a company that then was handing it over to a company with whom they had a maintenance contract. So the two top programmers for the company with many years of programming experience and one of whom had graduated from one of Canada's "Top" CS programs began asking me interesting questions about my code; the most critical question was "What are those odd arrays that you are using?" The odd arrays were associative arrays. So I just thought they used a different word so I began thinking of anything even slightly synonymous, hashtable? Lookup table, map, key/value pair? In the end I gave a more than one hour tutorial on the value of a using keys to find values and how the hashing and use of trees could make the underlying algorithm wicked fast even on massive data sets.

The questions went on and on. Won't that waste too much memory (it was a single purpose server with 8G and a data set well under 500M)? Won't scanning through that huge array take forever(I had explained the whole look up tree and showed that a typical lookup only took a tiny handful of steps and that it didn't matter as the system was wicked fast even with a test load many times the system growing far beyond the company's wildest dreams.)

But these two who I now refer to as the Hash Twins could talk the talk. They would blah blah about log files and in-line assembly and wanted to argue about commenting style but had massive gaps in fairly basic bits of programming. So these two would have interviewed quite well unless your interview was basically a coding test. A timed coding test also makes sense in that if you gave me the weekend to pass a basic test in a language that I am very unfamiliar with like Haskell I could probably pass but I would have to look up nearly every bit as I not only don't use Haskell and am unfamiliar with functional programming so I would not be able to step into a complicated Haskell project as the technical lead. I shouldn't pass.

But even after years of C++ I would fail if you gave me a test full of pedantic questions such as "List all the different basic ways you can use the keywords static and const, define the following keywords: explicit, compl, noexcept, and decltype." (I had to look those up)

So I would think that a basic filter test should be quite understandable where you give a person a machine and say, "In the language of your choice code me a function that takes an integer and returns and integer of that Fibonacci number." or for a database person say "Given this trio of tables(users, products, and product sales) cough up an SQL statement that pulls up a list of all the Canadian users and a count of product X that they bought and include those Canadian users who bought nothing." If you couldn't do something that simple in less than 20 minutes (assuming job relevant questions) then an interview would be a waste of everyone's time. I am also willing to bet that the two above questions would kill more than 80% of applicants around these parts. Also questions like the above allow the user to show their thinking; do they put in error conditions? Do they use stupid variables? Do they do it in 10 seconds really well or do they sweat out the whole 20 minutes and produce marginal crap? Do they show a lack of Mathematical ability by asking what the Fibonacci is (Not a deadly strike but a test to see how broad their knowledge is as great CS does use math. Plus if they don't know Fibonacci it only takes a few seconds to explain)

I would avoid such an employer. (0)

Anonymous Coward | about a year and a half ago | (#42544913)

This is just a variant on how to address the problem of having too many people apply for a position that too little time has been allocated to fill properly. Nobody can be arsed to talk properly anymore. Whatever your view on the validity of the test and its appropriateness - you are decreasing the risk of hiring an absolute numpty and increasing the risk that someone really useful or valuable to your company might be missed. 25years ago I sat a load of tests involving ticking boxes and drawing things. The very prestigious banking institution in question said I had an aptitude for 1) working in the computer centre or 2) working in the trade floor as a runner. Because I liked computers and knew almost nothing about the trade floor I chose option 1. Had I chosen option 2) I would have ended up undoubtedly financially better off (in a big way). I left the Bank after 18months and joined an outsourcing outfit. Who did this test serve best? Had someone with experience and who cared explained (i.e. spoken properly) to me about these choices then my career and arguable my value to the bank could have been much improved.

My point is: these test are just filters and probably offer very little value in terms of what your woking life will be like.

Deadlines (2, Insightful)

wisnoskij (1206448) | about a year and a half ago | (#42544933)

Are you telling us that you have never heard of job deadlines before?

Big "depends" (0)

Anonymous Coward | about a year and a half ago | (#42544935)

If you hire someone fresh from college, this test is of quite limited value in predicting future performance. If you hire someone with (self-claimed?) long experience on the job, it will give a rather good impression of how well his claimed skill set holds up to reality. If you hire someone for a research position, it is much more important how he deals with stuff he is not prepared for rather than otherwise. However, an oral examination is in this case much better suited for keeping the candidate on the brink of failure and seeing how he fares there.

The problem with doing it offline is that if he does not know where to start or does not know where to stop [wikipedia.org] , you don't really get the information you were after.

It's About Liability (1)

EmagGeek (574360) | about a year and a half ago | (#42544953)

With product liability lawyers dragging more and more people into court in product suits, they are looking deeper and deeper into a company's processes for developing those products, right down to hiring practices and whether a company was negligent in hiring the "best and brightest," arguing that failing to hire the "best" engineers amounts to negligence when the products they design fail.

Juries eat that shit up, too.

What matters most = thoughtful intelligent design (0)

Anonymous Coward | about a year and a half ago | (#42544955)

See subject-line above - You're NOT going to do that in 45 minutes on anything of complexity, or unfamiliar territory (be that in data, it's schema, or processing it for input to the required outputs - especially since, as I've seen in my case, NOBODY PROCESSES THEIR DATA EXACTLY THE SAME AS THE NEXT GUY, or has the same exact data either... it's just NOT happening (usually)).

* A 45-minute test WILL, however, probably show how well you know the particular language tool, & perhaps, how fast you can solve a problem with it... which is, again, NEGATED (somewhat) by the above.

(Tests like these are NOT the "end-all/be-all/do-all" in other words & don't tell the entire story - they're just another 'metric' which has SOME uses of course, but imperfect overall!).

That's my take on it, coming from nearly 18 yrs. total time professionally (& long before that academically) with the titles of software-engineer, programmer-analyst, & programmer...

This is why data-processing jobs (business programming) are SO prevalent, & the 'steady-eddy' part of the trade since nobody does the job exactly the same as the next guy, & things like data conversions, for example, are necessary (even between softwares used, let alone when you send or receive data from others).

APK

P.S.=> If we could solve every problem in the world in 45 minutes, there wouldn't BE any, in other words...

... apk

Yet another stupid trend (1)

JustNiz (692889) | about a year and a half ago | (#42544957)

One thing in moving to the US from UK I've noticed is how relatively low quality everything seems to be made in the US. Everything is designed cheap and weak and falls apart quickly.
This apparently also extends to software development. All the US companies I've worked at have been completely happy with very crap code as long as its done fast. I've not really seen anything like it in my previous 25 years of software contracting all over the EU. In fact the mentality there is the direct opposite. The mantra I'm more used to is that rework costs about 10X the cost of getting it right the first time so work smart.
I still can't believe this incredibly short term 'get it out quick' view in the US really pays off more, but as everyone over here seems to do it, I guess it must, at least for US markets.

Depends on medium and content of the test (1)

Korbeau (913903) | about a year and a half ago | (#42544959)

I would be nice to have more insights as to the content of the test and the medium used.

If it is a complex algorithm (for instance I don't know, writing a compression method, an digital filter etc.) these kind of questions are better suited for pen&paper where the candidate will mostly spill out pseudo code, ideas and comments. As others said, in such a test I would not expect the candidate to write flawless code, but it will give me an idea as to whether he knows basic concepts of programming, possibly of a particular language, and most importantly if he knows how to think.

On the other hand, if you are given 45 minutes in front of a computer, I would expect it to be a very very simple task. This could be for instance writing a form where the user inputs some values that get serialized on disk. I would more be interested to know if the candidate is as fluent as he says with the tools provided. I would expect him to finish on time and evaluate the real quality of the code. Or if he had problems this would be a base of discussion as to what happened. Of course, in such a context I don't see why this test would not be "open books" with access to the Internet and any reference documentation needed.

Both tests are valid enough I think. But mixing them (doing a pen&paper test for code that is meant to compile flawlessly or doing a seated test with a complex algorithm) is just malicious! Maybe that is what your friend encountered.

Better Test (0)

Anonymous Coward | about a year and a half ago | (#42544965)

It might be better if they had a "find the bug and fix it" timed test. Not a week goes by in my SaaS organization in which I don't have a time-sensitive bug that needs to be found and fixed.

Depends... (1)

siDDis (961791) | about a year and a half ago | (#42544991)

If you are testing if the person can program or not, then it's fine. But if you're looking for only the cream of super software developers, then something minor as the persons emotional state for the day can make a huge impact on the result.

For me personally, I think understanding the actual question is the most difficult part. Some people find bizarre mathematical puzzles fun. I prefer puzzles from the real world, like for example how to get two systems to talk together.

Depends on what you are coding for. (0)

Anonymous Coward | about a year and a half ago | (#42544997)

If it is Java or web language then certainly there is a call to code a routine quickly because the coding needs to be out the door much faster. Skilled web and data base coders are using higher level language and therefore do not need to think at the device level.

However if the coding is for chip level routines for things like device drivers then a speed test could be very counter productive.
If I were hiring a device programmer my thinking would be along the lines of how well does the individual understand the importance of using concise comments. More to the point how well do they incorporate sub routines in C and low level language like Assembly when called for.

The job will always get done right faster in the long run by the programmer who best understands when to slow down and think things through for device coding.

Just assume. Don't test! (0)

Anonymous Coward | about a year and a half ago | (#42545023)

NEVER administer an objective test under controlled conditions.

Instead, give them a subjective test; for example, consider the candidate's hair and how well they are dressed and how much fun it is chatting with them.

Also, have the candidate be judged in a group discussion so each judge can influence one another as the discussion unfolds. Don't ever collect independently created data.

My first job did this (0)

Anonymous Coward | about a year and a half ago | (#42545043)

My first job SORT OF did this. What they did was call in a bunch of entry level programmer applicants, give them a simple COBOL program to do (keep in mind this was a few decades ago), and then graded the results. The test was open book, and started out with unlimited time. Eventually they discovered that anyone who was there more than a couple of hours never passed the test, so they shortened it to 4 hours to weed out the really bad ones quicker.

The applicants came from Computer Learning Center and the local community colleges for the most part. I was in a group of about 80-100 test takers, and out of that group, only 3 had acceptable scores, and out of those 3, two were hired. One was not hired because the interviewer couldn't understand her and vice versa.

I talked to some of the candidates from a later group, after I had been hired and working there for about 2 years. He was from a community college, and said that he was comfortable with MOST of COBOL, but he did have trouble in his PROCEDURE DIVISION. I was polite enough not to bust out laughing.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>