Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

IT's Next Hot Job: Hadoop Guru

samzenpus posted more than 2 years ago | from the specialize-or-adapt dept.

Databases 112

gManZboy writes "JPMorgan Chase and other companies at this year's Hadoop World conference came begging for job applicants: They say they can't find enough IT pros with certain skills, including Hadoop MapReduce. That spells high pay. As for Hadoop's staying power as a career path (a la SQL 30 years ago), IBM, Microsoft and Oracle have all embraced Hadoop this year. Maybe the best news of all: 'Intelligent technologists will pick up Hadoop very quickly.'"

Sorry! There are no comments related to the filter you selected.

I'll start now! (3, Insightful)

loftwyr (36717) | more than 2 years ago | (#38004964)

After all, every other framework of the month has lasted for 30 years, Hadoop will have at least as much staying power as Ruby on Rails!

Re:I'll start now! (1)

ackthpt (218170) | more than 2 years ago | (#38005350)

After all, every other framework of the month has lasted for 30 years, Hadoop will have at least as much staying power as Ruby on Rails!

I wonder if you can learn how to create and maintain security in less than a week.

Re:I'll start now! (1)

Amouth (879122) | more than 2 years ago | (#38005520)

it's simple - rent a cement mixer and wire snips - start with pouring it over everything until it is fully encased - then go around with the wire snips and cut any communication cables coming out of the system..

wait - did you want to use it? then how could it possibly secure if you allow users to use it?

Re:I'll start now! (1)

jsnipy (913480) | more than 2 years ago | (#38005644)

Accessibility is an aspect of security

Re:I'll start now! (1)

MrEricSir (398214) | more than 2 years ago | (#38005852)

Is that the latest excuse as to why the "security solutions" out there today are complete crap?

Re:I'll start now! (1)

froggymana (1896008) | more than 2 years ago | (#38008868)

Is that the latest excuse as to why the "security solutions" out there today are complete crap?

Just put it in a VM. If it's in a VM it's secure, right?

Re:I'll start now! (1)

sexconker (1179573) | more than 2 years ago | (#38006168)

Accessibility is an aspect of security

No, it isn't.
Accessibility is a concession security types make.
If a thing is DDoSd, the security is actually improved.
The PHBs will whine and cry, though.

Re:I'll start now! (1)

Semyazza (881512) | more than 2 years ago | (#38013994)

In traditional physical security the only things limiting it are: Convenience, Comfort and Cost. This can be said the same for computer security.

Re:I'll start now! (1)

Niomosy (1503) | more than 2 years ago | (#38007228)

Which is why the goal should be user-unfriendly. If they can't figure out how to use it, you won't have to worry about security issues.

Re:I'll start now! (5, Funny)

Xyrus (755017) | more than 2 years ago | (#38008186)

Hadoop is geat, fast, and easy to use!*

*Statements are based on word count example and terrasort. Performance may vary greatly. May need to spend significant amounts of time to tune cluster for your particular data and applications to see any real performance. Applications may need to be specially designed to fit within the tuning constraints of the cluster. This statement does not apply if you are using binary data of significant size (BDOSS). Multiple data sets and apps may not perform equally well within the cluster. Data pre-processing, formatting, sequencing, and other such steps are not included in this statement. If you any problems, hope to $DIETY Google returns a hit. See your browser search bar for further details.

Bad learning resources (1)

Anonymous Coward | more than 2 years ago | (#38005066)

If you want a strong userbase, projects with good, easy to use learning resources do better. When you hit the hadoop main page, they tell you what it is, but not what you need to know in order to use it. They don't tell you what languages it supports. They give no examples of usage. Essentially, they don't do you any favours.

Re:Bad learning resources (4, Informative)

ackthpt (218170) | more than 2 years ago | (#38005304)

If you want a strong userbase, projects with good, easy to use learning resources do better. When you hit the hadoop main page, they tell you what it is, but not what you need to know in order to use it. They don't tell you what languages it supports. They give no examples of usage. Essentially, they don't do you any favours.

I spent some time trying to implement some nice free tools from IBM and Apache. I found I needed to download X and do a build of it, but half way through it wanted Y to complete the build. OK... So I go find Y and try doing a build on it, but need something else from Apache, which doesn't like the vesion of Apache I'm running. So I get the other Apache thing and find I can't get it to start up. I go research it and find conflicting and incomplete information all over the web. I throw in the towel.

One thing needed is One source for information and clear instructions for a basic, default build of a platform. Once that is reliable, then document ways to add foo and bar or even plugh if it suits you.

Re:Bad learning resources (1)

evil_aaronm (671521) | more than 2 years ago | (#38005616)

Agree completely. Describes my experience with Linux - which I've used and liked, for the most part, since '95 - very closely. I now use a Mac as my primary machine. I get the close-to-the-metal experience - when I want it - with the ease of double-click installation.

Re:Bad learning resources (4, Informative)

JonySuede (1908576) | more than 2 years ago | (#38005758)

drink the maven kool aid, and you worries will be beyond you.
To use hadoop :

        org.apache.hadoop
        hadoop-core
        0.20.205.0

in your pom.mxl

Then write 2 classes like those one:

class MyMap extends MapReduceBase implements Mapper<K1, V1, K2, V2 >...
class MyReduce extends MapReduceBase implements Reducer<K2, V2, K3, V3>...

Feed instances of those to a JobConf and feed that instance to a JobClient.

The rest should be obvious to a seasoned programmer, just by looking at the nomenclature of the classes hierarchy.

The great Ward Cunningham, is right, put two days into studying something and you are already half way to expert. [infoq.com]

Re:Bad learning resources (0)

Anonymous Coward | more than 2 years ago | (#38007240)

There a similar maven recipe for deploying a hadoop cluster? That's the part that struck me as really tedious, but I might be confusing that with an hbase install.

Hadoop is easy -- having the analytical skills to express a problem as mapreduce in the first place is the hard part.

Re:Bad learning resources (1)

JonySuede (1908576) | more than 2 years ago | (#38008236)

There a similar maven recipe for deploying a hadoop cluster

no, not now, but if you need it, you could easily fork http://mojo.codehaus.org/wagon-maven-plugin [codehaus.org] so that it could be used like that:mvn install deploy:deploy deploy:execute-on-remote.

Hadoop is easy -- having the analytical skills to express a problem as mapreduce in the first place is the hard part.

Agreed, math is more useful, my halfexpertise enables me to assert that the right way to express a problem for hadoop is it to formulate it into associative (a+b==b+a) and distributive (a*(b+c)==a*b+a*c) operators. But I only have two day of self-education on the subject, if someone more experienced would like to enlighten us, I would appreciate.

Re:Bad learning resources (1)

mr_stinky_britches (926212) | more than 2 years ago | (#38008898)

You forgot your oozie workdflows.mxl!!!1

And then what about your PIG UDF's... and wait, I actually want to do it in scala, mkay?

Re:Bad learning resources (2)

kelemvor4 (1980226) | more than 2 years ago | (#38007642)

I spent some time trying to implement some nice free tools from IBM and Apache. I found I needed to download X and do a build of it, but half way through it wanted Y to complete the build. OK... So I go find Y and try doing a build on it, but need something else from Apache, which doesn't like the vesion of Apache I'm running. So I get the other Apache thing and find I can't get it to start up. I go research it and find conflicting and incomplete information all over the web. I throw in the towel.

One thing needed is One source for information and clear instructions for a basic, default build of a platform. Once that is reliable, then document ways to add foo and bar or even plugh if it suits you.

Sounds like IBM all right. They make some decent products sometimes. I'm fairly certain that other times they go out of their way to make things a pain in the ass to use. Maybe it's supposed to be a joke on the rest of the world?

Right. (0)

Anonymous Coward | more than 2 years ago | (#38005092)

Because somehow remaking existing clustering ideas as a single point of failure framework written in one of the most inefficient languages ever is somehow revolutionary and necessary.

Of course, dumb IT managers will eat it up. What ever happened to simple, reliable techniques that didn't rely on the latest flavor of the month to execute?

Re:Right. (1)

Anonymous Coward | more than 2 years ago | (#38005478)

Java is one of the most inefficient languages ever? I take it you've never programmed in ruby, python, perl, etc. IIRC, Java benchmarks have shown it outpacing everything except for C/C++, FORTRAN and OCaml.

Re:Right. (4, Interesting)

ackthpt (218170) | more than 2 years ago | (#38005548)

Java is one of the most inefficient languages ever? I take it you've never programmed in ruby, python, perl, etc. IIRC, Java benchmarks have shown it outpacing everything except for C/C++, FORTRAN and OCaml.

On first execution (and compile) it's slow. On first creation of an instance it is slow. After that Java makes up for itself rather nicely. If well implemented it's a great way to go, though I wouldn't chose it for my 3D rendering or reconciling a fiscal year's worth of journal entries, it's not that kind of language.

Re:Right. (1)

bertok (226922) | more than 2 years ago | (#38007108)

On first execution (and compile) it's slow. On first creation of an instance it is slow.

But it doesn't have to be slow ever! Microsoft .NET doesn't have most of those problems, despite being otherwise mostly identical. That's because Microsoft applied this fantastic new technology that apparently Sun has never heard of called a "cache".

This is why Java fell flat on its face in the desktop world, because Sun couldn't wrap their heads around that fact that every launch will be a "first instance" because having dozens of simultaneously running instances of a single process is very rare on desktops. Oh, and of course, on top of this, Java doesn't share code between processes, so the few situations where there are many instances of a process running (e.g.: Citrix XenApp or Terminal Servers) can enjoy two to three times the memory usage compared to native and .NET applications.

Java was originally developed for set-top boxes where there's only one process running that starts on boot. It never really grew up to embrace the PC world, and works on servers as well as it does only coincidentally.

Re:Right. (1)

Spykk (823586) | more than 2 years ago | (#38007304)

though I wouldn't chose it for ... reconciling a fiscal year's worth of journal entries

You might be surprised how often java is used to do just that.

Re:Right. (0)

Anonymous Coward | more than 2 years ago | (#38005684)

I take it you don't know what the phrase "one of X" means, because you have perfectly described what I mean by "Java is ONE OF THE most inefficient languages ever" by listing more of them.

Are people really this dumb these days?

Re:Right. (1)

sexconker (1179573) | more than 2 years ago | (#38006220)

I take it you don't know what the phrase "one of X" means, because you have perfectly described what I mean by "Java is ONE OF THE most inefficient languages ever" by listing more of them.

Are people really this dumb these days?

Yes, yes they are.
All I hear is java this, python that. People care more about being able to throw shit on a wall and have it run than they do about performance, reliability, or functionality.

Getting the Experience (5, Informative)

geoffrobinson (109879) | more than 2 years ago | (#38005120)

The trick is going to be getting the appropriate experience without having learned it on the job already.

Yes, it can be done. However, this technology is geared towards environments with lots of nodes in big clusters. (which can run Linux) That's not the same as simply learning a language.

I got a job utilizing a "Big Data" database technology by being at the right place at the right time, when this technology was being rolled out. It's also hard to find people with that specialized experience.

So I would suggest to companies, hire people and train them. Just get quality people if you can't find someone with the specific skill set.

Re:Getting the Experience (1, Insightful)

theshowmecanuck (703852) | more than 2 years ago | (#38005220)

That's OK, it won't stop moron head hunters from stipulating in the coming weeks that they only want Hadoop programmers with at least 5 - 10 years experience. I remember seeing that for Java programmers... in 2000.

Re:Getting the Experience (1)

Questy (209818) | more than 2 years ago | (#38005262)

Or 10 years of Windows 2000 in 2002. The answer is always the same, too... "that's what my client wants" and my answer is also always the same: "Then your client is an idiot and you can't count."

Re:Getting the Experience (1)

MichaelKristopeit421 (2018882) | more than 2 years ago | (#38006024)

"Windows" is a product LINE that had certainly been around for 10 years in 2002... the client was obviously asking for 10 years of experience in the product line, culminating with experience with the latest available version.

you're an idiot and you can't infer.

Re:Getting the Experience (2)

raydobbs (99133) | more than 2 years ago | (#38006170)

Oh, how I wish this was how it's supposed to work... many employers really REALLY -MEAN- having ten years of experience in the product that JUST came out. I remember getting through the 'gauntlet' only to have the interviewer get really pissed that I didn't have the experience. Told me I 'wasted his time and mine, and that he would make sure I would NOT be considered for any position in that company EVER, just for being such a liar'.

Needless to say, I told him how awesome he was - and never worried about it again.

Some people ARE -those- kind of assholes, though.

Re:Getting the Experience (0)

MichaelKristopeit421 (2018882) | more than 2 years ago | (#38006276)

you are the -only- kind of liar. the lying kind.

go lie to someone else.

you're an idiot.

Re:Getting the Experience (1)

Aryden (1872756) | more than 2 years ago | (#38006408)

How would you know if he's lying? I've been asked the same kinds of questions by idiot managers, and the answer is always "No I do not have x years of experience with Y product that just launched"

Maybe you just don't get that there are seriously ignorant people out there in positions that somehow determine your eligibility for employment. Especially your average HR manager.

I interviewed for a gig working for an MMO developer, the HR guy spent 45 minutes talking about how "This workplace culture eludes him" and how he has never played computer games or worked for any company in the IT industry. He came to the company from a warehouse management job. Yet the only 2 technical questions he asked me, which I explained easily, he couldn't comprehend.

Re:Getting the Experience (1)

MichaelKristopeit422 (2018884) | more than 2 years ago | (#38006768)

considering he already claimed to have claimed to have experience that wasn't possible, and quoted someone else who claimed such action was an act of lying, it's irrelevant to determine if he is currently lying... i only stipulated that he is certainly a liar... because he is. he admitted it. you're an idiot.

how would you know if the HR guy really couldn't comprehend you? perhaps he was slyly testing your ability to cope with those who SEEMINGLY don't comprehend you.

you're an idiot.

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38006468)

Sea kelp.

Re:Getting the Experience (0)

MichaelKristopeit422 (2018884) | more than 2 years ago | (#38008580)

why do you cower in my shadow? what are you afraid of?

you're completely pathetic.

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38007006)

Just last year I was contacted by a headhunter demanding 5 years of Exchange 2010 experience...

Re:Getting the Experience (1, Interesting)

ackthpt (218170) | more than 2 years ago | (#38005424)

That's OK, it won't stop moron head hunters from stipulating in the coming weeks that they only want Hadoop programmers with at least 5 - 10 years experience. I remember seeing that for Java programmers... in 2000.

Blame HR departments. They need to spend some time with the internal department which needs the guru. I remember having a good laugh in 1999 when some ads were run, looking for people with at least 10 years Java experience. The sick thing is the HR department or Headhunter will use that as a screening device and only end up with liars applying -- like the contractor we had for 2 weeks, who claimed to be an expert in a staggering number of tools and languages, despite a rather young age -- yeah, he had to borrow my copy K & R to try to puzzle how to write a date verification function and in a week he still didn't have it done, so much for being proficient in c.

Re:Getting the Experience (1)

layer3switch (783864) | more than 2 years ago | (#38008796)

> I remember having a good laugh in 1999 ... how to write a date verification function

Y2K, good times. good times...

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38005998)

I don't know why that's supposed to be so unreasonable. I saw lots of JAVA IS TEH BEST ads in 1995. I suppose if I had give a shit then about Java I could have learned more and started getting experience.

Re:Getting the Experience (1)

xclr8r (658786) | more than 2 years ago | (#38006144)

Don't argue with them on the details.. think outside the box. Just give them dumb answers back.

20 years experience including the following:
a.
b.
c.
d.
e.
f.
etc. etc.

Re:Getting the Experience (1)

Tablizer (95088) | more than 2 years ago | (#38006500)

That's OK, it won't stop moron head hunters from stipulating in the coming weeks that they only want Hadoop programmers with at least 5 - 10 years experience. I remember seeing that for Java programmers... in 2000.

I've seen some of those time-travel ads myself. My colleagues said of this practice, "just lie to the HR people. It's what we did to get here."

Re:Getting the Experience (1)

geoffrobinson (109879) | more than 2 years ago | (#38007286)

I recently inquired about a side gig involving that rare database skill. Apparently, they weren't interested in part time person with this skill who was willing to do remote work.

No, the person had to be on-site... for a 3 month contract. I just told them "good luck finding the right candidate". But as you guys said, they probably wind up with liars.

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38008700)

It's just as getting a job offer from Google and then having to make silly high-school programming assignments as a way to asses your skills.
After working +20 years in the industry, you'll get used to all kinds of morons.

.

Re:Getting the Experience (1)

Builder (103701) | more than 2 years ago | (#38010480)

If you get an offer, why do you still need to assess your skills? Or did you mean an interview ?

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38006244)

The trick is going to be getting the appropriate experience without having learned it on the job already.

Yes, it can be done. However, this technology is geared towards environments with lots of nodes in big clusters. (which can run Linux) That's not the same as simply learning a language.

I got a job utilizing a "Big Data" database technology by being at the right place at the right time, when this technology was being rolled out. It's also hard to find people with that specialized experience.

So I would suggest to companies, hire people and train them. Just get quality people if you can't find someone with the specific skill set.

Words of wisdom sir!

IT needs apprenticeship with classes and real work (1)

Joe_Dragon (2206452) | more than 2 years ago | (#38006720)

and NOT just CS classes.

Take a tech school class load and add apprenticeships to it.

Re:Getting the Experience (1)

aliquis (678370) | more than 2 years ago | (#38007106)

People should just do a huge piracy network with it.

Re:Getting the Experience (0)

Anonymous Coward | more than 2 years ago | (#38008916)

The experience requirement is sometimes misinterpreted. It should say programmer with 5-10 years experience, that also knows Hadoop. They want 5-10 years of work experience, AND you know Hadoop. Not 5-10 years of Hadoop experience.

Re:Getting the Experience (1)

geoffrobinson (109879) | more than 2 years ago | (#38012116)

I understand. I'm speaking to getting the experience at all.

That spells high pay. (0)

Anonymous Coward | more than 2 years ago | (#38005130)

Typically when I read "That spells high pay." it usually isn't much higher than the industry average.
Bankers get $5.5 million in bonuses per year. Now that's high pay!

Become an analingus expert instead! (-1)

Anonymous Coward | more than 2 years ago | (#38005170)

It happened when I was 19, a guy I met a guy in my College library took to his dorm and turned me around having pulled pants down. I figured he wanted to eat me doggystyle, when he stuck his tongue up my anus...

7 years later and more than 30 partners of all shades; half of whom have performed analingus on me, has me thinking its perhaps the new cunnilingus and 10 years time it will be part of foreplay.

PS: I return the favour.

Your thoughts.

What about stability and uptime of their web?!? (1)

papaia (652949) | more than 2 years ago | (#38005360)

They should look into some resources able to make/redesign/re-architect their site to have ALL offered services available for more than a few hours at a time. From downtime of days, to on-and-off access through their mobile apps, to services unavailable at all hours of the day, via their "standard" web presence ... how much of this will Hadoop address?!?

general IT market fairly hot (3, Interesting)

peter303 (12292) | more than 2 years ago | (#38005392)

If I attend some public talk on a trendy subject its swarming with recruiters. Topics include no-sql, html5, mobile, etc. There seem to be at least ten job openings for everyone looking for something.

As someone looking for a new job .... (0)

Anonymous Coward | more than 2 years ago | (#38006112)

I have no interest in doing programming of any sort, much less for html5 or mobile. SQL? Why yes I do have experience with that and over a decade in IT engineer/support/dev.....

I'm guessing I'll be looked over.

Re:As someone looking for a new job .... (1)

sonamchauhan (587356) | more than 2 years ago | (#38007758)

At its core, 'programming' is you 'telling' computers what to do.

Since you are doing 'IT support', 'SQL', etc. you are already 'programming'.

The real problem is the Babel effect of multiple, heterogenous computer 'languages'. So why limit yourself? Pick one (say Perl, or Powershell) and then depend upon CPAN or PowerShell libraries to do the heavy lifting for you.

Re:general IT market fairly hot (1)

greg1104 (461138) | more than 2 years ago | (#38006446)

You didn't just describe the "general IT market"; what you're describing is commonly called the Web 2.0 Bubble. The same people who funded the .com bubble learned nothing from that, and tech company startups have repeating the same sort of overvalued silliness that leads to a bust again during the last few years. When we have ridiculous things like Groupon being "valued" at billions of dollars, of course there's a bunch of money hiring to build more companies in that space. All of that combined is still pretty small compared with the larger IT market though.

Re:general IT market fairly hot (0)

Anonymous Coward | more than 2 years ago | (#38012512)

I'll get off your lawn, old man.

If you're a coder working as a hired gun, you shouldn't necessarily give a shit about market valuation. If Groupon is willing to throw you 6 figures to sling code, stfu, collect your check and get it while the getting is good.

It floors me to see people whine about how coders are underpaid and then simultaneously lament the overvaluation of companies. Would you honestly mind working for a couple of years for them making a lot of scratch? Stop trying to act like you are on some kind of moral high ground. You want to be moral high ground? Write software in academia to solve really hard problems that have real effects on people's lives, while making significantly less money.

Re:general IT market fairly hot (1)

greg1104 (461138) | more than 2 years ago | (#38013472)

I was warning about what I see as high business risk around the current web+mobile boom (relative to traditional IT jobs), not making a moral commentary about either type of work. And I already work on open-source software that has real effects on people's lives, you cowardly troll.

Re:general IT market fairly hot (1)

Prof.Phreak (584152) | more than 2 years ago | (#38011650)

I think it's bubble mindset. It was similar atmosphere around 1999 or so. Not sure how this time it's different... Just flashier with different key-phrases---all hoping to cash in on the massive growth that ``advertising dollars'' will be bringing in as the economy of the world unravels.

"Gurus" need not apply (2)

pwizard2 (920421) | more than 2 years ago | (#38005522)

If I were a recruiter, I would automatically be wary of anyone who seriously refers to themselves as a "guru" of $language. Sure, you may be good at writing code and may know a particular library inside out, but anyone who calls themselves a guru probably has a very overinflated sense of their importance and actual skill level. These also tend to be the people who have the right buzzwords to get past HR filters and then proceed to bullshit their way through interviews.

Re:"Gurus" need not apply (4, Funny)

ackthpt (218170) | more than 2 years ago | (#38005654)

If I were a recruiter, I would automatically be wary of anyone who seriously refers to themselves as a "guru" of $language. Sure, you may be good at writing code and may know a particular library inside out, but anyone who calls themselves a guru probably has a very overinflated sense of their importance and actual skill level. These also tend to be the people who have the right buzzwords to get past HR filters and then proceed to bullshit their way through interviews.

"It says in your resume you were part of the initial development team and wrote one of the first reference books on $language."

"That is correct, I was also part of a team which worked to ensure cross-platform consistency and stability. I've also written tutorials in $language and developed several application examples which are included in the reference website."

"Anything else you'd like to add?"

"I also have chaired the past two Worldwide $language development conferences and am teaching an Introduction to $language at the local community college."

"That all sounds very good, but what development experience do you have developing $language in $businessEnvironment?"

"None, really. I think this will likely be the first instance of its kind using $language in $businessEnvironment."

"Sorry to hear that. We're looking for someone with more experience. Thank you for your time, there's the door."

Re:"Gurus" need not apply (0)

Anonymous Coward | more than 2 years ago | (#38006100)

I know you might laugh at this but I have heard a few times from employers were they have hired the very people who created the frameworks, software or languages they use only to find they don't have the business skills to cut it (such as Systems Analysis or simply the lack of ability to communicate with stakeholders properly). You might be able to write highly efficient and effective code but if you can't even get the client's requirements downpat then it's a waste.

Re:"Gurus" need not apply (0)

Aryden (1872756) | more than 2 years ago | (#38006472)

I have a guy who can write the hell out of C# and C++ but the only way you get anything out of him is to give me the most detailed SOW you can possibly provide. You try to get him to talk to any stakeholder, process owner or direct management and he's as useless as tits on a bull.

Re:"Gurus" need not apply (2)

ackthpt (218170) | more than 2 years ago | (#38006516)

I have a guy who can write the hell out of C# and C++ but the only way you get anything out of him is to give me the most detailed SOW you can possibly provide. You try to get him to talk to any stakeholder, process owner or direct management and he's as useless as tits on a bull.

Which is why you have a Systems Analyst as the go between, or at the very least, a Project Manager. Two shops I have worked in have cordoned off the developers from the users (including and particularly external customers.) Give the coder direction and let him/her go to it.

Re:"Gurus" need not apply (1)

Aryden (1872756) | more than 2 years ago | (#38007128)

And we do, which is why I have a job.

Re:"Gurus" need not apply (2)

roman_mir (125474) | more than 2 years ago | (#38006646)

which makes perfect sense. Just because you can make a lathe doesn't mean you have the necessary experience to build, say, an airplane turbofan engine, and if the company is looking for somebody who has turbofan experience, why would they rather hire the guy who built a lathe?

Re:"Gurus" need not apply (1)

corbettw (214229) | more than 2 years ago | (#38006832)

That's an apples to orangutans comparison. A better one would be, the company needs to hire someone who has built hinges using a lathe, why would they hire someone who's only experience is in designing and building lathes and teaching others to do so?

The correct answer would be, they'd be fools not to hire that person.

Re:"Gurus" need not apply (1)

roman_mir (125474) | more than 2 years ago | (#38008122)

No, it's your comparison that makes no sense. You are assuming that designing and building a language is a similar experience to designing and building some business specific application.

I will not argue on a point that a person who is experienced and smart enough to design and build a language is likely a person who can build a business app, however this is not a person who has experience building business apps, and while he has experience building a language, this is not the experience that is required.

Constructing a hammer (digging your own iron ore, smelting it, forming it, doing whatever a black smith would have to do, then making a wooden handle, etc.), this does not give an experience of using a hammer to build a house.

Those are different things - a tool and some specific use of that tool, and building a lathe and a turbofan engine is a good enough comparison, because just as using a lathe while building an engine is required, it's not the only experience that is required.

Designing an turbofan engine and making all parts fit and having the correct approach to that design and testing and installation procedures and whatever it takes to build a turbofan engine, is not the same requirement that goes into building a lathe.

Same with a computer language and some use of a computer language.

Re:"Gurus" need not apply (1)

CadentOrange (2429626) | more than 2 years ago | (#38011198)

That's an apples to orangutans comparison. A better one would be, the company needs to hire someone who has built hinges using a lathe, why would they hire someone who's only experience is in designing and building lathes and teaching others to do so?

The correct answer would be, they'd be fools not to hire that person.

They'd be fools to hire that person.

Let's think about this for a minute. Do you see any downside to hiring somebody who is clearly overqualified for the job?

How soon before this person finds the work uninteresting, gets bored, and then starts looking for a job elsewhere? If this person is over-qualified, that implies that they can (easily?) get a job that is more intellectually stimulating and better paying elsewhere. It's in everyone's interest that this over-qualified individual doesn't get hired.

Be were of Gurus before crying wolf (1)

dbIII (701233) | more than 2 years ago | (#38008202)

I think we need a system called Loup so arrogant idiots will call themselves Loup Gurus.
As the song goes:
Oooh ooh Loupgarou gonna get ya, betta run to the river or ya gonna be dead.

Re:"Gurus" need not apply (0)

Anonymous Coward | more than 2 years ago | (#38013970)

Hi, I'm the Director of Guru. I completely disagree with your HTML5 comment about the AJAX JQuery XML call that brings back dynamic no-sql from the file system over the web.

Novices and Experts? (1)

clinko (232501) | more than 2 years ago | (#38005662)

Quoting the article: "The company (JP Morgan) has been working with Hadoop for more than three years"

Then the article quotes the experts:

"The good news is that Hadoop experts aren't born, they're trained. "I'm sure companies that train their workforces on Hadoop will derive lots of benefits," said Jeremy Lizt, VP of engineering at Rapleaf, in a recent interview. A data provider that has been using Hadoop for nearly four years, Rapleaf was among the earliest adopters."

What a difference a few months makes...

Re:Novices and Experts? (1)

mjwalshe (1680392) | more than 2 years ago | (#38006396)

woo 3 years so long LN Risk of the arms of our parent has been working with big data for over a decade - hmm must put that on my CV :-)

from TFA... (1)

Anarchduke (1551707) | more than 2 years ago | (#38005936)

JP Morgan Chase has 25,000 IT employees, and it spends about $8 billion on IT each year--$4 billion on apps and $4 billion on infrastructure. The company has been working with Hadoop for more than three years

Wow they must be super experts!!!

Reading up on Hadoop... [wikipedia.org]
Stable release 0.20.204 / September 11, 2011; 57 days ago
Preview release 0.21.0 / August 23, 2010; 14 months ago

Re:from TFA... (1)

MyLongNickName (822545) | more than 2 years ago | (#38006014)

Also from wikipedia:

On February 19, 2008, Yahoo! Inc. launched what it claimed was the world's largest Hadoop production application. The Yahoo! Search Webmap is a Hadoop application that runs on more than 10,000 core Linux cluster and produces data that is now used in every Yahoo! Web search query.[23]

go figure that out...

Re:from TFA... (1)

MyLongNickName (822545) | more than 2 years ago | (#38006032)

This might explain it a little better:

link [apache.org]

Big data in a small world? (1)

bigsexyjoe (581721) | more than 2 years ago | (#38005978)

The thing I always wonder about Hadoop is how important can it get? It's only useful if you have too much data for an RDBMS, right? It seems like only JPMorgan and other giant companies could make use of it. Am I wrong?

Re:Big data in a small world? (2)

sexconker (1179573) | more than 2 years ago | (#38006278)

The thing I always wonder about Hadoop is how important can it get? It's only useful if you have too much data for an RDBMS, right? It seems like only JPMorgan and other giant companies could make use of it. Am I wrong?

There's no such thing as too much data for an RDBMS.
There is such a thing as poor database planning and a shitty schema, though.

Re:Big data in a small world? (1)

mjwalshe (1680392) | more than 2 years ago | (#38006412)

big publishers have a lot of documents - which is what my 20% project is at RBI is based on

Re:Big data in a small world? (1)

geoffrobinson (109879) | more than 2 years ago | (#38007360)

I believe it can be used to feed data into "Big Data" databases like Netezza, Vertica, etc.

So what you're saying is... (1)

superdude72 (322167) | more than 2 years ago | (#38006128)

...all us job-seekers who are already familar with several other languages and/or frameworks should read the Wikipedia page for Hadoop, bullshit our way past the HR person, then learn Hadoop on the job.

Re:So what you're saying is... (2)

corbettw (214229) | more than 2 years ago | (#38006858)

Sounds like the way I got my first Linux-based job in '95, except I used newsgroups instead of Wikipedia.

Re:So what you're saying is... (0)

Anonymous Coward | more than 2 years ago | (#38008400)

Same here. But I used blow jobs instead of bullshit.

Jobs? (0)

identity0 (77976) | more than 2 years ago | (#38006228)

Is it a bad sign that I saw 'Job' and thought 'Not another Steve Jobs story...'? :P

At least the Jobs frenzy seems to be dying down lately.

The Future (2)

Tablizer (95088) | more than 2 years ago | (#38006452)

SQL is a query language, not a database implementation technology. In the future Hadoop-style engines will probably be wrapped by SQL such that it will be an implementation detail or choice, similar to the MyIsam versus InnoDB choice in MySql.

I'm not saying this will make it a non-career, only that the career will morph to be more like that of an Oracle tuning specialist (who make good money still).

Re:The Future (1)

paitre (32242) | more than 2 years ago | (#38007662)

There already exist tools/frameworks to work with Hadoop and HBase using SQL :)

Hadoop Is Easy: MapReduce + plumbing (1)

curmudgeon99 (1040054) | more than 2 years ago | (#38006632)

This should be a no brainer. Hadoop is merely MapReduce plus the plumbing to care and feed for it. All the various nodes and TaskTrackers, it's not that complicated at all. You can learn the basics in a night and master it in a month.

"hot" trend? (1)

superwiz (655733) | more than 2 years ago | (#38007380)

Dice search for C++ yields 17k hits. Dice search for Java yields 18k hits. Dice search for hadoop yields ~600 hits. Of the "direct company" ads (266), 18 from amazon. That's about 7% of all hadoop direct-company hits. Not a single one of them mentions an investment bank. I smell self-promotion.

Re:"hot" trend? (1)

paitre (32242) | more than 2 years ago | (#38007714)

There are positions out there.

Most of the folks that are hiring Hadoop and HBase folks are doing it on the sly.

It's how I got my current job :)

Re:"hot" trend? (1)

Prof.Phreak (584152) | more than 2 years ago | (#38011702)

Many large corps have databases like Netezza (IBM) or Greenplum (EMC). To get better deals on their contracts, they'd like leverage of having an ``alternative''... Hadoop is often seen as that alternative (similar architecture, different mind-set) that can potentially be shoehorned into doing similar things that Netezza or Greenplum does---and not cost $bazillion dollars.

they're all tools (1)

misfit815 (875442) | more than 2 years ago | (#38007814)

After 20 years in the industry, in various forms, I've come to this realization: C++, Java, Hadoop, Ruby on Rails, PHP... all these things are the airgun and socket wrench and grinder and welder and all the other tools in the garage. What matters is if you have experience working on BMW's or Kenworths or IndyCars or Harley-Davidsons. In other words, have you written accounting systems, industrial control systems, customer-facing websites, etc. I don't want to work for someone who's going to hire me because I'm a C# guru. I want to work for someone who recognizes that my background in financial systems fits their need on a loan processing project. Ok, not really, because that would bore me to tears, but you get my point.

Re:they're all tools (0)

Anonymous Coward | more than 2 years ago | (#38010244)

Very true. All languages are tools. Admittedly, I have my favourites to work with, but all languages are there to get things done, not for flag waving.

Re:they're all tools (1)

zachie (2491880) | more than 2 years ago | (#38011256)

Absolutely. Too many people are overly worried about their expertise on some programming language, database, operating system, you name it. But who cares! These are just the tools you will use to do your thing. It is much more valuable to think in these terms: 'I know these languages, and I am particularly good with this one. That said, what actual programming language I end up coding in is just an implementation detail.' What you want to be valued for is your ability to solve such and such problems. Of course you are familiar with the tools you will require for that.

Back to 1975? (0)

Anonymous Coward | more than 2 years ago | (#38008172)

"There are limits to what Hadoop can do, he said. When applications are transactional, when they demand low latency or rapid response times, or when there's lots of query complexity or concurrent workloads, JPMorgan Chase's IT organization still recommends using conventional relational databases"

HHAHHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAH!

Handjob Guru? (0)

Anonymous Coward | more than 2 years ago | (#38008332)

I don't even want to know. I mean I do... but i don't.

Why Hadoop? (0)

Anonymous Coward | more than 2 years ago | (#38008616)

We have PLENTY of good distributed cluster file systems already (PVFS2, GFS etc) that have better fail over, redundancy. It's just a matter of providing a MapReduce framework and an NoSQL layer to utilize, why should we need a specific file system to do such kind of work?

The hype right now is because nobody else has implemented mapreduce or nosql on the other file systems?

Re:Why Hadoop? (0)

Anonymous Coward | more than 2 years ago | (#38009468)

I suspect there's one company that does mapreduce on some pretty big GFS clusters. Three guesses.
   

Who finds it difficult? (1)

Evets (629327) | more than 2 years ago | (#38009734)

Does anybody actually have a hard time learning Hadoop? In my experience its pretty easy to pick up and go with.

Can't have a career chasing hot skills (0)

Anonymous Coward | more than 2 years ago | (#38010782)

The problem with the computer industry is that there's always something new that there's a shortage in. You can't have a career when the next hot thing comes along on a yearly schedule. All the people learning the last hot thing (wasn't it mobile apps?) are left in the dust when the next hot thing (mapreduce) comes along. By the time anyone learns enough about Hadoop to be productive, the next hot thing will come along. The insanity has to stop. As a business, you can't have a perpetual shortage of the next hot thing and get anyhing done. As a professional, you can't constantly chase the next hot thing.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?