Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Microsoft Goes In For Hadoop

timothy posted more than 2 years ago | from the well-it-is-a-nice-name dept.

Software 67

Frankie70 writes that after more than three years, Microsoft has "finally learned to stop worrying and love Hadoop." Frankie70 excerpts from the linked Wired article: "Any aversion to Hadoop disappeared on Wednesday, when the company announced that it will integrate the platform with future versions of its relational database, SQL Server, and its platform cloud, Windows Azure, an online service for hosting and readily scaling applications. The company is now working to port the Hadoop platform to Windows."

cancel ×

67 comments

Sorry! There are no comments related to the filter you selected.

Hadoop is written in Java (5, Funny)

binarylarry (1338699) | more than 2 years ago | (#37701890)

So what they mean is, they're going to do a search and replace to make it compile as a C# application.

Re:Hadoop is written in Java (0)

Anonymous Coward | more than 2 years ago | (#37701916)

Not when there is IL that makes that port even easier.

Re:Hadoop is written in Java (-1)

Anonymous Coward | more than 2 years ago | (#37702294)

Hadoop is a weird name. It's like what niggers would call basketball if it didn't already have a name.

And how do you make a nigger self-destruct? Point out basketball was invented by a white man. You might want to stand back.

Re:Hadoop is written in Java (1)

allenw (33234) | more than 2 years ago | (#37705910)

Actually, there is an ever increasing amount of JNI (read: C) code in Hadoop that is in the critical path for security and performance features. Most of that code is not very portable. So either MS is going to pay for some major overhauling of that code, completely new code/branch to replicate that functionality or MS Hadoop is going to be severely lacking in features/performance.

Re:Hadoop is written in Java (1)

shutdown -p now (807394) | more than 2 years ago | (#37707092)

If that C code is well-written, it shouldn't be hard to port it over - the porting would have to be done at the Java/C boundary, and .NET actually has it much simpler thanks to P/Invoke.

Re:Hadoop is written in Java (1)

allenw (33234) | more than 2 years ago | (#37707784)

It isn't. There is an incredible overuse of glibc/Linux-isms to the point that even porting it to another UNIX is difficult.

Re:Hadoop is written in Java (1)

shutdown -p now (807394) | more than 2 years ago | (#37708128)

Well then, perhaps those guys will clean it up architecturally while they're porting it, and submit the changes upstream.

Re:Hadoop is written in Java (1)

Ed Avis (5917) | more than 2 years ago | (#37711726)

Since it's written in Java, they can just run it on the .NET virtual machine using IKVM [ikvm.net] .

We know, we know... (0)

Anonymous Coward | more than 2 years ago | (#37701894)

Embrace, extend, extinguish. Now, someone please say something insightful.

Re:We know, we know... (1)

Tsingi (870990) | more than 2 years ago | (#37701956)

How about Spiteful?
Does taking the source from an Apache licensed product and converting it from java to .net mean you no longer have to observe the license?

I'd say no, I wonder what M$ will say.

Re:We know, we know... (1)

MozeeToby (1163751) | more than 2 years ago | (#37701978)

I have to wonder where the first line of the article came from. If it's from the author, nothing to see here, but if it's a near direct quote from the source it sounds more like a shoutout to Dr. Strangelove, which could indicate that they are still terrified of it but are embracing it because they don't feel they have any choice in the matter.

Not wanting to put a dampener on things... (1)

bernywork (57298) | more than 2 years ago | (#37701972)

I think MS getting involved with Open Source is great, but....

We've seen the way that they work before, embrace and extend... This hasn't worked out that well for them before, but you have to ask if there is an alterior motive in there...

Re:Not wanting to put a dampener on things... (0)

Anonymous Coward | more than 2 years ago | (#37702220)

There is no question that there is an ulterior motive here by Microsoft. Everything every person ever does is always done with such. The real question is will Microsoft's motives be in direct conflict with what is the overall good for the rest of the users/developers of Hadoop. With MS, that is how it almost always plays out so it would be pretty naive to assume otherwise in this case. My suggestion, short Hadoop when MS gets it in their clutches.

Re:Not wanting to put a dampener on things... (1)

Attila Dimedici (1036002) | more than 2 years ago | (#37702226)

I am not sure what an alterior motive is, but I am quite sure that MS has an ulterior motive for this. The only question is whether or not that ulterior motive is detrimental to the Open Source community.

Re:Not wanting to put a dampener on things... (1)

Forbman (794277) | more than 2 years ago | (#37702798)

Sell more Windows Server & SQL Server Enterprise/Data Center licenses?

Re:Not wanting to put a dampener on things... (0)

cayenne8 (626475) | more than 2 years ago | (#37704150)

Sell more Windows Server & SQL Server Enterprise/Data Center licenses?

What major data center (working with large volumes of critical data) in its right mind would ever even consider using MS SQL Server as its database? Who'd consider running their critical server for any database on a windows box?!?!

Not in any major production player I've ever seen or worked at....

Re:Not wanting to put a dampener on things... (1)

Anonymous Coward | more than 2 years ago | (#37704854)

NASQAQ uses sql server http://www.computerworld.com/s/article/106050/Microsoft_unwraps_flagship_database_SQL_Server_2005

DirectEdge - 4th largest stock exchange uses sql server
http://blogs.technet.com/b/dataplatforminsider/archive/2011/06/03/fourth-largest-us-stock-exchange-direct-edge-looks-to-sql-server-parallel-data-warehouse-for-big-data-needs.aspx

Re:Not wanting to put a dampener on things... (2)

jbolden (176878) | more than 2 years ago | (#37702808)

I think they are telling the truth about their goals:

1) Get Hadoop to work on Windows servers
2) Create a Windows server management interface for Hadoop
3) Create SQL Server extension to manage Hadoop.

And the motive is:
a) Sell server licenses
b) Sell SQL Server licenses

Re:Not wanting to put a dampener on things... (1)

jimicus (737525) | more than 2 years ago | (#37703384)

You would be amazed how many people go nuts over the latest F/OSS platform du jour... and then complain that it runs first and foremost under Linux.

Even if they're never going to go anywhere near the underlying OS anyway, still that gets brought up.

Windows Server licensing is quite lucrative for Microsoft. So if they can now announce "Hadoop: Now certified for Windows (TM) Server" they can sell more licenses for Windows Server.

Re:Not wanting to put a dampener on things... (1)

aztracker1 (702135) | more than 2 years ago | (#37703666)

I think it's to integrate a map/reduce structure into SQL server... I haven't RTFA, but that is about it... I wouldn't necessarily expect them to use Hadoop directly, but to support Hadoop's interfaces. My $0.02 on this. I know a lot of people are using MongoDB, and other document centric datastores lately, and MS is moving to compete in their tool space. More power to them, doesn't mean it'll be my first choice.

Re:Not wanting to put a dampener on things... (1)

Thing 1 (178996) | more than 2 years ago | (#37724158)

I am not sure what an alterior motive is, but I am quite sure that MS has an ulterior motive for this.

My ex-girlfriend was an ulterior decorator.

Re:Not wanting to put a dampener on things... (1)

bernywork (57298) | more than 2 years ago | (#37702324)

> alterior

bad spelling, ulterior. Sorry, my bad.

Re:Not wanting to put a dampener on things... (1)

jbolden (176878) | more than 2 years ago | (#37702792)

They are quite publicly indicating their intention is to embrace and extend:

1) Get Hadoop to work on Windows servers
2) Create a Windows server management interface for Hadoop
3) Create SQL Server extension to manage Hadoop.

So we don't have to speculate, that's what they say they are doing. That being the case all that stuff might be useful for Hadoop.

Re:Not wanting to put a dampener on things... (1)

mandelbr0t (1015855) | more than 2 years ago | (#37703168)

This is a smart move by MS. Microsoft is not working internally on any sort of NoSQL server, so they support an existing project that complements their own product. The very obvious goal is to integrate Hadoop with SQL Server management tools. The upshot is that Hadoop gets a leg up on their competitors (Yahoo! PNUTS, Google BigTable) and Microsoft sells more SQL server licenses. Seems to me to be a win/win.

Denver Open Source Group meets at MicroSoft (1)

peter303 (12292) | more than 2 years ago | (#37703584)

In the beginning I thought it was ironic. But its very generous of them to provide a free meeting room to our open-source computer study group. And MSFT people attend, but dont speak often.

Re:Denver Open Source Group meets at MicroSoft (0)

Anonymous Coward | more than 2 years ago | (#37706644)

This is the same thing that happened to our mobile group here the east, where MSFT sponsored the events, but when time and time again, no one is actually developing on their platform, MSFT drop the sponsorship.

MSFT is here, because customer ask them to, not because they want it, it's all for short term results. MSFT is in no way committed to open-source, it's still a lip-service to me.

Re:Not wanting to put a dampener on things... (1)

allenw (33234) | more than 2 years ago | (#37706018)

This isn't about Microsfot getting involved with open source. This is about Microsoft not getting left out. Beyond the countless startups, Apache Hadoop already has major players like Amazon, Dell, EMC, HP, IBM, NetApp, Oracle, VMware, ... trying to make a dent in the community in some form or another. Hell, I have a SuperMicro catalog on my desk emblazoned with the Apache Hadoop logo all over it. Like Oracle, they are coming in very late to the party and now need to play catch-up. Buying off Hortonworks is a very fast way to do that.

First... (1)

knuthin (2255242) | more than 2 years ago | (#37702034)

First they ignore you.
Then they laugh at you.
Then they port you on their platform.
Then you win.

Original: http://bit.ly/o3V3cA [bit.ly] [Google Books]

Re:First... (0)

Anonymous Coward | more than 2 years ago | (#37702420)

Yeah, you win by... helping them!
Go, FOSS, go!

... Last (0)

Anonymous Coward | more than 2 years ago | (#37702476)

Then they embrace you.
Then they extend you.
Last they extinguish you.

Re:First... (1)

camperdave (969942) | more than 2 years ago | (#37702512)

First they ignore you.
Then they laugh at you.
Then they port you on their platform.
Then you win.

First they ignore you.
Then they laugh at you.
Then they port you on their platform.
Then they add some convenient feature that they only make available on their platform
Then they win

FTFY

Re:First... (1)

knuthin (2255242) | more than 2 years ago | (#37714406)

That was one another depressing possibility.

Think I will go into a corner and cry for a minute now. :/

Re:First... (0)

Anonymous Coward | more than 2 years ago | (#37703970)

First they ignore you.
Then they laugh at you.
Then they fight you.
Then you win.

  -- Gandhi

First they march you through hundreds of miles of jungle without food or water.
Then they shoot you.
Then they disembowel you.
Then you lose.

  -- Gandhi, had the Japs won WW2

Heh (4, Funny)

Hatta (162192) | more than 2 years ago | (#37702156)

Someone should trick Timothy into reposting this article. Then he'd be duped into posting a dupe about hadoop.

Re:Heh (-1)

Anonymous Coward | more than 2 years ago | (#37702194)

Someone should trick Timothy into reposting this article. Then he'd be duped into posting a dupe about hadoop.

Re:Heh (1)

darthdavid (835069) | more than 2 years ago | (#37702272)

Someone should trick Timothy into reposting this article. Then he'd be duped into posting a dupe about hadoop.

Re:Heh (1)

Anonymous Coward | more than 2 years ago | (#37702474)

Congrats, you've demonstrated enough brain-power for kindergarten. If a kid stacks a blue lego block on top of another, you are able to imitate and stack a 3rd blue lego block.

Re:Heh (0)

Anonymous Coward | more than 2 years ago | (#37702450)

You must be new here.

Re:Heh (0)

Anonymous Coward | more than 2 years ago | (#37702662)

I lost my book about object oriented programming for hadoop.

Got hadoopoop?

No, I hadhadoopoop.

Re:Heh (1)

Gilmoure (18428) | more than 2 years ago | (#37703540)

*golf clap*

Re:Heh (1)

Anomalyst (742352) | more than 2 years ago | (#37703954)

*golf clap*

Something you get from being in the showers with Tiger Woods?

Re:Heh (1)

robot_love (1089921) | more than 2 years ago | (#37703818)

And then I'd file a false complaint against it being a copyright violation in France.

It would be a Hadopi dupe dupe Hadoop.

But I digress...

Embrace. Extend. Extinguish. (3, Interesting)

mrflash818 (226638) | more than 2 years ago | (#37702236)

"Those that do not learn from history are doomed to repeat it."

"Embrace, extend and extinguish,"[1] also known as "Embrace, extend and exterminate,"[2] is a phrase that the U.S. Department of Justice found[3] was used internally by Microsoft[4] to describe its strategy for entering product categories involving widely used standards, extending those standards with proprietary capabilities, and then using those differences to disadvantage its competitors.

http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish [wikipedia.org]

Re:Embrace. Extend. Extinguish. (1)

mrflash818 (226638) | more than 2 years ago | (#37702332)

Microsoft allowed the pair to continue their contributions to the open source project, and Powerset, which was rolled into Redmond’s Bing search engine, continued to run atop Hadoop.

This made Bing one of the first “shipping” Microsoft products to actually include open source code. But somewhere along the way, Microsoft moved the engine onto a proprietary platform...

"Microsoft allowed the pair" -- Here

"which was rolled into Redmond's Bing search engine" -- It

"But somewhere along the way, Microsoft moved the engine _onto_a_proprietary_platform_" -- Comes *sigh*

Re:Embrace. Extend. Extinguish. (-1)

Anonymous Coward | more than 2 years ago | (#37705342)

The only way they can make it proprietary is if the original GPL authors *chooses* to re-license it. He is free to leave with his code whenever he wants. But ofcource.. he wanted the microsoft money.. like slashdot, which loves to run MS ads.

Its obvious to people with a functioning brain that microsoft made it proprietary because the GPL is a trojan horse which can force you to make your entire product open source if you are not careful. That is if its GPL V2. V3 is down right evil and anti-business.

Its similar to how Google has opensourced their search engine, gmail, google maps etc. oops !

But hey.. why don't you anti-ms trolls come up with something new? You are just plain boring.

as usual, the summary here is lacking (4, Insightful)

tomzyk (158497) | more than 2 years ago | (#37702248)

its relational database, SQL Server, and its platform cloud, Windows Azure, an online service for hosting and readily scaling applications

That's wonderful that the summary mentions what "SQL Server" and "Azure" are... but why no mention of wtf "Hadoop" is?
Why do I need to RTFA just to find out what we're talking about here?

Hadoop — an open source platform for crunching epic amounts of a data across an army of dirt-cheap servers

Re:as usual, the summary here is lacking (1)

slim (1652) | more than 2 years ago | (#37702380)

The Wired article tells you what Hadoop is, because it's written by journalists, and aimed at a broad readership.

The /. summary tells you what SQL Server and Azure are, because the descriptions arbitrarily happen to occur in the Wired paragraph that's been quoted.

The /. summary does not tell you what Hadoop is, because (unlike Wired readers), /. readers are expected to have some basic knowledge of the software world. If you think Hadoop is obscure, maybe this isn't the site for you?

Re:as usual, the summary here is lacking (1)

Anonymous Coward | more than 2 years ago | (#37702764)

I'm with the OP. Quoting an article is fine, even with descriptions one should know... but if an article is primarily about something that isn't common, then a nice little blurb about it would save AT LEAST TWO PEOPLE a wiki call. Hadoop isn't as ubiquitous as you believe. Coming from a heavy consulting background in coding, I've never seen this.

Oddly, by your logic, /. readers don't know what SQL Server and Azure are. I'll put dollars to doughnuts that more people USE SQL Server than KNOW what Hadoop is. Looking at the WIKI, I'm also willing to bet that you aren't pronouncing Hadoop correctly (not unlike the original GIF pronunciation).

Re:as usual, the summary here is lacking (1)

slim (1652) | more than 2 years ago | (#37713436)

Oddly, by your logic, /. readers don't know what SQL Server and Azure are.

Nope. As I said, the definitions for those arbitrarily happen to be in the quote chosen for the summary. But those definitions aren't the pertinent part of the quote.

Re:as usual, the summary here is lacking (1)

CannonballHead (842625) | more than 2 years ago | (#37704780)

The /. summary tells you what SQL Server and Azure are, because the descriptions arbitrarily happen to occur in the Wired paragraph that's been quoted.

So what you're saying is .... the person who wrote the summary doesn't know how to write a good summary or pick a good quote. :) (no, I'm not new here...)

Re:as usual, the summary here is lacking (0)

Anonymous Coward | more than 2 years ago | (#37702394)

Think of a search query where if you didnt get all the data back it is not that big of a deal. The term is eventual consistency.

For something like facebook page updates, web searches, data averaging this makes some sense. For something where "i must get the the data or all my results are wrong" type of things (think my receipt out of a cash register) not so good.

They are removing the C out of ACID to achieve speed.

Re:as usual, the summary here is lacking (1)

camperdave (969942) | more than 2 years ago | (#37702616)

Me too! I had to double click on Hadoop, then right click and choose 'Search Google for "Hadoop"' from the context menu. Then I had to switch tabs and scan for the wikipedia site, click on that, and then skim through the wiki for a few seconds just to figure out whether it was worth it to read anything beyond the summary.

Sadly, this is happening more and more with Slashdot.

Dont look behind that curtian! (1)

Lumpy (12016) | more than 2 years ago | (#37702322)

"the company plans to eventually release its work back to the open source community."

That is a bit too vague... because 100 years from now is "eventually"

Re:Dont look behind that curtian! (2)

jbolden (176878) | more than 2 years ago | (#37702856)

If you look at their todo list they could release it constantly it won't matter. What they are doing is essentially creating extensions for their commercial products that work with Hadoop. I think they have every intention of trying to get the small parts that need to be in Hadoop back into the main tree.

Amazing! (0)

Anonymous Coward | more than 2 years ago | (#37702568)

That's fantastic! ...I'd be even more excited if there had been a 1 liner about what in the allspark Hadoop is...

Really surprise? (0)

Anonymous Coward | more than 2 years ago | (#37702572)

Half a year ago Microsoft was headhunting Hadoop developers from all around the world, and this fact was pretty well known in the Hadoop community. Why such a surprise now?

One issue that bothers me is the license... (0)

Anonymous Coward | more than 2 years ago | (#37702602)

I don't trust big for-profit software companies. If they are involved, it's not good for the OSS/free/libre community other than to say "Look! Guess who is using our software!". Big for-profit companies generally do what is good for them and their bottom line. For once, I wish OSS/free/libre software authors would really look at what's behind a license before they select a license. I don't want certain entities to profit from open source when their actual goals go against the OSS/free/libre community. GPL3 is the way to go. Flame on should you wish, but I've been in this industry far too long to not care about the long-term implications of getting in bed with companies who say one thing and do another. Embrace and extend are the watchwords here. Just goes to show that in the end, OSS is always better and they use it because they can do no better. Let them use it, but keep them under the watchful eye of the GPL.

Dryad (1)

Utopia (149375) | more than 2 years ago | (#37703006)

Microsoft has a equivalent to Hadoop known as Dryad.
They should have open-sourced Dryad a long time ago.

I wonder what is going to happen to Dryad with this focus on Hadoop.

Re:Dryad (3, Informative)

mandelbr0t (1015855) | more than 2 years ago | (#37703500)

Dryad is not quite Hadoop. From their whitepaper:

We can map the whole relational algebra on top of Dryad, however Dryad is not a database engine: it does not include a query planner or optimizer; the system has no concept of data schemas or indices; and Dryad does not support transactions or logs

I can see how Hadoop would supplement their own research in this field.

YOUW FAIL IT (-1)

Anonymous Coward | more than 2 years ago | (#37703708)

troubled OS. Now reciprocating bad hype - BSD's NetBSD posts on of OpenBSD versus something done very distracting to Survive at all me if you'd like, wasn't on Steve's I know it sux0rs, All our times have ultimately, we BSD style.' In the Dying. All major there are only Www.anti-slash.org from the FreeBSD may also want Be fun. It used rotting corpse and other party us the Courtesy cycle; take a watershed eesay, what they think is BOUGHT THE FARM.... for the state of COME HERE BUT NOW Obtain a copy of

test from my kindle (0)

Anonymous Coward | more than 2 years ago | (#37704128)

tsting.....

So... (0)

Anonymous Coward | more than 2 years ago | (#37704872)

Does this mean MS will somehow kill Hadoop? Cause that'd be great.

oh gimme a f______g break (-1)

Anonymous Coward | more than 2 years ago | (#37706084)

They have NO business at all in the hadoop space - Azure is a joke and Hadoop doesn't even run on windows!!! where do they get off - stick to your damn mail and office apps that's the only thing you do well - a-holes!!!!!

Re:oh gimme a f______g break (0)

Anonymous Coward | more than 2 years ago | (#37707276)

Did you think that you could hide from us by going 100% FOSS, freetard? Muhahahahaaha... we are here to SPREAD EVIL and MAKE YOU SUFFER. If we have to write FOSS for that, we will do so. Resistance is futile. You will be mentally eviscerated.

Microsoft Goes Hadooken (1)

Spy Handler (822350) | more than 2 years ago | (#37706566)

that's what i thought it said...... MS going all street fighter

EMC Greenplum HD? (0)

Anonymous Coward | more than 2 years ago | (#37709846)

Admittedly it's Linux based, but the full price Greenplum HD runs on Hadoop ported to C. So it not only can be done, it can be done well.

The cool parts (1)

davevr (29843) | more than 2 years ago | (#37709974)

The two cool parts of this announcement:

1) They are contributing the bits needed to make it work on windows back to open source (Hortonworks is helping that make sure that goes smoothly)
2) They are making JavaScript a first-tier language for writing map/reduce jobs, and contributing THAT work back to the community.

That is awesome.
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>