×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

217 comments

GFS? (0, Redundant)

Anonymous Coward | more than 4 years ago | (#29046929)

Google File System.

Re:GFS? (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#29047487)

Google File System.

... that no one gives a FUCK about. In other news, two years into running the heat pump I have overhauled its air filters. Douchebags.

hmm (5, Funny)

gnarfel (1135055) | more than 4 years ago | (#29046933)

Well I'm no expert on Google's internal workings, but are any of these protocols or file systems they've developed been released outside of Google for public use?

Re:hmm (5, Funny)

buchner.johannes (1139593) | more than 4 years ago | (#29046991)

GFS is proprietary and for internal use only. The only released a paper describing how it works (don't know if that content is enough to rebuild it). I think GFS (global file system) from Redhat [redhat.com] and OpenGFS [sourceforge.net] is something differently. Hadoop is what you want. What would we do without the wiki [wikipedia.org]

Re:hmm (5, Funny)

ToadProphet (1148333) | more than 4 years ago | (#29047067)

Parent and GP modded funny? Am I missing the joke or are there some giddy drunks with mod points?

Re:hmm (4, Funny)

buchner.johannes (1139593) | more than 4 years ago | (#29047125)

Everyone is in a good mood. Why not :-)

Re:hmm (5, Funny)

ToadProphet (1148333) | more than 4 years ago | (#29047367)

Everyone is in a good mood. Why not :-)

Modded Troll... now that's delicious.

Re:hmm (5, Funny)

skine (1524819) | more than 4 years ago | (#29047687)

Woooooooooooooo!

(Apparently just entering "Woooooooooooooo!" creates an error. I have to explain that it's supposed to be a giddy mod, thus destroying any semblance of assuming intelligence present in at least part of the /. community).

Re:hmm (5, Funny)

Snarf You (1285360) | more than 4 years ago | (#29047327)

While I found both posts informative, I find it funny that they were modded funny. It's meta-funny. You know what else is funny, is that the word funny starts to sound funny after saying it enough times.

Re:hmm (5, Funny)

Afforess (1310263) | more than 4 years ago | (#29047435)

If you repeat a word too many times, quickly, your brain become tired of that word and it begins to become foreign to it. This event is similar in nature to looking at grid illusions [wikipedia.org] . Your brain becomes tired after a few moments and you see dots.

Where's meta-moderation when you need it? (5, Funny)

nobodyman (90587) | more than 4 years ago | (#29047817)

I'm impressed that all of these Reddit users had the attention span to stay long enough to get mod points. But nobody likes a guest who overstays their welcome. Besides, I think somebody posted an animated gif of an old man falling down or something. GO CHECK IT OUT!!!1!1one

Re:hmm (1)

marafa (745042) | more than 4 years ago | (#29048227)

funny funny funny -
  Your comment violated the "postercomment" compression filter. Try less whitespace and/or less repetition.

hey check it out .. you dont have to worry about that coz slashdot wont let you repeat it too many times!

Re:hmm (5, Funny)

Anonymous Coward | more than 4 years ago | (#29048273)

Many people think that Google's original claim to fame is PageRank. That's only partially true. Google became as successful as they are because of their systems-scalability work. That is, Google figured out how to build the biggest clusters, with the most storage space, the most computation capacity, and the lowest latency, for the least amount of money (compared to their competitors anyway). If you have 1000x times the computing power of your nearest competitor, then you can do 1000x as much data mining, which means that your search results (and ad relevancy) will be that much better.

For a long time, Google refused to release any information on their system infrastructure (it was their crown jewel, after all). The GFS paper was released in 2003, well after Google had put the filesystem (and its predecessors) to public use.

To sum it up: GFS has been one of the strongest contributing factors to Google's dominance. The idea that Google would voluntarily give this code to competitors is laughable.

Re:hmm (5, Funny)

Brian Gordon (987471) | more than 4 years ago | (#29047059)

No, they haven't. So why does the editor think we care? "Google Six Months Into Resurfacing Parking Lot"

Re:hmm (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#29047123)

I'm 6 inches into kathleen fent's smelly cunt.

Re:hmm (-1, Troll)

Anonymous Coward | more than 4 years ago | (#29047437)

You should grow some, then. Hell, I've seen midgets with bigger tools. I've seen CHILDREN with bigger tools!

Re:hmm (-1, Troll)

Anonymous Coward | more than 4 years ago | (#29047933)

Why are you looking at the willies of children?

Re:hmm (0)

Anonymous Coward | more than 4 years ago | (#29047515)

You cared enough to post in the thread...

Re:hmm (5, Funny)

mysidia (191772) | more than 4 years ago | (#29047061)

They have not, and apparently Google thinks of the Google FS as part of their secret sauce, such that they will probably never get it released. Although they seem happy to write papers about it.

It's actually really sad... Google has built an innovative platform for distributed computing, that solves quite a few problems, vastly superior to the state of the art in distributed computing, but they basically keep the filesystem and clustering implementations completely to themselves, it would seem.

They use the Linux platform to the absolute max, leveraging all the blood and sweat Linux developers poured into its development over the past 15 years, and yet, not contributing back any of their most significant enhancements.

I won't call it evil, as they're under no obligation to release GoogleFS or their map reduce implementations, it's just unkind.

I would equate it to an inventor creating the lightbulb, and their employer saw this, and decided instead of trying to sell the invention to the public, they decided to only allow their own factories to buy lightbulbs, thus netting them a competitive advantage over other factories whose workers had to operate in the dark or by candlelight.

No software product available to the public that even utilizes GoogleFS. Instead it's all software as a service (The Google search engine service, that is)

Re:hmm (5, Interesting)

MeatBag PussRocket (1475317) | more than 4 years ago | (#29047153)

They use the Linux platform to the absolute max, leveraging all the blood and sweat Linux developers poured into its development over the past 15 years, and yet, not contributing back any of their most significant enhancements.

i see your point, but its not like google isnt giving signifigantly in return. most people would be hard pressed to deny that Googles search engine was a game changer in the interweb. at its release it was leaps and bounds better tahn just about anything out there, and is still the gold standard for finding information. hell they gave us the verb "to google" we got a pretty decent browser out of it, gmail, google docs, google maps, and a whole bunch of other stuff they've generated. not to mention a forthcoming OS. at this point i can already hear critics screaming about Googles profits driving these services, and you know what, maybe they are, but i havent paid Google a dime, and most likely, neither have you. i dont care if they make money, theres nothing wrong with it, and i'm even happier that they make money without involving me whatsoever. in many ways i would think Google would be a champion to the FOSS community. so they want to keep a filesystem proprietary, frankly thats not so bad, competition is good but competitors arent usually. Google is a good counter balance to Microsoft and other would-be owners of the interwebs. are they "good" as in saintly? no, but they never claimed to be, they claimed "dont be evil" i'd say they're pretty far from that.

Re:hmm (5, Insightful)

lawpoop (604919) | more than 4 years ago | (#29047237)

They use the Linux platform to the absolute max, leveraging all the blood and sweat Linux developers poured into its development over the past 15 years, and yet, not contributing back any of their most significant enhancements.

Not contributing back!? Dude, they gave us *google*. Remember what it was like before google? When internet search was basically voo-doo crapshoots, that worked 25% of the time? They gave us a search engine that actually *worked*. Before that, you basically had to bookmark or memorize internet sites that you liked. Good luck actually finding what you were looking for without having an actual site in mind beforehand.

I think that alone has probably spurred the development of free software. Imagine being able to *find things* on the internet!

Re:hmm (5, Funny)

Night Goat (18437) | more than 4 years ago | (#29047315)

Yahoo worked fine for me before Google. I think you give it more credit than it deserves. The downside of Yahoo was its advertising and clutter. The searching part worked fine.

Re:hmm (5, Funny)

CharlyFoxtrot (1607527) | more than 4 years ago | (#29047355)

Altavista worked fine, HotBot too. I started using Google primarily because of the cached pages, not because the search was that much better. Plus like you say the Google interface was a breath of fresh air.

Re:hmm (5, Funny)

negRo_slim (636783) | more than 4 years ago | (#29047407)

Plus like you say the Google interface was a breath of fresh air.

Sometimes I wonder if Yahoo hadn't made their default page http://search.yahoo.com/ [yahoo.com] early on, if they wouldn't have done somewhat better for themselves.

Re:hmm (0)

Anonymous Coward | more than 4 years ago | (#29047753)

who remembers webcrawler?

Re:hmm (5, Informative)

mysidia (191772) | more than 4 years ago | (#29047377)

Yahoo was originally a web directory, not a conventional search engine. The search results were provided by others.

In 2000, they signed an agreement with Google, and Yahoo's search was powered by Google, in other words -- if you used Yahoo, you were using Google.

That didn't change until 2005, and after several other search engine company acquisitions, when they developed their own search technology.

Re:hmm (1)

negRo_slim (636783) | more than 4 years ago | (#29047479)

In 2000, they signed an agreement with Google, and Yahoo's search was powered by Google, in other words -- if you used Yahoo, you were using Google.

Let us not forget Inktomi [cnet.com] , I believe they used a few other providers during those years as well.

Re:hmm (5, Informative)

jcnnghm (538570) | more than 4 years ago | (#29047449)

You know from June 2000 to February 2004 Google was the backend for the Yahoo web page search. That was back when Yahoo was a web site "human directory" search first and foremost, and only secondarily a machine-powered internet search. Sort of like how Yahoo search is going to be powered by Bing in the future, and was powered by Inktomi before Google.

Re:hmm (5, Funny)

billcopc (196330) | more than 4 years ago | (#29047347)

You clearly weren't an Altavista user.

Google's results today are no better than the leading search engines 10 years ago. People were gaming the engines then, and Google came up with a smarter algorithm (Pagerank), but today's results page is again full of garbage because people learned how to game Pagerank. Combine that with the web 2.0 fad of scraping and regurgitating everyone else's content, and the resultant pile of URLs for any given keyword is utterly worthless. I call it "metapublishing", because the content is worthless, it's become a twisted game of outwitting Google to maximize ad revenue while providing zero value.

Searching has always been a game of finding the most specific yet least popular terms to define what you want, and then adding a bunch of negative keywords to filter out the junk. Google scored a hit, many many years ago, but they haven't been able (or willing) to maintain that lead, and all their competitors have pretty much died out anyway.

If Google hadn't come along when it did, someone else would have stepped up. Maybe Altavista, or Yahoo, or someone else. There was a need, and a provider to address that need. The only reason we don't have a new search engine to beat Google today is because, well, everyone is scared shitless of going head-to-head with Google, except Microsoft with their propaganda-laced Bing embarrassment. They're just not the golden child people seem to think they are.

Re:hmm (4, Insightful)

lawpoop (604919) | more than 4 years ago | (#29047483)

If Google hadn't come along when it did, someone else would have stepped up.

Doesn't change the fact that it *was* them, who was able to do it when nobody else had been able to. So I think that yes, they did contribute a lot to open source development. It's not enough to have a good idea, or believe that someone will eventually get around to it; someone actually has to sit down and *do* it. If google hadn't done it then, we would be that much further behind in internet search technology.

Re:hmm (5, Informative)

Mostly a lurker (634878) | more than 4 years ago | (#29047647)

Your recollections are different from mine. Prior to Google, I tended to use AltaVista and Hotbot. Searches took at least ten times as long. Results rarely included any recently created pages. The number of indexed pages was several orders of magnitude less than Google handles today (which in turn is one order of magnitude, or so, greater than current competitors). In spite of the fact that gaming of search engines is overwhelmingly targeted at Google, Google still does a relatively better job of finding the genuinely useful pages. Is Google perfect? No, of course not. Search is still only a partially solved problem. However, since its inception, Google has come up with most of the practical advances in the state of the art, as well as the best infrastructure for its implementation.

Re:hmm (4, Interesting)

Dahamma (304068) | more than 4 years ago | (#29047779)

You clearly weren't a daily Google user 10 years ago.

The moment I realized Google was completely superior to the others was when I was able to paste an obscure compile error for an equally obscure CPU architecture into Google and immediately get the answer back... the kind of utterly random error that a few years previous would have potentially taken hours to debug...

If Google hadn't come along when it did, someone else would have stepped up. Maybe Altavista, or Yahoo

And you were modded Insightful - sigh... So you are saying they decided "oh, well Google is pretty good at this - let's NOT STEP UP." Yeah, that's what companies do in that situation. Or maybe they do try, and fail (nothing wrong with trying and failing... but that's the REALITY of the situation).

Re:hmm (4, Insightful)

wdr1 (31310) | more than 4 years ago | (#29047855)

Put the crackpipe down!

I was an altavista user. A die-hard one, for most of the mid/late-nineties. In fact, I remember the day I finally convinced my boss to switch from Altavista to Google, because he had worked on Altavista.

Today's results completely blow away the search engines of 10 years ago. In fact, any of the major players -- Yahoo, Microsoft, even Ask & co. -- would blow away the search engines of 10 years ago.

(Add to the fact that the number of documents on the web that they need to crawl & rank have exploded.)

Your comment that "the resultant pile of URLs for any given keyword is utterly worthless" is itself hyperbolic nonsense. If that were true, nobody would use them.

Re:hmm (4, Insightful)

unity (1740) | more than 4 years ago | (#29048071)

The only thing you really missed there was the really simple, non-image intensive interface. That alone spurred people to use google.

Re:hmm (4, Insightful)

sootman (158191) | more than 4 years ago | (#29048299)

It really amuses me how all these different comments come up in every thread about search engines. Everyone's experience is different. Google is still very useful to me 99% of the time. As for AltaVista, I remember '96-'97 very well. I would usually use Yahoo first. If Yahoo only produced a small handful of results--literally, 10 or less, and no good ones--then I'd go to AltaVista and get tens of thousands of results. If I was lucky I'd find what I wanted in the first few pages, else I'd give up.

Google is still literally orders of magnitude than anything else I've tried. Disclaimer: I've pretty much used only Google for the last... um, however many years it's been since they came on the scene. I won't claim to have used it when they were still hosted at stanford.edu, but I heard about them early on (back when they had , probably from Slashdot, and I was impressed right away. I probably stopped using Yahoo altogether within a couple months.

Re:hmm (5, Funny)

MobileTatsu-NJG (946591) | more than 4 years ago | (#29047661)

They use the Linux platform to the absolute max, leveraging all the blood and sweat Linux developers poured into its development over the past 15 years, and yet, not contributing back any of their most significant enhancements.

Not contributing back!? Dude, they gave us *google*. Remember what it was like before google? When internet search was basically voo-doo crapshoots, that worked 25% of the time? They gave us a search engine that actually *worked*. Before that, you basically had to bookmark or memorize internet sites that you liked. Good luck actually finding what you were looking for without having an actual site in mind beforehand.

I think that alone has probably spurred the development of free software. Imagine being able to *find things* on the internet!

Are you kidding? Search for Quake? Porn. Search for a new version of Netscape? Porn. Google? PFtb. It always gave me Quake and Netscape. My pr0n searching was MUCH more productive before Google!

Re:hmm (1)

sootman (158191) | more than 4 years ago | (#29048317)

Not contributing back!? Dude, they gave us *google*.

And "just fucking google it" [justfuckinggoogleit.com] has started replacing RTFM. :-) See also "Let Me Google That For You." [lmgtfy.com]

A guy on a list I used to be on used to ask REALLY dumb, easily-googlable questions. I mean, you could literally take the message subject, plug it into google, and get the answer. I wrote (but never deployed) a script that would take the subject of his message, google it, and reply to the list with the first page of search results in the body. (Something like "links --dump http://www.google.com/search?q=$1+$2+$3+$4+$5 [google.com] | mail blah@example.org")

Re:hmm (0)

Anonymous Coward | more than 4 years ago | (#29047271)

You seem to be under the impression that GFS would be super useful to the public at large, like a lightbulb. It's not. It's quite specialized.

Besides search engines, I suppose there might be some scientific applications, but even that is not a given, said applications probably need their own specialization and may require lots of modification to be useful anyway.

Re:hmm (1)

KahabutDieDrake (1515139) | more than 4 years ago | (#29048267)

It's simply good business, and as we've been shown endlessly, there is no good or bad, no right and wrong in business. Simply what you can do, and what you can do and get away with.

Google isn't going to arm their competition with the very tools that set google apart. I don't think any rational person would actually expect them to. While I'd like to see their tools and platforms released to the community, it would probably help google's competitors more than it would the community itself. After all, when was the last time you deployed a distributed system with geo nodes across the world and needed absolutely fail safe data backup with no down time? It's just not something anybody does very often.

It's not really GFS (5, Insightful)

mysidia (191772) | more than 4 years ago | (#29046963)

It's GoogleFS.

GFS refers to the Global File System [wikipedia.org] , which is commonly used in Linux clustering environments.

By comparison, GoogleFS came second, is basically a no-name filesystem unknown to most of the IT world, because it's not available for use, hasn't been released as a product, compared to the well-established global filesystem.

It would certainly seem like the Global File system would have priority claim over the name GFS...

So let's stop calling Google's filesystem, which we'll probably never get to use GFS :)

Re:It's not really GFS (1)

fatalwall (873645) | more than 4 years ago | (#29047005)

its just a tla doesnt have any meaning what so ever until you give it context. By saying google its assumed there talking about googles tla gfs. Thats why when ever you use a tla its good to tell people what is at least once.

tla = three letter acrylic

Re:It's not really GFS (5, Funny)

mysidia (191772) | more than 4 years ago | (#29047103)

That is a problem that may be getting corrected [ietf.org] by the IANA TLA registry :)

Re:It's not really GFS (1)

fatalwall (873645) | more than 4 years ago | (#29047381)

for those that havent read the above link do.. should give you a chuckle or two.. some one must have ocd to have fully made that!!

Re:It's not really GFS (1)

skine (1524819) | more than 4 years ago | (#29047773)

some one must have ocd to have fully made that!!

Yes, people who do things without monetary reward have a disease.

Re:It's not really GFS (4, Funny)

corsec67 (627446) | more than 4 years ago | (#29047423)

Tell that to the NWA. Wrestling and a wildlife foundation should be even easier to tell apart, they both aren't as similar as two file systems.

Re:It's not really GFS (1)

icannotthinkofaname (1480543) | more than 4 years ago | (#29047899)

tla = three letter acrylic

I'm sorry, I'm not getting the joke. Are you deliberately defining "TLA" in an unexpected way, as you demonstrated in the content of your post that you could, or did you mean to type "three-letter acronym"?

Re:It's not really GFS (0)

Anonymous Coward | more than 4 years ago | (#29047173)

GFS is commonly seen in clusters only in the sense that it's "commonly seen in research environment and throwaway test setups" as it's not really all that stable, has major consistency issues, only compiles when there's a full moon and lacks serious documentation

Google is IT done right... (5, Funny)

Alien Being (18488) | more than 4 years ago | (#29046981)

but God help us all if they ever do turn evil.

Re:Google is IT done right... (3, Funny)

PhrostyMcByte (589271) | more than 4 years ago | (#29047109)

Google is what happens when developers and IT talk to each other correctly. Normally there is a brick wall separating the two, with IT guys being at the mercy of whatever the well-meaning but typically oblivious (to IT problems) devs cook up.

Re:Google is IT done right... (1)

Alien Being (18488) | more than 4 years ago | (#29047155)

Developers aren't IT?

Re:Google is IT done right... (1)

east coast (590680) | more than 4 years ago | (#29047171)

A lot of places don't see devs as IT. They see anything hardware as IT. In your environment it may be screwy, in another it's accepted.

Re:Google is IT done right... (4, Funny)

ObitMan (550793) | more than 4 years ago | (#29047199)

not on your life.
Developers constantly ruin perfectly good infrastructure.

Re:Google is IT done right... (2, Interesting)

Antique Geekmeister (740220) | more than 4 years ago | (#29047275)

Because they're not allowed to share their ideas with IT, and vice versa. I can't list the number of times developers have published brain-scrambled vomit as part of their projects, because it didn't interest them and no one with experience was around to explain the inevitable problems. The maintenance model for subversion where you have to completely rebuild the repository to completely delete an accidentally stored DVD image is a classic example.

Conversely, I've expressed extreme doubts about projects that turned out to be effective and workable because my knowledge of file system behavior or hardware limitations was 3 weeks behind the times. I even spiked a project once for such reasons, although when the developer and I spoke without the confused manager in the way it became clear that the hardware _could_ support his needs.

Re:Google is IT done right... (5, Funny)

Nefarious Wheel (628136) | more than 4 years ago | (#29047225)

Developers aren't IT?

Not really, no. It's kind of like the difference between a doctor and a patient. Or to use a car analogy, the difference between being an automotive engineer and the guy who takes money for candy bars, magazines and fuel.

Disclosure: I was a developer for about thirty years before I took a step down and moved into marketing. I learned a lot of languages but was stopped when I discovered I was having trouble mastering Hindi.

Re:Google is IT done right... (0)

Anonymous Coward | more than 4 years ago | (#29047597)

It's more like the janitors (IT) and the guys who use the men's room and have bad aim (Developers).

I was both, for many years after being a real engineer (mechanical), consulting (in several fields), research, security, and finally management (for the big money). Love the Valley, man. Never bored. Anyway, I stand by my analogy.

FYI: Helpdesk is like the people who clean out bed pans.

Re:Developers aren't IT? (1)

networkzombie (921324) | more than 4 years ago | (#29047695)

You need to change your terminology, because if you are saying that developers are not part of Information Technology, you are full of shit.

Re:Developers aren't IT? (1)

skine (1524819) | more than 4 years ago | (#29047837)

Let me put it this way:

Poorly run companies have HMOs and patients.

Less poorly run companies have HMOs, doctors and patients.

Well run companies hide the fact that there are HMOs.

Re:Developers aren't IT? (1)

Nefarious Wheel (628136) | more than 4 years ago | (#29047957)

You need to change your terminology, because if you are saying that developers are not part of Information Technology, you are full of shit.

Oh dear dear dear. I may be uh, compostally privileged, but I am pretty much across the industry. I even started a bit of it (not hard when you enter the industry in 1969). "IT" or "ICT" is commonly used to denote network, server and desktop infrastructure these days. Development, that is COTS package development or bespoke customisation is usually abstracted and referred to as -- well, "Development".

I asked my daughter once what genus Dragons were most closely related to - were they a type of bird? "No, silly daddy! Dragons are Dragon-type".

So I'd classify Developers as Developer-type, and leave the term IT to the wire-pullers and server-pluggers.

Re:Google is IT done right... (5, Interesting)

Duncan3 (10537) | more than 4 years ago | (#29047811)

Not really, it's IT done by not letting anyone over 30 or with any experience into the room. Every single issue they had to learn and fix mentioned in the article is quite literally standard textbook stuff in distributed systems, and has been for over 40 years. The failure model, the huge chunk sized, the single master problems... etc. Nobody who had taken even one decent class would have ever considered the original design viable.

They really should just stick to buying their tech pre-made like everything else Google is known for - acquisitions [wikipedia.org] . Other companies are willing to hire experienced people. You know, those old lazy bastards that only work 40 hours a week because they have families, cost way too much to provide health insurance to, but get things done 5x as fast because they have done it before :)

Re:Google is IT done right... (2, Insightful)

Daniel Phillips (238627) | more than 4 years ago | (#29048111)

Not really, it's IT done by not letting anyone over 30 or with any experience into the room. Every single issue they had to learn and fix mentioned in the article is quite literally standard textbook stuff in distributed systems, and has been for over 40 years. The failure model, the huge chunk sized, the single master problems... etc. Nobody who had taken even one decent class would have ever considered the original design viable. They really should just stick to buying their tech pre-made like everything else Google is known for - acquisitions [wikipedia.org]. Other companies are willing to hire experienced people. You know, those old lazy bastards that only work 40 hours a week because they have families, cost way too much to provide health insurance to, but get things done 5x as fast because they have done it before :)

You hit the nail right on the head. The original GFS is pretty lame, as Google folks freely admit (full disclosure: I'm a fomer Googler, but I'm not telling you anything you can't find on ahem Google). The new GFS will also be pretty lame, because as you correctly point out, Larry, Sergey and Eric don't quite get the concept of experienced people who have done it before. All that standard clustering stuff has to be reinvented by Googlers, who frankly, have gotten a little soft over the years, now so used to working. We will see, but I'm predicting that the new GFS will still be a research project two years from now.

Curiously (5, Insightful)

ShooterNeo (555040) | more than 4 years ago | (#29047001)

In the article, it's stated that the load on the google file system has grown orders of magnitude greater than it was ever intended to handle. And one of the algorithm changes is that the chunks in the new file system are 1 megabyte in size rather than 64 megabytes. This is to reduce latency, which makes logical sense...but dividing a gigantic database into pieces that are 64 time smaller doesn't make intuitive sense...

Re:Curiously (2, Interesting)

Alien Being (18488) | more than 4 years ago | (#29047069)

"..but dividing a gigantic database into pieces that are 64 time smaller doesn't make intuitive sense..."

It does if it was 64x too big to begin with. Live and learn.

Re:Curiously (4, Funny)

martin-boundary (547041) | more than 4 years ago | (#29047333)

It does if it was 64x too big to begin with. Live and learn.

No need to learn. 64x should be enough for anybody, dammit!

Re:Curiously (1)

nthitz (840462) | more than 4 years ago | (#29047127)

Perhaps 64 MB is good for Search data where they are doing massive analyzation of sites, but for content such as short youtube videos or text email messages it may work quite nicely.

Re:Curiously (1)

MeatBag PussRocket (1475317) | more than 4 years ago | (#29047167)

would a highway have faster traffic if it were semi-trucks only or motorcycles only?

Re:Curiously (2, Insightful)

Russianspi (1129469) | more than 4 years ago | (#29047253)

I don't know if that is so much the question, as much as: Would a highway get more people to where they are going faster if it were semis only or motorcycles only? The answer to this question (and probably to Google's question, too) is "It depends on a lot of other factors".

Re:Curiously (0)

Anonymous Coward | more than 4 years ago | (#29047365)

That's actually a very poor example, mostly because none of the required data points are provided.

Such as origin, destination, speed limit, pay load.

There's an old saying, "never underestimate the bandwidth of a station wagon full of backup tapes".

If you were moving pianos motor cycles are the worst possible transport.

If you're moving a million court filings from a million places motor cycles, a million court filings from one office to another would be trucks.

If the speed limit for moving something is possible by both transports, they're equal, motor cycles only win in congestion.

Re:Curiously (4, Funny)

Runaway1956 (1322357) | more than 4 years ago | (#29047587)

The question is based on assumptions. I've personally pushed an 18-wheeler over 150 mph. I've pushed a bike over 170 mph. In both cases, the limiting factors were all the 4-wheelers. Take the cars off the roads, and let the bikes and the trucks run.

Oh yeah - one more thing. Mandate that cop cars have square wheels. They already have radio, they need a handicap to make things fair.

Re:Curiously (1)

lazyforker (957705) | more than 4 years ago | (#29047175)

Maybe it allows them to distribute the load across a much larger number of spindles. Also they'll use space more efficiently: so a 1K file now only takes 1MB instead of 64.

Re:Curiously (0)

Anonymous Coward | more than 4 years ago | (#29047523)

"but dividing a gigantic database into pieces that are 64 time smaller doesn't make intuitive sense..."

dude, u got it wrong, they don't want to basically break everything 64 times smaller. the idea is to support 1MB chunks.

"Rapid growth also put pressure on another key parameter of the original GFS design: the choice to establish 64 MB as the standard chunk size. That, of course, was much larger than the typical file-system block size, but only because the files generated by Google's crawling and indexing system were unusually large. As the application mix changed over time, however, ways had to be found to let the system deal efficiently with large numbers of files requiring far less than 64 MB (think in terms of Gmail, for example). The problem was not so much with the number of files itself, but rather with the memory demands all of those files made on the centralized master, thus exposing one of the bottleneck risks inherent in the original GFS design."

"My gut feeling is that if you design for an average 1-MB file size, then that should provide for a much larger class of things than does a design that assumes a 64-MB average file size. Ideally, you would like to imagine a system that goes all the way down to much smaller file sizes, but 1 MB seems a reasonable compromise in our environment."

i hear open source fanbois like it in the ass (-1, Troll)

Anonymous Coward | more than 4 years ago | (#29047129)

faggots.

re: hmm (0)

Anonymous Coward | more than 4 years ago | (#29047233)

nope, read the Wikipedia article on GFS

Quality of comments going downhill... (5, Funny)

s0litaire (1205168) | more than 4 years ago | (#29047243)

There's over 25 comments and not one has attempted to call it "Goatse File System"!

Whats up with you trolls! You guys on a union break or what!!

Re:Quality of comments going downhill... (0)

Anonymous Coward | more than 4 years ago | (#29047471)

Goatse File System would be open source. VERY open source, if you get my meaning...

Re:Quality of comments going downhill... (0)

Anonymous Coward | more than 4 years ago | (#29047511)

Union rules require one troll to type each sentence, and then one more to hit the submit button. It would be simple as that, but the change in the slashdot UI created an undefined amount of time between "Preview" and "Submit", so they're still renegotiating the contract to account for that. Sorry.

Seriously Folks (1)

Nom du Keyboard (633989) | more than 4 years ago | (#29047439)

Seriously folks, is there no already existing file system that can already meet these needs? If not, then what are Google's competitors using?

Is that no one else has yet to face up to this issues properly and this is a huge competitive advantage for Google, or is it simply NIH?

Re:Seriously Folks (2, Insightful)

MikeBabcock (65886) | more than 4 years ago | (#29047495)

Google has competitors?

Seriously, Microsoft has been promising a database driven filesystem for its server OS for years without delivering anything substantial to date, and it doesn't seem like they're running anything different internally either.

Re:Seriously Folks (1)

jpkotta (1495893) | more than 4 years ago | (#29047651)

Seriously folks, is there no already existing file system that can already meet these needs? If not, then what are Google's competitors using? Is that no one else has yet to face up to this issues properly and this is a huge competitive advantage for Google, or is it simply NIH?

FTFA:

One thing that helped tremendously was that Google built not only the file system but also all of the applications running on top of it. While adjustments were continually made in GFS to make it more accommodating to all the new use cases, the applications themselves were also developed with the various strengths and weaknesses of GFS in mind. "Because we built everything, we were free to cheat whenever we wanted to," Gobioff neatly summarized. "We could push problems back and forth between the application space and the file-system space, and then work out accommodations between the two."

TFS fucking summary (0)

Anonymous Coward | more than 4 years ago | (#29048167)

... shouldn't list both the acronym and its constituent words.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...