Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

"Slacker DBs" vs. Old-Guard DBs

kdawson posted more than 5 years ago | from the close-enough-for-web-work dept.

Databases 267

snydeq writes "Non-relational upstarts — tools that tack the letters 'db' onto a 'pile of code that breaks with the traditional relational model' — have grabbed attention in large part because they willfully ignore many of the rules that codify the hard lessons learned by the old database masters. Doing away with JOINs and introducing phrases like 'eventual consistency,' these 'slacker DBs' offer greater simplicity and improved means of storing data for Web apps, yet remain toys in the eyes of old guard DB admins. 'This distinction between immediate and eventual consistency is deeply philosophical and depends on how important the data happens to be,' writes InfoWorld's Peter Wayner, who let down his old-guard leanings and tested slacker DBs — Amazon SimpleDB, Apache CouchDB, Google App Engine, and Persevere — to see how they are affecting the evolution of modern IT."

cancel ×

267 comments

Sorry! There are no comments related to the filter you selected.

first? (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#27315517)

first?

Re:first? (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#27315577)

You seem unsure.

Hackers. (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#27315683)

Was the best movie of all time.

Re:Hackers. (1, Funny)

TheSpoom (715771) | more than 5 years ago | (#27315705)

If I could do a security audit on a website by flying through a psychedelic 3D futurescape, I might just become a workaholic.

SELECT * FROM SNARKY_COMMENTS (4, Funny)

billstewart (78916) | more than 5 years ago | (#27316191)

Can't quite fit the whole query into the title box, but if you were using one of those databases that Wayner's article talked about, you'd be able to query and find out if you were first...

Obama Policies Will Bankrupt the US Tsarkon Report (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#27315529)

Obama Policies Will Bankrupt the US Tsarkon Reports
s
- This just in, March 23 2009 Chinas central bank on Monday proposed replacing the US dollar as the international reserve currency with a new global system controlled by the International Monetary Fund [ft.com]

- Russia too: It is necessary to work out and adopt internationally recognized standards for macroeconomic and budget policy, which are binding for the leading world economies, including the countries issuing reserve currencies - the Kremlin proposals read. [en.rian.ru]

- President Barak "The Teleprompter" Obama is deeply connected to corruption, Rahm Emanuel (Radical authoritarian Statist whose father was part of the Murderous Civilian Killing Israeli Terrorist Organization known as IRGUN), Connected to Rod Blagojevich (Rahm inherited Rod's federal-congress seat), Connected to Ayers, a man who promotes the concept that civilian collateral damage is ok in a war against freedom, Preacher Jeremiah Wright, who is himself a black-elitist who wants all the people who largely "pay the freight" to suffer, 31 million on food stamps, more blacks are in prison and on food-stamps per capita than anyone else. The problem with Wright is simply this: the facts are "racist."
- Obama: Racist, AIPAC-bootlicker, Corrupted to the bone Chicago style and a Traitor to the US Constitution and a Liar who can't even produce a valid birth certificate (which is not a certificate of live birth)
- Raytheon lobbyist in Pentagon, lots lobbyists getting exemptions even though he promised not to have them.
- Goldman Sachs insider second in command at Treasury. Bumbling tax cheat idiot in "command" of Treasury with 17 positions unfilled as of late March 2009.
- Cabinet has had several nominees and appointees with multiple tax fraud issues.
- Lied about having a new degree of accountability and a SUNSHINE period of new laws, he has signed bills with little or no review at whitehouse.gov as promised.
- Appointed a second amendment violating Rich-pardoning treasonist Eric Holder as AG, the top cop of the USA, a man who helped a fugitive evade justice.
- Has not put a dime in for a single new nuclear power plant but wants to help bridges and roads to promote more driving.
- Obama, Blagojevich and Rahm Emanuel have a LOT to hide. They literally lived next to each other, Rahm had (until being Chairman Obama's Chief of staff) Blagojevich's old federal congressional seat. Blagojevich helped Chairman "The Teleprompter" Obama cheat his way to the Illinois senate by getting other candidates thrown off the ballot in Illinois. Why do you think Blagojevich was so mad? Obama DID owe him, big time. Rahm and Obama are using Blagojevich and trying to cut his head off to keep him away.
- Tony Rezko, Iraqi Arms Dealer Nahdmi Auchi, and of course Aiham Alsammarae. Chairman "The Teleprompter" Hussein Obama is so corrupted its a joke.
- Fools and "useful idiots" twist the pie charts by leaving welfare, workfare, interest on debt, social security, Medicare and Medicaid out and focusing only on non-whole "discretionary" pie charts.
2007 high level pie chart, Federal Budget, USA [wikimedia.org]
2009 Pie chart, detailed, Federal Budget, USA [wikimedia.org]
- Chairman Obama is drastically increasing spending and creating more entitlements that will make the US less competitive (especially against China, India, East Europe/Russia). This will be a huge disaster and change you can believe in will strap you and your grandkids with more debt. No taxation without representation? Obama is spending money for the next two-three generations and they can't even vote yet, or even have been born.
- An alternative to the dollar and a forex and a reserve currency came up at the last G20 meeting. The world will not take faith in Obama's liar-socialist spending and welfare state, why should the taxpayers (plebian citizen-slaves of a police state).
- The spending going on now vastly eclipses all previous spending. In fact, the massive trillion plus debts is a thing of the 80's onwards. Congress signs the checks, remember that Year after year, as egregious as the pentagon spending is, that the social spending is completely a waste of money and it is unfunded over the long term. Eisenhower built the interstates, the US could build a new power infrastructure with this money but instead is being pissed into creating more of an entitlement system that is STILL unfunded, and without massive poll-taxes and far more aggressive progressive taxes, could NEVER be funded.
- The budgeting being done today were recently reported by a non-partisan auditing commission will lead to about 10 TRILLION in new debt over the next 10 years. Obama is going to double the national debt while doing nothing to address the unfunded debt obligations of Social Security.
- Clinton appointed David Walker of the GAO, he quit, the unfunded debt obligations have rendered the USA insolvent according to accounting standards.
Taxpayers on the hook for $59 trillion [usatoday.com]
US Public Debt Unfunded Debt Obligations [wikipedia.org]
- Most of the world population gets NOTHING from their governments, or a very bare minimum or services that benefit only the upper echelons of society. However, the liar Chairman Obama says we need his universal "state-hospital" rationed health care to be competitive. Bull. China and India give nothing, and they are the biggest threat to the American worker. By forcing healthcare and higher taxes, Americans will be less competitive.
- If you think 60% tax rates end to end (income, accounts receivable tax, building permit tax, CDL tax, cigarette tax, corporate income tax, dog license tax, federal income tax, unemployment tax, gasoline tax, hunting license tax, fishing license tax, waterfowl stamp tax, inheritance tax, inventory tax, liquor tax, luxury tax, Medicare tax, city, school and county property tax (up 33 percent last 4 years), real estate tax, social security tax, road usage tax, toll road tax, state and city sales tax, recreational vehicle tax, excise tax, state franchise tax, state unemployment tax, telephone federal excise tax, telephone federal state and local surcharge tax, telephone minimum usage surcharge tax, telephone state and local tax, utility tax, vehicle license registration tax, capital gains tax, lease severance tax, oil and gas assessment tax, misc internet sales tax and many more taxes that I can't recall at the moment) will make the US competitive, along with compulsory programs to provide everyone with health care is going to make the US competitive in the age of India and China, you are a joke.
- As the US nationalizes (read: rations healthcare) to the least common denominator of affordability without regard to efficacy, people with money will simply look into medical tourism so those with money can go to medical parks in India and get real health care. Those who have lived in Canada or in the UK can tell you "free" healthcare is NOT a panacea. If you think this, you are again, a useful idiot. The NHS in the UK has given bad blood and Hepatitis and AIDS blood to people, and Jade Goody who just died was misdiagnosed twice resulting in her death (She was all cleared twice of cervical cancer which she just died of). The NHS in the UK is not able to be sued or held accountable. Neither will Chairman Obama's rationed health care service for America.
- Sorry to bust the socialist bubble-lie, but support of these types of policies will simply lower the standard of living in the USA, particularly for the middle class. At least at the end of the Eisenhower projects the USA got roads to show for the spending, and with this new spending, the USA could have built power plants that get the USA out of the middle east, but the age of government for the sake of government is upon us, and the useful idiots line up and believe empty promises.
The pentagon (and Bechtel, Kroll, Bluewater, Halliburton, etc) could get less than half of what they get today, but that will fix nothing fundamental in terms of government spending. It is simply not enough to make a difference when compared to the Medicare, Medicaid, welfare, workfare and social security entitlements.
See: YouTube - US Government Immorality Will Lead to Bankruptcy [youtube.com]
- If Obama thinks its ok to lie to 300 million people about being able to "take care of them" without even being honest about what that care would look like, then being an idiot and believing in Obama is for you.
- The head of the IRS and the head of the Treasury, Geithner, is a Tax Cheat
- Lied about no lobbyists - their numbers are growing.
- The US Government already have over 50% of the budget on Medicare, Medicaid, welfare, workfare and social security. Socialists: Good job on that one, its working great. Solution to the current near-collapse-due-to-over-spending: add more unfunded entitlements!
- You Socialist-liars can break my spirit and my financial back to force me to "need" a federal government that is turning this country into a police state and turn it into a quasi-socialist lie, but I will, I must put up a fight. I have kids to educate and feed, and the stuff you sell (which is failing to various degrees everywhere else as implemented) is simply forcing a culture of failure on a once great, libertarian free country.
- I will not be complacent with your "change," and there will be a point where civil war will become an option. See how hard you can push before you get it. How much more than half can the truly productive workers in this country afford to pay. Keep pushing to find out how to start a civil war.
- The socialist-lie of a plan will not work, its not fundable, it WILL destroy the currency to fund it, and its really as simple as this: if this insanity is funded by borrowing from the US's economic and military adversaries then Obama and his socialist cabal is NOT fit to administrate society. Rome fell. Kings who mis-manged their treasuries all fell. Every example of unhinged spending leads to the same result: systemic collapse.
- Obama and his sycophantic lunatics would want to have a civil war to get Chairman Obama's way and force the socialist-lie system on my already tax paying law abiding ass. And as far as "no new taxes" for those under 250k, its a lie, the tax is called inflation, which is set to begin just about now that the Chinese wont want the USA's worthless treasuries to fund the socialist-lie fantasy (one that COMMUNIST China doesn't even try and sell to its people!)
- Chairman Obama's numbers don't add up. There is a $59 trillion dollar hole (UFDO) in social security alone. AIG $150 billion here, TARP $350 billion there. $800 billion for a highly dubious stimulus package. Another one on the way. $59 trillion hole in the balance sheet IGNORED. China saying they aren't going to buy treasuries, Clinton clamoring to find buyers now. $3.6 trillion dollar budget, potential military action on Mexico, Iran still a "terrorist state" at the behest of the AIPAC, spending up, dollar about to fall, inflation over time since Breton Woods extremely easy to document, yet, the socialist-liars question when the numbers (the Federal Government numbers) simply don't add up to the point where if the US-GOV was a company it would be insolvent.
  -How dare the taxpayers question what Chairman Obama's drastic spending increases are going to do to the purchasing power of our savings because Chairman Obama wants to recklessly spend and try to maintain and American empire AND guarantee a standard of living, and Chairman Obama doesn't even want to build a single nuclear power plant to do it? Chairman Obama must be a complete and total lunatic moron.
- Obama is either a negligent idiot or an unhinged maniac with delusional fantasies. Meanwhile, Chainman Obama's tax dodging Treasury Secretary has 17 unfilled positions, the Treasury Dept. isn't even functioning at this point.
- "General welfare" in the constitution was, according to the man who wrote it, Madison, meant to be extremely limited in scope. The federal government per the constitution doesn't even have the enumerated POWER to deal with economic messes. A lot of these "POWERS" were created while there is a crisis to dupe the public into accepting an un-constitutional authoritarian regime as the government and to usurp authority over the people.
- The USA is a constitutional republic. A democracy is two wolves and a sheep voting to eat a sheep. Also a constitutional republic isn't about using a barely-majority or a plurality to stuff your (un-fundable disastrous) crap down the disenfranchised other-half's throat.
- With Obama's authoritarian corrupted criminal (aiding and abetting a criminal in flight of prosecution, Rich case) Eric Holder in charge, we won't have our inalienable and enumerated rights to firearms much longer. For a constitutional law expert, Obama must have never read the federalist papers or he would simply hand himself as a traitor.
- The arbitrary expansion of "general welfare" is not only unconstitutional, it may very well lead to a serious conflict on the issue.
- Here is a debate on general welfare and how stuff like this came to pass, but was clearly no intended by the authors of the document of root law.
In Federalist No. 41, James Madison asked rhetorically: "For what purpose could the enumeration of particular powers be inserted, if these and all others were meant to be included in the preceding general power?" (In reference to the general welfare clause)
So strongly did the founders believe that "general welfare" wouldn't be expanded as written:
In Federalist No. 84, Alexander Hamilton indirectly confirmed Madison's point. (That the "general welfare" clause was "clearly" nota free pass for government)
Hamilton argued that a bill of rights, which many were clamoring for, would be not only unnecessary, but dangerous. Since the federal government was given only a few specific powers, there was no need to add prohibitions: it was implicitly prohibited by the listed powers. If a proposed law a relief act, for instance wasn't covered by any of these powers, it was unconstitutional.
"why declare that things shall not be done which there is no power to do? Why, for instance, should it be said, that the liberty of the press shall not be restrained when no power is given by which restrictions may be imposed?"
Hamilton goes on to argue that making Amendments (e.g., enumerating Free speech, press and assembly) and enumerating the 'right' would have the following effect:
(A bill of rights) "would furnish, to men disposed to usurp, a plausible pretence for claiming that power that is, a power to regulate the press, short of actually shutting it down. "
"With respect to the words 'general welfare,' I have always regarded them as qualified by the detail of powers (enumerated in the Constitution) connected with them. To take them in a literal and unlimited sense would be a metamorphosis of the Constitution into a character which there is a host of proofs was not contemplated by its creators." --James Madison [The US Supreme Court has found the meaning of "general welfare" in the Constitution to be much more elastic than did Mr. Madison. But as the "author of the Constitution," what does he know?]
James Madison, when asked if the "general welfare" clause was a grant of power, replied in 1792, in a letter to Henry Lee,

If not only the means but the objects are unlimited, the parchment [the Constitution] should be thrown into the fire at once.

"...We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed. That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it...it is their right, it is their duty, to throw off such Government , and to provide new Guards for their future security. ...--The Declaration of Independence
- Wrong, monetizing failures causes more. Japan showed us this for decades. But hey, Chairman Obama thinks you can fix a problem DECADES in the making with a quick fixer-upper, he is screwed in the head.
- The complaints are with the Federal government (in general) since Breton Woods. The Federal Government and Obama's minions STILL didn't listen to David Walker, a Clinton appointee and former head of the GAO. This isn't about political parties anymore morons!
- Show me a single federal budget that was less than the previous. If this $3.6T budget goes, its never coming back barring systemic collapse.
- The United States Federal Government, The United States Federal Reserve, and the banks which were enabled to continue down reckless paths by a quasi government agency known as the Federal Reserve whose actions are not subject to congress and whose members are unelected. This situation is untenable and unconstitutional.
- Every inflationary road taken in history ends in collapse. Keynesian policies are widely regarded as no longer workable.
- Inflation is a tax: What ignorant tax and spenders don't take into account here is the relative percentages of people's wealth (both net and gross) and the costs of owning and maintaining houses, cars, standards of living.
- Inflation via deficit spending is going to make it such that you will be paying a lot more by percentage of your income to maintain a given standard of living. Obama's arguments are so poorly thought out and seek to blame "Republicans" for the mess, its really simply laughable - the needs cleanup now, not worsening.
- You can't spend your way out of a hole if the creditors (e.g. China) start telling the USA they won't buy. It is that simple. Now America starts to have to collateralize the debt with assets. The USA will be selling off chunks of American assets to back the new debt. One day, it may even be necessary to sell Alaska back to Russia because no one will take greenbacks to prop up a failing version of a modern Rome.
- Ah, here we go with the Matthew Lesko arguments. [lesko.com]
Interest rates were on the rise before the government stepped in with free money for everyone (the fine print of course indicate massive strings attached).
Other economies, for example, India, have the central rates set to far more reasonable/realistic rates (at the moment ~ 8+%), which is still tends to be too low, but shows that if you need someone else capital you need to pay a premium for it, and given that capital is in short supply, it would stand to reason that a premium must be charged for it.
The problem is the unrealistic growth rates of mature economies don't allow for profiting via growth projections (rather than simply earning money). So the government steps in, turns on the free money spigot, gets the interest rates for savings down in the 1-2% range while diluting the value of the whole currency in order to prop up dying companies that ran the business like a Madhoff Ponzi scheme.
- The Republicans aren't solely responsible for the crisis as Obama's minions would have you believe, congress is (no particular congress), the Executive of the US government (no particular one) and the US Federal Reserve System are all at fault.
- Fundamentally, the government is trying to fix the prices of various things to "make it all work." This pulling on the invisible hand is a fools venture. It was predicted long ago the housing collapse (and those, such as myself, in the know, wished while realizing the housing collapse coming that we were wrong for everyone's sake - but the truth is the truth) . It may be that the Austrian (von Mises) economists will ultimately be proven right.
- We are a nation of partially educated whiney grabby idiots, and we got the government that represents this. The Chinese, India and other up and coming nations will show no mercy for this arrogant abuse of our status as the world's forex reserves.
- War and asset sales will continue to be the only option for this scheme until it is corrected at the core. And to say that the government has already averted a depression by doing what they did (most of the monies injected wont be "felt" for some time), is just arrogance and stupidity. Price fixing prolonged the Great Depression. Price-fixing (or attempting to) houses will do the same, but probably worse.
- Obama's minions simply don't care if the US is bankrupted and rendered insolvent, they just want a say in how its done, presumably to "feel safe." Rather selfish.

"They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety." AND "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (Possibly Richard Jackson)

- Everyone better realize that inflation will pay a major role in funding un-fundable fantasies, wiping the savers and the middle class out. The problem is, that other countries are growing tired of making our Federal Reserve notes worth something by buying our debt as treasuries. Obama's minions talk about spending, but in order to "get what YOU want" you will sell debt to potential economic and military adversaries? Real bright. What's really sad is that despite David Walker being an authority on these issues, people refuse to even watch him and listen to what he is saying.
- On the success of Canada and its form of Socialism: A huge country like Canada with massive amounts of uranium and tar sands and natural resources and a huge land mass with a scant 30 million people is an order of magnitude less of a problem to manage than a country with 10x its population, a serious leaky southern border, backfiring aggressive foreign policy, particularly with Iran, and the US is competing with countries like India and China whose middle classes are larger than the US's entire population. The top 5 students in every Indian and Chinese primary school out numbers all the kids in primary school in the US. Canada is a idyllic island, the USA is front and center in an all out economic and political clash of ideologies.
- Cap and trade (and pollution control for solving global problems) will never work unless the top 10 countries in the world (in terms of both GDP and manufacturing capacity and population) are on board. Period end. If the world doesn't quickly move to nuclear now and fusion shortly, it is OVER possibly not if every home on the planet gets a wind vane, but that seems unlikely to happen (since its possible now).
- Keynes calls it "the paradox of thrift" and suggested that policies forcing people not to save is a "good idea." The guy wanted people spending all the time, or if he didn't, he never conveyed that to his protégés well enough for them to not do what they are doing. Right now the plebeians in the US are actually stashing cash, and everyone from Obama to the media is trying to get people to spend spend spend. The best thing for the long term is for people to prepare for the coming hell, not set out with no reserves.
- I have seen Keynes invoked to justify nearly every bad move in the past decade, and its warming up to be a potential currency collapse, the collapse of the US Treasury and Federal Reserve notes, and a collapse of the NYSE. And then they invoke Keynes to suggest the best way out of the mess is to spend out of an already near-critically debt massed black hole.
- A house is run like a town is run like a country or business is run like a state is run like a government. If there are things the government is doing that would either force your home into bankruptcy or into jail via fraud charges, then the government and banks shouldn't be operating in that fashion. A certain degree of stretchy liquidity is in order, but in terms of percent of GDP, there is no way of justifying what they US has now.
- Iceland failed at 850 percent debt to GDP. The US is at 350 and rising. It is not a good thing at all.
- What is happening to the dollar as a forex standard. [youtube.com]
- March 19, 2009 C-SPAN - "Let's Quit Destroying Our Dollar!" [youtube.com]
- HR 1207 (A bill to make the Fed more accountable and to answer questions regarding the dollar policy) [loc.gov]

Title: Obama sidetracked by fiscal mess, but presses on [yahoo.com]
"Being heard above the din may prove difficult. Lawmakers are wrangling over taxing people who got big bonuses and worrying the president's budget could generate $9.3 trillion in red ink over the next decade."
- Kremlin to pitch new global currency [infowars.com]
Russia proposes creation of global super-reserve currency

Holy crap, even the Russians and Chinese get it. Strange days are here.

slashdot insult? :( (5, Funny)

FlashBuster3000 (319616) | more than 5 years ago | (#27315597)

FTA: "The world won't end if some snarky, anonymous comment on Slashdot disappears."
What? Nothing more important than anonymous slashdot trolls to moderate :/

Re:slashdot insult? :( (0, Insightful)

Anonymous Coward | more than 5 years ago | (#27316625)

Well, fuck them very much!

--AC

*mods article -1, Flamebait* (5, Insightful)

TheSpoom (715771) | more than 5 years ago | (#27315603)

Is it just me or did this article go out of its way to insult people who use "traditional" RDBMSs?

I mean, I'm well versed in SQL and data consistency et al, but I'm still more than willing to consider new technologies. What the hell?

Re:*mods article -1, Flamebait* (0)

Anonymous Coward | more than 5 years ago | (#27315939)

lol, insult to injury with the flamebait tag. sorry man.

Re:*mods article -1, Flamebait* (1)

Just Some Guy (3352) | more than 5 years ago | (#27315943)

I read it exactly the other way, that they were slagging on the newcomers in favor of us old fogies (PostgreSQL FTW!).

Re:*mods article -1, Flamebait* (1)

peterwayner (266189) | more than 5 years ago | (#27316655)

Phew--- At least I know that I was reasonably balanced. I guess it depends upon how you feel about the intensity of insult buried inside the use of words like "slacker", "codger" etc. If they're all about the same level, well, I've threaded the needle.

Re:*mods article -1, Flamebait* (3, Funny)

Just Some Guy (3352) | more than 5 years ago | (#27316777)

You seem to imply there was more to the story than the summary. This confuses me.

Re:*mods article -1, Flamebait* (1)

Hognoxious (631665) | more than 5 years ago | (#27315967)

It did. Are you one of those fossilised old farts who insists on using a remote control as a remote control? [slashdot.org]

Old vs. New Simple DB's (2, Funny)

billstewart (78916) | more than 5 years ago | (#27316469)

Wayner's usually a good writer, and did some good theoretical-computer-science work back in the day, but this article was too short to answer the questions he asks at the beginning, and he mostly highlighted the new shiny things from big ASPs, which is generally what Infoworld wants.

I'm particularly disappointed that while he referred to the name and history of Berkeley DB, aka Sleepycat, aka Oracle Renamed-foo, he didn't actually talk about using it. (OTOH, Infoworld did review one version of it in 2005 [infoworld.com] .) I no longer have my 4.1BSD manual on the shelf, but it was useful if you wanted something faster than using grep/sed/awk/look on tab-separated text files (which were the canonical Unix database format, and what I normally used for databases.)

These days if I want a lightweight database, I usually just put build tables in Excel, and then bitch about how it doesn't have a join or even decent text-editing and filtering capabilities, and occasionally have to save it as a CSV file and install vim on Yet Another Work-owned Windows box so I can get some bloody work done. I supposed if Excel did have a join function there'd be fewer people buying MS Access...

Re:*mods article -1, Flamebait* (1)

teknopurge (199509) | more than 5 years ago | (#27316631)

Cloud Zealots(google employees, amazon EC2 developers, Wall St. analysts that are trying to attach their name to the "Next Big Thing", etc.) tend to have an insulting tone to their rhetoric.

Normalization doesn't exist to save disk space (5, Insightful)

qoncept (599709) | more than 5 years ago | (#27315613)

Now that disk space is so cheap and many of the data models don't benefit as much from normalization, ...

You don't want to store the same data in multiple places. Your query might run faster, but your data integrity is going to suck.

And, uh, I have the pleasure of working now with a huge data warehouse that hasn't normalized status codes, so instead of quickly searching for an integer, the queries run slow as hell scanning char fields. It's not good.

Re:Normalization doesn't exist to save disk space (1)

TheSpoom (715771) | more than 5 years ago | (#27315647)

Couldn't you index a char just as easily as you could an int? Or are you saying their status codes are strings?

Re:Normalization doesn't exist to save disk space (2, Informative)

qoncept (599709) | more than 5 years ago | (#27315719)

"CHAR(50)"

Oracle doesn't have a "string" datatype.

Re:Normalization doesn't exist to save disk space (2, Informative)

TheSpoom (715771) | more than 5 years ago | (#27315801)

Ah, my apologies. Really, it should be an indexed enum (or whatever Oracle equivalent there is... it's been a while since I used it) if there's no additional data to go along with the status code... or another table if there is additional data.

Re:Normalization doesn't exist to save disk space (2, Interesting)

qoncept (599709) | more than 5 years ago | (#27315867)

My point exactly. :) There are a lot of things are data warehouse should be and it's not. We're working on redesigning it now though so we should be resolving a lot of the issues. But most people aren't just about to redesign their databases because it's a huge deal. We have 8 different apps using the warehouse, hundreds of reports and people hitting it we don't even know about that will all be obsolete. The cost to redesign is huge, and we only have the opportunity now because a project it is dependent on is being redesigned.

Re:Normalization doesn't exist to save disk space (1)

prozaker (1261190) | more than 5 years ago | (#27315695)

i once had a project once that wasn't normalized and on top of that was written in access, and vba code for forms. high level shit.

Re:Normalization doesn't exist to save disk space (5, Funny)

Hognoxious (631665) | more than 5 years ago | (#27316063)

You don't want to store the same data in multiple places.

But if one of them is wrong, you can check the others and correct it.

My boss - a lead senior senior lead developer from Android Whorehouse & Douche - several years back, when I tried explaining "why I'd missed some fields out of one of the tables".

Re:Normalization doesn't exist to save disk space (4, Funny)

mooingyak (720677) | more than 5 years ago | (#27316165)

But if one of them is wrong, you can check the others and correct it.

My boss - a lead senior senior lead developer from Android Whorehouse & Douche - several years back, when I tried explaining "why I'd missed some fields out of one of the tables".

I was about to post something explaining to you why that's bad, and then I reread your post and the whooshing noise around me quieted down.

Re:Normalization doesn't exist to save disk space (1)

Hognoxious (631665) | more than 5 years ago | (#27316343)

To be fair, I should have quoted the first bit. Maybe that's why I never made it to "lead senior senior lead developer".

Re:Normalization doesn't exist to save disk space (4, Informative)

qoncept (599709) | more than 5 years ago | (#27316379)

Right, and, boss, which one is right?

People that haven't done it don't realize how easy it is to end up in that situation. Say, I write reports about people, and Robin writes reports about assets, whose owners are people, and puts a person's name in her table to make it faster. Someone gets married, their name changes, and now Robin's reports are wrong.

Re:Normalization doesn't exist to save disk space (1)

Abcd1234 (188840) | more than 5 years ago | (#27316293)

And, uh, I have the pleasure of working now with a huge data warehouse that hasn't normalized status codes, so instead of quickly searching for an integer, the queries run slow as hell scanning char fields. It's not good.

What the hell does that have to do with schema normalization? Normalization has to do with how you architect your tables and relations. The types you use for the columns, and how you standardize their values, is an entirely different, though somewhat related, discussion.

And as it happens, in a data warehouse application, it's not at all unusual to denormalize tables, in order to speed up query performance during data mining operations.

Re:Normalization doesn't exist to save disk space (1)

Hognoxious (631665) | more than 5 years ago | (#27316441)

He probably means standardised rather than normalised, but I'm guessing it's pretty hard doing joins where the common field has different values meaning the same thing, or same values meaning different things.

Re:Normalization doesn't exist to save disk space (1)

Abcd1234 (188840) | more than 5 years ago | (#27316509)

If that's true, then his comments have nothing to do with the quoted section about how, "Now that disk space is so cheap and many of the data models don't benefit as much from normalization, ...". Because that comment is *specifically* about schema normalization in the formal sense.

Re:Normalization doesn't exist to save disk space (1)

rackserverdeals (1503561) | more than 5 years ago | (#27316773)

You don't want to store the same data in multiple places. Your query might run faster, but your data integrity is going to suck.

I wonder if normalization is going to be less important as de-duplication [sun.com] gets integrated into more file systems.

It's going to be a part of ZFS later this year I think. It looks like it's going to be a block level implementation within ZFS rather than file level. Since ZFS uses a copy on write model it seems fairly easy for them to implement compared to other file systems.

Databases that work on top of existing filesystems could benefit from this whereas databases that use block level addressing (Oracle) may not, but they might be dealing with this in other ways.

It kinda makes sense. Build your DB for the best performance and let the underlying file system handle normalization.

I don't know enough about the tech to determine if this would be an applicable usage but I don't see why not.

Mod this down (5, Funny)

Anonymous Coward | more than 5 years ago | (#27315615)

Like the article says, "The world won't end if some snarky, anonymous comment on Slashdot disappears."

moral of the story (1)

Em Emalb (452530) | more than 5 years ago | (#27315621)

even technical folks sometimes don't like change.

Also, calling people who've worked on DBs for a long time codgers and younger DBAs "twerps" is stupid.

Haha, yeah, tongue in cheek. I get it.

Still lame.

Now get off my patch of closely-cropped ground cover, you callous jerks. ;D

Re:moral of the story (1, Interesting)

Anonymous Coward | more than 5 years ago | (#27315951)

I think the question is "to what end will this change benefit anyone".

I view these new "ideas" on data storage and retrieval as a dumbing down of DBs the way higher level languages have dumbed down programming. On the one hand, it's much, much nicer to be able to whip together a working PHP app in a day than it is to have to constantly comb C code to make sure every little exception has been handled and every little bit of data checked. On the other hand, I don't feel quite the same way as I'm constantly zapping little bugs introduced by laziness in type checking or data validation.

It's all well and good that this allows people with less knowledge of the field to put together "good enough" applications, but sometimes I wonder if we're really that much more productive, or if we just shifted all our workload from building stable apps to constantly maintaining buggy ones.

Re:moral of the story (1)

Ninnle Labs, LLC (1486095) | more than 5 years ago | (#27316279)

I view these new "ideas" on data storage and retrieval as a dumbing down of DBs the way higher level languages have dumbed down programming.

Yeah, seriously. All you noobs who don't write in machine code should get the fuck off my lawn.

Re:moral of the story (0)

Anonymous Coward | more than 5 years ago | (#27316723)

Was that a joke or did you knot your panties up by not bothering to read beyond the first sentence?

I can't tell with this place sometimes...

Re:moral of the story (1)

Mr. Slippery (47854) | more than 5 years ago | (#27316261)

even technical folks sometimes don't like change.

There are two types of fool. One who says, "this is old and therefore good", and one who says "this is new and therefore better".

Change for its own sake is great when you're dealing with your air or water supplies, but is a lousy problem-solving strategy. These "slacker DBs" don t seem to introduce anything new. They're keyed value storage, dbm [wikipedia.org] with glitter and racing stripes glued on.

Laziness Rules (5, Insightful)

ergo98 (9391) | more than 5 years ago | (#27315651)

Slacker DBs like CouchDB and SimpleDB, have taken off for the simple reason that most developers have absolutely mediocre database knowledge or skills, and rather than learning it's just as easy to just wave it all off as obsolete.

It's no surprise that the creator of CouchDB, for instance, hadn't a clue about databases when he began his project. All of that built up knowledge just ignored while someone invented their own, and it's as rational as rolling your own encryption from scratch without the slightest clue about encryption algorithms or theories.

Re:Laziness Rules (3, Interesting)

Samschnooks (1415697) | more than 5 years ago | (#27315833)

... and rather than learning it's just as easy to just wave it all off as obsolete.

I don't know about that. But maybe these slacker DBs are perfect for what they're doing? Glancing at the those mentioned in the FA, it just looks like their simple tools to do simple things.

Don't get me wrong. I once had the pleasure of working with an Oracle god. This dude was about to take his final Oracle exam in a series of exams and he turned my Join that took ten seconds into a Join that took less than a thousandth. I have no idea what he did to this day, but it took several lines of PL/SQL. We were dealing with tens of millions of rows that had to be processed every night.

My point is if it's something simple to do, why all the RDBM overhead? Many times, just a simple flatfile is all you need and maybe a little more.

Re:Laziness Rules (3, Interesting)

KagatoLNX (141673) | more than 5 years ago | (#27315973)

In the end, the problem is that people just want a "default tool". They don't want to think about their requirements for data consistency. The really scary bit is that while RDBMses are the "default tool" of yesterday and slacker DBs are the "default tool" of tomorrow, neither of them are really the "problem".

The "default tool" attitude IS the problem. Unless you carefully weigh your data consistency requirements, you shouldn't be making that call at all.

I welcome the slackers and all of their new options along the spectrum of speed versus consistency. It's just that most of the people developing applications scare the shit out of me. They're so cavalier (or should I say, "agile", or maybe "pragmatic") about requirements that it's truly disturbing.

That said, if you're really interested in all of the options, I also recommend checking out memcachedb, memcacheq, and redis.

Re:Laziness Rules (2, Insightful)

phoenix321 (734987) | more than 5 years ago | (#27315991)

Problem is, you're re-inventing the wheel several times over in the process. Hint: "a flatfile and maybe a little more" could very well be all the storage technology invented today only a few years down the road.

At first, all you need is to store key:value pairs. That works with a flat file or with Oracle. Then you need some consistency checks, which are can be modelled fast in Oracle or reasonably fast in your software. Then you need some triggers, which could be written fast in Oracle and not-so fast in your software. And so on until you have progressed through the whole platform effect with several squeaky wheels invented and thousands of hours wasted.

Any project worth doing that involves storing key:value pairs is worth a real database. Take the tiniest, lowliest member of the crowd as long as it can somehow speak SQL and allows to be linked and unlinked into the project. Everything else will require at least a medium rewrite at some point when you switch over to a real database. You could of course extend everything upon a glorified flatfile until your reinvented wheels strangles all your progress.

Re:Laziness Rules (4, Insightful)

Ambiguous Puzuma (1134017) | more than 5 years ago | (#27316161)

If you want "a little more" than a simple flat file, perhaps SQLite [sqlite.org] is the answer? The people on the Firefox team seem to think so, for example.

SQLite has been a pleasure to use for a small personal project involving a few Perl scripts. Granted my background is with SQL Server and Oracle, so perhaps I'm not the target audience, but I found it extremely easy to use and surprisingly efficient--and I didn't need to set up a server or anything. I didn't even need to explicitly create a database!

Re:Laziness Rules (1)

jellomizer (103300) | more than 5 years ago | (#27316187)

For the most part the Overhead of running a Real DB is usually made up over time.

Small Apps tend to grow to big ones over time. Having Babbie Databases can become a stubmling block to your application. As well as for the organization. They may want to wharehouse your application data for making better business decisions and integration across apps. So you little 1 million record database will need to be integrated into a billion/trillion record database.

Re:Laziness Rules (1)

AkiraRoberts (1097025) | more than 5 years ago | (#27316381)

Because many times, you start off needing a simple flat file and then you need a little more and a little more and a little more and suddenly that simple flatfile is the foundation of a steaming mound of crap and your life is horrible. I'd rather think things through a bit more at the start. Yes, sometimes that simple flat file might do the trick. But sometimes you might want to put together something that, right in the moment, will seem like a bit much and then will prove, 2 years from now, exactly what was needed.

Re:Laziness Rules (1)

samkass (174571) | more than 5 years ago | (#27316855)

The fundamental problem of enforced consistency with our system is that it requires hard locking, and that is at odds with distributed scalability. When you're going over a few satellite hops and you've got users real-time deep collaborating (ie. they're all heavy writers) with >2s latency, a traditional DB isn't going to scale very well. Even the Facebook MySQL+memcached is going to break down in that environment.

Re:Laziness Rules (4, Informative)

metalhed77 (250273) | more than 5 years ago | (#27316003)

Damien Katz, CouchDB's creator, worked at MySQL prior to writing CouchDB, and worked on Lotus Notes prior to that...

Re:Laziness Rules (3, Interesting)

ergo98 (9391) | more than 5 years ago | (#27316149)

I'm just going on the statements he made about his own (lack of) knowledge in this video [infoq.com] .

Re:Laziness Rules (5, Funny)

Anonymous Coward | more than 5 years ago | (#27316275)

Thanks for validating the OP comments....

Re:Laziness Rules (4, Insightful)

diamondsw (685967) | more than 5 years ago | (#27316365)

>Damien Katz, CouchDB's creator ... worked on Lotus Notes prior to that...

That's not exactly a ringing endorsement.

Re:Laziness Rules (1)

morgan_greywolf (835522) | more than 5 years ago | (#27316383)

Well, as ergo98 said, he hadn't a clue about databases when he began his project.

*ducking*

Re:Laziness Rules (0)

Anonymous Coward | more than 5 years ago | (#27316549)

MySQL. Lotus Notes. I rest my case.

Re:Laziness Rules (2, Informative)

Anonymous Coward | more than 5 years ago | (#27316781)

Damien Katz, CouchDB's creator, worked at MySQL prior to writing CouchDB, and worked on Lotus Notes prior to that...

He started work on CouchDB in 2005. Prior to that he was a Notes grunt of little significance.

He started at MySQL in 2007.

The point holds.

Worse is better (1)

oGMo (379) | more than 5 years ago | (#27316227)

...especially when you don't know what "better" is and you're too lazy to learn: unwillingness to learn is stupidity. Like the quote says, ignorance is curable; stupidity is terminal.

People who use these things and think they're great and that they're doing amazing things don't realize that the time they're taking and the problems they're struggling with are long-solved trivialities. Nothing new. Nothing cool. It's like someone struggling with a bunch of complicated excel formulas to make their spreadsheet do something that you could do in a few lines of your favorite scripting language.

Unfortunately this is something that afflicts most of the industry these days, and we end up thinking half-assed pieces of crap are cool just because you see them in your browser.

Re:Laziness Rules (2, Informative)

sl0ppy (454532) | more than 5 years ago | (#27316321)

first some context. i architect data warehouses for a living. i also live in a world of building fairly specialized frameworks to deal with data warehouses architected as star and snowflake schemas. i tend spend quite a lot of time in pseudo-relational databases [wikipedia.org] that don't fully implement codd's rules [wikipedia.org] .

for fun, i like to spend some time toying with couchdb, using it for loose data warehousing, extending it, and generally enjoying the application development freedom it gives me.

that said, let me respond to some of your points:

Slacker DBs like CouchDB and SimpleDB, have taken off for the simple reason that most developers have absolutely mediocre database knowledge or skills, and rather than learning it's just as easy to just wave it all off as obsolete.

map/reduce solves a specific problem in data warehousing - column based lookups given specific rules, able to be broken down into atomics and performed in massive parallel. this allows for very cheap horizontal scaling over a large dataset.

It's no surprise that the creator of CouchDB, for instance, hadn't a clue about databases when he began his project.

this just shows ignorance. even just a cursory scan of damien's resume [209.85.173.132] says otherwise.

Re:Laziness Rules (1)

ergo98 (9391) | more than 5 years ago | (#27316685)

map/reduce solves a specific problem in data warehousing - column based lookups given specific rules, able to be broken down into atomics and performed in massive parallel. this allows for very cheap horizontal scaling over a large dataset.

That's great. No one said they had no use. Do you think you're refuting something? You aren't. Linking to Wikipedia doesn't really make you authoritative, as an aside.

this just shows ignorance. even just a cursory scan of damien's resume [209.85.173.132] says otherwise.

Ignorance indeed. Again, as stated otherwise, I was basically quoting from he himself said in a presentation that he gave. But to humor your point, both a cursory and a intensive look at that resume gives me zero hints that he has any credible database knowledge prior to CouchDB. Do you mind guiding where it says otherwise?

Re:Laziness Rules (1)

sl0ppy (454532) | more than 5 years ago | (#27316775)

Linking to Wikipedia doesn't really make you authoritative, as an aside.

no, its puts things in context, and helps to make sure that we are using common terminology via definition.

anyways, your attempts at flamebait won't draw me in - you've already shown in this, and previous comments that you're just out looking for a fight.

if you were willing to learn about something, and not just make inflammatory comments, that would be something else entirely. i feel kind of sorry for you.

Re:Laziness Rules (1)

ergo98 (9391) | more than 5 years ago | (#27316833)

anyways, your attempts at flamebait won't draw me in - you've already shown in this, and previous comments that you're just out looking for a fight.

You added some noise dressed up with faux authoritarianism, and you consider being called on it a demand for a fight?

Hardly.

if you were willing to learn about something, and not just make inflammatory comments, that would be something else entirely. i feel kind of sorry for you.

Every word of your two posts have been pure troll. I enjoy trolls (it's a weakness), so yes, I do respond.

Tell me again what in Katz' resume is so impressive that my absolutely true statement, which was derived from the lips of Katz himself, is "ignorance"?

Re:Laziness Rules (1)

0xdeadbeef (28836) | more than 5 years ago | (#27316401)

It's no surprise that the creator of CouchDB, for instance, hadn't a clue about databases when he began his project. All of that built up knowledge just ignored while someone invented their own, and it's as rational as rolling your own encryption from scratch without the slightest clue about encryption algorithms or theories.

It's funny how some people react by attributing ignorance to others when confronted with things they themselves don't understand.

Re:Laziness Rules (1)

0xdeadbeef (28836) | more than 5 years ago | (#27316693)

Nevermind, I watched part of that video. Yikes!

But that underscores the validity of these schema-less database systems. Due to the technical depth required to use relational databases effectively, many people are better off not using them at all. They will get farther faster and with fewer bugs with something that is flexible and forgiving.

Well, it's like... (3, Funny)

oldhack (1037484) | more than 5 years ago | (#27315657)

Either is cool with me, as long they are cool and takes care of business, you know what I am saying?

It's all good.

Old guard, new guard, right guard... who cares? (1, Insightful)

Anonymous Coward | more than 5 years ago | (#27315667)

Why the need to make it 'old guard' vs 'new guard'... seems like flamebait for fanboys.
"tastes great" vs "less filling", or just explain the merits of both and leave it there.
It's like forum kiddies arguing raid 5 vs raid 1+0.

Yes, if the database is important, you want the most CAREFUL management available. Obviously.
But if these -db apps work fine, and your data isn't corporate mission critical, who cares?

Seems to me convenience and interoperability score higher for most small datasets, am I wrong?

DBs they may be ... (1)

Anonymous Coward | more than 5 years ago | (#27315715)

... it's when they get referred to with "relational" or "management system". DB fine. RDBMS they are not.

Cartesian products are GONE!!! YAYYYYY...... (1)

bodland (522967) | more than 5 years ago | (#27315723)

What is a Cartesian?....is that the water in Olympia Beer?

Re:Cartesian products are GONE!!! YAYYYYY...... (1)

KagatoLNX (141673) | more than 5 years ago | (#27316015)

Actually, cartesian products and joins aren't gone. It turns out that they just end up being done client side.

It's the tragedy of "join-less" databases. Joins do something that you need. The lack of joins forces people to correctly normalize, which ironically they should have been doing anyway. It doesn't take away any genuine need to join though. :(

Re:Cartesian products are GONE!!! YAYYYYY...... (1)

aquatone282 (905179) | more than 5 years ago | (#27316641)

What is a Cartesian?....is that the water in Olympia Beer?

That wasn't water the Cartesians put the beer. . .

Ask anyone who ever drank Oly and they'll tell you what it was.

Signed,

A former Olympia resident.

a base of data (4, Insightful)

poot_rootbeer (188613) | more than 5 years ago | (#27315743)

"tools that tack the letters 'db' onto a 'pile of code that breaks with the traditional relational model"'

If "database" were intended to mean only "relational database", we wouldn't have had any need for the latter term...

who needs transactions? (3, Insightful)

alen (225700) | more than 5 years ago | (#27315781)

the article is right that in some cases it doesn't matter if a transaction is lost. but in any case where money is involved it's a must. you can't just start a fund from your Oracle or SQL Server savings to pay for mistakes because it will kill your brand and you may lose a lot of future business. and any savings will be eaten up by the extra cost to hire people to solve all the data problems

i've seen this. no constraints on the data that is orginally put in, not enough referential integrity and you get customers opening up a lot of trouble tickets and you end up hiring people to clean up the data every time a mistake is found

distributed databases and P2P (4, Informative)

thanasakis (225405) | more than 5 years ago | (#27315851)

The problem of distributed consistency has kept researchers occupied for quite a while. For example, see project Scalaris [onscale.de] . They are using a distributed hash table to distribute data among many nodes. This should be relatively easy, at least once you have a good hashing function on your hands. But a lot of research has been done on P2P networks during the last decade, so there is quite a lot of stuff to read and take ideas from.
The interesting part is that it can maintain consistency and support ACID properties. From the site it appears that they accomplish that by using a modified Paxos Algorithm [wikipedia.org] which basically is a way to maintain consensus among many different peers in a non-Byzantine system (this means that there are no malevolent peers in the system -- peers can break down and cease working but not sabotage the system). Leslie Lamport [lamport.org] of Microsoft Research has done a lot of work on this, anyone interested may take a look at his papers, very advanced stuff there.

Who? (0)

Jaysyn (203771) | more than 5 years ago | (#27315911)

Aside from Google, I've never even heard of those "upstarts".

Re:Who? (1)

DigitalSorceress (156609) | more than 5 years ago | (#27316373)

... and from the description, the Google solution reminds me a LOT of Zope.

For those who don't know, Zope is a CMS written in Python and the database is pretty much a really big object. (In fact, I think it's technically an OODBMS instead of an RDBMS, but I could just be talking out my arse.

Zope doesn't have a learning curve, it has a learning butte, but once you scratch and claw your way to the summit, the view is kinda nice... It's just that when you get there, you can't help but to notice that there's a whole tour bus of geriatric patients that got a nice air conditioned ride to the same spot on the SQL express.

I think I had a point in there somewhere.

Oh yeah, I remember now... GET OFF MY LAWN!

MySQL not listed as toy ? (1)

nicolas.kassis (875270) | more than 5 years ago | (#27315923)

Seriously any Old Guard DBA will put MySQL in the toy category.

Re:MySQL not listed as toy ? (3, Insightful)

dacut (243842) | more than 5 years ago | (#27316257)

MySQL strives to provide RDBMS and ACID semantics, though its quality of service (QoS) may fall short. By contrast, these "slacker" databases don't even try to support RDBMS or ACID; even if they operated perfectly, they won't provide RDBMS/ACID.

I work for one of the companies in question (no, I don't speak for them). We rely heavily on a combination of these "slacker" dbs, Berkeley dbs, memcached, Oracle, flat files, and tape backups. Each fills a niche. I wish these articles would quit trying to create a false dichotomy.

Re:MySQL not listed as toy ? (0)

Anonymous Coward | more than 5 years ago | (#27316635)

At least every "Old Guard DBA" will be relieved to know that you are here to speak in their name.

List of non-relation DBs (1)

captainclever (568610) | more than 5 years ago | (#27315959)

I wrote an article about non-relation databases, and there were some interesting comments about the various tradeoffs etc: http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/ [metabrew.com]

Re:List of non-relation DBs (0)

Anonymous Coward | more than 5 years ago | (#27316397)

I wrote an article about non-relation databases [....]

Yeah, I wrote one, too. But I stored it in my non-relational DB and now it's gone.

*sigh*

You young whippersnappers don't know nothing! (4, Interesting)

www.sorehands.com (142825) | more than 5 years ago | (#27315961)

Relational DB? People forget Network Model Databases (http://en.wikipedia.org/wiki/Network_model) and flat databases.

Network model databases will outperform relational all the time. You just don't have the same flexibility.

Newer models are not based on the design or performance issue, but the distribution of the data. These are not invalid reasons, but the old issues still apply.

I have had arguments with people who consider PC programming different from mainframe. The same rules apply. The difference is that many PC programmers are just sloppier. When you have cheap CPU and memory, people don't analyze and optimize as much.

I've never understood the UNIX world's fascination (5, Informative)

Richard Steiner (1585) | more than 5 years ago | (#27315971)

I've never understood the UNIX world's fascination with relational databases.

Speaking as a programmer in mainframe online transaction environments for the past 20+ years, I've become very familiar with very fast and simple database systems like the "freespace" files we use on the Unisys mainframe platform.

We don't need relations for real-time processing. Most programs just need a place to keep data, and a simple key to retrieve that data. Some efficiency in disk usage is nice, but the primary design factor is performance.

A freespace file is a collection of pre-allocated fixed-length records of various sizes (e.g. 256 bytes, 512 bytes, 1024 bytes, 2048 bytes, 4096 bytes, and 8192 bytes). Each record size is a assigned a type number (e.g., 1 through 6 in the above case), and a given file is created and pre-allocated with a mix of various records depending on the usage pater for that particular file. If you know all you need is tiny records, create a file containing a few hundred or thousand type 1 and maybe 2 records.

Records not allocated are filled with a deallocated fill pattern.

A program uses a record by performing a Write New operation. That tells the database manager to find a record in that file closest and >= to the size required, stick the presented buffer in the record, save it, and return a key to that record to the calling program. Typical key format is where Record Number is a number from 1 ... n. If your file has 1000 Type 3 records, it'd be from 1...1000 or 0...999.

To read a record, use a key from a previous Write New (stored away somewhere), perhaps in another file) to read that record from a file. Length is not required.

Programs use a very simple read-and-lock mechanism when modifying existing records. If one program has a record locked, another program must wait. Not a problem with intelligent coding.

We've used this system in airline systems for 40+ years. It works well. Sometimes an environment has robust commit and rollback/recovery features to allow for an entire series of changes to be rolled back on error, sometimes not. It doesn't seem to matter that much, especially for transient data like weather, flight schedule data, etc.

I would LOVE to see a freespace database ported to Solaris, personally. We'd use it heavily. :-)

Re:I've never understood the UNIX world's fascinat (0)

Anonymous Coward | more than 5 years ago | (#27316067)

What does this have to do with UNIX? Relational DBs were invented in the mainframe world.

Re:I've never understood the UNIX world's fascinat (1)

Richard Steiner (1585) | more than 5 years ago | (#27316349)

In my experience, most UNIX programmers tend to assume a relational database for almost everything if it isn't a vanilla flat file. That includes programmers in realtime applications, C people, Java people, etc. I can't begin to tell you how many applications I've seen written to use Oracle, Sybase, etc., just to store a simple static table of information. There's no POINT to that!

Most mainframe environments, on the other hand, have many established options, and relational is usually only considered if you actually need that type of functionality. Normally, systems are written with a mix of different database types. Could be flat files, could be freespace, could be RDMS, or could (in our case) be DMS, a network database with some types of set-linking properties but not really table-based.

Again in my experience, working in the airline industry, I've seen a bias towards RDMS for enterprise applications that I simply haven't seen on the mainframe (in my case Unisys transaction processing) side of life.

A web site is a tranctions processing facility. Just replace fancy 3270 or Uniscope screens with HTML. Same idea, forms, etc. It wants in and out, fast. Why use something not really made for that?

Re:I've never understood the UNIX world's fascinat (1)

Richard Steiner (1585) | more than 5 years ago | (#27316077)

Oops. Forgot that brackets get eaten. Typical record format is RECORDTYPE/FILENUMBER/RECORDNUMBER. The first Type 1 record for File 100 might look like 01-0127-0001 or whatever (specific binary representation in hex or octal would obviously vary depending on implementation and preference).

In our case, it's a 36-bit word shown as 12 octal digits, probably not a popular choice with UNIX folks. :-)

Re:I've never understood the UNIX world's fascinat (1)

Richard Steiner (1585) | more than 5 years ago | (#27316215)

01-0127-0001 is the first type 1 record for file 127. 01-0100-0001 would be the first for file 100. That's what I get for doing patchwork re-editing an existing message before sending it...

Re:I've never understood the UNIX world's fascinat (4, Insightful)

dcowart (13321) | more than 5 years ago | (#27316573)

How does it work for searching though? If I just have my "freespace" file and my pointers to records, does a search for some piece of user requested data have to hit every record or is there a hash somewhere for the data contained in the record? You don't mention it in your description.

It seems that the biggest advantage to a relational DB is that the syntax for accessing it is well known, SQL. It has a human read-able interface and while sometimes whonky to work with for complex operations, it provides the simplest cross-platform way to access data. I don't need to know which data blocks hold the data, I just ask the database for them "SELECT slashdotid, name FROM users where slashdotid 20000"... and I get rows of data.

Could I just read it from a file? Yes. Would it be simpler? Maybe. But what if I have 200001 records, then I have to do some magic sorting in my program, and I have to manage memory for them, and disk space, etc. It is simpler to let the DB handle that mess and I just ask for the data I need.

It breaks up the process of programming into data storage and data manipulation/presentation. DB's for storage, my bad python for manipulation and presentation.

--Donald

Re:I've never understood the UNIX world's fascinat (1)

dcowart (13321) | more than 5 years ago | (#27316627)

In Re: to my Re:, I like sqlite for simple DB applications, I get DB functionality with a very low overhead. Otherwise I use postgresql.

I have used Oracle and some others before now, but those are my two current DB's (sql-engines?) of choice.

Re:I've never understood the UNIX world's fascinat (0)

Anonymous Coward | more than 5 years ago | (#27316619)

I would LOVE to see a freespace database ported to Solaris

Sound like you might want to check out the "Berkely Database". It is very fast and has been in wide use for many years.

"http://en.wikipedia.org/wiki/Berkeley_DB

Berkeley_DB is also one of the unlaying data store methods that MySQL can use. I think MySQL can use either Barkely or a raw file system.

Going way back IBM had a system like this call ISAM. All of these are very simple

Re:I've never understood the UNIX world's fascinat (1, Insightful)

Anonymous Coward | more than 5 years ago | (#27316735)

Maybe the fascination with relational databases is that you can easily work with the data in there.

What you describe just sounds like a file system. A specialized one, but it doesn't really support more than a filesystem does. Everything works fine if you have the key to the data. You can read the data, do your stuff, and update the data. But what if your problem it to find the key? Like you want to know which orders are overdue? Doesn't sound like the freespace file will help me there. Sounds like I have to implement the whole searching by myself.

When I am searching for a database solution then probably because I really need that searching and I want it to be fast, and I don't want to do it myself.

What you suggest doesn't sound like a database. It sounds more like an allocation scheme any database could use under the hood. What you suggest may suffice if my requirement is a high performance filesystem. But I don't see how it supports even the most basic database operations. I don't say your solution is bad. It just doesn't solve the same problem as a database.

Re:I've never understood the UNIX world's fascinat (2, Insightful)

LWATCDR (28044) | more than 5 years ago | (#27316827)

Okay how do you find the data without a record number? I can see the value of the system but it also seems very inflexable.
I do agree that way to many programmer use MySQL for a file system, flat files, configs, and goodness knows what else.
 

When is a database not a database? (1)

MonolithicX (656642) | more than 5 years ago | (#27316035)

I think the question here isn't New DB or Old DB but when do you stop considering any data store a database? There are plenty of ways to write data to disk fast as hell but God help you if you want to do anything with the data later. I see these as specialty data stores - get the data in fast and then batch it out to your "old school" relational database to perform analytics on it later. Relationally Yours, MonoX.

Harsh? (2, Insightful)

Bobb Sledd (307434) | more than 5 years ago | (#27316115)

I'm a DB admin, and I use things that aren't toys; but what I've heard here is kinda harsh.

Look, it's all about "right tool for the right job." Why do you need a nuclear-powered drill that can make a tunnel from here to China, when really all you needed was a shovel?

For most daily projects that have small amounts of data, they may be using something like Crystal Reports or Excel or SPSS that just does all the number-crunching client-side anyway. You don't always need Oracle or [favorite DB flavor] for that.

Re:Harsh? (1)

mattygabe (1457601) | more than 5 years ago | (#27316719)

I'm a DB admin, and I use things that aren't toys; but what I've heard here is kinda harsh.

Look, it's all about "right tool for the right job." Why do you need a nuclear-powered drill that can make a tunnel from here to China, when really all you needed was a shovel?

For most daily projects that have small amounts of data, they may be using something like Crystal Reports or Excel or SPSS that just does all the number-crunching client-side anyway. You don't always need Oracle or [favorite DB flavor] for that.

What?! We shouldn't suggest that our company buy the biggest, baddest, best-performing supermachine just based on the cost efficiency that it can achieved if pushed to it's optimum limit? Even if we only need a shovel? You're no fun!

I feel old (2, Informative)

a2wflc (705508) | more than 5 years ago | (#27316167)

When I saw the title I thought "I'm old-guard". Then I read the article and JOINs are a key concept to the old-guard.

My first few DB apps involved using a b-tree or ISAM library (or writing our own). Then the "new guys" started wanting to pay for a server that did JOINs. We did JOINs, just at the app layer and without the guaranteed consitency that a good relational design gives you. And getting a server that does it was expensive.

I wouldn't want to go back to pre-relational server days, but am also very thankful that I did write my own DBs from the ground up. I will probably never need to use the entire experience, but can often use bits and pieces of it, and I appreciate a good key/value store.

Re:I feel old (3, Funny)

nextekcarl (1402899) | more than 5 years ago | (#27316377)

I was listening to the radio (didn't pay attention the the station it was on) one day and generally liking the music I was listening to on it. Then the station ID came across between songs. It was the "oldies" station. I suddenly felt like I needed a cane (or perhaps a walker). Why does that happen? And is it going to happen every 10 years or so? I don't think I can take too many more of those moments.

Berkeley DB is awesome (4, Interesting)

IGnatius T Foobar (4328) | more than 5 years ago | (#27316263)

I can't believe there hasn't been any mention of Berkeley DB [oracle.com] yet. Guess what, folks: sometimes you just don't need the features of a full relational database. Sometimes all you need is fast, robust, reliable storage of indexed key/value pairs.

I can attest that Berkeley DB does exactly that, and does it really, really well. We use Berkeley DB for all of the data storage in the Citadel [citadel.org] system, including the mailboxes themselves. Some sites have tens of gigabytes or even hundreds of gigabytes of data, and Berkeley DB just keeps chugging along, happily and reliably doing its thing. Our biggest problem? People who point at it and say "storing email in a database is unreliable" because they know it constantly explodes when Exchange does it. Well guess what, folks: Berkeley DB ain't the Exchange database (actually, maybe Exchange wouldn't be so unreliable if they switched to Berkeley DB).

Eschewing the full set of RDBMS features isn't slacking. It's choosing the right tool for the job.

"eventual consistency"? WTF? (1)

wardk (3037) | more than 5 years ago | (#27316351)

yeah, who wants consistent data, that's for old guard type people

gee I deposited my paycheck three weeks ago and it's not in my account yet?

bank: don't worry, it will be there eventually.

It's about the foundation (1)

fluffernutter (1411889) | more than 5 years ago | (#27316363)

I dunno.. You know what.. I am all about using the tool that gets the job done but if I'm spending the time to develop something I'm not going to take a chance. If I'm going to build a house am I going to build on a cracked foundation because it's convenient and cheap. I'm going to spend money on a sound foundation. If I am going to go into months of development I'm going to use something fundamentally sound like a relational. Besides you don't really need to know SQL inside and out anymore anyway. That's what ORMs are for.

The problem is growth... (1)

alexhmit01 (104757) | more than 5 years ago | (#27316637)

Non-normalized databases are fine, and might be faster, for small sites, but when things scale, the sloppy databases (or worse, sloppy frameworks like Ruby's Active Record) just cause problems.

A scalable, normalized database means consistent data, when you have multiple applications hitting it.

For a web forum, sure, a relational database may be the wrong tool, because all you care about is speed on new stuff, the archive can crawl, etc.

However, what happens when your web forum adds some actual data, and then a few years down the road, you need new tools to talk to that data? You can abstract everything through code, and post into your webserver and let Perl/PHP manage it, but then that's a new piece of legacy code to maintain.

I keep all my stuff in a PostgreSQL database, and build Schemas with Views for web apps, etc. So when a new piece of functionality is needed, it's kept segmented off. So you can Prototype a Ruby app, maintain a PHP Web App, and even build custom tools in VB or other environments that talks directly to the database for manipulation. The spreadsheet guys ALWAYS loved when I could setup an ODBC connection, and they could pull real time data into Excel, instead of needing to go through a web interface and grab CSV pulls. Hell, I had a simple Excel spreadsheet that went out to my PostgreSQL database, got the necessary data, prepped it (all in Excel), and then stuck the data into Quickbooks via the SDK (using VBA of all technologies) to prevent needing to double enter.

If you were on a real GL powered with DB2 or Oracle, you could do even fancier things.

RDBMS skills are a good thing to develop. The overhead is pretty minor for starting off, and it gives you great flexibility down the road.

Now, if you have a technology REASON to want a non-relational database, go nuts, new tech can do new things. But if it's a refusal to learn relational theory, pick up a good book and learn the mathematics behind it.

Alex

why I love non-relational databases. (-1)

Anonymous Coward | more than 5 years ago | (#27316643)

"Flat is better than nested". Screw you normalization.
scroll through my phone's phonebook and you'll see stuff like:
daycareSusan
familyDad
familyJacky
familyMom
Jack
Steve
moverBob
workTed
workBill
etc.

Flat rocks! Go ahead and normalize down to the letter of the alphabet the characters in the name consist of if you want, but my theory is that those who live by the sql query die by it.

Peter Wayner... (1)

RemoWilliams84 (1348761) | more than 5 years ago | (#27316649)

I mean seriously, really, did they make that up? I bet he caught a lot of crap growing up.

Not all databases have to be relational... (1)

shoppa (464619) | more than 5 years ago | (#27316659)

For the vast majority of web applications, the "key-value pair" class of databases work fine.

I think the real problem is that the "relational database weenies" look down on the key-value pair databases, and there are a lot of non-DB-weenies out there who like using true relational databases as nothing more than key-value pair. It degenerates into name calling, instead of getting the job done, pretty fast.

Ignore The Rules At Your Peril (2, Insightful)

Prototerm (762512) | more than 5 years ago | (#27316687)

You may have seen in the news recently how in the last decade or so Wall Street ignored some of the hard-won regulations and guidelines developed in the wake of the Great Depression.

We all know what happened as a result.

The same is true when dealing with data. You don't ignore the rules completely, or follow them only when you feel like it, or when you have time. As the old joke goes, Quality is *not* Job 1.1.

If the data isn't important enough to store correctly, then it's not important enough to be stored at all.

Just data structures (4, Insightful)

Thaelon (250687) | more than 5 years ago | (#27316699)

Databases at a very abstract level are just data structures. Choosing a relational database when you don't need that much functionality is just as wrong as choosing a flat file when you need a database.

Knowing the ins & outs of your data structures is still a vital skill of programming.

The problem is scaling (2, Insightful)

plopez (54068) | more than 5 years ago | (#27316755)

so you start a small project, "we just need a few hundred/thousand records, a few key value links and the occasional transaction". so you start with a slacker DB. A slacker DB far too often implies a slacker hack software d00d.

Then it grows. Instead of educating themselves (Q: what's the difference between those who can't read and those who don't? A: nothing. ) and finding a better DB solution they thrash around trying to hack in DB functions into their code.

So they lose consistency etc. Soon they have a polluted DB that breaks all the time. Often they are proud of the heroics of the wasted effort they put into it. A good programmer know how to be correct form of lazy: do not reinvent the wheel.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>