Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Moving From CouchDB To MySQL

Unknown Lamer posted more than 2 years ago | from the hep-cats-just-use-postgres dept.

Databases 283

itwbennett writes "Sauce Labs had outgrown CouchDB and too much unplanned downtime made them switch to MySQL. With 20-20 hindsight they wrote about their CouchDB experience. But Sauce certainly isn't the first organization to switch databases. Back in 2009, Till Klampaeckel wrote a series of blog posts about moving in the opposite direction — from MySQL to CouchDB. Klampaeckel said the decision was about 'using the right tool for the job.' But the real story may be that programmers are never satisfied with the tool they have." Of course, then they say things like: "We have a TEXT column on all our tables that holds JSON, which our model layer silently treats the same as real columns for most purposes. The idea is the same as Rails' ActiveRecord::Store. It’s not super well integrated with MySQL's feature set — MySQL can’t really operate on those JSON fields at all — but it’s still a great idea that gets us close to the joy of schemaless DBs."

Sorry! There are no comments related to the filter you selected.

Not getting RDMS (5, Insightful)

Anonymous Coward | more than 2 years ago | (#40016041)

And in another three years they will switch to whatever is the coolest up-and-coming storage solution. Incompetent developers will always be incompetent developers.

Re:Not getting RDMS (5, Insightful)

gbjbaanb (229885) | more than 2 years ago | (#40016533)

true, just reading their blog

Things like SQL injection attacks simply should not exist.

HTTP API. Being able to query the DB from anything that could speak HTTP (or run curl) was handy.

so sql injection is real bad, bad design of SQL... yet allowing any old HTTP javascript queries is somehow ok. Yes, incompetent developers indeed.

They also say

Why are we still querying our databases by constructing strings of code in a language most closely related to freaking COBOL, which after being constructed have to be parsed for every single query?

apart from the concepts of query caches - and stored procedures - so what if the language is related to COBOL, javascript is closely related to C which is almost as old. And that has plenty of relations to Algol which is even older.

So yes, it sounds like they havn't really got a clue. Great advert for their business!

Re:Not getting RDMS (-1, Redundant)

Bill, Shooter of Bul (629286) | more than 2 years ago | (#40016573)

To be fair: COBOL is a *bad* language that no one wants to use anymore. C is a good language that's just too difficult for some people's children, but everyone wants to use something that's kind of looks like it.

Not quite true (4, Informative)

Viol8 (599362) | more than 2 years ago | (#40016781)

If all your application is ever going to do is read and write to fixed sized record structured data with little relational (or any) attributes then COBOL will suit you fine as that's what it was designed for. Unfortunatly those sorts of apps are few and far between these days, but in its ever decreasing niche COBOL is still good.

Re:Not getting RDMS (2)

Lisias (447563) | more than 2 years ago | (#40017299)

COBOL can be a bad language, but the best paid jobs around here are for COBOL programmers.

It's hard to find a position (someone must die in order to open up a position), but once you get it, it's for life. =]

Re:Not getting RDMS (2)

arth1 (260657) | more than 2 years ago | (#40016847)

so sql injection is real bad, bad design of SQL... yet allowing any old HTTP javascript queries is somehow ok.

HTTP isn't a subset of javascript - no javascript queries are needed for HTTP. Even for JSON and other javascript objects.

That said, yes, the developers don't seem to "get it". An object/method based database query language, which they seem to want, has already been tried. Look where Informix is right now.

Yes, parsing can be a bitch, and which is why using a structured database isn't always the right choice to start with. If you're just using it for data storage, it rarely makes sense.

Re:Not getting RDMS (0)

Anonymous Coward | more than 2 years ago | (#40017001)

Javascript is not the query language. In CouchDB queries are REST calls; the query parameters are either passed via. HTTP headers or you can do an HTTP POST and pass them as a JSON document.

Javascript is used for Map/Reduce functions, but it's unlikely you'd ever dynamically generate a Map/Reduce and embed user input in it. That would sort of be like dynamically generating a stored procedure and executing it in an RDBMS.

Just in case anyone is wondering, I don't actually like CouchDB.

Re:Not getting RDMS (5, Insightful)

Xest (935314) | more than 2 years ago | (#40016881)

"Why are we still querying our databases by constructing strings of code in a language most closely related to freaking COBOL, which after being constructed have to be parsed for every single query?"

I couldn't agree with you more, this quote makes me want to vomit. Is this really how low the average competence of today's web developer has stooped? Between PHP developers not getting why PHP is a pretty shitly designed and developed language and stuff like this, I barely get how the web even runs anymore.

To answer the original quote, the reason we're "still querying our databases by constructing strings of code in a language most closely related to freaking COBOL, which after being constructed have to be parsed for every single query?" is because SQL is a language based on mathematically sound principles, and which is supported widely, and known widely, and is processed by database engines across the globe that have literally decades of stability behind them, data in them and so forth.

There's absolutely no reason to change SQL, because if you build a new query language that is based on the same mathematically sound principles of relational algebra then it will er... look just like SQL. The fact the kiddie (I can only assume he's a kiddie due to his blatant lack of knowledge and/or experience in the field) who wrote that blog post doesn't get this suggests he should absolutely not be trusted with your data as he'll only lose it.

This is a classic example of someone bitching about something not because it's bad, but because they simply don't understand it and believe that rather than learn about it properly, it's better to bitch and hope you can somehow effect change by bitching.

The advantage of most SQL/RDBMS is that they do adhere to the ACID principles, and for people who want to be able to have some degree of trust in their data source that's pretty fucking important. It's no surprise that they've moved over to MySQL though as it's one of the few RDBMS that is completely shit at adhering to the ACID principles and keeping uptodate with solid, stable implementations of modern database functionality.

Re:Not getting RDMS (0)

Anonymous Coward | more than 2 years ago | (#40017281)

Well stated!

Re:Not getting RDMS (1)

thisisfutile (2640809) | more than 2 years ago | (#40017351)

Well stated! (There, I'm not an "anonymous coward" anymore) ;-)

Re:Not getting RDMS (1)

speculatrix (678524) | more than 2 years ago | (#40017393)

mod parent up

Re:Not getting RDMS (4, Insightful)

serviscope_minor (664417) | more than 2 years ago | (#40017155)

so sql injection is real bad, bad design of SQL...

SQL injection actually has nothing to do with SQL.

Exactly the same attacks happen in any system where you build up a string from user data and pass it off to an interpreter. SQL has nothing to do with it.

Exactly the same thing used to happen with sudo shell scripts.

Exactly the same thing happened with javascript injection in very early webmail systems.

There are plenty of opportunities for code injection on poorly written PHP, too.

Re:Not getting RDMS (1)

Lisias (447563) | more than 2 years ago | (#40017277)

But yet, are these same developers that are being *highly* paid on these Web 2.0 times.

Serious. I was of of them - but got kicked out because I made the huge mistake of pointing the obvious: you must be a skilled programmer to do programs right. Ruby On Rails will not make a good coder from a dumb ass.

The dumb asses joined up em kick me out. =D

Re:Not getting RDMS (4, Insightful)

gorzek (647352) | more than 2 years ago | (#40016557)

I think the main problem is application developers not understanding anything about database theory. The vast majority of databases I encounter are not normalized at all, and it's almost always because they were designed by a developer with no database background.

Granted, I didn't come into this field with that background, either, but I made a point to learn it, and now I'm very cognizant of implementing sound database designs. This whole idea of throwing random strings of structured text into a database column, and then relying entirely on the program code to parse and use it... well, why the hell even use a relational database, then?

Relational databases aren't suitable for every application, nor are "bigtable" and other NoSQL implementations. The problem is that developers use a particular kind of database without really understanding how to use it properly. If they can get data in, and get data out, that's basically all they care about. Never mind if they make it a maintenance nightmare in the process.

Normalisation isn't a panacea (1)

Viol8 (599362) | more than 2 years ago | (#40016861)

Yes it makes sense up to a point , but it starts to suffer from the law of diminishing returns and at some point having to do complicated multi-table joins actually slows down your queries so much that it becomes simpler and faster to suffer duplicate data than normalise to the Nth degree.

Re:Normalisation isn't a panacea (2)

Xest (935314) | more than 2 years ago | (#40016943)

It depends on the task though, I'd wager 90% of SQL work that is done by developers day to day isn't in such a performance sensitive environment that it needs to favour performance over normalisation, and I agree with the GP, there's far too many developers out there that just don't do it and hence simply don't have the performance excuse. It really is just bad database design as a result of incompetence most the time.

Re:Normalisation isn't a panacea (2)

gorzek (647352) | more than 2 years ago | (#40017069)

I can definitely see the value in making an informed tradeoff, but like you said, a lot of the time it's not an informed decision--they just do it to make it work and don't really have the expertise to know which is the right way to go. I've definitely seen enough bad database designs to know that most developers just have no clue how to design them. The worst I've seen had bad designs and poor performance, and were built in a completely ad hoc manner without any eye toward maintainability, performance, or data integrity/consistency. The philosophy was just "make it work."

I think developers need to realize that databases are a lot like code: first you prototype, then you throw away the prototype and do it right. (Then again, plenty of developers just keep the prototype and use it for production.)

Re:Normalisation isn't a panacea (4, Insightful)

gorzek (647352) | more than 2 years ago | (#40017143)

Yeah, it really depends on what you are doing. But any time you break normalization there should be a good reason. Performance is certainly a valid reason. "I'm too lazy to make a well-designed database," however, is not.

If you find yourself breaking normalization all the time, then you've probably found a use case where a relational database isn't the best tool for the job.

While there is a "right" way to use a given tool, there is no one tool that is right for every situation. People who get this backwards are zealots and will often make poor decisions.

Re:Not getting RDMS (4, Insightful)

SQLGuru (980662) | more than 2 years ago | (#40017119)

I completely agree. A lot of non-DB centric people think that they can do more in the app tier, effectively using their databases as glorified file stores. Why even have a database server in those instances? I'm not saying that everything should be done in the database, either, but take advantage of every tool you have.

NoSQL has a place, so does relational. Learn their strengths and determine which is the best fit for your project. Then, learn how to use the tool to its fullest.

The decision the simple (1)

Anonymous Coward | more than 2 years ago | (#40016047)

If you want to hire twentysomething engineers who have party during the week and have gadgets falling out of their pockets, go with CouchDB or Cassandra or MongoDB.

If you'd rather have people in their 30's and 40's with families, then go with MySQL or Oracle.

Re:The decision the simple (4, Interesting)

Sarten-X (1102295) | more than 2 years ago | (#40016109)

That's actually a rather insightful point...

If your application fits well with the methodologies of a traditional RDBMS, use a traditional RDBMS, and hire people who are trained and experienced in using those methodologies to their full potential.

If you're dealing with the latest Big Data paradigms and designs, where you can sacrifice some of the rigidity of a RDBMS to gain some flexibility and cheaper scalability, use a NoSQL database, and hire people who aren't stuck in their old RDBMS ways.

Re:The decision the simple (5, Insightful)

Anonymous Coward | more than 2 years ago | (#40016213)

That's actually a rather insightful point...

If your application fits well with the methodologies of a traditional RDBMS, use a traditional RDBMS, and hire people who are trained and experienced in using those methodologies to their full potential.

If you're dealing with the latest Big Data paradigms and designs, where you can sacrifice some of the rigidity of a RDBMS to gain some flexibility and cheaper scalability, use a NoSQL database, and hire people who aren't stuck in their old RDBMS ways.

The real key is for the person doing the hiring to understand which of those of methodologies fits their application.

Re:The decision the simple (4, Informative)

TheSpoom (715771) | more than 2 years ago | (#40016357)

The real key is for the person doing the hiring to understand which of those of methodologies fits their application.

This is insighful. I've worked extensively with RDBMS solutions and now quite a bit with NoSQL technologies. They each have their place. An entire article could be written on where each fits most naturally, but in general if you don't need to join between tables, need to throw data to your store at a high velocity (e.g. logging), and/or need a loose schema, a NoSQL solution works best. If what you're doing can be naturally modeled (i.e. users HAVE AND BELONG TO stations, stations HAVE MANY playlists, etc. etc.), use an RDBMS.

One can see in the subtext of the GP that they may not get this, with their comment that people using RDBMS solutions are "stuck in old ways". It seems like they are saying that NoSQL is effectively always best. I'm curious why they think that. Nail, hammer, etc...

Re:The decision the simple (1)

Sarten-X (1102295) | more than 2 years ago | (#40017131)

people using RDBMS solutions are "stuck in old ways". It seems like they are saying that NoSQL is effectively always best.

No, no, no, no, no, no, no, no, no, and hell no.

I'm referring somewhat-sarcastically to the RDBMS proponents who reject NoSQL out of hand. The ones who see "database" and think it must have a rigid structure, where all connections are made with JOINs. The ones who don't accept that NoSQL databases are inherently different and must be designed differently. If a programmer is actually stuck thinking in terms of an RDBMS, they should not be working in a NoSQL database. If the programmer is flexible enough to do both, good for them! They're hired!

It's all about having the right tool for the job, and hiring someone who knows how to use the tool in question. The process for soldering pipes together is very different from soldering electrical components, but that does not prevent someone from being able to do both.

For the record, a major part of my NoSQL experience was building a web application for medical research. We pulled data into a Hadoop/HBase store (for speed and cheap parallelism), ran processing on it, and put results into MySQL (for rigid structure and maturity). The web app ran entirely off the MySQL instance. It was the right tool for the job.

Re:The decision the simple (1)

TheSpoom (715771) | more than 2 years ago | (#40017367)

Fair enough, sorry if I misunderstood.

Re:The decision the simple (1)

gstoddart (321705) | more than 2 years ago | (#40017495)

The ones who see "database" and think it must have a rigid structure, where all connections are made with JOINs

So, I'm actually curious about this part.

I've worked in RDB's, and I've worked in things that are more based on Berkeley DB ... but I am actually having a hard time thinking of specific examples of where I'd want something database-ish and not have the need for JOINs.

Berkeley gives you key value pairs, but the product I worked on which was based on it allowed us to do searching on multiple of those, which was kind of join-ish.

I just can't think of a specific case in which I have data structured enough to be in a database, but unstructured enough to never have joins, schemas, or the rest of the RDB trappings. But it may just be that I've not encountered any yet.

Re:The decision the simple (2)

gstoddart (321705) | more than 2 years ago | (#40016269)

And most importantly, make sure you know the difference.

Because I should think someone who thinks you should ditch your RDBMS when it's the thing you need to keep using is going to cause you more problems than they're worth. Of course, the opposite is true ... I remember someone who insisted in writing ER diagrams to describe our system, despite it not being an RDB, and not being accurately described by ER diagrams -- but to him everything was an ER diagram.

It's not uncommon for geeks to push to use the latest stuff simply because it's the latest. (Or, as you point out, use something because that's what they've always used)

I've actually seen someone suggesting we scrap an architecture to go with something he'd read recently -- despite having insisted we switch to the current architecture after reading about that.

After a certain point, you just realize they're a technology magpie and tell them to STFU if they're not providing solid reasoning for why this is better in this context. After a while "because it's newer and better" becomes code for "shiny and pretty". Especially if these whims happen in shorter periods than your development lifecycle.

Re:The decision the simple (3, Insightful)

sycodon (149926) | more than 2 years ago | (#40016323)

RDBMS systems can be flexible also. It just takes a bit of planning, a good understanding of your data and a well designed application...which you should do/have regardless of your storage solution.

Call me set in my old RDBMS ways, but if I'm supporting it then I want to know what the hell is gong on with the data.

Re:The decision the simple (1)

sycodon (149926) | more than 2 years ago | (#40016435)

going, going, gong!

The Uncle Larry feature (1)

symbolset (646467) | more than 2 years ago | (#40016125)

This makes me want to avoid your suggestion. If you presently don't have the Larry problem it's best avoided.

Re:The Uncle Larry feature (0)

Anonymous Coward | more than 2 years ago | (#40016229)

This makes me want to avoid your suggestion. If you presently don't have the Larry problem it's best avoided.

huh?

Re:The decision the simple (3, Insightful)

h4rr4r (612664) | more than 2 years ago | (#40016623)

Or use a better DB like Postgres. How the MySQL still is popular I will never know. I think it is a conspiracy to prove FREE DBs suck.

Re:The decision the simple (1)

The Dancing Panda (1321121) | more than 2 years ago | (#40016841)

Not all of the 20 something engineers are idiots. I'm 27, and I'd much rather use a SQL Database for most things. Also, I know what I'm doing.

Has to be said (0)

Anonymous Coward | more than 2 years ago | (#40016101)

Couch DB is *not* webscale!

Re:Has to be said (5, Funny)

Anonymous Coward | more than 2 years ago | (#40016135)

MongoDB is Webscale. MySQL is not Webscale, because it uses joins. SQL also has impetus mismatch.

Wikipedia and Slashdot use MySQL (3, Insightful)

tepples (727027) | more than 2 years ago | (#40016305)

Anonymous Coward wrote:

MySQL is not Webscale, because it uses joins.

Then how does a non-webscale database power popular web sites such as Wikipedia and Slashdot? If you don't do joins in the database, you'll probably end up doing the equivalent of joins (using one value as the key in another table) in your application.

Re:Wikipedia and Slashdot use MySQL (0)

Anonymous Coward | more than 2 years ago | (#40016369)

MongoDB is a document database that doesn't need JOINs. It uses map/reduce.

Then what's it called instead of a join? (5, Insightful)

tepples (727027) | more than 2 years ago | (#40016461)

In NoSQL systems such as MongoDB and CouchDB, what do you call the operation where you retrieve one document, pull an identifier out of that document, and use that identifier as the key to retrieve another document?

Re:Then what's it called instead of a join? (3, Interesting)

mj1856 (589031) | more than 2 years ago | (#40016643)

Not sure about MongoDB or CouchDB, but I have experience with RavenDB, which is absolutely fantastic. Instead of "joins" you have "includes" or "live projections". See http://ravendb.net/docs/client-api/querying/handling-document-relationships [ravendb.net]

Re:Then what's it called instead of a join? (5, Funny)

Anonymous Coward | more than 2 years ago | (#40016653)

Witchcraft.

Re:Then what's it called instead of a join? (1)

Eponymous Coward (6097) | more than 2 years ago | (#40016959)

If you have a lot of related data, then you should probably be using a relational db, no?

Re:Then what's it called instead of a join? (0)

Anonymous Coward | more than 2 years ago | (#40017461)

"performance disaster"

Re:Wikipedia and Slashdot use MySQL (1)

emorning (2465220) | more than 2 years ago | (#40016469)

like he said, you end up doing joins in your application

Re:Wikipedia and Slashdot use MySQL (4, Funny)

Hognoxious (631665) | more than 2 years ago | (#40016681)

MongoDB can write its data to /dev/nul/ for extra performance.

Re:Wikipedia and Slashdot use MySQL (3, Funny)

Anonymous Coward | more than 2 years ago | (#40017537)

If /dev/null is webscale then I will use it.

Re:Has to be said (1)

TheSpoom (715771) | more than 2 years ago | (#40016423)

I hadn't read the article this was based on before, thanks for the laugh. I encourage others to Google "webscale" :^)

Re:Has to be said (1)

emorning (2465220) | more than 2 years ago | (#40016535)

MongoDB is Webscale. MySQL is not Webscale, because it uses joins.

I see this comment a lot and I don't get it. Somehow Map/Reduce is considered *webscale* and joins are not. But I can implement a join with Map/Reduce so why don't the webscale databases provide join functionalty that is built on top of Map/Reduce and save us all the trouble of reinventing the wheel?

Re:Has to be said (-1)

Anonymous Coward | more than 2 years ago | (#40016853)

Join runs inside one process, so inside one query in one process boundary. Map/reduce runs across multuple process boundaries, can operate across thousands of machines, on petabytes of data, distributed across all those machines.

M/R is thousands of times more scalable than an sql based aggregation query.

Re:Has to be said (1)

emorning (2465220) | more than 2 years ago | (#40017259)

Yes, traditional joins run in a single process but AFAIK there no reason why a join could not be distributed across many processes by implementing the join with Map/Reduce. Right?

Re:Has to be said (1)

i_ate_god (899684) | more than 2 years ago | (#40016909)

I read this entire thread wondering if "Webscale" is really some kind of valid term or not. I'm not convinced that it has any valid meaning whatsoever.

Re:Has to be said (0)

Anonymous Coward | more than 2 years ago | (#40017301)

"Webscale" is a marketing term. It has no meaning beyond "bawww joins are hard."

Re:Has to be said (1)

dririan (1131339) | more than 2 years ago | (#40017289)

It's a quote. [youtube.com]

Re:Has to be said (1)

The Moof (859402) | more than 2 years ago | (#40017415)

I see this comment a lot and I don't get it.

Here you go. [youtube.com] Now you'll get why it's funny, and not serious.

Why not PostgreSQL? (5, Interesting)

JamesA (164074) | more than 2 years ago | (#40016117)

Re:Why not PostgreSQL? (-1)

Anonymous Coward | more than 2 years ago | (#40016387)

Postgres or Berkeley for engineers, MySQL or 'NoSQL' for clueless, self-promoting, gob-shites who like to talk about solutions to problems they wouldn't have if they'd selected the correct tool from the outset.

Re:Why not PostgreSQL? (1)

sourcerror (1718066) | more than 2 years ago | (#40016859)

MySql-s MyIsam is much faster with reads than PostgreSql. I think for the things people use NoSql, MyIsam is perfect. And when you want to move better ACID support, you can effortlessly switch to InnoDB.

Re:Why not PostgreSQL? (2)

vadim_t (324782) | more than 2 years ago | (#40016937)

That's all fine until you need to actually write to that table. With myISAM any write needs a table lock, and that makes performance drop like a rock.

Re:Why not PostgreSQL? (1)

sourcerror (1718066) | more than 2 years ago | (#40016997)

And how is it worse than NoSQL?
Anyway, that's when you move to InnoDB.

Re:Why not PostgreSQL? (1)

Lisias (447563) | more than 2 years ago | (#40017389)

Not really.

MySQL is good enough for 80% of the currently web sites. Damn it, even PHP is good enough for a small site.

As someone below states, if you need to be serious about ACID, you can switch to InnoDB.

Are you pissed of with Oracle? Go for MariaDB.

I like PostGRES, and acknowledge its technical superiority on every single aspect of MySQL.

But I'm using MariaDB on my site: I (still) don't need the PostGRES superiority, and MariaDB is easier to maintain (not to mention its smaller memory footprint!).

Re:Why not PostgreSQL? (0)

Anonymous Coward | more than 2 years ago | (#40017523)

Are you fucking kidding me?

If you need to be serious about data you don't use MySQL, InnoDB just doesn't cut it. Try triggers on cascading events *whoops* go back to start. Try constraints on cascading events *whoops* go back to start. Try increasing the size of the binary log for InnoDB, restart the database and create a table asking for engine InnoDB - if you are lucky your program will notice the warning, if you are unlucky your program will happily run with the MyISAM table you just made.

MySQL will always try to do best effort, that simply won't do when you are handling mission critical data. If I ask you to put garbage into a column, you bloody well don't say "hey, that doesn't fit, but this nice default value will do just fine - and oh, while I'm at it, how about updating that first timestamp on this row even though you didn't ask for it"; if you get garbage you reject the data.

Re:Why not PostgreSQL? (5, Funny)

squiggleslash (241428) | more than 2 years ago | (#40016449)

Because it's an urban myth.

The reality is there are only two SQL databases in the entire universe: MySQL and Oracle. You might have been told others exist, hell, you might even have worked on something called "SQL Server" in your .NET shop, but in reality: they don't. They're all figments on your imagination. Your imagination is SO determined to find better, more robust, faster, powerful, alternatives to MySQL and Oracle that an entire fantasy world comprised of "a successor to Ingres that makes MySQL look like a piece of crap" and "A Microsoft product that doesn't feel like a thirty year old mainframe product hacked onto a modern platform" develops in your head.

C'mon, if these mythical products actually existed, sites like Slashdot wouldn't ignore them, right? Right?

Re:Why not PostgreSQL? (1)

aclarke (307017) | more than 2 years ago | (#40016799)

I'm not generally a Microsoft fan, but I love SQL Server. However, I haven't started a new project with it in years, I guess since pricing for SQL Server 2008 was announced. I've not been in a situation where I could justify the costs as the project (hopefully) was successful and scaled up. I also don't like being forced to run my database server on Windows. For these reasons, I just don't use it any more except in projects where it was selected years ago. I know you have to look at TCO, but I still can't generally justify the costs for SQL Server and Windows Server.

If I was working in a corporate environment with big budgets to throw at projects, then I'd probably still be using it SQL Server at times. In my world though, it generally just doesn't add up.

Re:Why not PostgreSQL? (0)

Anonymous Coward | more than 2 years ago | (#40016827)

They like insert performance? Postgres is nice but it has a few flaws

Nosql in Postgres (4, Interesting)

rla3rd (596810) | more than 2 years ago | (#40016123)

You can get json support using the PLV8 extension http://code.google.com/p/plv8js/wiki/PLV8 [google.com]

or altenatively you can use the hstore data type.

XML field? (0)

Richard_at_work (517087) | more than 2 years ago | (#40016173)

I haven't touched MySQL in years, but do they have an XML field type? MS SQL Server does (as does many other RDBMSes I bet, I mention SQL Server as that's where my current experience lies) which allows you to essentially keep the table schema-less but still allows you to perform complex queries on the contained data.

Wouldn't that be better than a TEXT field type?

How is XML indexed? (1)

tepples (727027) | more than 2 years ago | (#40016335)

do they have an XML field type? MS SQL Server does [...] which allows you to essentially keep the table schema-less but still allows you to perform complex queries on the contained data.

But how does it index the data in the XML or JSON fields? How does it, say, tell an element containing a number from an element containing text? Does it act like SQLite, which is dynamically typed (and thus can store text in any field) but can be told to prefer to compare and index certain columns as numbers, dates, text with Unicode collation, or binary data?

Re:How is XML indexed? (0)

Richard_at_work (517087) | more than 2 years ago | (#40016615)

Not sure about indexes as I've not used this beyond briefly having to support it in someone else's app, but when selecting you can define what types the fields are at that point.

Re:How is XML indexed? (1)

serviscope_minor (664417) | more than 2 years ago | (#40017231)

Does it act like SQLite, which is dynamically typed

That's one of the annoying thing to me about SQLite, and the justification is a marvellous case of doucle standards. It goes something like this:

Static typing is bad because it limits flexibility and you can check elsewhere if you need to. We will never add it ever because we are right about this.

Then, in release foo.bar of SQLite

We've added foreign keys!

Either constraints are good or they are bad. Typing is just another constraint, like foreign keys, and various other domain constraints. I cannot see any valid argument for having foreign keys but not type constraints. It jest seems bizarre to me. It's not like they could be optional or anything.

programmers don't know how to store data (2)

vlm (69642) | more than 2 years ago | (#40016197)

But the real story may be that programmers are never satisfied with the tool they have.

Ah typo

But the real story may be that programmers don't know how to store data

They many not know because no one knows the business needs, but more often because they have no idea what they're doing WRT to data storage.

IT training tends to cover data manipulation pretty well "how to add two numbers'
IT training gets shakey on data structures "So, in junior level class we will talk about data structures, which is too bad because you've already developed at least two years of bad habits first"
IT training tends to pretty much skip data storage "In a senior level class, you might talk about scalability, maybe in an optional class. Or maybe you'll take a semester of cobol instead"

Re:programmers don't know how to store data (0)

Anonymous Coward | more than 2 years ago | (#40016293)

Appreciate the concern – but:

1) I know my business needs.
2) We are still in the CouchDB universe. Just using BigCouch. :)

--Till

Re:programmers don't know how to store data (2)

Zocalo (252965) | more than 2 years ago | (#40016295)

But the real story may be that programmers are never satisfied with the tool they have.

Ah typo

Possibly, but given how quick many programmers are to get into a fruitless pissing match over their favourite language it's quite apropos, no?

Maturity issues sitting on the couchDB? (0)

Anonymous Coward | more than 2 years ago | (#40016209)

"...affected by a lot of the same maturity issues that made CouchDB tough for us to work with"

The matury issues were sitting behind the keyboard.

Re:Maturity issues sitting on the couchDB? (0)

Anonymous Coward | more than 2 years ago | (#40016251)

No shit, right. CouchDB sounds like something you'd use if you have or want to have "maturity issues".

Re:Maturity issues sitting on the couchDB? (0)

Anonymous Coward | more than 2 years ago | (#40016327)

No, but if you wonder why there are schema's and complain that an RDBMS is hard - then you are choosing CouchDB for a wrong reasons. To choose uninformed for the obscure, to switch to mainstream and get all kind of advantages ...

Re:Maturity issues sitting on the couchDB? (0)

Anonymous Coward | more than 2 years ago | (#40016709)

Hah. Yep. And if you write drivel like "guize, like it's totally like 2012, why are we still using SQL guize?", then you're a fucking nincompoop.

Is a DB even needed sometimes? (1, Interesting)

Viol8 (599362) | more than 2 years ago | (#40016255)

It seems to be a knee jerk reaction amongst a lot of developers and designers that as soon as your app starts requiring persistent data beyond ini values a database is needed. Why? For large but simply structured data something like json or XML or even a flat csv file is perfectly adequate. Performance can be an issue during searches but if for example you have a fixed record size with key sorted data then finding a given key is simple (binary chop or similar).

It seems to me that reaching for a DB is the easy way out taken by a lot of oders and they end up paying for it in maintanability, bugs and support.

Re:Is a DB even needed sometimes? (1)

neorush (1103917) | more than 2 years ago | (#40016401)

This logic works just fine until the customer goes, oh! lets add our customer's information, transactions, etc. to that completely static tool that we never planned to need a real database for. Even very simple tools and services we at least use SQLite, with PDO connections it's (mostly) painless to move the system up a level or two in relational database capability. I've found that schemaless databases have there place, but the jump on the schemaless data band wagon has been premature for many software projects. That said, look at draw something's expansion, zero downtime as they increase capacity because they used a scalable schemaless database.

Re:Is a DB even needed sometimes? (1)

TheSpoom (715771) | more than 2 years ago | (#40016459)

Starting with a database avoids the pain of migrating flat files to a database later when the database is needed (and if your app gets at all popular, it will be).

Sure, if you're only ever expecting 10k rows of data with very little concurrent access, go nuts with your flat files.

Re:Is a DB even needed sometimes? (4, Informative)

rtaylor (70602) | more than 2 years ago | (#40016521)

A CSV or XML or JSON file is a db (a DB is just structured data).

Are relational DBs always required? Certainly not.

The big benefit to a relational DB with lots of enforcement at the data layer is that you can have one or more applications reading/writing to it with minimal concern of data corruption.

What isn't obvious is that second application is often aggregate reporting for management. "How many customers are using $foo and where do they live geographically". With a relational DB, I might knock that query out in a few minutes across millions of customers.

With a flat XML file per customer spread across a number of servers, this could take days to assemble, particularly if $foo is nested deep in the structure.

Having spent far too much time writing one-off scripts to gather customer data because the middleware didn't support that type of query, I've actually gone the other way and started shoving some business logic into the DB.

Functions such as isCustomerPaymentOverdue are now in the relational DB with a very thin model in the middleware to allow for much easier and faster reporting.

Re:Is a DB even needed sometimes? (4, Insightful)

serviscope_minor (664417) | more than 2 years ago | (#40017241)

The big benefit to a relational DB with lots of enforcement at the data layer is that you can have one or more applications reading/writing to it with minimal concern of data corruption.

Not just that, but good use of relations and normalization makes whole classes of bug impossible.

Re:Is a DB even needed sometimes? (1)

Anonymous Coward | more than 2 years ago | (#40016751)

...For large but simply structured data something like json or XML...

You wouldn't happen to be the developer whose code I inherited, would you?

Because if you are, I'd just like you to know that I hate you with ever fibre of my being.

Re:Is a DB even needed sometimes? (0)

Anonymous Coward | more than 2 years ago | (#40017503)

So instead of using a database you invent your own database?

Do everyone a favor. Use a database from the start.

all db solutions have problems at some point (1)

Torvac (691504) | more than 2 years ago | (#40016341)

with big data comes great responsibility.

Re:all db solutions have problems at some point (1)

JustOK (667959) | more than 2 years ago | (#40016537)

SELECT * and ye shall find.

y;ou Fa1l It (-1)

Anonymous Coward | more than 2 years ago | (#40016405)

40,000 coming start a holy war as tHose non gay,

Re:y;ou Fa1l It (0)

Anonymous Coward | more than 2 years ago | (#40016731)

39,999 actually, sorry. Dave had an appointment. He sends his regrets and all that.

Native JSON fields (2)

Xanni (29201) | more than 2 years ago | (#40016411)

PostgreSQL 9.2 (now in beta) includes native JSON fields:

http://www.h-online.com/open/news/item/PostgreSQL-9-2-beta-improves-scalability-adds-JSON-1573815.html [h-online.com]

It's also available as an extension for the current 9.1 release:

http://people.planetpostgresql.org/andrew/index.php?/archives/255-JSON-for-PG-9.2-...-and-now-for-9.1!.html [planetpostgresql.org]

Re:Native JSON fields (1)

DiogoBiazus (1216102) | more than 2 years ago | (#40016467)

You can also use the hstore contrib module in the current PostgreSQL versions, which has a gem (https://github.com/softa/activerecord-postgres-hstore) for ActiveRecord and Heroku support (at least in some database plans). Seriously, PostgreSQL is the best Not-Quite-So-SQL DBMS out there.

PICK (2)

kibbey (96367) | more than 2 years ago | (#40016489)

Hop into the wayback machine and fire up any flavor of PICK. The database where schema is applied on use, not on storage. No length limits on fields and very fast on old hardware (really fast on new). Storing bits of xml and code are no problem. And for those users who simply must have SQL, many versions will support that too (UniData and UniVerse are two examples). It's not cool, not new, but it does work.

Riak (1)

platypusfriend (1956218) | more than 2 years ago | (#40016659)

We're using a four node Riak cluster, in a production environment spanning two datacenters, and have zero downtime because we can lose two nodes (for example, a whole datacenter) and still read/write. Maybe CouchDB can do that; maybe it can't. Maybe our setup isn't even the best. But, as many people have already pointed out, there is a time and a place for a NoSQL solution, and you can have "near zero" (or, so far for us, zero) downtime. Here's the catch: We are using Secondary Indexing and MapReduce, in Riak, but for searchable analytics we fire off a stripped dataset to a smaller, less-redundant MySQL store.

Urban Airship (3, Interesting)

jjohnson (62583) | more than 2 years ago | (#40016685)

Urban Airship went PostgreSQL to MongoDB to Cassandra to PostgreSQL. http://wiki.postgresql.org/images/7/7f/Adam-lowry-postgresopen2011.pdf [postgresql.org]

It's a good presentation because they're in love with none of them and are moving for specific reasons each time, handling different issues. It's not coders chasing the new hotness.

Kneejerk Reaction (1)

mj1856 (589031) | more than 2 years ago | (#40016705)

So they love NoSQL but had a bad experience with CouchDB. And the solution is... move away from NoSQL??? Ridiculous. CouchDB is certainly NOT the only solution out there, and they all have their strengths and weaknesses.

I've recently switched from MS SQL Server to RavenDB http://www.ravendb.net/ [ravendb.net] and I am never going back! Many of the things they found wrong with CouchDB and MongoDB are superior in RavenDB.

Re:Kneejerk Reaction (1)

gbjbaanb (229885) | more than 2 years ago | (#40016883)

what did they find wrong with MongoDB, apart from rumour.

I doubt RavenDB is any different really, it looks and feels the exact same (except written in .NET so you have a lot of garbage and resource issues to deal with), same as Cassandra (written in java).

Maybe erlang isn't as good as they say, or maybe CouchDB isn't as well written as they say.

Oh, the joy! (1)

pz (113803) | more than 2 years ago | (#40016939)

... the joy of schemaless DBs.

You mean working with a file system and not using a DB at all, not needing to pay a DBA, not dealing with corrupted databases, not using arcane tools, etc.?

I jest, but not entirely. Clearly there are purposes for which databases are the right toold for the job. I'm most definitely not convinced that big blob storage is one of them.

Re:Oh, the joy! (2)

The Moof (859402) | more than 2 years ago | (#40017345)

not using arcane tools

I know database concepts are difficult for some people, but it's by no means magic.

Couchdb is great when it's used right (1)

thanosv (2640765) | more than 2 years ago | (#40016985)

Clues to the source of some of Sauce Labs' problems can be gleamed from their list of Maintenance headaches:

View indexes are only updated when queried — insertion does not update the index. That means you have to write a script to periodically run all your views, unless you want them to be surprisingly slow when they haven’t been queried in a while. In practice we always preferred view availability to any performance boost obtained by not updating indexes on insertion, but writing reliable scripts to keep view indexes up to date was tricky.

Oh please doing an HTTP GET periodically is tricky ??? No it's not. With couchdb if your database is very dynamic you should either index periodically and very frequently. This creates a quantified and controlled performance demand on the server. Ideally read servers should never be the write server and replication should be filtered. Using Couchdb naively will lead to failure. Don't use javascript views, python views are 3-4 x faster, erlang views are 7-10 faster. Used in the right way and following the many tips that you can get from the Couchdb community will make Couchdb not just a great database but a great application platform.

And yes 1.2 is a great improvement.

Currently I'm using coucdb, mongodb, and MySQL all in one high profile project handling terabytes of data and millions of hits. Each has it's use. When it comes to reliability and performance all three DBs are NOT my problem.

Re:Couchdb is great when it's used right (0)

Anonymous Coward | more than 2 years ago | (#40017167)

1) terabytes of data is not big data by any stretch
2) millions of hits per month is hardly impressive nowadays, tons of independent wordpress blogs clock this alone
3) anyone who would architect a solution using three different databases isn't impressing me, two yea, but three? bad systems design

CouchDB just didn't work (2)

Animats (122034) | more than 2 years ago | (#40017449)

a majority of our unplanned downtime was due to CouchDB issues

Nowhere on the CouchDB home page [apache.org] is reliability even mentioned. And that's the real issue. Developing a reliable database system is a difficult design and programming task. It requires real software engineering. The hacks who write PHP and use JSON aren't up to a job like that. The "aw, we'll fix it in the next release" attitude doesn't cut it in databases.

So what do they do again? (1)

FilmedInNoir (1392323) | more than 2 years ago | (#40017457)

They do cross-browser testing. Right? I can't really figure out what they sell or why I would want it. The real question, "Why haven't I started a mismanaged company with ambiguous products?"
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?