Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

MemSQL Makers Say They've Created the Fastest Database On the Planet

timothy posted more than 2 years ago | from the before-you-even-think-it dept.

Databases 377

mikejuk writes "Two former Facebook developers have created a new database that they say is the world's fastest and it is MySQL compatible. According to Eric Frenkiel and Nikita Shamgunov, MemSQL, the database they have developed over the past year, is thirty times faster than conventional disk-based databases. MemSQL has put together a video showing MySQL versus MemSQL carrying out a sequence of queries, in which MySQL performs at around 3,500 queries per second, while MemSQL achieves around 80,000 queries per second. The documentation says that MemSQL writes back to disk/SSD as soon as the transaction is acknowledged in memory, and that using a combination of write-ahead logging and snapshotting ensures your data is secure. There is a free version but so far how much a full version will cost isn't given." (See also this article at SlashBI.)

Sorry! There are no comments related to the filter you selected.

BS !! (-1)

Anonymous Coward | more than 2 years ago | (#40433205)

Not fastest !! No way !!

Top coder (4, Interesting)

Taco Cowboy (5327) | about 2 years ago | (#40434045)

They did have an ad to lure in "Top Coders" at http://developers.memsql.com/blog/ [memsql.com]

Apart from their ad, what they said about Top Coders was interesting - with the exception of top coders memorizing who books filled with algorithms, because top coders do not memorize nothing - top coders do not get to be top coders by memorizing.

Instead, top coders have that instinct to _know_ which algorithm to adapt and apply, and top coders know where (and how to) look for the algorithm (either from their own archive, from books, from old magazines, or from some strange corners on the Web)
 

Re:Top coder (1)

gweihir (88907) | about 2 years ago | (#40434553)

Quite true. That is also what competent computer scientists do: Learn the rough border conditions of a problem and its solutions and look up details when needed. Be able to construct something reasonable when no solution can be found in the literature. Committing details to memory is only for those weak of mind. Of which there are many.

Re:Top coder (4, Interesting)

Surt (22457) | about 2 years ago | (#40434653)

All of the best developers I've met had phenomenal memories. I think both a natural reasoning ability and great memory are assets. If you are missing one, you aren't going to be as strong as someone who has both.

nice slashvertisement, guys (0, Troll)

Anonymous Coward | more than 2 years ago | (#40433233)

too bad there wasn't mention of Apple or Raspberry Pi. That'd be sure to get even more people to click through!

A nice approach perhaps... (4, Interesting)

alphatel (1450715) | more than 2 years ago | (#40433251)

It sounds cool, but we can get 200k iops on Raid10 SSD without degradation.

Re:A nice approach perhaps... (4, Insightful)

tomhath (637240) | more than 2 years ago | (#40433427)

Price/performance is a better question. If it's fast enough that you don't need the Raid 10 SSD then it could be a good choice. Throw hardware at any DBMS and you'll get good throughput.

Re:A nice approach perhaps... (2)

hcs_$reboot (1536101) | about 2 years ago | (#40433919)

So, two guys meet and churn out some code that cache most of the DB work in memory. Great. But MySQL has a MEMORY engine and is pretty well optimized (eg keep indexes in memory, does some caching as well)... the hardest part is probably its setup: setting the right options in MySQL to achieve top performance is not easy.
Besides, the "caching" or equivalent work is not the most difficult part of a DBMS, by far: What about the algorithms to "compile" queries in order to use indexes and perform the JOINS optimally. A good algorithm beats the architectural benefits of a "cache" oriented DBMS.

Re:A nice approach perhaps... (2)

guruevi (827432) | about 2 years ago | (#40434191)

You can get more than 10M IOPS on certain RAM-based SSD, they're just mighty expensive.

Re:A nice approach perhaps... (0, Flamebait)

Anonymous Coward | about 2 years ago | (#40434343)

In other news: The *brightest* oil lamp!
Seriously, SQL is unbearably primitive...

wnata goa (-1)

Anonymous Coward | more than 2 years ago | (#40433255)

zupper hacka norma

Ya Don't Say! (5, Insightful)

Rary (566291) | more than 2 years ago | (#40433271)

Really? Accessing RAM is faster than accessing a disk? What a novel discovery!

It seems to me that MySQL can also be run in memory. Apparently that's how the clustered database works (or used to work). I've never tried it, but let's see some benchmarks between MemSQL and an entirely memory-based MySQL.

Re:Ya Don't Say! (1)

JimboFBX (1097277) | more than 2 years ago | (#40433327)

I was going to say, how does this perform on large queries in a large database.

Re:Ya Don't Say! (2)

lucifuge31337 (529072) | more than 2 years ago | (#40433791)

It seems to me that MySQL can also be run in memory. Apparently that's how the clustered database works (or used to work).

Absolutely correct. NDB Cluster. It's quite fast, even on older hardware providing you have enough RAM to hold your database.

Re:Ya Don't Say! (0)

Anonymous Coward | about 2 years ago | (#40434555)

No foreign keys with NDB

Re:Ya Don't Say! (2)

lucifuge31337 (529072) | about 2 years ago | (#40434597)

No foreign keys with NDB

Also correct. Your point? Other than that specific tools were made for specific jobs.

Re:Ya Don't Say! (3, Insightful)

errandum (2014454) | more than 2 years ago | (#40433813)

That and memcached (I think that's the name).

This comparison is far from fair... Is it ACID? Or eventually synchs up? How does it compare with other memory based DB's?

Comparing it with a slow relational DB will not give you any kind of credibility.

Re:Ya Don't Say! (1)

MobileTatsu-NJG (946591) | about 2 years ago | (#40433893)

Isn't the implocation that in this case there's a lot less time between the transaction getting made and that data being committed to non-volitile memory?

Re:Ya Don't Say! (3, Interesting)

bill_mcgonigle (4333) | about 2 years ago | (#40434063)

Not just that - you can get a FusionIO ramdisk device for really big databases and get performance that's somewhere between SSD and memory. Those are all battery backed and such, so no monkeying around with whether the ACID was done right or not.

Re:Ya Don't Say! (0)

Anonymous Coward | about 2 years ago | (#40434195)

I am a ramdisc fan since Mac+. The article implies a structural code change that is faster, by time of recognition, which may also be improved by disc speed. I am not qualified to parse this, but I invite those who are, to employ these criterion in spoon feeding us mere mortals. I want to be the spoon-feed-ee for a change.

Re:Ya Don't Say! (4, Interesting)

gman003 (1693318) | about 2 years ago | (#40434449)

It's a bit more complex. There's four main ways to do MySQL storage in RAM (which I know of because my current work project is a MySQL application).

First, the NDB Cluster system is there, which is what you've mentioned. That's basically just a MySQL frontend to a distributed, memory-based NoSQL database, though. Convenient, but not truly "MySQL".

The second is using the "Memory" storage engine, where it actually stores a normal MyISAM table in memory. However, this is a surprisingly crappy option, because it uses table-level locks for writing, so parallel write performance is only marginally faster than disk.

The third is to store regular InnoDB tables on a ramdisk. This can be crazy fast, but it also means that if your server crashes or loses power, you're *fucked*

The fourth is to use Memcached, which isn't really a MySQL thing at all. You're basically just caching data in a memory-only NoSQL database, at the application level. This is actually what we ended up doing, because all the others are pretty crappy options - Cluster is the best one, but the hardware requirements are higher than we could justify spending given our performance requirements. Shoving memcached onto the web server (which has RAM to spare) and setting certain queries to cache their results there sped things up significantly, at minimal cost.

As far as I can tell from the summary (I refuse to read the articles for such a blantant slashvertisement), this "MemSQL" doesn't do anything you can't do by configuring MySQL properly, although they likely optimized some rarely-used modules to make them faster.

Re:Ya Don't Say! (4, Informative)

arth1 (260657) | about 2 years ago | (#40434729)

The third is to store regular InnoDB tables on a ramdisk. This can be crazy fast, but it also means that if your server crashes or loses power, you're *fucked*

Not necessarily. There are battery-backed volatile RAM devices that can last for days, and also non-volatile RAM devices like F-RAM and MRAM.
Battery backed volatile RAM can even be considered "cheap" - if the bottleneck are in tables small enough to fit on these, or the amount of overall writes is so high that placing the innodb logs there makes sense, it can be cheaper than a RAID10 or 50 of high-speed SAS drives.

The HyperCard / ACARD drives, for example, are only $300 plus RAM. And if the worst happens, you can even dump the RAM to a CF card before the battery runs out.

Re:Ya Don't Say! (0)

Anonymous Coward | about 2 years ago | (#40434685)

I'm sure no one would ever think about doing the same thing using a ramdisk partiton and a disk partition connected with Linux software RAID 1 and having the rotational media partition marked as "write mostly" so that most reads would come from the ramdisk and writes would happen on the disk as soon as they could - but would happen in memory faster. I mean, that would take all of (walks over to a random machine and does it) a couple of minutes to set up.

I tried reading the article to replicate the benchmarks. The information isn't in there. Then I tried watching the video. Not only is that embedded player too small to read the information and not only did they write their own benchmarking tool (really? Couldn't find an accepted tool compatible with MySQL, like, say, mysqlbench?), but I had to jam a screwdriver through my speakers so I wouldn't hear that dumbass say "sequel" another time. Look, Facebook engineer / fucktard, SQL is pronounced as three letters. Return of the Jedi is a sequel. Unless you have Ewoks fetching your data, you're not using "sequel" in your queries.

Looks good for testing (1)

All_One_Mind (945389) | more than 2 years ago | (#40433349)

MemSQL is definitely good news, and hopefully it will encourage the MySQL team to play catch up with it's performance. Maybe it will provide an improved web experience if it gets wide adoption and deployment. As a long time SysAd/webmaster/developer, I'm certainly interested, but for obvious reasons I'm not putting any business critical servers onto something this fresh and new, regardless of performance benefits. I think I'll download a copy and use it locally for testing, but like any software, there are going to be bugs, maybe even data loss or security issues that may emerge on certain server setups. I'll see how the changelog looks in 6 months or a year before considering it for my mission critical servers. Regardless, kudos to the developers. Grabbing my download before/if it gets /.'ed

Re:Looks good for testing (5, Insightful)

realityimpaired (1668397) | more than 2 years ago | (#40433611)

As a long time SysAd/webmaster/developer, I'm certainly interested

At the risk of sounding incredibly condescending....

If you were really a sysadmin who could benefit from that kind of speed improvement, you'd know that it's possible to achieve that level of performance with MySQL already, by either running it from memory or by using a fast hard drive array. The simplest/cheapest option to drastically improve MySQL performance is to throw a large amount of RAM at a system and point MySQL at the memory. MySQL can be configured to keep the database in active memory and sync to the disk on a regular basis, which is almost exactly the kind of behaviour described for MemSQL... for an exceptionally large database that can't be stored in system memory, I imagine that the advantage that MemSQL is boasting would evapourate. There are other ways to go about doing it, such as running a fast disk array or a cluster, in order to get around the limitations of using RAM, but ultimately the prime determining factor for speed in MySQL is speed of access to the database file itself.

Re:Looks good for testing (2)

errandum (2014454) | more than 2 years ago | (#40433825)

I'd love to see their tests when this DB needs to go into swap / pagefile. It's double the slowdown, needs to write into the swap (disk I/O) and then sync the DB (disk I/O again).

I can't, for the life of me, understand where this will be better than the already available options.

Re:Looks good for testing (1)

hawguy (1600213) | about 2 years ago | (#40433897)

I'd love to see their tests when this DB needs to go into swap / pagefile. It's double the slowdown, needs to write into the swap (disk I/O) and then sync the DB (disk I/O again).

I can't, for the life of me, understand where this will be better than the already available options.

I think the point of an in-memory database is that you size your machine so it does *not* need to swap in normal use. Otherwise, since as you said, you lose all of the speed - worse because the operating system decides what to swap out, and may not make the most efficient choice. (though they probably just mlock() the memory buffers into RAM and prevent any of the database RAM from being swapped out at all.)

But if the architect did expect the machine to swap at times, he probably wouldn't put the swap and data on the same physical disk, so concurrent I/O's to both devices wouldn't take twice the time.

Re:Looks good for testing (0)

Anonymous Coward | about 2 years ago | (#40434019)

Just say no to swap. It's pointless, except as a crutch for broken software. And it's dangerous on a server. If an application wants disk-backed VM, it can use mmap.

Re:Looks good for testing (1)

errandum (2014454) | about 2 years ago | (#40434141)

I see your point, but I disagree. I consider it better to have the webserver running slowly than having it crash because it ran out of memory. To do that might be a choice, but you could just go here ( http://unixfoo.blogspot.pt/2007/11/linux-performance-tuning.html [blogspot.pt] ).

Re:Looks good for testing (2)

hawguy (1600213) | about 2 years ago | (#40434709)

Just say no to swap. It's pointless, except as a crutch for broken software. And it's dangerous on a server. If an application wants disk-backed VM, it can use mmap.

Swap isn't just a crutch for broken software (though it can be), sufficient RAM is not always available. In a perfect world, all servers would have more RAM than their applications ever need, more cores than the processes can take advantage of, and all disks would be RAID-10 arrays of SSD's.

But back in the real world where most of us have to live, swap does come into use at times to let a server accommodate loads that it otherwise couldn't handle due the memory footprint of the software it's running. Swap doesn't have to be a death knell for the server - some light or even moderate swapping can mean that you're using the server more efficiently - especially when you're running on VMWare and it wants to reclaim some RAM of its own via the memory balloon driver. When VMWare is under memory pressure, it's better to let Linux decide what to swap out than to let VMware swap memory out from under the virtual machine.

Re:Looks good for testing (2)

errandum (2014454) | about 2 years ago | (#40434111)

I meant 2 disk access, some or another. From what I read they would never be simultaneous anyways.

Either way, this would be useful (actually IS, some solutions do this) in the Business Intelligence field. But the whole point of keeping everything in memory is moot when you have petabytes of information that you need to process during your ETL. What matters in this database is, how well does it behave in a cluster and how would it handle concurrency (ACID? Eventually synchronized?).

I doubt this is all that useful for common DB applications like websites and the like. Relational DB's have been proving to be enough for everything (ex: Youtube uses mysql shards - or used to) purely web related for a while now, I doubt this is a gamechanger at all.

Re:Looks good for testing (2, Insightful)

Anonymous Coward | about 2 years ago | (#40434211)


If you were really a sysadmin who could benefit from that kind of speed improvement, you'd know that it's possible to achieve that level of performance with MySQL already, by either running it from memory or by using a fast hard drive array.

The guys that wrote it are former Facebook employees. So I have to assume they know how to get the best performance out of MySQL, and that itdoesn't suit their needs for whatever reason.

The article doesn't really go into much detail about why, but my point is really about not jumping to conclusions and admonishing someone because you think you know more than they do. Maybe this whole product is useless, and maybe it's brilliant and useful, but you can't determine that soley from this article.

okay...? (4, Funny)

bhcompy (1877290) | more than 2 years ago | (#40433359)

When I think of fast databases to compare to, the first thing I think of is MySQL.

/Actually, I'd rather see a comparison to Pick or other lightning fast MV dbs

Re:okay...? (4, Insightful)

Kergan (780543) | more than 2 years ago | (#40433489)

MySQL is the last thing I think of, personally. It sucks as soon as you make it ACID compliant and start hitting it with thousands of concurrent requests. You're much better off with PostgreSQL.

Re:okay...? (4, Funny)

LurkerXXX (667952) | more than 2 years ago | (#40433573)

But with MySQL you can get a wrong answer REAL FAST!!!

Re:okay...? (3, Informative)

Ziekheid (1427027) | more than 2 years ago | (#40433589)

He was being sarcastic..

Re:okay...? (3, Insightful)

evilviper (135110) | more than 2 years ago | (#40433643)

When I think of fast databases to compare to, the first thing I think of is MySQL.

MySQL is actually very fast under light loads / one-off queries, and if you choose to leave it at the non-ACID compliant default settings, and similar. eg. "innodb_flush_log_at_trx_commit"

That's probably the only reason why it got popular... There weren't any open source NoSQL DBs at the time, and MySQL seems fast when tested with a basic, shallow benchmark. Of course others like PostgreSQL completely leave it in the dust once there's some real load, or complex queries, or you WANT to be absolutely sure transactions were committed to disk before returning.

As a single point of evidence, I give you Zabbix... It supports the use of all the major databases (Postgresql, DB2, Oracle, SQLite, etc.) as backends, yet MySQL is recommended as it performs the fastest.
http://www.zabbix.com/documentation/1.8/manual/performance_tuning [zabbix.com]

/Actually, I'd rather see a comparison to Pick or other lightning fast MV dbs

Level-2 overflow! Resize analysis! Change the modulo! Ahhhh!

I've done the PICK-OS thing for a few years, and I'm not a big fan. I'm infinitely happier administering PostgreSQL DBs.

Besides, you don't have to go to something as exotic as PICK to get away from SQL. Try ages-old Berkley DB (db4), or any of the newer NoSQL options.

Re:okay...? (1)

julesh (229690) | more than 2 years ago | (#40433817)

That's probably the only reason why it got popular... There weren't any open source NoSQL DBs at the time

Zope? BDB? Both of these were available at the time MySQL became popular.

Re:okay...? (0)

Anonymous Coward | about 2 years ago | (#40433921)

And then there's how PICK crumbles for large datasets. I remember doing circles around a rather hefty HP box running PICK with a tiny 2 core server running Postgres.

Ahhhh, Pick! (4, Interesting)

hedronist (233240) | about 2 years ago | (#40433891)

The most over-the-top DB God I know started in Pick-land (ca 1972?). Although he does (is forced to?) use SQL nowadays, he thinks in ways that do not come out of any SQL DBA handbook. As a result he gets DBMSs to do things that are ... unnatural.

He is currently doing some data-cubing stuff for us that I didn't think could be done with something less than a DOD budget. He says his touchstone is thinking in Pick and then 'translating' to SQL.

I still think that the 2 missing courses from any CS degree program are 1) how to debug, and 2) history of computing.

Re:Ahhhh, Pick! (1)

Samantha Wright (1324923) | about 2 years ago | (#40434377)

I am intrigued by your ideas. Let us start a newsletter.

Re:Ahhhh, Pick! (4, Insightful)

Zenin (266666) | about 2 years ago | (#40434713)

I still think that the 2 missing courses from any CS degree program are 1) how to debug, and 2) history of computing.

Practical software engineering is mostly about debugging. An actual course in debugging would imply that Computer Science curriculum had something to do with practical software engineering, which we're all painfully away it hasn't in the slightest.

Re:okay...? (0)

Anonymous Coward | about 2 years ago | (#40434101)

KDB [kx.com] anyone?

Show me vs a real DB engine (4, Interesting)

Kergan (780543) | more than 2 years ago | (#40433395)

Show me benchmarks vs Oracle, PostgreSQL or SQLServer. Spare me the comparison with MySQL or some other toy.

vs DB2 (1)

Hyperhaplo (575219) | more than 2 years ago | (#40433679)

I would like to see the compare againsr DB2. Midrange DB2 if you really want like for like, mainframe if you have guts :)

Re:Show me vs a real DB engine (4, Informative)

BitterOak (537666) | more than 2 years ago | (#40433821)

Show me benchmarks vs Oracle, PostgreSQL or SQLServer. Spare me the comparison with MySQL or some other toy.

I think the reason the comparison to MySQL is appropriate is that this database is supposed to be MySQL compatible.

Re:Show me vs a real DB engine (4, Informative)

symbolset (646467) | about 2 years ago | (#40434027)

This i supposed to be funny. Oracle prohibits private benchmark publication in their license.

Re:Show me vs a real DB engine (1)

guruevi (827432) | about 2 years ago | (#40434225)

MySQL is not a toy (anymore). It's been very good for at least half a decade and has been ACID compliant if you have a half-way competent DBA. Also, MySQL is the fastest of the set you just mentioned in the most basic SELECT/INSERT/UPDATE benchmarks (although each of the rest do excel in solving some really specific problems).

Re:Show me vs a real DB engine (0)

antifoidulus (807088) | about 2 years ago | (#40434249)

SQLServer is included in the "not a toy" list? Its a toy DB engine that only runs on a toy OS, its much more of a toy than MySQL.

Re:Show me vs a real DB engine (3, Funny)

gman003 (1693318) | about 2 years ago | (#40434461)

Ah, but it's an "enterprise-grade" toy.

Re:Show me vs a real DB engine (0)

Anonymous Coward | about 2 years ago | (#40434697)

SQLServer is included in the "not a toy" list? Its a toy DB engine that only runs on a toy OS, its much more of a toy than MySQL.

Nope, the MS SQL Server DB engine was originally created by Sybase on Unix & VMS [wikipedia.org] . Not a toy by any means.

You must be thinking about the JET DB engine of MS Access [wikipedia.org] .

Re:Show me vs a real DB engine (2, Funny)

Anonymous Coward | about 2 years ago | (#40434453)

PostgreSQL is definitely the best database software out there.

Re:Show me vs a real DB engine (1)

jmactacular (1755734) | about 2 years ago | (#40434593)

Just wish we could easily re-order columns with FKs. =)

Re:Show me vs a real DB engine (2)

gweihir (88907) | about 2 years ago | (#40434579)

MySQL is not a toy. Oracle is a bloated monster that only survives by locking-in their customers. I know a lot of high-end customers that would ditch Oracle immediately if that would not mean rewriting a lot of software.

Re:Show me vs a real DB engine (0)

93 Escort Wagon (326346) | about 2 years ago | (#40434655)

Show me benchmarks vs Oracle, PostgreSQL or SQLServer. Spare me the comparison with MySQL or some other toy.

Fanboys are cute, in an annoying immature kind of way.

Err... what? (3, Interesting)

Splab (574204) | more than 2 years ago | (#40433399)

Ok, so both article and video is extremely thin on details, the explanation for the massive performance is pretty much gibberish and their argumentation for ACID compliance is bullshit.

Just leaves me with the question, what are they trying to get out of this BS?

Re:Err... what? (4, Informative)

viperidaenz (2515578) | more than 2 years ago | (#40433503)

Just leaves me with the question, what are they trying to get out of this BS?

Your money, its not a free piece of software.

Re:Err... what? (0)

Anonymous Coward | more than 2 years ago | (#40433627)

> their argumentation for ACID compliance is bullshit.

There's no ACID standard so that's literally unimportant. Isolation and Durability are only necessary for comparing states (when written to disk), otherwise those concepts can be disregarded (as with many systems that are eventually consistent-persistent). Memory only experiments that disregard ACID and are useful, has been a long time coming. Many systems already do this with data, but for some unknown reason ... database applications have remained "scary" and non-ACID consideration "heretical".

Re:Err... what? (0)

Anonymous Coward | about 2 years ago | (#40434001)

Many systems already do this with data

No. Many systems already do this with your porn or your email. Our shit is important.

Re:Err... what? (0)

Anonymous Coward | more than 2 years ago | (#40433767)

From what I have read on the subject all scaling performance in new DB engines is gained by gaming acid compliance. SQL just isn't scalable. Don't start with it unless the application has limited scale.

Re:Err... what? (1)

gweihir (88907) | about 2 years ago | (#40434669)

Quite true. Of course this is something the "enterprise" DB vendors are desperate to hide and there are still enough people that do not have enough of a clue about database-theory. The problem is not SQL though, but the relational database model in general.

Re:Err... what? (2)

gweihir (88907) | about 2 years ago | (#40434607)

Self-aggrandizement and money. When somebody claims they are better than everybody else, they are usually lying and knowing it.

Most secure? Best tested? Easiest to use? (0)

Anonymous Coward | more than 2 years ago | (#40433405)

Oh, but those questions don't involve ego, and could involve butthurt.

Bah - I had the SAME basic idea in 1996 onwards (0)

Anonymous Coward | more than 2 years ago | (#40433425)

Windows NT Magazine (now Windows IT Pro) April 1997 "BACK OFFICE PERFORMANCE" issue, page 61

(&, for work done for EEC Systems/SuperSpeed.com on PAID CONTRACT (writing portions of their SuperCache program increasing its performance by up to 40% via my work) albeit, for their SuperDisk & HOW TO APPLY IT, took them to a finalist position @ MS Tech Ed, two years in a row 2000-2002, in its HARDEST CATEGORY: SQLServer Performance Enhancement).

---

* Now, decades LATER, they're only doing an imitation of what I did then? Please... lol!

(Hey - to that? Well... All I can say, is "imitation is the sincerest form of flattery", & "Great Minds Think ALIKE!"...)

APK

P.S.=> I used to design software ramdisk based softwares (drivers + interfaces in GUI etc./et al) & was featured as front page on sites like CENATEK &/or EEC Systems for them, per the above article... but I went elsewhere? Where?? HARDWARE, per this article imitating my idea...

Currently - I use a TRUE "SSD" here, in the Gigabyte IRAM 4gb DDR2/SATA bus (to do the following for systemwide performance enhancements currently (& before it a CENATEK RocketDrive to do the same, after I moved away from ramdisk softwares usage to do the same using dedicated hardwares that have always been FAST ON WRITES & long-lasting keeping their performance level consistent, unlike modern "SSD"s based on FLASH RAM technologies)):

---

1.) Pagefile.sys placement
2.) Custom hosts file placement
3.) Print Spooling
4.) Logging (from the OS eventlogs and any apps that allow movement of logs)
5.) %Temp% environment ops placement
6.) %Tmp% environment ops placement
7.) Opera, WaterFox/Palemoon 64-bit browser placement (all histories & such kept here also, + sessions & bookmarks etc.)
8.) %Comspec% environment var placement (cmd.exe here)

---

... & more!

(They occur faster there, better SEEK/ACCESS by far, & also help by offloading my main C: drive of those duties too, effectively speeding it up by relocating those tasks elsewhere to a faster media & that disk is on a WD Velociraptor 300gb SATA II 16mb buffer 10,000 rpm disk riding on a Promise Ex-8350 PCI-e 128mb ECC RAM caching RAID controller too, for speed...)

apk

Re:Bah - I had the SAME basic idea in 1996 onwards (1)

Anonymous Coward | more than 2 years ago | (#40433559)

The hosts file thing can make networking hugely efficient, given how much time is eaten up on slow DNS servers.

I can easily see a DNS solution using P2P Bittorrent transfers and cryptographic signing that both allows distributed DNS, eliminates spoofing, and creates massive performance gains by mirroring hosts files to clients.

Re:Bah - I had the SAME basic idea in 1996 onwards (2)

icebraining (1313345) | more than 2 years ago | (#40433667)

History

The ARPANET, the predecessor of the Internet, had no distributed host name database. Each network node maintained its own map of the network nodes as needed and assigned them names that were memorable to the users of the system. There was no method for ensuring that all references to a given node in a network were using the same name, nor was there a way to read the hosts file of another computer to automatically obtain a copy.

The small size of the ARPANET kept the administrative overhead small to maintain an accurate hosts file. Network nodes typically had one address and could have many names. As local area TCP/IP computer networks gained popularity, however, the maintenance of hosts files became a larger burden on system administrators as networks and network nodes were being added to the system with increasing frequency.

http://en.wikipedia.org/wiki/Hosts_(file) [wikipedia.org]

Re:Bah - I had the SAME basic idea in 1996 onwards (1, Funny)

Anonymous Coward | more than 2 years ago | (#40433625)

Windows NT Magazine (now Windows IT Pro) April 1997 "BACK OFFICE PERFORMANCE" issue, page 61

(&, for work done for EEC Systems/SuperSpeed.com on PAID CONTRACT (writing portions of their SuperCache program increasing its performance by up to 40% via my work) albeit, for their SuperDisk & HOW TO APPLY IT, took them to a finalist position @ MS Tech Ed, two years in a row 2000-2002, in its HARDEST CATEGORY: SQLServer Performance Enhancement).

This episode of "Tech Tips With Timecube Guy" brought to you by the letters W, T, and F, and by CapsCORP: If Bizarro Caps Are Your Thing, cALl uS!

Meh. (5, Insightful)

hey! (33014) | more than 2 years ago | (#40433435)

Give me fast enough, robust, easy to administer and standards compliant. Maybe a little less fast means you throw more hardware at a problem, but it doesn't matter if overall the overall cost and risk is inflated. A platform decision boils down to three things: (1) is it good enough; (2) is it economical; (3) if we decide later this doesn't work for us, are we totally screwed.

In any case, there's no meaningful way you can make a claim that a database management system is the fastest on the planet. All you have is benchmarks, and different benchmarks apply to different use-cases.

Pedant alert! (2)

PPH (736903) | more than 2 years ago | (#40433497)

What you have there is (or may be) the fastest database management system.

I have the worlds fastest database. One table, one record, and one field (NULL).

Facebook engineers? Gah! (3, Funny)

TheMiddleRoad (1153113) | more than 2 years ago | (#40433531)

I wouldn't run my toaster on software engineered by someone from Facebook, let alone a database. I'd have to spend ten minutes searching for my toast, and it would show up the following week.

Re:Facebook engineers? Gah! (5, Funny)

duk242 (1412949) | more than 2 years ago | (#40433571)

And then the next week, your toast would have changed from white bread to wholegrain and you're just going to have to get used to it.

Re:Facebook engineers? Gah! (1)

kiore (734594) | about 2 years ago | (#40433869)

Not only would it tell all your "friends" and relatives what you are eating and when but the control for turning notifications off would be deeply buried next to the mains power wire and mysteriously switch itself back on at random intervals.

Re:Facebook engineers? Gah! (0)

Anonymous Coward | about 2 years ago | (#40433955)

At least you would have the ability to "Like" the toast :)

But not dislike the toast. (1)

TheMiddleRoad (1153113) | about 2 years ago | (#40434537)

But not dislike the toast.

Re:Facebook engineers? Gah! (0)

Anonymous Coward | about 2 years ago | (#40434209)

and meanwhileyour toaster would have alerted all your friends that you're looking for your toastand an update would go out when you found it! :P

Re:Facebook engineers? Gah! (2)

Rary (566291) | about 2 years ago | (#40434243)

Oh but come on. Their engineers are super leet! To work at Facebook, you have to win a drunken speed-hacking contest just to be a PHP coder!

Faster then MUMPS? (2)

stanlyb (1839382) | more than 2 years ago | (#40433617)

Or its nowadays name: CACHE? The best, the fastest, and the most reliable commercial database on the planet? Common, guys, get real.

Re:Faster then MUMPS? (0)

Anonymous Coward | about 2 years ago | (#40434267)

Don't you want the uncommon guys to get real, too?

What's the magic? (0)

Anonymous Coward | more than 2 years ago | (#40433711)

It can work for small amount of data, but how it "handles terabyte-scale workloads", as claimed in memsql.com, without paging and disk access?

Ok, they complete the sentence with "by connecting MemSQL AND MySQL NODES TOGETHER". It seems Ashton Kutcher & Cia money wasn't enough and they are trying to get more with not-so-technical investors.

Re:What's the magic? (2)

symbolset (646467) | about 2 years ago | (#40434057)

Newsflash: servers come with up to 2 TB RAM now.

Nothing to see here, move along, folks. (1)

140Mandak262Jamuna (970587) | more than 2 years ago | (#40433713)

Some clever tricks and cache management. All the speed improvement seems to be coming via read/write speeds rather than any fundamental breakthrough or parallel implementation or massively parallel database of any such thing. And the test was the standard test but some hand picked data base and their own queries. Probably the original funders are planning to sell it down to the next set of chumps.

How do they write to disk faster? (2)

mounthood (993037) | more than 2 years ago | (#40433777)

They're durable and synchronously log all changes to disk, so what makes them faster? They do say this, from: http://developers.memsql.com/docs/1b/durability.html [memsql.com]

Reconfigure the server to use a faster disk. MemSQL exclusively relies on sequential (not random) disk writes, so using an SSD will dramatically improve durability write performance.

Are SSDs better at sequential writes? I thought their advantage was random reads, and they weren't any faster at writes then HDDs. Also, the data would become hopelessly out of order by only doing sequential writes, unless they're periodically re-writing all the data in order, which would mean lots more I/O then a typical DB.

Re:How do they write to disk faster? (1)

hawguy (1600213) | about 2 years ago | (#40433975)

Are SSDs better at sequential writes? I thought their advantage was random reads, and they weren't any faster at writes then HDDs. Also, the data would become hopelessly out of order by only doing sequential writes, unless they're periodically re-writing all the data in order, which would mean lots more I/O then a typical DB.

They say they rely on snapshots and logging. I'm assuming that it periodically writes a snapshot of RAM to disk, then logs transactions in the log for recovery. Hopefully it snapshots different portions of RAM at different times so there's not one huge snapshot being written to disk every time.

Though if I had a database where I needed 80,000 query/second performance, I'd probably want a cluster of these so if one machine goes down, the other machine can take over so I don't have to wait for the service to restart, then a 100GB snapshot to be read and gigabytes of logs to be applied before the database is back online.

Wondering about GPL violation (0)

Anonymous Coward | more than 2 years ago | (#40433783)

Since MemSQL is MySQL compatible, I'm wondering if they re-implemented the query parser or if they copied GPL code from MySQL. If so, they'd be violating the GPL and Oracle could take them to court. It's possible this whole thing is above board, but I really do wonder.

The other problem is that it's only for Linux. Some of us run databases on other operating systems.

Facebook you say? (1)

outsider007 (115534) | more than 2 years ago | (#40433831)

Is it written in php by any chance??

VoltDB (0)

Anonymous Coward | more than 2 years ago | (#40433837)

In-memory databases already exist. See VoltDB http://en.wikipedia.org/wiki/VoltDB
An in-memory database that supports SQL and is ACID compliant. Full rewrite implementation of SQL with no legacy code.

Completely in memory. Uses replication with failover for recovery instead of write-ahead log to avoid hitting slow disk with write-ahead log. Assign each core a different segment of memory and run single threaded writes to avoid overhead of locking and latching.

Supposedly can do 50x faster than traditional systems already.

See http://www.youtube.com/watch?v=uhDM4fcI2aI for an interesting talk on it.

Re:VoltDB (1)

hlavac (914630) | about 2 years ago | (#40434569)

In memory database and ACID? More like ACIohshitpowerwentoff

Speed vs. speed (4, Interesting)

Todd Knarr (15451) | about 2 years ago | (#40433903)

Speed's fine, but what kind? Or more specifically, over what timeframe? High transaction rates are fine, but they don't do any good if you can only sustain them for a few seconds or minutes before the whole thing collapses. I want to know the transaction rate the thing can sustain over 24 hours of continuous operation. In the real world you have to be able to keep processing transactions continuously.

That long-time-period test also shows up another potential problem area: disk bottleneck. In-memory's fine, but few serious databases are small enough to fit completely in memory. And even if it will fit, you can't lose your database when you shut down to upgrade the software so eventually the data has to be written to disk. And that becomes a bottleneck. If your system can't flush to disk at least as rapidly as you're handling transactions, your disk writes start to lag behind. Sooner or later that'll cause a collapse as the buffers needed to hold data waiting to be written to disk compete for memory with the actual data. You can play algorithmic games to minimize the competition, but sooner or later you run up against the hard wall of disk throughput. And the higher your transactions rates are, the harder you're going to hit that wall.

Re:Speed vs. speed (1)

symbolset (646467) | about 2 years ago | (#40434069)

FusionIO

Re:Speed vs. speed (1)

ByronHope (2669333) | about 2 years ago | (#40434463)

+1, IO doesn't have to be a bottleneck.

Re:Speed vs. speed (0)

Anonymous Coward | about 2 years ago | (#40434333)

I can buy servers with over a Terabyte of ram, mutiple power supplies and 4 x 10G interfaces for FCOE.
What is a disk again other than to boot from.

"MySQL compatible" (1)

Ignacio (1465) | about 2 years ago | (#40433925)

Uh huh. MySQL is missing huge chunks of functionality by default, so this is not all that impressive. Wake me up when it's PostgreSQL-compatible.

Basic Math? (0)

Anonymous Coward | about 2 years ago | (#40433957)

Sounds like fewer and few people can do basic math.
80/3.5 = 30? try 22.8 they are off by over 31.5 %

mod dowN (-1)

Anonymous Coward | about 2 years ago | (#40434257)

is not prone To since then. More

TimesTen Database (2, Interesting)

Anonymous Coward | about 2 years ago | (#40434323)

So what is the difference between MemSQL and TimesTen [wikipedia.org] ?

Other than the 16 years TimesTen has been out longer, the fact that Oracle now owns TimesTen, that it runs on both 32bit and 64bit Linux and Windows, that it can run in front of another database engine to give it a boost, and that it has customer installations up to the Terabyte range.

Just another lame attempt to reinvent the wheel.

Wholesale Authentic Cheap Nike NFL Jerseys NHL Jer (-1)

Anonymous Coward | about 2 years ago | (#40434385)

Wholesale Cheap Authentic NFL Jerseys USA: Customized NFL NHL MLB NBA jerseys,
Cheap Nike NFL jerseys, Women MLB hats Caps, youth soccer jerseys, NHL kids jerseys,
Reebok NHL jerseys and NBA jerseys for sale CheapJerseys.com

http://www.6pmjerseys.com

Filesystem anyone? (2, Informative)

Anonymous Coward | about 2 years ago | (#40434407)

Remember the good old days, when XYZ-db wasn't always available (or even disirable)? we used to use files.

Yea, files. Novel concept, these days, mention ISAM to someone and they don't know what you're talking about!

If you really need speed, maybe a database isn't your best bet. Maybe, just maybe, you should consider structuring the data in a way that makes sense for your application using files.

Looks like that old Prevayler "database" (1)

Lisias (447563) | about 2 years ago | (#40434639)

"No more porridge". Right.

This thing is ACID at least?

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?