Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ask Slashdot: Do you want a local object store (i.e. a flat file system)?

DidgetMaster (2739009) writes | about a year and a half ago


DidgetMaster writes "Object stores have been around for a while now. For example, Amazon S3 storage is a set of buckets in the cloud in which you can store millions of files as "objects". Local file systems have been around much longer. For nearly 50 years now, we have been stuck with the traditional hierarchical file system storage model (e.g. a tree-like structure of folders or directories). Both systems make it easy to store a ton of data and to find a single item very quickly if you know its unique ID (full path for files, key for objects). But both systems are terrible at searching for all data that have certain features. If you have 2 million files on a file system volume and you want to find all pictures (*.jpg, *.png, *.ico, etc.), then it takes forever to scan the whole system looking for them. Cloud systems are not any better at search. Adding extra metadata (e.g. tags or extended attributes) can help distinguish one file or object from another, but searching for things based on those is even slower. "Find all documents where Author=John" is only fast if all the metadata has been collected and stored in a separate database, otherwise go to lunch while you wait for the results.
The Didget Management System wants to change all that by introducing a new object storage model designed to replace file systems. A Didget (Data Widget) is like a file that can contain any unstructured data stream up to 16 TB in length, but it is also like a row in a NoSQL database table where lots of searchable structured tags can be attached to it. Structured and unstructured data can be stored side-by-side within the same container and both types can be returned as the result of a query. Note: this is NOT like other indexing systems like Spotlight or Windows Search. See the 5 minute video demo at
Is the ability to instantly find "All photographs taken in Hawaii in 2011" when there are 100 of them among 5 million other pieces of data, enough for you to want to replace your file system with something new?"

Link to Original Source

cancel ×


Sorry! There are no comments related to the filter you selected.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>