Brief update: calc, dvm, distributed filesystem

Journal TTK Ciar's Journal: Brief update: calc, dvm, distributed filesystem 2

Journal by TTK Ciar on Sunday July 20, 2008 @01:21AM

Quick Rundown

Progress on the new house is .. well, progressing. We're having a really hard time of it, but may be getting more help soon. At least our contractor is doing a brisk job on his end -- the deathtrap electrical system has been redone, and cat6 connects patch panels in various rooms to a master panel in the room that will be my office, and the repaired bathroom will have its fixtures installed soon. We got a riding lawn mower, which is kind of annoying to use but also thrilling at times -- there are some pronounced hills in our front yard. I need to trim some low-hanging tree branches before next using it, though.

Calc3 is almost, but not quite, useful. This despite its being 150% the size of calc2. I've been abstracting things nicely and putting in hooks for all the things I want it to eventually become, which eats time and lines of code. I'm also impatient enough, though, that next time I get to it, I'll just focus on getting the calculator mode minimally useful. If it's useful enough that I start using it for my day-to-day work, it should become obvious what my priorities should be, and what features need rethinking.

Some time ago, I was hot after developing a distributed filesystem, not only because I needed one for my own use, but because I wanted to make some money. I know of a guy who runs a business which would benefit tremendously from being able to offer a distributed filesystem on his products, and figured he might be willing to license the thing, if I could get something working.

When I switched jobs, money ceased to be so much of an issue (did I mention that The Archive pays its employees about 2/3 what they're worth? Consider it mentioned), so I backburnered the idea. Lately I've been revisiting it because, well, I still need one. The original plan was to develop a "tolerably functional" interim solution fast, then work on the "preferred" solution.

My idea for the interim solution was to use linux's "network device" (nnd) driver to make really big network-mounted RAID5 and/or RAID6 filesystems, then use SAMBA to export them, and write some automated monitoring and control logic to manage the system and to provide users and administrative interface. This would have the advantage of being relatively fast and easy to develop, and would leverage linux's existing software-RAID technology to make the cluster's disks look like one huge filesystem (or a few). It would also have several disadvantages, though: Users' accesses to each distributed RAID array would be funnelled through a single server, making it both a SPOF and a performance bottleneck. Adding new space to existing filesystems would not be seamless, necessitating significant service disruption as the array rebuilt itself. Linux's RAID6 bites, performance-wise, and would limit the robustness of the cluster -- if three hosts went down, or even the wrong three disks, the filesystem would cease functioning. Finally, it turns out that nnd is not really ready for production use, so basing the filesystem on nnd would be like building a house on sand.

On top of it all, very little (if any) of the work I'd put into this interim solution could then be used to develop the preferred solution. It would all be throwaway code. The only real appeal was that I'd perhaps get a sellable product sooner, and be making a little money (hopefully) while developing the preferred solution.

Since I'm no longer feeling pressured to get something out fast, I'd might as well just start work on the preferred solution, which would provide distributed access (so users' connections could use any host's network connection for their accesses, alleviating the most severe bottleneck), better-than-RAID6 redundancy, and seamless addition of new space to existing filesystems.

Towards this goal, it makes sense to start with a project I've already been working on: DVM. It would provide the robust messaging system the cluster would need to chat amongst itself, and I need it for other projects anyway. It also makes sense to get DVM working first, because things might happen in the meantime which makes the distributed filesystem project moot. Google might publish its GFS and make it free for everyone to use. Ceph might come out with a better distributed filesystem than mine (I had sought a job there, but it didn't work out -- they have more technical talent than business-running talent). EMC or some other company might open-source their solution. Hell, I might even lose interest. If any of these things happen, then any work I've done which is specific to my distributed filesystem project will be wasted effort. But if I've spent my time working on DVM, then none of my work will have been wasted, because DVM has many uses.

Sometimes it seems like all roads lead back to DVM. Some of the functionality I want for calc3 will want DVM, too. So I'm working on DVM. This iteration is being developed in C, so it's slow going. I have a hash library which implements perl-like hash tables ("dictionaries", for the philistines amongst my readership), which helps speed things up, but it's still going to be a long while before I get it to a useful state. One advantage to my frustration is that it's encouraging me to really, *really* avoid early optimizations and unnecessary features (which one should avoid anyway -- it's just good engineering). Those can come later. This iteration is looking a lot more like conventional old-school unixy networking code than my other attempts, with big select()-and-loop constructs and the like. Maybe that's a good thing. We'll see.

-- TTK

This discussion has been archived. No new comments can be posted.