Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

How Do You Sync & Manage Your Home Directories?

kdawson posted more than 5 years ago | from the kitchen-sync dept.

Data Storage 421

digitalderbs writes "A problem plaguing most people with multiple computers is the arduous task of synchronizing files between them: documents, pictures, code, or data. Everyone seems to have their own strategies, whether they involve USB drives, emailed attachments, rsync, or a distributed management system, all of which have varying degrees of success in implementing fast synchronization, interoperability, redundancy and versioning, and encryption. Myself, I've used unison for file synchronization and rsnapshot for backups between two Linux servers and a Mac OS X laptop. I've recently considered adding some sophistication by implementing a version control system like subversion, git, or bazaar, but have found some shortcomings in automating commits and pushing updates to all systems. What system do you use to manage your home directories, and how have they worked for you for managing small files (e.g. dot configs) and large (gigabyte binaries of data) together?"

cancel ×

421 comments

Svn (0, Redundant)

WillKemp (1338605) | more than 5 years ago | (#28443419)

Subversion.

Re:Svn (2, Insightful)

Anonymous Coward | more than 5 years ago | (#28443505)

Now that you have first post with that entirely informative response, do you care to elaborate on how you have implemented it?

Re:Svn (3, Informative)

WillKemp (1338605) | more than 5 years ago | (#28443885)

If i'd elaborated, i wouldn't have made first post!

However, i use subversion for two things - backup and syncing my development system with my remotely hosted web server. Neither of which is really how i "sync and manage home directories", but if i needed to do that subversion is what i would use.

Some months back, i foolishly pointed to my web hosting service that there was a serious security hole in the way their system (cpanel) was configured for subversion - and they killed the subversion service and haven't reinstated it. So i have to do 'svn update' over an sshfs virtual file system, which is mildly irritating.

Anyway, i've got a single repository set up on my system and i check in all new web sites i'm working on. Then i check them out onto the server - and update the files on the server with 'svn update'. It's easy, reliable, and reasonably fast. It also makes backup nice and easy, as i just sync the repository with a mirror on an external hdd.

Re:Svn (4, Interesting)

MaskedSlacker (911878) | more than 5 years ago | (#28443659)

I use git, with flashbake and cron to automate commits, and a simply cron job to automatically update a backup copy on an external hard drive.

Re:Svn (0)

walshy007 (906710) | more than 5 years ago | (#28443903)

seconded

And the Large Files? (2, Informative)

eldavojohn (898314) | more than 5 years ago | (#28443699)

I've used subversion quite a bit and we simply avoid committing Java archives and instead use Maven2 to get those. This is because it seems to take up a lot of space and time with large files. Maybe this is typical of any versioning system but I do not know enough about git. From Subversion's best practices:

Be patient with large files

A nice feature of Subversion is that by design, there is no limit to the size of files it can handle. Files are sent "streamily" in both directions between Subversion client and server, using a small, constant amount of memory on each side of the network.

Of course, there are a number of practical issues to consider. While there's no need to worry about files in the kilobyte-sized range (e.g. typical source-code files), committing larger files can take a tremendous amount of both time and space (e.g. files that are dozens or hundreds of megabytes large.)

To begin with, remember that your Subversion working copy stores pristine copies of all version-controlled files in the .svn/text-base/ area. This means that your working copy takes up at least twice as much disk space as the original dataset. Beyond that, the Subversion client follows a (currently unadjustable) algorithm for committing files:

* Copies the file to .svn/tmp/ (can take a while, and temporarily uses extra disk space))
* Performs a binary diff between the tmpfile and the pristine copy, or between the tmpfile and an empty-file if newly added. (can take a very long time to compute, even though only a small amount of data might ultimately be sent over the network)
* Sends the diff to the server, then moves the tmpfile into .svn/text-base/

So while there's no theoretical limit to the size of your files, you'll need to be aware that very large files may require quite a bit of patient waiting while your client chugs away. You can rest assured, however, that unlike CVS, your large files won't incapacitate the server or affect other users.

Really, I think he's asking for one tool to do both small files and large files when (in my mind) it makes more sense to back up ISOs and MP3s over longer periods of time than my source code or documents that I may edit and change daily.

Subversion for source control. A simple script that pushes large files to an external drive. That's all I do. Bulletproof? No way. But it sounds like he's devoting a lot of time to this. I guess he must have a lot more computers than I do.

Re:Svn (1)

dword (735428) | more than 5 years ago | (#28444039)

Yes, I wonder what the shortcomings of CVS are. I'm using it myself and it does exactly what I need, so I assume that "shortcomings" depend on the individual (no penis joke pun intended).

CVS (0)

Anonymous Coward | more than 5 years ago | (#28444063)

I've been using CVS for over a decade now. Generally with small files like .cshrc, .emacs, etc.

test (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#28443429)

test

Re:test (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#28443685)

did it work?

Re:test (2, Funny)

WillKemp (1338605) | more than 5 years ago | (#28444167)

No - he was trying for 5, but only got -1!

Dropbox (4, Interesting)

snl2587 (1177409) | more than 5 years ago | (#28443469)

I recently started playing around with Dropbox for some smaller folders than my entire home directory and haven't yet run into any major problems. And the versioning it provides is nice as well, and as a plus they don't consider the deleted files that they still retain versions of as part of the quota.

Re:Dropbox (-1, Troll)

Anonymous Coward | more than 5 years ago | (#28443613)

If you haven't already, sign up using this link and get 250MB more free and, obviously, I get 250MB more as well.

https://www.getdropbox.com/referrals/NTg0NDY4OQ

Re:Dropbox (1)

Stinky Fartface (852045) | more than 5 years ago | (#28443781)

I'll second this. I like my Dropbox. I referred enough people to get up to 3GB free, and that is surprisingly useful. I fantasize about my entire desktop running from my Dropbox but I can't afford that level of service.

Re:Dropbox (1)

digitalderbs (718388) | more than 5 years ago | (#28443835)

This does seem like a very viable option. For those not aware, it's cloud server that you upload from /download to. For
It appears to be a very clean system, but I would be concerned about having open/unencrypted files on an uncontrolled server. Dropbox would be great if you could manage your own server, which doesn't appear to be the case.

thanks for the link.

Re:Dropbox (1)

bobstreo (1320787) | more than 5 years ago | (#28443861)

I'll ++ dropbox especially for locked down "workputers" and links you want to look at later.

For bookmarks, I use spurl and the RSS feed.

For RSS reading and browsing, I use netvibes.

For all the big stuff. I have a NAS box so I don't have to store all the big files
everywhere.

Re:Dropbox (4, Informative)

buchner.johannes (1139593) | more than 5 years ago | (#28443909)

Have a look at Jake [slashdot.org] . Official website: Jake [jakeapp.com]

It is aimed for the average user (no server setup needed) and provides a syncing solution across the Internet with a nice UI. Free and open source, available for all operating systems.

Check it out!

Re:Dropbox (1)

compro01 (777531) | more than 5 years ago | (#28444029)

Is it just me or has that site gone down already?

Re:Dropbox (1)

syphax (189065) | more than 5 years ago | (#28443983)

I pay Dropbox for 50GB of space (I'm around 1/2 full, mostly pictures).

It's not perfect, but it's pretty damn good and I'm not looking back.

I'm too old to screw around with DIY approaches.

Re:Dropbox (1)

bastion_xx (233612) | more than 5 years ago | (#28444117)

I love Dropbox. The first thing I normally do is *not* store my home directory files in the default locations. It's easier to create a Vista/WIn7 favortite c:\My Dropbox, An OS X folder /home/user/My Dropbox, and then make entries in Explorer / Finder to make it easier to access.

I'm on the free plan at present, but Dropbox will get my money.

The ability to go back and restore files is nice too.

rsync (1)

timdrijvers (1583565) | more than 5 years ago | (#28443475)

rsync + hardlinks

time machine is better (4, Informative)

goombah99 (560566) | more than 5 years ago | (#28444165)

for backups I used to swear by rsync plus hardlinks. But since time machine came out it's oh so much much better. For one thing rsync is still a bit unstable on huge directory trees that contain lots of hard links. And it also boofs on some extended type attributes, forks and file types, though it keeps getting better (perhaps it's perfect now). Rsync + hardlinks also does not retain the ownership and privledges and ACL faithfully either.

But even if Rsync + hardlinks didn't have those troubles, time machine is so flawless it's just the thing to use. What is especially nice about time machine is the recovery and inspection process. it's not too hard to figure out what files chaged (there's even a 3rd party gui application for this) and because this info is stored in meta data it's faster and more relaible to retreive than a massive FIND command looking at time stamps. The time machine interface for partial recoveries is intuitive and easy to drill down. In many cases it's even application aware so you can drill not on the file system itself but on say your mail folders in the mail application. this is actually a pretty stunning achievement that needs to be seen to be believed how paradigm shifting it is.

And full recoveries could not be easier. you just boot off the CD and within ten clicks you have picked the souece and destination and it has done a series of idiot checks. While that might not seem too amazing, it sure is comforting. It's a mildly nerve wracking process of trying to recover from a back-up cause there's lots of ways to goof and maybe even wreck your original ( like oops, I didn't do a -delete, or I didn't tell it to reassign links, or worse I copied the wrong direction).

Here's a super nice tip: you can have two disks operating with time machine that you rotate. Actually the best way i've found is to have one constantly attached then on fridays attach the other one, redirect time machine to it, let it back up all the changes since last friday, then detatch it and let time machine go back to the main disk.

You can even use this as a way to sync your two computers though it's better as a backup than as a synch. have time machine back up just your home directory to a thumb drive, take this from home to work. plug it to the drive at work, back it up. then revert this to the backup from home. now home and work are synced plus, if there was one special file or two that was newer at work, well you have that in the backup you made! ( by the way to do this kind of thing requires fiddling with the backup cookie so two computers can share the same repository. google this if you want o know how)

Dropbox (1)

princessproton (1362559) | more than 5 years ago | (#28443495)

Since I don't have a lot of content to sync, Dropbox [getdropbox.com] meets my needs perfectly (and it's free!).

Myself... (2, Interesting)

Darkness404 (1287218) | more than 5 years ago | (#28443499)

Myself I simply store contact info "in the cloud", use my MP3 player to hold all my music and occasionally sync it to all computers to have a copy, any needed documents are either somewhere on my e-mail or on a USB drive, same with code. I have different computers for different purposes and have different data on each one. I never really liked the idea of having the same everything on all computers, most of my computers have different OSes, distros and desktop environments.

Re:Myself... (1)

IANAAC (692242) | more than 5 years ago | (#28443873)

Myself I simply store contact info "in the cloud", use my MP3 player to hold all my music and occasionally sync it to all computers to have a copy, any needed documents are either somewhere on my e-mail or on a USB drive,

I do the opposite with my mp3s.. I have a media server in the living room that has my entire collection. from there I mount those (NFS, in my case) to whichever system I happen to be working on. Or I copy different songs onto my mp3 player before leaving the house with it.

For documents, I use Opengoo (http://www.opengoo.org). It was super easy to set up on my hosted server, and for what I do - translation - it allows me to let my clients interact with the documents as well, should I want/need them to. I highly recommend it.

I do pretty much the same as you for my contacts, and sync them to my BB.

always mount your home dir with NFS (-1, Troll)

Anonymous Coward | more than 5 years ago | (#28443525)

so I can write an shosts/rhosts file with + + and voila! You're totally pwned. Suckers.

When it comes to hacking, you bitches be playin' checkers and I'm playin' chess.

Re:always mount your home dir with NFS (3, Insightful)

Foldarn (1152051) | more than 5 years ago | (#28443779)

And when the server hosting your NFS share dies, so does your entire home directory on every PC. Check and mate.

Re:always mount your home dir with NFS (1)

venom85 (1399525) | more than 5 years ago | (#28444091)

And when the server hosting your NFS share dies, so does your entire home directory on every PC. Check and mate.

It's not necessarily as world-ending as you make it sound. I use NFS to mount home directories as well as a couple others. I also have redundant hard drives on the Linux server, and each night rsync makes the backup drive a mirror image of the main drive. I could schedule that to run more often, but I don't change things often enough to make more than once a day necessary. If the main hard drive dies, I can very quickly and easily switch over to the backup drive temporarily until I get a new drive with all the data on it. If the server itself dies, it wouldn't take significant time to get it set back up. The only way I'm really stuck is if the main OS hard drive (separate drive altogether) dies, and even that only keeps me without my home directory for a week tops. For some applications, that may be too long, but in those cases you could simply have the rsync nightly backup also copy everything to a backup directory on a second server that could be your backup NFS host. The main article said that the user has two Linux servers at his disposal right now, so that could easily work. It would take all of about 5 minutes to get the backup server to point /home to the backup directory and get NFS shares on the laptop to point to the new NFS host. If all you're looking for is syncing between hosts, NFS is the way to go. If you're looking for versioning too, then it may not fit the bill quite as well.

Re:always mount your home dir with NFS (1)

Foldarn (1152051) | more than 5 years ago | (#28444199)

I agree wholeheartedly. I was just knocking the person before me down a notch or 2. Apparently using NFS shares is 'hacking'. I would prefer rsync by FAR over a simple NFS share. It provides redundancy as well as maintains deletions, etc.

"Distributed homedirs" or "CVS'd configs"? (1)

Slipped_Disk (532132) | more than 5 years ago | (#28443533)

I'd be interested in answers for this from the "I want the same homedir contents everywhere, but NFS makes me vomit" standpoint :)

For managing the default profiles around the office we use git - the dotfiles & such are managed, and the rest is left as an exercise for the user.
It's not ideal (I hate it), but it's what we've got...

Re:"Distributed homedirs" or "CVS'd configs"? (1)

hierofalcon (1233282) | more than 5 years ago | (#28443853)

Out of curiosity, what's wrong with NFS?

Re:"Distributed homedirs" or "CVS'd configs"? (2)

eln (21727) | more than 5 years ago | (#28443961)

It got a bad reputation due to some very nasty security holes 15 years ago, and so lots of people don't want to use it anymore. That and it's possible to make it highly insecure through export options, but if you're dumb enough to export your stuff to the world read/write with root privs, you probably shouldn't be allowed to operate a computer much less set up a home network.

Personally, I would think for a simple home network, having an NFS shared directory is the easiest and most practical solution. Sure you can't just take your laptop away and have all of your content, but given the storage capacity of your average laptop that's unlikely to be realistic anyway.

Of course, make sure you have a good backup solution for your file server, and make sure the file server is well protected behind your firewall, but other than that NFS should be just fine.

Re:"Distributed homedirs" or "CVS'd configs"? (1)

short (66530) | more than 5 years ago | (#28444071)

Using $HOME in CVS since 2001, it works perfectly, it is public:
cvs -d :pserver:pserver:@dyn.jankratochvil.net/cvs co nethome
Checking it out across the world on various machines. If I find I miss something on some host, I do `cvs update' by hand. Not a rocket science.
I have only one host I consider secure enough so there is no point in distributed mails.
I was using my Gecko bookmarks in $HOME but I had to create a small script [jankratochvil.net] to "normalize" them. Otherwise they contain a lot of useless info (timestamps, whether expanded etc.) making both history diffs useless and conflict merges difficult/impossible.

SSHFS (2, Informative)

WheelDweller (108946) | more than 5 years ago | (#28444189)

Look into sshfs. Keep your home machine ssh-reachable (isn't it already?) and you'll be able to ssh into it, mounting the filesystem on your desktop. It's convenient, secure, and effective. Works anywhere ssh does. Good stuff!

Chuck Norris (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#28443541)

Works flawlessly

Cloud (1)

Fastball (91927) | more than 5 years ago | (#28443565)

I have been considering this more of late as I utilize my new netbook. It's my first portable computer ever; I've clutched to my desktop since the beginning of time.

My comfort with this is directly related to who I can trust to store my data with. Right now, it's not a big enough need, so I'm reluctant. However, I can see my needs growing to a point where I'm going to take the plunge.

Any recommendations or horror stories anyone wants to share are welcome here.

SmartSync (1)

jbeale53 (1451655) | more than 5 years ago | (#28443567)

I've used a program called SmartSync since the early part of this decade. I think it was around $30, but it does a great job of synchronizing, including deletions. I do believe it's a Windows-only app, though, so while it may not work for your situation, some other slashdotters may find it of some use...

Windows users? (1, Informative)

DoofusOfDeath (636671) | more than 5 years ago | (#28443571)

Do Windows users even have anything like this, aside from Active Directory?

Re:Windows users? (1)

iMaple (769378) | more than 5 years ago | (#28443727)

Check out windows live mesh ( http://www.mesh.com/ [mesh.com] ) . It works with Macs and windows mobile phones (no linux support :( ) and you get 5GB of cloud storage too.

Re:Windows users? (3, Insightful)

iamhigh (1252742) | more than 5 years ago | (#28443891)

Out of curiosity, what do you think AD does that provides anything close to what the author is asking?

Re:Windows users? (1)

jtdennis (77869) | more than 5 years ago | (#28444203)

AD can set up Roaming profiles or use group policies and redirect profile folders to a central server. The benefit is everyone's data is in one place which can be backed up easily.
Roaming profiles are very bandwidth intensive, but the folder redirection isn't that bad, and I've used it for years at work.

Re:Windows users? (3, Informative)

cdub1900 (1167623) | more than 5 years ago | (#28444197)

Windows Live Mesh
https://www.mesh.com/ [mesh.com]

"With Live Mesh, you can synchronize files with all of your devices, so you always have the latest versions handy. Access your files from any device or from the web, easily share them with others, and get notified whenever someone changes a file.

Working on one computer, but need a program from another? No problem. Use Live Mesh to connect to your other computer and access its desktop as if you were sitting right in front of it. "

Not strictly "home directory", but... (2, Interesting)

langelgjm (860756) | more than 5 years ago | (#28443575)

Strictly speaking this isn't about my "home directories", but I use a combination of things to sync the information I need between my desktop and laptop:

1. Windows Live Sync (formerly known as Foldershare). Yeah, I know, it's from Microsoft, and only available for Windows and OS X, but it works pretty seamlessly and without much intervention on my part. I sync specific directories of projects I am working on using this; it's a set it and forget it tool.

2. Foxmarks for bookmarks.

3. Zotero, version 1.5 of which has the ability to sync resources between multiple computers. This is a godsend.

DRBD (1)

schmaustech (766915) | more than 5 years ago | (#28443583)

Replication with DRBD.

The old-fashioned way? (4, Interesting)

clang_jangle (975789) | more than 5 years ago | (#28443587)

I use multiple OS X, Linux, and FreeBSD machines daily. One cannot sync all home directory files, as all the config stuff differs between Gentoo, Debian, FreeBSD, Tiger, and Leopard. So it's mostly down to documents, graphics, and a few audio and video files. For the larger ones, I use a usb stick, the smaller ones I email to myself so they're always available via IMAP servers. But most of all I have a bootable, customized version of systemrescuecd installed on a 16GB usb stick, which at any given moment has all the currently important stuff I need. It works well enough for me.

Re:The old-fashioned way? (1)

Slipped_Disk (532132) | more than 5 years ago | (#28443787)

I use multiple OSX, Linux and FreeBSD machines daily - The Mac is in my office, the rest are about 45 miles away in our datacenter. I spend 80% of the day logged into various machines remotely and spend a substantial chunk of time copying files from host to host. There's also the headache when dotfiles get out of sync and machines behave differently for what should be identical commands (beyond the usual BSD-vs-SysV crap)

All by way of saying "Gee, wouldn't it be swell if I didn't have to manually sync these or painfully suffer through git merges?" :)

Different tools for different purposes (5, Insightful)

joe_cot (1011355) | more than 5 years ago | (#28443603)

  • If you're keeping track of code, use a code repository. Subversion, GIT, Bazaar, etc.
  • If you're trying to keep config files, documents, pictures, etc synced, use DropBox [getdropbox.com] .
  • For bookmarks, use one of the numerous Firefox bookmark syncing extensions, or the Del.icio.us extension (or use DropBox to sync your .mozilla/firefox folder).
  • For multi-GB files, use a portable hard drive, or rsync with a file server in your house/office

I wouldn't recommend using one tool for every purpose. I wouldn't want to store multi-GB files in SVN, and I wouldn't want to store all my code on an external hard drive. Maybe using DropBox, or rsyncing with a server somewhere would work.

Re:Different tools for different purposes (1)

644bd346996 (1012333) | more than 5 years ago | (#28443713)

For bookmarks (and history, etc.), I've found Mozilla Weave to be a great replacement for the abandoned Google Browser Sync.

Re:Different tools for different purposes (1)

Braino420 (896819) | more than 5 years ago | (#28444119)

My Google Browser Sync replacement is Xmarks (formerly foxmarks). I only use it on Firefox, but it works on IE and Safari too. The only thing I miss from GBS is how it saves the last windows you closed and then opens them up next time you start the browser on any computer, does Weave do this? Xmarks also gives you a nice page to manage your bookmarks and create different profiles, etc. You can also run your own Xmarks server so you can store your own passwords. I think it uses WebDAV, which could also be of use to the OP.

Windows Live Sync (1)

halfabee (685633) | more than 5 years ago | (#28443609)

Live Sync [live.com] does a fair job of syncing folders in Windows (sorry) hosts. Whenever both computers have internet access, the folders sync. Live ID required (again, sorry.) Few limits on data (no files larger than 4GB, no more than 20,000 or so folders?)

Live Mesh is a similar Microsoft offering, but also give you web access to your files and stores a copy on the cloud. Sadly, there is a 5GB limit on total data.

Store everything relevant on the home server (1)

dastrike (458983) | more than 5 years ago | (#28443617)

I just store anything saveworthy on the server at home. That way the "client" computers (my main "work"station/gamestation and laptops) contain only relatively expendable data. And if I need to access the stuff from some other location I just SSH into the box.

And backups are handled from the server manually occasionally to an external USB drive. I know, I need to improve on that part.

Xmarks for bookmarks (1)

sean_nestor (781844) | more than 5 years ago | (#28443633)

Not sure how much you use bookmarks, but I maintain a good archive of organized chaos myself. At one time, bookmarking a site was a pretty pointless endeavor, since I couldn't keep track of whether I had bookmarked it at work, on the bedroom computer, downstairs computer, laptop, etc.

So a while ago, I looked up a plug-in for Firefox called Xmarks (formerly FoxMarks) that was pretty much exactly what I was looking for. Very easy to set up and use, cross-platform, etc.

Just thought it was useful enough to mention here.

backintime, and rdiff-backup (4, Interesting)

gardyloo (512791) | more than 5 years ago | (#28443645)

For small backups, every ten minutes, I use backintime (based on rsync). For larger, nightly or more rare backups, I use rdiff-backup. Both work over the LAN, or to locally-mounted hard drives.

USB drive (4, Interesting)

the_rajah (749499) | more than 5 years ago | (#28443649)

I carry a 16 Gig USB flash drive with my working files on it. I've using this method since the days of 100 Meg Zip drives and just keep upgrading the media. My flash drive is automatically backed up to my backup server at home in the middle of the night so, if I forget it at the office, I'm only a few hours behind. Besides, I can use free Logmein to log into the office computer and transfer a file if it's got new and important information on it. It works the same way in reverse if I forget it at home. Since my working files are on the USB drive which is also compatible with my Linux machines, it really doesn't make much difference which machine I plug it into. Did I mention encryption? That's a good idea in case you lose the drive if you've got any sensitive information on it.

What about a FUSE FS powered by a MySQL DB? (3, Interesting)

necro351 (593591) | more than 5 years ago | (#28443653)

FAST 2009 has a paper on semantic data management using a file system built on top of an object store powered by MySQL. Performance isn't great, but it uses a distributed file system solution to solve the synchronization issue in a very nice way (e.g., synchronize all albums with my iPod, all photos with my laptop and computer, etc...). You can specify rules and I liked it when I heard about it. However performance is actually important, despite their claim :). Perspective: Semantic Data Management for the Home Brandon Salmon, Carnegie Mellon University; Steven W. Schlosser, Intel Research Pittsburgh; Lorrie Faith Cranor and Gregory R. Ganger, Carnegie Mellon University HTML Paper http://www.usenix.org/events/fast09/tech/full_papers/salmon/salmon_html/index.html [usenix.org] PDF Paper http://www.usenix.org/events/fast09/tech/full_papers/salmon/salmon.pdf [usenix.org] Slides http://www.usenix.org/events/fast09/tech/slides/salmon.pdf [usenix.org]

Dropbox (1)

gphilip (1155691) | more than 5 years ago | (#28443673)

http://www.getdropbox.com/ [getdropbox.com] Works perfectly for me. I use the free option, which has a 2 GB limit, which is more than enough for me to keep all my important stuff in sync. It has a client that integrates nicely with the Nautilus file browser on Ubuntu, which is what I use at home and office. Whatever you put (or symlink) into a designated folder (which you can choose) gets mirrored to their server, from where it gets synced to every other system where you have installed the client, the next time you connect that system to the internet. They also give web-based access to the stored files. There is the issue of privacy for the really paranoid, but I am not very concerned about that with the files I currently choose to mirror. I am more worried about the chance that their client develops a bug that wipes out my files, but I guess I'll take that risk.

AllwaySync (1)

Ritorix (668826) | more than 5 years ago | (#28443679)

freeware @ http://allwaysync.com/ [allwaysync.com]

I was playing with this for the first time last night, it gets the job done. Sync software with a nice GUI, and I was easily able to backup my systems to a 1tb backup drive in a reasonable time period. It has the usual features and can sync in multiple directions (one to many, bidirectional or one way).

Keep only one home directory (1)

Akir (878284) | more than 5 years ago | (#28443701)

There are numerous ways to do this, but I would use Samba (best used with VPN), because it can pretty much be used everywhere, even on computers with crappy OSes like Vista. There are numerous other solutions for the same thing; FTP, AFS, maybe iSCSI.

File server? (0)

Anonymous Coward | more than 5 years ago | (#28443707)

All my data goes on a Samba file server in the garage (which is separated from the house so hopefully in the event of a fire I don't lose both) and my various machines access this, with rights according to the user used to mount the Samba shares. This runs a nightly incremental tar to an external eSATA drive. Every time I copy any data I really care about (like new photos) onto the server I also run rsync to sync it to my machine in the house. Future plans include FreeNAS with ZFS (and hopefully snapshot support), and a number of external hard drives that I can keep at parents houses.

portable hard drive (1)

Compunexus (711717) | more than 5 years ago | (#28443723)

I have found that a portable hard drive with an encrypted partition works best. Put DSL in the bootable partition in case I have to work at an untrusted PC. FEBE and gmarks cover the worst of my needs on Firefox. If someone swipes my drive, they don't get my data.

Beyond Compare (3, Informative)

Anonymous Coward | more than 5 years ago | (#28443729)

On the windows side there is a great utility called Beyond Compare, around $30, that I have used to do this. I even had a small client once that could not afford a real backup software, so we faked the backup using portable USB hard drives and the Beyond Compare utility to sync her server and desktop to the drives. Worked quite great and the while thing was done for under $200.

Home Server (1)

therapyreject (1582611) | more than 5 years ago | (#28443747)

I save any data I might need shared to my server at home, and just map a drive on all my pc's I use. I only use Windows at home, so its pretty simple and works just fine.

Windows - SyncBack (4, Interesting)

Anonymous Coward | more than 5 years ago | (#28443755)

I spent a long time tackling this, as I am situated at different locations on different days.

I have 2 desktops and a laptop which must remain sync'd and encrypted. I use TrueCrypt for the encryption.

On my Windows boxes - SyncBack handles it. It can be triggered on write or on insertion, or just periodically. Has version control support. Will sync over FTP (poorly) and can create zip files or burn Cds etc. It's a swiss army knife of sync tools.

The key for getting the most out of a sync program is granularity. Inevitably, you'll have exceptions, and you don't want a PASS/FAIL result for your entire backup set. It works much better to sort files into categories and sync the individual groups than to try to make one profile that does your entire disk array. My 2 cents.

USB Drive. (0)

Anonymous Coward | more than 5 years ago | (#28443757)

If you keep everything really organized a usb stick and an external hd suffice. Occasionally overwriting old files on the hd to keep "backups" of the usb stick.
The problem is with my music collection is 30GB and my usb stick is 2GB :) I'm looking forward to replacing my usb stick for a 32+ GB usb stick so that I don't have to worry about extra hd's.

Mobile Home Directory (2, Informative)

Drizzt Do'Urden (226671) | more than 5 years ago | (#28443765)

At home, I've got a Linux server hosting an LDAP structure to mimic MacOS X Server's config. It is sharing my home directory via NFS. My Macs sync this home directory on login and logout, so all my personnal data is centalized for easy backup and available on any Mac I happen to add to my home network.

Re:Mobile Home Directory (1)

Drizzt Do'Urden (226671) | more than 5 years ago | (#28443821)

Some info about the settings here [menzonet.org] (in french)

Re:Mobile Home Directory (1)

dburkland (1526971) | more than 5 years ago | (#28443953)

I have something similar setup in that I got a FreeBSD setup with jails (openldap, NFS, Apache, etc) which hosts all of the home directories for my machines. Works quite nicely

Rsync (1)

Perl-Pusher (555592) | more than 5 years ago | (#28443777)

Whenever I'm on my workplace network I have cron job that uses rsync to sync my Documents directory between my linux desktop & mac laptop. This way the latest file is always on both machines. The Desktop is also rsync'd to a backup server daily, and weekly for off site storage at bank vault. And the Laptop uses Time Machine at my home. This allows me the flexibility of grabbing older files if I need them. For mail I use imap with SSL. So short of a nuclear holocaust, it would be pretty hard to lose a file.

Partway there, but data safety first (1)

hawk (1151) | more than 5 years ago | (#28443843)

Right now, actual synchronization tends to be entirely manual, with scp of subdirectories and possibly a tar -c | tar -x combo to not overwrite newer copies.

I decided to work on data integrity first--but then I have client info to consider.

I have a 3x1.5T zraid array using full disks on the main machine, and an external 1.5T for backup (I'll grab another, so I can have two alternating backups). These will stay disconnected when not backing up, and in two other rooms of the house. I'll probably copy zfs snapshots of /home, probably filtered for any /cache/ and so forth. I also have offsite backup that i haven't gotten around to enabling ):

I'm planning on actually figuring out rsync, and from there specifying the parts of /home/* that get synchronized (possibly to an nfs mount off the main machine?)

hawk

FTP (2, Interesting)

NineNine (235196) | more than 5 years ago | (#28443849)

FTP back and forth, select the root and overwrite whatever's newer. Unless the time on the files gets screwed up, it works fine. Worst case scenario, which is the dates/times getting messed up, the FTP client downloads everything. No big deal. I do it daily for all kinds of files.

As older, wiser programmers than myself have always told me: KISS: Keep It Simple, Stupid.

DropBox + symlinks (1)

Zortrium (1251080) | more than 5 years ago | (#28443851)

I use DropBox to keep (small) files synced across several machines -- I know some people object to keeping data 'in the cloud', but that's what backups are for. DropBox means no forgetting to svn update or commit and has never really gotten in my way. I keep things like my Firefox profile (bookmarks, addons, etc) in sync by symlinking relevant files in my Firefox profile folders to files in my DropBox. It's a bit of a pain to set up initially but only needs to be done once and then my browsing session is seamless from one machine to the next. I also use this method to sync stuff like my calendar, address book (no $99 per year to Apple for MobileMe, thanks), and SSH and bash config files (always symlinked so that I never need to actually move things).

rsync (2, Informative)

john_a_smith (1366597) | more than 5 years ago | (#28443875)

Primitive, but it works for me.

Re:rsync (1)

sexconker (1179573) | more than 5 years ago | (#28444009)

And it'll work for 99.9% of people.

I really don't get the desire for people to make shit more complicated and less reliable than it needs to be.

Very rarely would anyone need anything other than a simple sync job, manual or automated.

Arduous Task? (1)

sexconker (1179573) | more than 5 years ago | (#28443881)

What?
If you have enough files and move between systems frequently enough that this is an issue, consider a USB flash drive.

Store your files there.
Keep backups on each machine.

Want versioning? Seems to me that files typically have a datestamp for when they were last modified. That's all the versioning people need 99.9% of the time.

If you're in a situation where this is a big problem (many users many files many machines), you want your damned files stored in a more permanent location, such as on, I don't know, a file server?

Save your fucking files to the server. If you're away from the server, upload your file when you get back. When you get to another machine, grab the files you intend to work on.

Hell, Windows has supported this shit for ages with offline files and the whole "Briefcase" bullshit.

http://www.microsoft.com/windowsxp/using/mobility/learnmore/offlinefiles.mspx [microsoft.com]
http://support.microsoft.com/kb/307885 [microsoft.com]

And yes, Windows does a simple versioning and backup with shadow copies.

In your case, 2 Linux servers and OS X, just sync shit on a schedule when all machines are on the network if you want. The 2 servers should always be in sync with each other (since they're servers and should always be up and networked).

The laptop is the only issue, and should sync when connecting to and before disconnecting from the network.

Any way you want to go about syncing files is fine. Might I suggest a simple GUI drag and drop to/from the server? Seems to me most users can handle that, as long as you beat into them how to know which direction to do it. You could simplify this by making a simple script users could run. This script could include making backups on the server so we don't have issues of people going the wrong way, and so you can timestamp each old version (useful for keeping files for various projects grouped together, so people can grab old versions of a project if that's what they're working on).

You don't need services to handle this for you.
You have 2 servers and 1 laptop. I would say you don't need anything to handle this for you. I wouldn't even go as far as to keep a flash drive laying around. Just, you know, remember to grab files you're going to work on before leaving with your laptop, and remember to reupload those you've changed when you get back to the network.

If you've got a more complicated setup (multiple users accessing and modifying the same files at the same time) THEN you need a version control / checkout service running, and even then, none of them are intuitive, and users WILL get confused and break shit. Especially when you're dealing with mobile users who will be away from the network for unspecified periods of time.

Re:Arduous Task? (1)

sexconker (1179573) | more than 5 years ago | (#28443963)

Oh, and I'm assuming we don't trust /rely on "the cloud" or any 3rd-party online storage system.

(Because they shouldn't be trusted/relied on, and because in many cases they can't be, legally.)

rsync (1)

CaptSaltyJack (1275472) | more than 5 years ago | (#28443883)

I use Mac (a MacBook Pro and a Mac Pro), so I just do rsync -Cavz ~/Documents/SomeDir/ /Volumes/myusername/Documents/SomeDir/ and then usually an rsync the other direction, and I'm done.

The internet never forgets. (5, Funny)

kylben (1008989) | more than 5 years ago | (#28443923)

I embed all my documents in porn and post them on various web forums. The recovery procedure involves spidering my spam folder. I recently found my high school history term paper in a jpg of Marylin Chambers.

Unison (4, Interesting)

ashayh (636057) | more than 5 years ago | (#28443941)

Unison [upenn.edu]

Re:Unison (1)

Bubba (11258) | more than 5 years ago | (#28444031)

Mod parent up. Rsync with brains. Unison is the key.

Various Tools for syncing I use are... (1)

Stinky Fartface (852045) | more than 5 years ago | (#28443957)

I use the following tools to stay synced in various ways: Plaxo offers a fairly good Outlook sync utility for free which keeps my Address Book, Calendar and Tasks synced on my work and home desktop computers. Dropbox offers 2-3 GB of free storage that automatically syncs to any computer logged into that account. I keep all sorts of stuff in there. Photoshop prefs and tools, automation scripts, encrypted password database, etc. I set up a cheap home server with FTP. It goes to sleep if unused and I can wake it up with a Magic Packet remotely before doing file transfers. For large libraries of software, music, etc. I have a portable hard drive that I sync on either end using Directory Toolkit about every week or two depending. And Foxmarks for Firefox.

Subversion (1)

ChaoticCoyote (195677) | more than 5 years ago | (#28443989)

I've used Subversion for years to sync my various systems. I have four different machines (2 Vista, 2 Linux) and 20GB of data that must be kept in sync.

Of course, there could be somethign much better out there. I'm just very comfortable with Subversion, and it works.

iFolder (0)

Anonymous Coward | more than 5 years ago | (#28443991)

iFolder seems to work best for me. http://www.kablink.org/ifolder I use the Novell/Suse version though.

Webdav, SVN, etc (2, Informative)

fermion (181285) | more than 5 years ago | (#28443999)

For smaller files, I keep everything controlled using SVN. That is code, office type files, that sort of thing. I have a BASH script that pretty reliably works to commit, add, and update. Everyone once in a while I have to go in and manually fix something. I suppose I could put a chron entry in to make it automatic, but it is just as easy to go to the shell and update everything. Setting up the server was no issue, and it is an offsite backup.

For items that are larger, or that do not change so often, I use iDisk. This is just a fancy Webdav server that I do not have to manage.

I keep programs on an external hard disk. This is where I also keep my photo library and music and videos. I use one machine for Photos, so I do not really have anything to sync there. My music is not synced either, but I have used some third party software to hel with that.

It is getting to the point where if something goes wrong with a machine, I can have new one set up will all my data in a day. In normal circumstances, I can use any one of three machines and prety much have up to date information.

server and source control (1)

Is0m0rph (819726) | more than 5 years ago | (#28444045)

I have 1 server computer that stores all videos, photos, music, etc. My other computers and Xbox 360 connect to that one. For work related documents and source code the same server also runs Source Safe.

Briefcase (1)

TurboNed (1370389) | more than 5 years ago | (#28444103)

Isn't that what everyone uses Microsoft's desktop Briefcase for? I thought that's the ultimate in synchronization tools.

Panix (1)

Saint Stephen (19450) | more than 5 years ago | (#28444125)

I pay $100 a year for a Panix shell account and keep my data there. My own little server in the sky :-)

iFolder and Open Enterprise Server 2 from Novell (0)

Anonymous Coward | more than 5 years ago | (#28444127)

I setup Novell Open Enterprise Server 2 on a mini Linux server and use iFolder for syncing folders and files across several Windows and Linux desktops and laptops for myself and my wife.

I just don't sync (0)

Anonymous Coward | more than 5 years ago | (#28444129)

Between my Desktop, Server, Laptop, and iPhone I just keep whatever data on them I want and move what I want when I want. I can always access any other machine from any machine (mobile internet) so I always have access to the files I want or need.

google docs works well for me (1)

goffster (1104287) | more than 5 years ago | (#28444139)

Allows me to share with others as well.

If you're willing to spend some $ (2, Informative)

Binkleyz (175773) | more than 5 years ago | (#28444149)

Doubletake software makes an enterprise ready, real time replication suite.

It does block level replication, so only the changed bits of, say, a 10Gb databse gets changed.. It uses on the fly en/decryption so that the data streams are somewhat smaller than they would be otherwise..

I work for a Fortune 10 company, and when we have a need for real-time data replication, this is what we use.

Combination SVN and Shared Drive (1)

rainmaestro (996549) | more than 5 years ago | (#28444169)

When dealing with source code for the software projects I work on, I store everything on SVN repos hosted on my quasi-server box (SVN, CUPS, assorted DB's, Trac, etc). Whatever machine I need to be on (dev machine, XP build machine, etc), I just sync the checkout.

For regular files, I keep most of my stuff on a 1TB drive that is NFS'ed to all my machines. If I need to do a lot of work on a big file, I pull down a copy, edit it locally, then push it back to the share. This handles all of my needs for the most part. My Linux, BSD and OSX machines are good to go, my two XP machines aren't. One is only for playing games that choke in Wine, the other is an old box that just runs a build script for my apps, so I've never bothered to try to get them configured to handle NFS shares.

Both methods are cheap, simple, and fairly pain-free if you keep Windows out of the mixed-mode environment.

Re:Combination SVN and Shared Drive (1)

rainmaestro (996549) | more than 5 years ago | (#28444229)

As far as .config stuff goes, I don't sync anything. I run multiple distros, so it's nontrivial to share config files between them. I store backup copies of all my configs on the NFS drive, but apart from that, each distro has it's own set of configs.

del.icio.us, TrueCrypt, USB stick, and iPod (1)

cavemanf16 (303184) | more than 5 years ago | (#28444179)

del.icio.us has plugins for IE, Firefox, and there is a less useful third-party plugin for Safari which make it easy to "sync" your bookmarks across computers, so I use that for my browser synchronization.
TrueCrypt keeps my really important data (passwords, resume, other sensitive personally identifiable info) safe and secure on my USB drive.
My USB stick on my keychain holds a copy of TrueCrypt to boot from directly when you plug it into a USB port (you need admin authority on the computer you're using to use this feature though), and then some other miscellaneous documents I wouldn't want to lose but aren't sensitive sit on my USB stick in generic folders.
And lastly, my iPod holds a copy of all of the music I care to not lose. (My wife and I also have a 750GB backup drive attached to our iMac at home to keep all of our media files, like photos and video, backed up)

Everything else is either done "in the cloud" online for us, or is proprietary or sensitive data that shouldn't be getting moved off of the primary computer it is on anyway.

NFS or AFP mounted home directories (0)

Anonymous Coward | more than 5 years ago | (#28444181)

All my Macs in the house (5) mount their home directories from a central server. This server was originally a Linux box and the mounting was done via NFS. About a year ago I got fed up enough with Linux to convert that server to a Mac as well and it now shares all home directories via AFP.

On the notebooks I have two accounts: one that mounts the home direcotory when I am home and in wireless range and another account that is a separate desktop for when I am not home. I'll occasionally rsync files from the non-central home directory to my server. I make sure that I stay on top of it so that files do not get forgotten (set a specific desktop folder for content you want to keep/sync).

HTH

http://www.gbridge.com/ (1)

jsnipy (913480) | more than 5 years ago | (#28444221)

http://www.gbridge.com/ [gbridge.com] very underrated

bittorrent (1)

bl8n8r (649187) | more than 5 years ago | (#28444233)

tar cvf - /home | bzip2 | aespipe -w 10 -K /etc/homekey.gpg | bittorrent archive.`date +%s`.foo

Just started using DirSyncPro... (1)

zarozarozaro (756135) | more than 5 years ago | (#28444237)

DirSyncPro is FOSS, I just started using it a few months ago, and I love it. My company now uses it to keep our two programmers synced(sp?) to the server. Works great, I hope they package it better for MS soon, there is no installer yet, just an executable.

unison/sshfs/rsync (2, Informative)

Cocoronixx (551128) | more than 5 years ago | (#28444247)

My current setup is a combo of unison/sshfs/rsync.. I've been using it for quite a while at this point, and it works so well that I don't even give it any thought anymore.

I have a Media Center/Fileserver box at my house that is always on, and acts as the 'master' copy of the home directory, on all my workstations my .xinitrc/.xsession calls unison to sync my home dir with the server root as the preferred copy, then calls my WM, after my WM exits, unison is called again with the local homedir as the authoritative copy. A well-crafted ignore list is crucial, ignoring things like temporary file patterns, mozilla cache, machine-specific data, obscenely large files, etc.

I use sshfs-fuse for any of the above mentioned ignored files, sshfs allws you to mount remote filesystems locally, tunneling through ssh (or something like that).

Finally, rsync can be used to give time machine like backups. In my case, I backup to USB drive connected to the fileserver, which I only turn on when running backups.

This is for the most part Linux-only, but unison has a windows binary available, and could probably be used to sync to windows automagically.

AD with redirected my documents (1)

mordred99 (895063) | more than 5 years ago | (#28444249)

I have AD on my file server (and DNS, exchange, etc.) but setup through group policy folder re-direction of the "my documents" to a \\server\user\mydocs directory. Every user that has a domain gets these files created automatically. I also created the local settings folders as well (thus outlook and things like that) in the \\server\user\settings directories. For my linux box I create a folder /mnt/user then in the /etc/fstab create a mount there with full rights. Then finally in my linux box home directory, I have a documents folder, which I create a link to the mount point. For MP3 and other files they are stored in a separate directory on the same NAS on the windows box, create mounts for them in linux, and can access them via windows shares published in AD. However I use Jiznora as the front end to access all my music and files. It is not a lot of work to setup, and took maybe 30 minutes once I had the servers up and an AD setup. All pcs can connect in my house (wireless, etc.) and also my xbox 360.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...