Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Monitoring What Files Your Applications Leave Behind?

Cliff posted more than 13 years ago | from the package-manifests-GOOOOOD! dept.

Privacy 131

GoRK asks: "I have to install a commercial application on one of my servers. The application refuses to locate itself anywhere other than under the /usr tree, and I am concerned with it munging with my configuration under /etc as it automatically configures its daemons and whatnot. I am also a bit concerned with the method it goes about to verify its license on install. Is there any way I can run the installer in some sort of wrapper that allows me to monitor what files and network sockets get read/written to during the change so that I can monitor what data on my machine is getting moved around and also build a catalogue of every last little bit of the app in case I ever have to remove it?" In my opinion, a sensible software installer should have some form of user accessible package manifest included. Why should consumers trust third party software to "do the right thing" in the right locations, especially when installing software you don't have the source to.

"I am a big fan of MD5 sums and package management as a very reliable integrity check and I value the fact that every file on my system save user documents, etc. belongs to a package and I can verify its authenticity. I need to make sure that I know about each and every system file that gets modified during the installation. It would also be nice to see and control if it accesses anything under /dev/ just out of curiosity.

I've never been a diehard security freak before, but I just feel like it's 'time to do things right' so to speak. Is there a tool that will assist me here?"

cancel ×

131 comments

Sorry! There are no comments related to the filter you selected.

If you were really concerned (1)

Anonymous Coward | more than 13 years ago | (#194879)

You'd have named this mysterious "commercial application". After all, someone else reading slashdot may have installed it too, and know what it does.

Of course, you could also ask the company that makes the software. I have a small hunch that they would also know.

That presumes of course, that this software actually exists, and this isn't an Open Source Troll Story.

FAM (1)

Anonymous Coward | more than 13 years ago | (#194880)

I dunno what OS you use, but on FreeBSD FAM is quite useful. You could combine it with a NIDS system.

Re:Simple. Geesh. (1)

Anonymous Coward | more than 13 years ago | (#194881)

No.

The guy who modded it down probably just has a clue as to how "locate" works. Something which is obviously lacking in yourself, and "Amp", despite his low UID.

Note that he got modded up to 5 anyway, and the downmod was "overrated". Exactly the mod that "Amp" deserves. Ontopic, not a troll, nor flamebait, but a a factually incorect pile of drivel.

Good to know there ARE some clueful moderators still. Bad to know that he was outvoted by four clueLESS idiots.

chroot (4)

Anonymous Coward | more than 13 years ago | (#194883)

Run the application in a chroot'd sandbox.

protopkg (2)

dangermouse (2242) | more than 13 years ago | (#194886)

Grab protopkg [slackware.com] from the Slackware FTP site and write a prototype that wraps the install process. (No, protopkg doesn't care how the software is installed, or even that you're installing software... it just watches the system change and packs up whatever's new. How the system changes is left to you, the prototype author.)

Or, (protopkg trick of the week, kids), write a prototype that just has "sleep 10" in the compile() function. When protopkg goes to sleep, hit ctrl-z to stop it, and do whatever you want to manually. Then when you're done, give that shell an 'fg' to let protopkg finish its work.

Idunno about monitoring the network sockets... that's kinda weird.

Simple. Geesh. (4)

Amphigory (2375) | more than 13 years ago | (#194887)

bash# locate /usr/ > /tmp/usrfiles.old

do the install

bash# locate /usr/ > /tmp/usrfiles.new

bash# diff /tmp/usrfiles.old /tmp/usrfiles.new

Or, if you wanted to get fancier and check for changed files:

for i in `locate /usr/`
do
echo $i:`sum $i` >> /tmp/usrfiles.old
done

install

for i in `locate /usr/`
do
if grep -v $i:`sum $i` /tmp/usrfiles.old; then
echo $i >> /tmp/usrfiles.changed
fi
done

###

Shell scripting is your friend. Learn it well.

--

Re:strace is your friend (2)

peter (3389) | more than 13 years ago | (#194888)

Linux strace doesn't trace fork(2)/clone(2) by default. You have to use -f for that. Read that man page, and practice by tracing a simple shell script to see what it does and what you can make of it. Use -s to show longer strings if you want. (The default is to show ... after 32 character strings, except for file names, which are always shown in full.)
I recomment you strace all system calls from the installer (i.e. don't use -e), and filter later with grep | less, so you make sure you don't miss any interesting data.
#define X(x,y) x##y

Useful tools... (1)

Omniscient Ferret (4208) | more than 13 years ago | (#194889)

If you want a list of modified files, you could use checksumming utilities such as Tripwire [tripwire.com] or Aide [cs.tut.fi] .

If you want to see what filehandles are open - as in, files & sockets - lsof [purdue.edu] is useful.

The checksummers take a couple of minutes to check timestamps, and at least a few minutes for checksumming; lsof could be scripted to run in a loop, I guess. These are tools for use at intervals. If you want to get a continual log, look at strace. If you want to be able to reverse the changes, you could try chroot, or back up your system, or use a test system.

Tripwire (1)

GTM (4337) | more than 13 years ago | (#194890)

This package is more security oriented, but Tripwire is the way to go, since you have no standard way of monitoring events on the file system under Un*x.

strace (1)

khuber (5664) | more than 13 years ago | (#194891)

you could run strace -o<file> <installer>
on it, but you'll get a lot of detail to
wade through unless you give more options

you'd see all file accesses though

-Kevin

or truss Re:strace (1)

khuber (5664) | more than 13 years ago | (#194892)

should've mentioned that. most systems call
what Linux's strace is truss

-Kevin

Re:my opinion (2)

raphi (7620) | more than 13 years ago | (#194894)

1) Yes, there is closed source software. (Somewhere on other peoples computers.)
2) Yes, Open-Source Software has bugs just like any other software. But having the source enables you to fix it.
3) Yes, you have to trust many people, if you use any kind of software. But it's easier to trust software engineers, who don't hide their source.
--
Raphael Wegmann

Shareware mode (1)

BrookHarty (9119) | more than 13 years ago | (#194895)

Alot of installers dont want you to know whats going on behind the scenes. They want to install time limited keys, so you get nagged about purchasing the products. If you could just rm the key, you basically bypass the time limit. This is where hacking 101 comes in, trussing a file, then using lsof and logging all files. Plus not to mention what files it alters, what about ports it opens?

You should always play around with the application on a non-production machine before you go live.

BTW, TripWire will catalog your entire system, and let you know if any new files are added or altered. Very good method for security also.

VMware (1)

Gerv (15179) | more than 13 years ago | (#194897)

VMware [vmware.com] has cool stuff like non-permanent disks so you can install it, see what happened, and then roll your disk back to where it was before the install. Just what you need.

Gerv

Re:This is where NT admins have it good :) (1)

Todd Knarr (15451) | more than 13 years ago | (#194898)

Hate to say it, but we do have that. It's called Tripwire. Been on Sourceforge since 1998, and I can recall mention of it back for years before that. Thing's a lifesaver when it comes to detecting anything messing with system directories.

Re:This is where NT admins have it good :) (2)

Todd Knarr (15451) | more than 13 years ago | (#194899)

True, but then on Unix all the configuration's on the filesystem anyway. Usually all you want in the end's a list of what changes the program made. Detailed step-by-step traces are useful only rarely, in debugging arcane problems. If I need more detail than Tripwire gives by default, I usually just diff the new files against a CD-RW backup from just before the install ( yes, I'm spoiled by being able to quickly back up my configuration that way ).

OpenBSD never screws /etc or /dev (1)

Alex Farber (16133) | more than 13 years ago | (#194900)

Not an answer to the original question, but do you guys know that OpenBSD policy is to never touch the /etc (neither from the ports, nor during the system upgrades)?

As for logging the changed files, how about looking at mtree [openbsd.org] and tripwire?

If your using Debian... (1)

meridian (16189) | more than 13 years ago | (#194901)

You could run the install in a chroot and the package the install into a .deb file using the deb-make utilities and install that which would let you know if it is trying to overwrite any files that have already been installed and fail unless you send switches to force overwrites. If the original rpm or tgz simply needs to be unpacked to / to install you just need to alient packagename to change it to debian format. You can then get a list of the files installed by this package using dpkg -L packagename
But if your using Redhat or other Linux distros using rpm formats i guess such an elegant solution
is not available for you

Re:This is where NT admins have it good :) (1)

RedGuard (16401) | more than 13 years ago | (#194902)

The tools the previous poster mentioned actually
intercept system calls to record exactly what
a program did, tripwire only reports the
difference between two snapshots of a filesystem.

Re:Test it first! (2)

dillon_rinker (17944) | more than 13 years ago | (#194904)

Yup. Minimum for any shop that considers itself professional:

Production (it works)
Test (we think it works)
Development (we're trying to make it work)
Training (the users are trying to figur out how it works)

Our test platforms sometimes double as training platforms; nothing reveals incorrect assumptions quicker than a clueless user banging at an app.

"What do you mean I shouldn't close while it says 'Committing transaction...'? If I shouldn't do it, why does it let me?"

Designed for Windows Program (1)

NeoMage (29426) | more than 13 years ago | (#194909)

Isn't this why Microsoft has the "Designed for Windows" logo program, so that the consumer can have some form of belief that the program will install in a certain manner (ie. own DLLs in it's own directory, registry keys done properly).

I wonder can there be a similar program made up for Linux? It would probably be fairly straight forward, with some complications as to where you prefer apps to install.

Re:This makes a good point... (2)

rcw-work (30090) | more than 13 years ago | (#194910)

make -n install used to be really useful, but now most apps that use autoconf have an install target that iterates over several directories and output several hundred lines of junk. You really have to know what you're doing to find out what it's doing. I'd hate to have to use that feature to determine if something malicious was in there.

For those kind of apps, it's faster to build a chroot (Debian chroots are really simple to make - unpack the base2_2.tgz to a dir somewhere, cd to it, and do chroot . bin/bash), make a copy of it, then do rsync --dry-run --verbose or diff -u --recursive on the two dirs to find out what changed.

Tripwire (2)

rips (34200) | more than 13 years ago | (#194911)

Tripwire is designed as a security tool to tell you what files have been added, deleted and changed on your system but it sounds like it would easily the job you're looking for.

You just run it once to generate a database of files on your system and again after installation to see what has changed. Easy!

Re:Use ext2 to your advantage (1)

Kingpin (40003) | more than 13 years ago | (#194913)


Indeed! I should have heeded your advice earlier today. I tried to GRIP a CD, the application hung like a horse, so I tried ALT+CTRL+F2 to try and get a prompt. I got burried in output about DriveSeek and horror and error++, so I logged in as root and rebooted. Needless to say I spent the rest of the day recovering my HDD. Initially it didn't even show up on fdisk. Bugger.

Re:my opinion (2)

JimDabell (42870) | more than 13 years ago | (#194915)

You are thinking of Reflections on Trusting Trust [acm.org] , and it wasn't a compromise of gcc, it was just a "what if" situation that Ken Thompson talked about once.

Re:my opinion (1)

weecol (48120) | more than 13 years ago | (#194916)

I do believe I have herd at least on instance of someone introducing suspect code into gcc which caused login to be compiled with a backdoor password. I dont remember any web referances but I have seen it discussed in the professional Linux programming: published buy wrox search the index for security.

Virtual Machines! (1)

joeytsai (49613) | more than 13 years ago | (#194917)

Use VMWare or Plex86. Install your application on the virtual machine, and there you can actually see your machine with all the changes. You can write the session to disk to try later, or just kill the VM if the app isn't what you want.

Although I've personally never used one, there's dozens of interesing uses for VMs.

Re:my opinion (2)

Dwonis (52652) | more than 13 years ago | (#194918)

What makes you think that your clients can't change the microcode anyway, just because you don't give them source code?
------

Re:Installshield for Linux (2)

Dwonis (52652) | more than 13 years ago | (#194919)

Too bad the Linux companies won't use it.

No, the InstallShield way of doing things is a POS that is the reason why Windows programs can trash DLL versions and such.

Most of the modern unices have a central package management system that I would never trade for any per-package installation program. In fact, anyone could distribute a deb package, and all you'd have to do is run dpkg --force-depends -i commercialapp.deb && apt-get -f -y install, and the package and all its dependencies would get installed.
------

DMAPI? (2)

1010011010 (53039) | more than 13 years ago | (#194920)

Does your Unix support DMAPI? Perhaps you could get a look at what's its doing using that. Maybe a plugin for ReiserFS, if you're using that, to get a look at what the changes are that it makes.

- - - - -

Installshield for Linux (2)

pipeb0mb (60758) | more than 13 years ago | (#194923)

InstallShield offers a Linux version now...has for awhile. Too bad the Linux companies won't use it. It is Java based, and it is pretty damned cool. And, of course, you can view files and thier install paths with the setup list. Would probably help in cases like this...

http://www.installshield.com/iemp/specs/default.as p/a? [installshield.com]

strace is your friend (3)

Convergence (64135) | more than 13 years ago | (#194924)

strace, which uses posix tracing, can trace every system call made by an application.

Actually, I frequently use it when debugging. Program doesn't start up correctly, I run it under strace to see if there's a file it can't find.

It'll show you every subprocess, every kernel call, every file access, network access, etc.

'strace netscape'

or

'strace -eopen netscape'

Ask your vendor (2)

Nailer (69468) | more than 13 years ago | (#194925)

I have to install a commercial application on one of my servers. The application refuses to locate itself anywhere other than under the /usr tree,

You didn't say whether the program was Open Source or proprietary, just that it was commercial. However, I'll assume that like most slashdotters you've never looked up the dictionary definition of commercial, or the Free Software Foundations confusing words list, and mean `closed source' instead. In reality a programs status as Free Software or Open Source or otherwise has no bearing on whether it is commercial or not.

Either way, whether its pay per license proprietary software or Open / Free software that is produced for commercial reasons (meaning a support contract is avaliable), complain to your vendor.
The FHS specifies RPM 3.05 as the (current) standard for installing software on Linux systems. nearly everyone who provides software on Linux (Open or proprietary) provides packages in this form, and the overwhelming majority of users use them. Get your moneys worth (either from licensing if its a closed source or support agreements otherwise) and tell the vendor you want packages.

Where to put the app is another argument. I have no problem with a packaged app that wants to live in /usr. being `part of the OS' is fairly hard to determine on a platform without a standard distribution, and such arbitray information is useless to base a filesystem standard on.

tripwire will tell you what changed... (2)

mnot (71203) | more than 13 years ago | (#194926)

and then you can restore whatever got wiped out from backups.

You are using tripwire, right? And keeping good backups?

*grin*

Re:Simple. Geesh. (3)

err666 (91660) | more than 13 years ago | (#194928)

locate only finds files that are already in its database. You'll need to use find instead for this to work.

Simple way of seeing what files were create/modded (4)

dburr (92379) | more than 13 years ago | (#194929)

$ touch /tmp/instdate.<package-name>
$ <run the appropriate install procedure>
$ find <tree where it installed itself, or if unknown, /> -newer /tmp/instdate.<package-name> -print

Works for me... :)

--

Re:Tripwire? (2)

tve (95573) | more than 13 years ago | (#194930)

It's capable of detecting new files, changes to existing files and files that were deleted.

Tripwire? (3)

tve (95573) | more than 13 years ago | (#194931)

Well, perhaps tripwire [tripwire.org] would be an option.

chroot and diffs between ls -R (1)

he-sk (103163) | more than 13 years ago | (#194933)

Why don't you create a minimal chroot environment. Before you install you do a `ls -lR`, as well as after the install.

Now just diff the ls-lR files and you see which files have been modified and which are newly created.

(Just curious, isn't this something for a mailing list rather than an Ask Slashdot?)

Re:One Technique I have used... (1)

he-sk (103163) | more than 13 years ago | (#194934)

If this is an RPM on a linux box use "--root somedir" to prefix things like /usr with "somdir". Other install methods like SVR4 "pkgadd" have similar relocate facilities.

That would be --prefix somedir , and unfortunatelly it's not always an option, because some RPMs are not relocateable.

Re:One Technique I have used... (1)

he-sk (103163) | more than 13 years ago | (#194935)

Umm, I guess, you're right.

And that's all that I have to say about that. -- Forest Gump

Re:Simple way of seeing what files were create/mod (2)

kjhambrick (111698) | more than 13 years ago | (#194939)

This is more-or-less my method. However a
couple comments.

I make a directory ( say, /root/changelog )
and place all such package installation
find-listings in that directory.

The only other change is that most times,
the installed files retain their original
dates. To overcome this, I recommend using
the find-flag -cnewer ( changed newer )
sequence ( assume 'some-new-package' is the
new software package )

touch ./root/changelog/foo
# install your software now
find / -cnewer /root/changelog/foo -print \
|tee /root/changelog/some-new-package-file.lst

The file /root/changelog/some-new-package-file.lst
will contain a list of all changed files on the
system.

hth

-- kjh

p.s. This is also useful for figuring out what
files a GooEY admin menu affects.

Do the touch ; Do the GooEY ; do the find-command

Package management thoughts (2)

Michael Woodhams (112247) | more than 13 years ago | (#194940)

This is only peripherally related to the question, but I have some thoughts on how I would like package management systems in Unix to work.

I want large tracts of the directory structure to be modified only by packages installing their standard files. To completely restore these areas, all you need know is what packages (and which versions) were installed.

The ideal is something like this: I backup /etc, /home and /usr/local only. If my hard drive goes to Silicon Heaven, I buy a new one, restore onto it, and then run a program that looks at the list (saved in /etc) of packages I had installed and helps me reinstall them all.

CHROOT (1)

jfmiller (119037) | more than 13 years ago | (#194942)

the chroot command was designed to do just this read the howto

chroot() (1)

friscolr (124774) | more than 13 years ago | (#194943)

man chroot
try chrooting to a safe location first and see what it does. That will help verify that the script won't modify/have access to any files you don't want it to have access to.
of course there's a good chance the script won't work either.

A good firewall can monitor what network connections it starts up.

-f

Re:This makes a good point... (1)

friscolr (124774) | more than 13 years ago | (#194944)

That's the beauty of make -n install which will just print the commands that make install will be running.

Of course that won't work for install scripts that don't do make install's

and for the original post...
If you're *really* unsure about what the install scripts that the closed source programs you're installing will fuck up, then why are you trusting their actual app to run correctly?

2 things-
1- make sure you have good tech support from the company.
2- install on a test server first, then on your production machine.

-f

/proc/####/fd (1)

xee (128376) | more than 13 years ago | (#194945)

The /proc filesystem should tell you what files a process has open. Get the program's PID, then go to /proc/PID/fd and do an 'ls -l' to see what files (and sockets) the process has open. This will probably not give you all of the info you want, but it should be a good start. You can also try doing a 'find' on files modified since a minute before you ran the installer. I know that these are quick fixes, but, until you find something better...


-------

Encap / Epkg (1)

nivfreak (131107) | more than 13 years ago | (#194946)

Give this tool a shot, it may help, I used this with slackware in order to have _some_ kind of package managment for thrid party binarys/source builds: http://www.encap.org/ Good luck

Re:chroot (3)

BMazurek (137285) | more than 13 years ago | (#194948)

FreeBSD extends the ideas behind chroot with it's jail [freebsd.org] functionality.

Using this, you could set up a virtual machine, similar to the way the Openroot Project [sektor7.ath.cx] does.

Re:chroot (2)

langed (142123) | more than 13 years ago | (#194949)

Offtopic? At least it gets a positive rating.

I would have suggested the very same thing, for determining what files get installed/removed. Make a basic filesystem and chroot into it, and then install/run the commercial app. This way your system would be protected.

Of course, my other suggestion would put things into a little more perspective:
1. Make a complete system backup.
2. as root: md5sum `find / -name \*` > /files.1
3. ls -l `find / -name \*` >> /files.1
4. install the commercial app.
5. repeat steps 2 and 3, to different filenames(files.2).
6. diff /files.1 /files.2

the output of #6, if you were feeling properly bored, could be parsed by a perl or shell script if you wanted it pretty, otherwise you'd know what files were created, deleted, or modified.
Of course, this solution wouldn't have the foresight to note the differences in other system changes, such as the log files... But you could deal with that yourself.
btw, I assume bash as your shell... you can probably avoid the backslash escape before the asterisk in other shells. heh

Use ext2 to your advantage (1)

quickquack (152245) | more than 13 years ago | (#194951)

Install it normally, then see if it changed /etc. If it did, then just pull the plug and everything will be fine, thanks to our friend, ext2.
------------

Re:Use ext2 to your advantage (1)

quickquack (152245) | more than 13 years ago | (#194952)

Yeah--the pull-the-plug trick is a good one. One time, someone erased illegal files right in front of my eyes. I, of course, was trying to stop them. So once they had dropped the files in the Trash (this was a Mac) and went to Special->Empty Trash, I pulled the plug. They walked away, thinking they had foiled my plan.

No way! I turned on the computer and found the files in their original location. Sometimes, asynchronous FSs can be helpful :-)
------------

Tracing system calls (1)

rmst (157328) | more than 13 years ago | (#194953)

If you want to trace what system calls are being made, have a look at strace, though if you want to make sure it doesn't actually modify anything, the idea of chrooting is probably a good one.

This makes a good point... (3)

TotallyUseless (157895) | more than 13 years ago | (#194954)

Many software installers will leave an installer log when they are done, but by then it can be too late. It seems rare that a software installer actually tells you what it is going to do before it does it. It is things like this that lead to unneeded tedium for the end user, such as backing up your configs before an install, 'just in case.' I just don't think we should be forced to go through this hassle, and I will make sure to clue the user in during/before the install in any software I write in the future. It had previously never occurred to me to do this, and I think it is the same way with most developers, be they commercial, open source, shareware, or whatever..

VMWare (1)

cromano (162540) | more than 13 years ago | (#194957)

Sounds to me like a textbook application for VMWare - if you have a box big enough, you can create a virtual machine and install you package there. If you take a snapshot before and after, comparisons are easy.

Also, the VMWare wrapper will let you monitor anything that goes in or out the virtual machine's network connections, thus enabling you to identify the licence confirmation mechanism.

I'll go back to lurking now.
--
If you want to live in a country ruled by religion, move to Iran.

Re:This is where NT admins have it good :) (1)

Eric Gibson (166760) | more than 13 years ago | (#194958)

Eh, even if there WAS SUCH TOOLS FOR UNIX, if you asked any good UNIX admin if you could install a tool on his/her system to do something as everyday and simple like this. He'd let out an evil laugh and run three commands that did the exact same thing as these tools before you could even download them. I always thought it was funny when I was a UNIX admin, and I worked along side some NT admins. They just think so in-the-box, I don't think it's thier fault though they don't even have an adequate shell! Things that I would have automated the hell out of they point/click and do manually every single day. Things that I could have written a simple 10 line shell script to do, they pay 1000 dollars for a site license for some 3rd party tool. It never ceased to amaze me...

Re:This is where NT admins have it good :) (1)

Eric Gibson (166760) | more than 13 years ago | (#194959)

Well yeah, tripwire. But really, you don't need something that elaborate. Just get a test box and:

find / > file.list1 ; ./install ; find / > file.list2 ; diff file.list1 file.list2

So maybe it's changing some files, and not only adding them. Turn off services that would be automatically changing files for a minute(you do all the services that are running on your system and what they are changing and when right?!). Install the app, check modification times for all the files on the system if they are later then the time you intalled the app, the app probably changed them. If I were admining a unix box, and someone suggested installing a FileMon, or SysDiff type tool on my system I'd laugh at them and tell them to write a script just for the point of the matter! I mean come on!

chroot or truss (1)

wren337 (182018) | more than 13 years ago | (#194960)

I've never tried it, but might running in a chroot'ed shell allow you to install it where you like, at least for a test run? Also you may be able to truss the pkgadd tool and log all of the system calls. Give it a shot with something you trust and see if it works out first. Don't forget to send a nastygram to the package author for not allowing configurability.

Pretty simple question (2)

Kernel Kurtz (182424) | more than 13 years ago | (#194961)

Why not just "find / > beforeinstall.txt" and "find / > afterinstall.txt" and use "diff" to compare them afterwards?

You can use diff to check files in /etc or wherever if you have backups as well.....

Re:installwatch - checkinstall (1)

Kazymyr (190114) | more than 13 years ago | (#194962)

I concur - I've been using installwatch [mayams.net] for quite a while, with excellent results; including with installers from commercial packages. Very nice utility. Use it as installwatch -o logfile [your installer name here].

Re:You're taking this too seriously (1)

MadCamel (193459) | more than 13 years ago | (#194963)

Perhaps I should state that I run my own buisness, and admin my own servers. If it was sombody else's system, there is no way I would spend that much time on it. These servers are my pride and joy, and I work hard to make sure every one going up for production use is *PERFECT*. This means delving in to the kernel and changing things around, libc modifications, (re)compiling everything optomised for the particular CPU the machine has, etc.

This course of action has really paid off. Every server I have put up has not needed any sort of modification/tweaking to this day. The machine goes up, and stays up, reliable as the hardware it is on.

If I were for-hire, I would probably charge per-job, as my machines are very close to self maintaining. Once it's set up, a low-paid lackie could handle it.

Re:You're taking this too seriously (2)

MadCamel (193459) | more than 13 years ago | (#194964)

Some people (noteably the best UNIX admins) take things like this VERY seriously, for a good reason. After putting 600 man-hours in to configuring a machine from the ground up, I know I wouldn't just go install somthing when I don't know what it will do to my system. Sure I have backups, but it should never come to that. In order to effectivly run a production server, you have to know how _everything_ installed on it works, and interacts with everything else. If you do not know, you waste lots of time with trial and error, and any modifications you make may cause other applications to misbehave, because you simply do not know how things interact. I agree 100% with the posters concern.

Re:my opinion (2)

garett_spencley (193892) | more than 13 years ago | (#194965)

As part of a company that sells multi-million dollar hardware under warranty. We certainly DO NOT want someone who doesn't understand the hardware architecture, going in and 'tweeking' our proven microcode. Potentially damaging hardware and forcing *US* to pay for their incompetence.

Come on now. That's just FUD. All warranties have certain things that will void them. For example, if you buy a new radio and it breaks you may be able to replace it, but if it broke because your 5 year old smashes it with a sledge hammer then you can't replace it because the warranty doesn't cover that.

You simply say something in the warranty like "If you use drivers for this device that were not written by either <your company name> or a company or person who <your company name> has certified, then the warranty on this device will be void." And presto! You are not forced to pay to replace any hardware that someone uses a 3rd party driver on.

Why open source the drivers then? So your consumers have more confidence in the quality, and they can fix bugs themselves and give them back to you.

I realize that this contract prevents users from applying bug fixes that they wrote if they want the warranty, but I think it's a fair compromise. You give the bug fix back to the vendor and they can review it and roll it out in the next version of their driver. Probably won't be as fast as you'd like, but hey, the bug could have not been discovered and fixed at all.

--
Garett

Re:Test it first! (2)

eclecticIO (195600) | more than 13 years ago | (#194966)

You should listen to this. This is some of the best advice you're likely to get short of wasting your money on a "consultant." Where I work, we have three stages to production. In addition to the test box, we have a more tightly controlled "production test" environment and then changes get promoted to production.
On the other hand, if you're a smaller company with limited resources and you happen to be running Linux, you may want to give serious consideration User-mode Linux [sourceforge.net] . From the site:

"User-Mode Linux gives you a virtual machine that may have more hardware and software virtual resources than your actual, physical computer."

I've played with it a bit and it gives you a complete (and completely sealed off) environment. It creates the entire environment within a file. So you could create the environment you want and then simply make duplicates using cp.
It does require quite a bit of disk space since each VM is a complete system. So, if you want a virutal system with a 2GB filesystem you'll need +2GB of disk space, but heh, disk space is one of the cheaper components, certainly cheaper than a new system.
Also keep in mind that it does "split up" your real system resources so you'll want to make sure you have plenty of RAM if you do any real work with it.
Anyway, check it out and see if will help. Either way, you really do need to seperate test and new "stuff" from production.

SUBTERFUGUE could do this... (1)

mkcmkc (197982) | more than 13 years ago | (#194967)

You might find SUBTERFUGUE [subterfugue.org] interesting. It's kind of alpha, but it could do this, given the right trick.

(Disclosure: I'm the principal author.)

--Mike

save the original directories (1)

PineHall (206441) | more than 13 years ago | (#194969)

It does not take much space to save some of your system directories. Since you have to install the software, copy system directories and then afterwards compare the before and after directories. Admittedly this is not the ideal solution, but at least you know what has changed.

Another idea that may work is to create an account that uses chroot and then install over your copied directories and then compare before installing in the real /usr.

virtual machine (1)

foobar104 (206452) | more than 13 years ago | (#194970)

Lots of discussion about testing servers. If you only have one server, consider running a virtual machine on it for testing. You can evaluate VMware for, I think, a month, and that should give you enough time to figure out what you want to do. Since VMware is fairly inexpensive, you might think about keeping it running for just this sort of thing.

On my server, I have a VMware GSX license, and I keep a clean virtual-disk with RedHat 7.1 + XFS (my dist of choice) on line at all times. When I need to test something-- anything, an app, a new kernel, whatever-- I clone off the test virtual-disk, boot the VM, and go.

The eval download page for VMware GSX is here [vmware.com] .

maybe this is too simple but... (1)

awptic (211411) | more than 13 years ago | (#194971)

ls -lR / > /tmp/before run the application for a while ls -lR . > /tmp/after diff /tmp/before /tmp/after ?... sure, alot of files would be changed due to other processes, but figuring out which shouldn't change isn't that hard.

Installers (1)

mordwin (212427) | more than 13 years ago | (#194973)

If there was one thing Commodore-Amiga ever got right, it was producing a standard script driven installer with logging and dry-run features, selection of levels of competence, etc. Its one thing I really miss on other systems. The script langage was a bit nasty being loosley lisp based and full of brackets, but otherwise...

Re:More details, please... (1)

Spinality (214521) | more than 13 years ago | (#194974)

Have you read the installation manuals/instructions at all?...Clueless and lazy admins are pretty abundant, it seems.... -- SwiftKick

Jeez, guy, no reason to flame what sounded like a reasonable general question, independent of the installation details you mention. Yeah, to deal with a particular install, all that stuff is relevant; and maybe the poster was in fact a 'clueless lazy admin' after all; but I don't think you have to be clueless and lazy to asking that question.

In fact, I think it points up a systematic defect in most OS configurations. In my ideal world, file system primitives would be available that let me checkpoint and undo/unroll installations and other actions. The existence of some good tools to help with this is a Good Thing, but I don't think we should have to take extra steps to protect ourselves from misbehaved packages and bad installers. I'd like my OS to be smarter than the packages it installs, and I'd like those protections to be comfortable wrappers like ip_masquerade that keep me safe even if I'm careless.

JMHO -- Trevor

You're taking this too seriously (1)

Grishnakh (216268) | more than 13 years ago | (#194976)

>Why should consumers trust third party software to "do the right thing" in the right locations, especially when installing software you don't have the source to. I think you're worrying about this too much. All Windows applications work this way, and everyone loves it. It's so easy: you just click a few buttons in a wizard and it does everything for you. People have been doing this for years on Windows machines and they're all happy. Sure, sometimes you have DLL conflicts, or you can't uninstall stuff properly, or it makes your OS blue-screen a lot, but that's ok. Software is so complicated these days you just can't expect a computer to work more than an hour or so without crashing. Just don't ask any questions, take your soma, and be happy.

Re:You're taking this too seriously (1)

Grishnakh (216268) | more than 13 years ago | (#194977)

Ah, good ol' Windows... you pay a lot of money for it, but it doesn't even come with something as basic as a package manifest to make uninstalling software easy, so you have to shell out even more money for a commercial program which supposedly cleans up after all the other crappy software you paid money for which can't uninstall itself properly.

Where does the spending end?

Re:my opinion (2)

Grishnakh (216268) | more than 13 years ago | (#194978)

So you're saying that commercial applications simply cannot use a sensible installation scheme, and must instead resort to hiding everything from the system administrator? So open-source really is superior, apparently.

Re:You're taking this too seriously (5)

Grishnakh (216268) | more than 13 years ago | (#194979)

Oh please. You don't need to know how everything works. Microsoft made Windows easy to use and reliable, so all you have to do is use the install wizard and everything works great. If you're really curious about the inner workings, though, part of the way they achieved this unparalleled reliability and ease-of-use is through architectural decisions.
1. The Registry. This piece of utter genious is crucial in an easily-maintained system. Instead of having a bunch of separate configuration files to keep track of, you just have one big binary file which all the install wizards access and change. Sure, sometimes one application screws it up for everything else, but that's no problem. Just re-install everything, no big deal. In this day and age, it's simply ridiculous to consider going back to ASCII text configuration files; it's just too difficult for certified system administrators to read these and edit them. Besides, we can always pass the blame to one of the third-party software vendors. I know that every time I have a computer problem at work and I need to meet a deadline, I can just tell my boss how some non-Microsoft program screwed up my whole system and he'll say "no problem, take as long you need. As long as we're using reliable MS software, we don't need to worry about deadlines."
2. A central location for DLL's. By keeping all the DLL's in one or two directories, and allowing all applications to modify or overwrite them, we achieve breathtaking gains in system efficiency and reliability. Sure, sometimes you'll have minor problems with incompatible versions of the same DLL, but we can just blame that on the non-MS vendor and reinstall everything.

Honestly, you talk like your time is precious or something. What's the big deal if you spend days setting up a system and some big-$$$ application screws it up? You just call their tech support (for $300+ per call) and tell them about your problem, then spend the next week or two redoing everything. And if it messes up that time, do it again. Using the intuitive and attractive Microsoft Windows user interface is such a pleasure, those hours (days, weeks...) will pass in no time. Plus, you can have Clippy (tm) help you! What other software company has such a fantastic, useful innovation?

Re:my opinion (1)

michaelo (224201) | more than 13 years ago | (#194980)

IMO you miss the point here.
i think the main issue here is that if it was open-sourced you could easily changed the installation procedure which is maybe impossible in this case of it is closed source.
J.

chroot (2)

michaelo (224201) | more than 13 years ago | (#194981)

what about doing the whole thing in a chroot environment and then comparing it with the "original" tree?
Just my 0.02 euros..
J.

my opinion (1)

unformed (225214) | more than 13 years ago | (#194982)

Why should consumers trust third party software to "do the right thing" in the right locations, especially when installing software you don't have the source to.

Okay, this whole open-source thing is going a little too far. Yes, I appreciate and fully support open-source, but:
1) Not all programs can be open-sourced
2) Having something open-sourced isn't going to verify that it does nothing wrong. I forgot the name of the program, but about a year ago, someone found a huge security leak in a program that had been open-sourced for years.
3) You don't HAVE TO "trust third party software to "do the right thing" in the right locations." You can NOT BUY IT. If you can't trust third-party programs, WRITE YOUR OWN PROGRAM.

Re:You're taking this too seriously (1)

Kalabajoui (232671) | more than 13 years ago | (#194983)

I'm one of those crazy people who have better things to do with my time than hunt down every small change a program makes while installing itself. There are programs for Windows like CleanSweep and such that automate this tedious task. As far as I'm concerned every commercial program should come with an uninstaller that comepletely removes any trace of the program and undoes any alterations the installer made. Too bad that not all comercial entities have the consumer's best interests at heart, otherwise there wouldn't be any spyware or a need for utilities like Cleansweep in the first place.

Re:You're taking this too seriously (1)

Kalabajoui (232671) | more than 13 years ago | (#194984)

"Where does the spending end?"

Ha ha ha ha! Good one, what makes you assume that it ever began?!

Seriously, I am not an advocate for Windows beyond the fact that it suits my current purposes just fine as a gamer and home user. Now .NET and other pay per-use-ware will probably drive me to Linux eventually. Besides, I think the whole installation/uninstallation situation is pretty much a cross platform problem for which there are not a whole lot of solutions that don't involve tedious and uneccesary work for the end user or administrator. Granted, an admin running a production box is going to have to pay more attention to details than a home user like myself.

installwatch - checkinstall (5)

nsrbrake (233425) | more than 13 years ago | (#194985)

Checkinstall is a script that uses installwatch and rpm to build rpms from a source install. Do a search on freshmeat.net for checkinstall, the author also maintains installwatch. You only need to run installwatch as a wrapper to the install program and it will give you a listing of the files that it creates.

This is where NT admins have it good :) (2)

whizzmo (239423) | more than 13 years ago | (#194986)

We NT folk have FileMon [sysinternals.com] , RegMon [sysinternals.com] , and Sysdiff/SMS Packager [microsoft.com] at our disposal.

It's not often there exists something in the NT world that doesn't have a parallel in *nix land, but this is one of 'em
---
nuclear presidential echelon assassination encryption virulent strain

Change your mount points (3)

ConsumedByTV (243497) | more than 13 years ago | (#194987)

And then install it. Make a second /usr for the install. Or try using a small shell script that will traverse the file system and tell you all the recently modified files. Or just use a virtual machine.


The Lottery:

A solution (1)

npcole (251514) | more than 13 years ago | (#194988)

Why not pipe the output of a find command to a file, and then use find to get a list of modified/new files after the install?

How is it packaged? (1)

kireK (254264) | more than 13 years ago | (#194990)

If it is a Sun package you can look into the package and see what it does. Look for the preinstall and postinstall scripts for "goodies". You can also do the same with RPMs but it is a little harder. One place where Sun is a little better than Linux IMHO.

No test server? (1)

bloo9298 (258454) | more than 13 years ago | (#194992)

If you don't have a test server, and you can't use chroot for whatever reason, you could try installing it in User-Mode Linux [sourceforge.net] . Follow the brief descriptions link and look at "As a secure sandbox or jail".

It will allow you to see how the software interacts with other files in a reasonable installation. It might be easier than trying to simulate it with chroot.

checkinstall (1)

dossen (306388) | more than 13 years ago | (#194993)

check out checkinstall [mayams.net] . It's a tool to build packages from tarball installs, it's build on something called Installwatch, maybe this could be of use, especially if the chroot approach fails, since this just watches what happens, and makes it possible to do a roll-back. Sure, if it turn ugly you still might need to bring parts of the system up from backups, but with a little luck, this might give some idea about what goes on, and maybe then you can move things around your-self....

Re:checkinstall... To lat it seems ;-) (1)

dossen (306388) | more than 13 years ago | (#194994)

Never read /. while doing other stuff... ;-) You find a nice topic, read the replies, think you have something to add, get the facts straight (wait, this must be where I went wrong *g*), do some stuff that needs doing, post your reply.... and notice, that you've been beaten to it [slashdot.org] .... by a long shot ;-)
Well I guess that's it, no more working while /.'ing ;-)

Re:my opinion (1)

ryants (310088) | more than 13 years ago | (#194995)

1) Not all programs can be open-sourced

Why not?

but about a year ago, someone found a huge security leak in a program that had been open-sourced for years.

It was found and (presumably) fixed. Score another one for open source.

At least your point #3 makes sense, EVEN IF you use "nested "scare" quotes" and GRATUITIOUS SHOUTING. :)

Ryan T. Sammartino

Test it first! (5)

JediTrainer (314273) | more than 13 years ago | (#194996)

I agree that it's probably a good idea to have a look at what it does (by running in some sort of sandbox), but here's my take on the whole situation in general...

First off, you shouldn't be installing anything untested onto a production server. What your company should have is a box identical in configuration to your production box (or at least a development server. You DO have those, right?).

Install the package on this server first. See what it touches. See if other packages misbehave after this is installed. Above all else, do not touch a production box period unless you've already seen what this program does.

If your staging box blows up, so what? It's happened to me. Nobody really relies on it, and that's what the box is for anyway. No big deal. Document everything that happens when you install on this machine. Since it's exactly the same as your production box, if everything works, then all you have to do is follow your documentation to install onto your production server, and all will work fine.

Trying to install stuff (even in sandboxes or wrappers) onto a server without testing it in a closed environment or staging area first is asking for trouble.

Re:Test it first! (2)

trifixion (318304) | more than 13 years ago | (#194997)

That's good in theory, but when you have application packages that install a wide variety of libraries and required portions of the program in different locations, or which go around and edit your configuration files, you ought to have some way of automatically cataloging what files were added, and what files were changed(and in what way) without having to fish through install scripts or hundreds of makefiles in a source package.

This has been one of my biggest gripes in using anything with an autonomous installation mechanism. We all know that Windows machines can operate under the heavy load of junk and poorly formatted configuration files and dirty system registries. The case is not so simple with a Linux box, especially with a person like me who only has about 5 hours of spare time a day(only a fraction of which I ever really have free to use my computer). It's not quite enough time to go about and map all my .files and figure out which one a given package has skewed to the point where after I remove what files I know to have been installed into a package X breaks, or I can't use PPP anymore(both cases have happened).

In the interest of fairness, I believe that in any case where someone is recieving money for some piece of software they have authored, there should be some way to remove the program without breaking the system.

You can't ask a normal person to go about trying to find what configuration files a given package has disturbed, or what libraries it has upended or added. And installing development libraries causes whole new realms of problems.. But I think it's obvious where I'm going with this.

Making sure anyone has the opportunity to use a stable operating system platform is the main reason I want to get into linux development.. :D

TRiFIXION

Re:How very defensive, let me clarify... (2)

flippety_gibbet (323926) | more than 13 years ago | (#194998)

While not wishing to fan these flames further, there are some (IMHO) good utilities from sysinternals [htp] which allow you to do this for free.

filemon [sysinternals.com] will report file update/access and regmon will report registry update/access. [sysinternals.com]

I haven't tried your examples, but I have found that even if the output from these is rather verbose, given some judicious regular expressions the output can be cut down to manageable size.

use tripwire (1)

mveloso (325617) | more than 13 years ago | (#194999)

or something of that ilk to baseline your system. Install the app, then figure out what changed using the provided tools. This is really the only way to do it.

Ideally, you'd put your system in a quiescent state before doing this so you can isolate the changes to your system to one app. You probably want to pull the various temp directories out of the watch list (/tmp, /var/tmp, /var/run, etc) since they tend to bear the brunt of changes.

The most common places that'll be changed are: /etc/rc scripts (for startup), /var (for a telltale), /etc/services (maybe), and /var/pkg (or the equivalent).

Re:One Technique I have used... (1)

IBitOBear (410965) | more than 13 years ago | (#195000)

Odd, my manual page for RPM, in the install section, says "--root", "--prefix" being reserved for parts of scripts where the variable "PREFIX" is used, while "--root" is for prepending to all paths (presumably using chroot I guess).

Take that Mr. Smarty-pants

--

Re:One Technique I have used... (1)

IBitOBear (410965) | more than 13 years ago | (#195001)

Also My Bad. On a reread, it wasn't totally clear that I was still talking about installing to a mock-up system in a sub directory. --prefix is what you use if the package is relocatable and you want the install to "work" in the overall live system. Let's split the 0-point karma we'll get for the thread 8-)

--

How very defensive, let me clarify... (1)

IBitOBear (410965) | more than 13 years ago | (#195002)

Pardon me. My bad. It should have been: "There is no I-DIDNT-HAVE-TO-BUY-A-$100-PACKAGE-IT-WAS-JUST-BUI LT-IN method to do this on a windoze install wizard platform.

You ever try to *READ* or *USE* the "recordings" made by CleanSweep etc? I have. Since the aforementioned products don't actually follow the actions of a hierarchy of programs (they, typically, record a delta on the system between two time points) you don't get a picture of what an installer did. You get a picture of all the system changes that happened while an install was running.

The difference being that if you didn't completely quiet every other potential action on the system you get a set of changes that are potentially larger than what the installer is responsible for.

Even worse, if the installing app spanws sub-installs (e.g. if the Thumbs Plus installer, for a randomly chosen concrete example, preforms a sub-install of ODBC updates) then you get fragmentary logs. Some of "what does this do" is in one place while the rest of it is in another.

But clearly I have no experience in these matters.

--

It not that good in your pigpen... 8-) (1)

IBitOBear (410965) | more than 13 years ago | (#195003)

Its great you can monitor something trashing your Registry. We don't have a registry to trash. Which is better?

I have seen mentions of "Tripwire" but I don't know what it does. I have *used* "truss" on Solaris and since it will (optionally) follow eihter or both branches of a fork and all internal threads if you use it wisely, and have any skill as chosing options and processing text files with "grep" etc I would be it beats the hell out of FileMon.

SMS will always mean "Shitcan My System" to me, but I may be biased. 8-)

As always, reguardless of platform or action is us about the admin knowing what question to ask more than it is about the platforms ability to magically disgorge "just the right answers".

MY general rule is, If I can phrase the question, likely someone with more drive than I has already written a program to provide the answer. If not, it's time to whip out the old compiler and do my share for the war effort...

--

One Technique I have used... (4)

IBitOBear (410965) | more than 13 years ago | (#195004)

I used to keep a mock-up of a minimal system (or you can use a disposable separate computer) in a sub-directory and I would do test installs to that sub-directory using a chroot(ed) shell. This gave the opportunity to deconstruct things nicely. It was only used after other suspicous nonsense happened with a package or source.

If this is an RPM on a linux box use "--root somedir" to prefix things like /usr with "somdir". Other install methods like SVR4 "pkgadd" have similar relocate facilities.

If it is just a tar/cpio archive and an associated script to install, read the scripts.

In short, there is no one tool to do this stuff, but it *is* doable by a number of means (on a *nix box) where it is otherwize impossible to do it on a windoze install-wizzard bundle.

Hope this helps some.

Rob.

PS I have also had some luck by looking at the errors generated after trying to run the install as a normal, non priveliged, user.

--

Tripwire (1)

jbr1 (412476) | more than 13 years ago | (#195005)

Look at tripwire (http:\\tripwire.org). It lets you compare file system deltas

Re:chroot and diffs between ls -R (1)

Arcturax (454188) | more than 13 years ago | (#195007)

> (Just curious, isn't this something for a mailing list rather than an Ask Slashdot?) I think it was a valid question. One reason they are posted here is not only to answer the question, but to discuss the issue around it, in this case, installers doing funny stuff to your computer without your knowledge. Its also nice to hear about other ways to do it as well. BTW: I didn't know about chroot until I saw this, so it teaches some of us about it as well. I wonder if this works in Mac OS X...

turn on scripting & bastille as a 3rd party (1)

discogravy (455376) | more than 13 years ago | (#195008)

as others have said, running the prog in chroot jail will do the trick, you could also:
a) make sure your backups -- you do have backups, right?-- are current;
b) turn on scripting to see what files get changed
c) install.
d) compare and reconfigure.

alternately, Bastille is a great resource (last i checked it was red-hat(-and-derived) dependent, but that was a while ago, so that may have changed.

--
"If you're really evil, let's see you EAT THIS KITTEN!"
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>