Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Watching Under The Hood Of Tiger's Spotlight

Hemos posted more than 9 years ago | from the looking-at-it dept.

OS X 43

jaketheitguy writes "Over at KernelThread.com, Amit Singh has released a commandline app called FSLogger for looking under the hood of Tiger's Spotlight. You can watch all kinds of filesystem changes going on in realtime. The utility apparently intercepts and displays filesystem change data as it goes out to Spotlight from the kernel. It even tells you which app is making the changes. Looks like Apple has included some pretty powerful API's in Tiger and there may be some othre really interesting uses of this API as mentioned on the app's page. I for one would really like to be able to tell if somebody changed ANY files on my system without my knowledge. I think you can do that with Singh's program, but how do you make sure somebody cannot disable the program?"

Sorry! There are no comments related to the filter you selected.

Two in a row? (1, Offtopic)

jack_call (742032) | more than 9 years ago | (#12610915)

Although it's two different areas, isn't two articles on spotlight a little extreme?
Come on Hemos, lets have a hattrick :-)

and oh... I for one welcome our new Spotlight overlords

This story seems utterly confused. (1, Informative)

Anonymous Coward | more than 9 years ago | (#12611189)

First there always has been a program called /usr/bin/fs_usage which monitors file system access. Second is the story writer worried about someone altering his files or about spotlight. How spotlight functions has been the subject of many detailed articles. Any time you change a file, spotlight calls the appropriate indexing program and collects and stores the metadata. It is not going to alter the data fork of your document but the data has to get stored somewhere. So relax. this story is paranoid delusional.

No Silver Search Bullet (0, Insightful)

Anonymous Coward | more than 9 years ago | (#12610922)

So all the article says is that the Silver Bullet or Holy Grail of Searching didn't turn out to be something one could create simply by telling the programmers to do it?

Apple (and MS for that matter) try to create a system where you don't have to keep any order on your computer and find anything you want instantly. I am sure I am not the only one with a gut-feeling that this is closer to the area of unsolvable problems, right with "Making Software Idiotproof" and "Creating the perfect user-interface everyone can use without any prior computer experience" and "Creating a 100% secure computer on the internet",...
[ Reply to This | ]

Spotlight changed my life. (3, Funny)

duffbeer703 (177751) | more than 9 years ago | (#12610935)

I used to be a lonely nerd, but thanks to Spotlight I can:

- Run Faster
- Jump Higher
- Score with the chicks
- Regrow lost hair!

Re:Spotlight changed my life. (2, Funny)

zero_offset (200586) | more than 9 years ago | (#12610938)

Disclaimer: If any of these conditions persist for more than four hours, seek medical attention immediately.

Re:Spotlight changed my life. (1)

valkraider (611225) | more than 9 years ago | (#12625149)

Ask you doctor if "Spotlight" may be right for you!

Possible side effects may include but are not limited to data loss, computer malfunction, loss of electricity, rugburn, high phone bills, cataracts, auto repossession, and in rare cases death and/or dismemberment and eternal damnation. Use only as directed.

Two stories in 20 minutes? (1)

Mensa Babe (675349) | more than 9 years ago | (#12610965)

It seems that the Spotlight is in the spotlight. Contrary to what I said [slashdot.org] before, the AC might have been right about Spotlight being overhyped in the extremes. It is overhyped to the max.

Re:Two stories in 20 minutes? (0)

Anonymous Coward | more than 9 years ago | (#12611717)

"Karma: Positive (probably because of superiour intellect)"

Are you sure about that?

Re:Two stories in 20 minutes? (1, Funny)

Anonymous Coward | more than 8 years ago | (#12615134)

Heu's froum Canadau. Theuy speull wourds wiuth louts ouf euxtra U's iun theum theureu.

Re:Two stories in 20 minutes? (0)

Anonymous Coward | more than 9 years ago | (#12619715)

Yeah, except all those words had U's in them to begin with, and the Americans just *had* to be different and remove them...

Recursion (2, Funny)

dangitman (862676) | more than 9 years ago | (#12610983)

Amit Singh has released a commandline app called FSLogger for looking under the hood of Tiger's Spotlight. You can watch all kinds of filesystem changes going on in realtime.

So, this application would shine a spotlight on Spotlight? Is that anything like when you point a video camera at a monitor hooked up to the camera's output?

Re:Recursion (1)

wealthychef (584778) | more than 9 years ago | (#12629224)

Honestly, if you were to redirect the output of FSLogger to a file... wouldn't it then bog the system down?

Run 2 versions (0)

Anonymous Coward | more than 9 years ago | (#12611173)

You could control changes to the app itself by having 2 editions of an fslogger based tripwire-app running together and then have the one go off when some changes happens to the other and vice versa.

IDS Potential (1)

Sentry21 (8183) | more than 9 years ago | (#12611286)

This has a lot of potential in the server market. Imagine an IDS that monitors certain files for changes and notifies the sysadmin immediately whenever a static file is updated. The system could have scheduled periods for upgrades, during which it doesn't send a thousand warnings to you, but other than that, it could monitor all disk activity at a low level without being subverted by e.g. changing the IDS's file hashes before it does its next check.

Interesting idea.

Tripwire (3, Informative)

@madeus (24818) | more than 9 years ago | (#12611633)

Actually you can get this functionality already in a long standing Unix utility called Tripwire.

http://www.tripwire.com/ [tripwire.com]
http://sourceforge.net/projects/tripwire/ [sourceforge.net]

There is even a Mac OS X version now it seems:
http://www.macguru.net/~frodo/Tripwire-osx.html [macguru.net]

Of course you'd probably then want an OS that implements some form of relevant Mandatory Access Control / POSIX.1e (e.g. LIDS for Linux, Trusted Solaris, or Argus Pitbull (Linux/Solaris)) to help prevent the intruder from interfering with Tripwire itself.

Re:Tripwire (2, Informative)

womby (30405) | more than 9 years ago | (#12621218)

I am going to assume you didn't read the article and provide a small description of what fslogger is doing and how it has nothing in common with tripwire.

Fslogger runs continuously and registers itself with the kernel, when a filesystem change event happens details about it are announced to all registered apps and fslogger displays the information it receives in a useful (if verbose) manner.

Tripwire is a fantastically useful app which I run on every one of the servers I admin, and perhaps the OSX version could be extended to make use of the same kernel interfaces that fslogger is using.

Tripwire runs once per day (however often you wish to run it), and scans the filesystem checking each file to see if it matches a checksum calculated at some known time in the past. This is useful on mission critical servers because outside of data / user directories changes should happen very infrequently. Tripwire is a robust way of confirming that a server has the same configuration on a day to day basis.

The startling difference between tripwire and fslogger should be obvious, tripwire has no mechinism to know when a file has changed except by looking at the file directly, fslogger has no mechinism to know if the event is important or not and no mechinism to notify an administrator of the event short of scrolling it by in a terminal.

With tripwire you could delete a file, recreate it from backup and so long as you was careful tripwire would never know. fslogger would display every step you took but would not know if the final step returned the filesystem to its original state.

Different tools with different behaviours for different target users presenting different information in a different manner.

So, to dispute your original assertation,
"Actually you can get this functionality already in a long standing Unix utility called Tripwire."
No, nothing fslogger does is replicated by tripwire and nothing tripwire does is replicated by fslogger.

Re:Tripwire (1)

womby (30405) | more than 9 years ago | (#12621295)

I didn't notice that you was replying to somebody who suggested creating tripwire, I thought you had posted a comment specifically about flogger.

Sorry if it the reply came across as harsh.

Re:Tripwire (1)

@madeus (24818) | more than 9 years ago | (#12626313)

Hehe, easily done. :)

I think the idea of having Tripwire hooks so that it's automatically informed of changes real time when on Mac OS X is certainly interesting and I'd think eminently doable.

I think true real time updates may actually have been a feature of a commercial implimentation (for Solaris), but that would be going back 7-8 years ago now, so I'm not certain (it could have been just a daemon that periodically checked for changes, or I may have remembered wrongly).

PS: I hadn't heard the name 'fslogger' used for the real time file monitoring feature in the Dawrin kernel - though I've not actually had a name for it all (it seems weird that it wouldn't have one, given it impacts not just things like Spotlight, but Automator and even the Finder, which now updates Windows contents in real time) - but is it perhaps just the name for a utility that lets you view the change history?

Tripwire (2, Informative)

mithran8 (186371) | more than 8 years ago | (#12615505)

You may be shocked to know how often files change on your system... without a good policy defining the scope of your monitoring, you're asking for a world of hurt. As @madeus mentions, there is an OS X build of Tripwire which gives you a good deal of this functionality. Two caveats, however:

- Tripwire is not a real-time service, it's scheduled to run at specific (user-defined) times.

- Tripwire does not prevent anyone from making changes - it merely ensures that any changes to the OS are recorded and made visible to you.

That said, Tripwire is a very functional tool with excellent scripting and integration potential. Plus, it has a good amount of internal security - unless you know the relevant passphrases, you cannot subvert the product. If you root the box you can always uninstall it, but you can't tamper with the database or policy to hide your changes or trick Tripwire into sending a false 'all clear' message.

As with all such tools, it's best to check it out and evaluate it for yourself to see how it works for you.

Where's "As Seen on TV" ... (1)

GaryPatterson (852699) | more than 9 years ago | (#12611615)

... when you need him?

He was very vocal about this sort of thing, and now he's gone very quiet. Almost as if he was an Apple employee who was given The Warning (tm) or... (obligatory Star Wars reference being used in shameless Karma whoring) ... as if a million of his posts were made, and then suddenly silenced. Hmm...

When I get some time, I'll read the article (thus breaking a long-running streak for me) and compare to ASoT's statements.

Re:Where's "As Seen on TV" ... (1, Informative)

As Seen On TV (857673) | more than 9 years ago | (#12612606)

No, I'm just out of the office this week.

Re:Where's "As Seen on TV" ... (1)

Elwood P Dowd (16933) | more than 9 years ago | (#12613314)

So, not only do you talk sense about unreleased Apple products while being an Apple employee, you do so from your office at Apple?

Awesome.

Re:Where's "As Seen on TV" ... (0)

Anonymous Coward | more than 8 years ago | (#12615885)

Yes, he does. I have seen the traffic reports of links posted in replies to his comments, and he is hitting those links from Apple IP addresses in the 17.* range.

Re:Where's "As Seen on TV" ... (1)

SYFer (617415) | more than 8 years ago | (#12618330)

Although I cannot believe these posts are from Jobs himself (as the poster below and others have suggested), he's either astroturfing /. in a provocative way on behalf of Apple--probably with Jobs' express consent, or he's truly doing it on the sly. Given Apple's secretive nature, I'd say that if it's the latter, he'd be a fool to post from the office as he would be easily identified via logs and, I can only assume, be shown the door. And he does not strike me as a fool.

If the AC's observation is correct and he is surfing from 1 Infinite Loop, it's clear to me that he's doing it with the blessings of the company. On the other hand, the appearance of Apple IPs in the logs could be from anyone at Apple--/. is a major tech site and I'm sure it's read by a great many of the employees. Hell, even a security detail assigned to rooting him out could generate the hits.

I guess you'd have to build a sort of ASOTV honeypot (or "AsottyPot") to track clicks, but I'm not exactly sure how you would set it up or sift the results. You'd have to post compelling honeypot links in response to his comments then maybe look for hits that closely match his posting times and look for consistent patterns. Joe Apple Lunch Hour Surfer would not consistently know when every ASOTV post appeared, so if a pattern emerged, you might have something.

Bottom line though, it would be very easy to trace him internally if he were posting from campus. He's also given a few specific personal details over time (the island thing, for example), but those could be clever red herrings, of course. I'm leaning toward his being an authorized insider poster and frankly think it's a smart thing. Apple is always concerned about the online zeitgeist, so why not join in the fray. Are they bound by the Prime Directive?

Re:Where's "As Seen on TV" ... (1)

That's Unpossible! (722232) | more than 9 years ago | (#12623883)

I do like the insider's look at Apple, also, but I think Apple would be opening itself up to shareholder lawsuits and SEC troubles if it has officially told an employee it is OK to post "insider info." For example, the information ASOT posted in the discussion of the video capabilities in iTunes could be considered vital information that should not be divulged to only a small group of people.

Re:Where's "As Seen on TV" ... (0)

Anonymous Coward | more than 9 years ago | (#12665896)

Bullshit. Why is it that any time that anyone provides corroborating evidence for ASOTV's employment at Apple, said evidence is invariably given anonymously? Post under your username (so that we can check your credibility by means of your account history), or better yet, post with a verifiable link.

ASOTV is a troll who has managed to dupe plenty of Slashbots into believing he actually works for Apple. No more.

Re:Where's "As Seen on TV" ... (0)

Anonymous Coward | more than 8 years ago | (#12618034)

Well, if you subscribe to the "he's an Apple shill" school of thought, then of course he posts from work. It's his job. He probably posts to Fark or SA on his own time.

Re:Where's "As Seen on TV" ... (3, Funny)

yanndug (832046) | more than 9 years ago | (#12613728)

Interesting. Steve Jobs is out of the office these days... http://www.thinksecret.com/news/0505itunes49.html [thinksecret.com]

Re:Where's "As Seen on TV" ... (2, Funny)

Reaperducer (871695) | more than 9 years ago | (#12619813)

I really really really really really really hope that Steve Jobs has something better to do than read Slashdot. If not, then there's no hope for the rest of us.

Re:Where's "As Seen on TV" ... (0)

Anonymous Coward | more than 9 years ago | (#12620138)

Of course he does. He's busy with lawsuits, book purges and washing machine selections. Time to read one of the world's most popular technology sites? I should think not!

ASOT is not Steve Jobs (2, Insightful)

That's Unpossible! (722232) | more than 9 years ago | (#12623679)

Here are three reasons why:

1. ASOT is too familiar with the technical underpinnings of Apple technology. Steve Jobs is smart smart smart, a great businessman, but there is no way he is this familiar with all the technical details. That was what Woz was for, remember? (No I'm not implying this is Woz, since he clearly no longer has this much access to Apple.)

2. There's no way the CEO of a public company would risk the MAJOR, MAJOR, MAJOR lawsuits and trouble that could be caused from the SEC and shareholders by divulging valuable information on Slashdot. There are rules the company officers must strictly follow in regards to how they divulge information previously unknown to the public. The information must reasonably be made publically available, not posted anonymously on Slashdot.

3. Steve Jobs gets more bang for his buck by keeping things top secret until the next time he's doing a keynote.

Re:ASOT is not Steve Jobs (1)

Decipherer's Dream (886385) | more than 9 years ago | (#12643465)

Has anyone noticed yet that As Seen On TV is an anagram for Steve's Anon? (Anon could mean Anonymous).

I don't know of any Steve at Apple except Jobs, so what should we conclude?

Re:ASOT is not Steve Jobs (1)

cvdwl (642180) | more than 9 years ago | (#12660762)

I don't know of any Steve at Apple except Jobs, so what should we conclude?

That ASOT is a witch!!

Re:ASOT is not Steve Jobs (1)

CableModemSniper (556285) | more than 9 years ago | (#12672841)

Burn her! Burn her!

Re:Where's "As Seen on TV" ... (1)

GaryPatterson (852699) | more than 9 years ago | (#12621160)

Cool bananas. You've been quiet lately, so I wondered...

Re:Where's "As Seen on TV" ... (1)

paco_loco (884415) | more than 9 years ago | (#12621488)

ASoTV doesn't seem too bothered about giving out details of his movements does he? If anyone at Apple really wanted to identify him it wouldn't be too hard if you read all his posts...?

Re:Where's "As Seen on TV" ... (0)

Anonymous Coward | more than 9 years ago | (#12642031)

Except of course, that he actually doesn't work here...

Tracking changes to the file system (3, Informative)

Simon Spero (10945) | more than 9 years ago | (#12612701)

There's a system call that lets user-space programs subscribe to a lot of interesting kernel level events.

Take a look at the kqueue(2) man page.

There are more details available at http://people.freebsd.org/~jlemon/papers/kqueue.pd f [freebsd.org]

Re:Tracking changes to the file system (2, Interesting)

argent (18001) | more than 8 years ago | (#12616040)

Yeh, when I heard about this I assumed that Apple would use kqueue and watch changes in the vnodes. It would require some extension to kqueue, because there's no "EVFILT_FS" or "EVFILT_VOP" filter that would monitor VOPs on more than a single file. But they needed to extend HFS+, too, so that's not really a big deal. You do have to be careful with this, because trying to monitor VOP_WRITE would be like drinking from a firehose... but you wouldn't actually need to track file content changes that closely for something like Spotlight that only needs to know that a file has changed recently. The key is that file content changes are reflected in the file modification time, so changes to the stat structure would be good enough if you delayed the actual examination of the file more than the granularity of time_t (one second), so by monitoring a few key VOPs you could get Spotlight working efficiently over any file system.

In fact, you could even just track inode changes and VOP_OPEN, VOP_MMAP, and VOP_CLOSE, and periodically peek at files that are open a long time to see if they're changed. The main thing is to be able to tell where to look without having to regularly traverse the whole file system.

Why they decided to use HFS+ instead of doing it at the vnode layer, I don't know. I can make some pretty good guesses, of course, because after all HFS+ is their baby and they really don't care much about supporting other file systems.

It's a shame. I really don't trust HFS+, and I wish they'd do more to support UFS transparently.

Physical security essential (1)

davidwr (791652) | more than 9 years ago | (#12613101)

"how do you make sure somebody cannot disable the program?"

You can't, not withint guarenteeing physical security to the box.

If someone can pull your hard disk OR boot with their own media, all is lost.

Short of that, your question amounts to "how do I keep from getting rootkitted."

Just use fs_usage (0)

Anonymous Coward | more than 9 years ago | (#12631123)

Seems like fs_usage does the same thing, and is already installed with OS X. From the man page:

NAME
fs_usage -- report system calls and page faults related to filesystem
activity in real-time

SYNOPSIS
fs_usage [-e] [-w] [-f mode [-f mode] ...] [pid|cmd [pid|cmd] ...]

DESCRIPTION
The fs_usage utility presents an ongoing display of system call usage
information pertaining to filesystem activity. It requires root privi-
leges due to the kernel tracing facility it uses to operate. By default
the activity monitored includes all system processes except the running
fs_usage process, Terminal, telnetd, sshd, rlogind, tcsh, csh and sh.
These defaults can be overridden such that output is limited to include
or exclude a list of processes specified by the user.

Re:Just use fs_usage (1)

Lucractius (649116) | more than 9 years ago | (#12631767)

On a side note this kind of thing is (though i cant check exactly im just going from the info i have on what spotlight does cause Well. I dont have a mac :P ) also available for windows machines through programs available at www.sysinternals.com

These guys are utter LORDS of the nt OS by any definiton. ( read their "About us" section and see just how A class it is. A Microsoft Most Valued Proffesional no less )

Anyway. There are filesystem access and notification tools around for nearly any os and its good to see OS X realy making a push with them instead of the way theyer usualy swept under the rug in most OSes publicity stuff (not that many oses have publicity to speak of lol )

Re:Just use fs_usage (0)

Anonymous Coward | more than 9 years ago | (#12632142)

fs_usage is not the same as fslogger. They use different API's, and report seemingly similar, but different things.

Sorry, you have to do the reading yourself (read the man page of fs_usage carefully, and then RTFA).

Hint: why doesn't Spotlight use the same mechanism as fs_usage?
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?