Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

How To Guarantee Malware Detection

CmdrTaco posted more than 4 years ago | from the pardon-my-skepticism dept.

Security 410

itwbennett writes "Dr. Markus Jakobsson, Principal Scientist at PARC, explains how it is possible to guarantee the detection of malware, including zero-day attacks and rootkits and even malware that infected a device before the detection program was installed. The solution comes down to this, says Jakobsson: 'Any program — good or bad — that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte.'"

cancel ×

410 comments

Sorry! There are no comments related to the filter you selected.

At least one byte (3, Insightful)

BadAnalogyGuy (945258) | more than 4 years ago | (#31483136)

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Hell, Windows is a whole slew of malware that masquerades as the whole OS.

Re:At least one byte (1, Interesting)

sopssa (1498795) | more than 4 years ago | (#31483168)

It doesn't need to do even that.

They forgot that malware code can reside inside another process and it's memory space, in which case comparing and writing random bytes to free RAM is a moot point.

Re:At least one byte (4, Informative)

palegray.net (1195047) | more than 4 years ago | (#31483222)

I don't normally quote huge chunks of articles, but I'm making an exception on grounds that I don't believe you RTFA before replying:

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

Why a delay, you ask? Because secondary storage is slower than RAM. Especially if the order of the reads and writes are done in a manner that intentionally causes huge delays if diverted to flash, hard drives, etc.

There's more details in RTFA.

Re:At least one byte (2, Insightful)

sopssa (1498795) | more than 4 years ago | (#31483344)

I did, and he doesn't say anything about this point.

Regarding making a keyed hash of the entire memory content, how would that even work? Every program modifies it's memory all the time. Then there's the programs like copy protections and Skype etc that modify it's own code in real-time too.

Refuting the imaginary article in your head (5, Informative)

spun (1352) | more than 4 years ago | (#31483496)

Still haven't read the article, eh? The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory. I could explain it all in greater detail, but, you know, there's this article, already there. Please do try to constrain your criticisms to things that actually apply to the article that was written, you know, the one we can all read. Refuting the imaginary article in your head does nothing for the rest of us.

Re:Refuting the imaginary article in your head (1, Interesting)

causality (777677) | more than 4 years ago | (#31483614)

Still haven't read the article, eh? The technique is to swap everything out except the scanner, then write random bits to the entire memory space, then hash the memory. I could explain it all in greater detail, but, you know, there's this article, already there. Please do try to constrain your criticisms to things that actually apply to the article that was written, you know, the one we can all read. Refuting the imaginary article in your head does nothing for the rest of us.

I'm glad you guys have managed to work out what the article says.

I have one glaring problem with this system, and all other systems designed to detect running malware: no focus on prevention. I'm glad we have a new tool to detect malware executing on a machine that's already compromised, but that's what all of the new tools I read about intend to do. I don't see much progress being made in terms of the design decisions and best practices that prevent (Windows) machines from getting compromised in the first place.

As good as this technique may or may not be, it's not security. It's damage control. It has this in common with almost every malware "solution" that is ever posted to Slashdot. Where are the researchers who want to prevent intrusion in the first place instead of having lots of clever techniques for identifying/limiting its damage after it happens?

Re:Refuting the imaginary article in your head (4, Insightful)

spun (1352) | more than 4 years ago | (#31483716)

Protection from malware should function like the immune system, with many lines of defense and many avenues of detection and counter attack. Prevention will never be perfect by itself.

Re:Refuting the imaginary article in your head (4, Insightful)

HungryHobo (1314109) | more than 4 years ago | (#31483758)

After reading TFA I'm still not seeing how this is supposed to detect unknown malware.
As far as I can see it would decide that a new install of any kind was a virus.

Sure if you know every program which is supposed to be installed and none of them do wierd things in memory(a big if) then you might be able to spot when some kind of change has been made but if you can do that then you have a situation where you might as well just re-image the machine from ROM every now and then.

I don't see any amazing new ideas in TFA

Re:Refuting the imaginary article in your head (0)

Anonymous Coward | more than 4 years ago | (#31483670)

Ok, you can swap everything out except the scanner. So assume that the malware is hiding inside another process, then it will be swapped out.
Then the hash will be computed etc., and... (surprise) nothing turns out to be wrong.

And the malware will go undetected.
Now, do you understand TF point?

Re:Refuting the imaginary article in your head (1)

spun (1352) | more than 4 years ago | (#31483768)

That is a different point. And also not valid. If the malware lets itself get swapped out, it can not interfere with a scan.

Re:Refuting the imaginary article in your head (1)

Java Pimp (98454) | more than 4 years ago | (#31483824)

If the malware gets swapped out it won't be detected in the scan. Which was sopssa's original point at the top of the thread.

Re:Refuting the imaginary article in your head (1)

clone53421 (1310749) | more than 4 years ago | (#31483830)

If the malware lets itself get swapped out, it can not interfere with a scan.

No. It can not. But you still have no way of guaranteeing that, even if the scan is not interfered with, it will find the malware.

In fact, I will guarantee that no scan will detect 100% of malware, even if the scan isn’t interfered with (by masking the malware in RAM or on the secondary memory).

Re:Refuting the imaginary article in your head (4, Insightful)

Romancer (19668) | more than 4 years ago | (#31483724)

I'm not an expert and not sure if I'm missing something obvious here but what is confusing me is the part about "swap everything out except the scanner". Wouldn't you then just be moving the malware too? Into a protected space that you then have to scan and know what to look for?

If it's a zero day infection then you don't know what to look for and you swapped it out of memory for nothing really. I do get that if it tries to protect itself it will look suspicious but what if it looks like a normal program? A service or scheduled task that could be normal. What if it takes on the guise of an adobe update program in size/hash and function until it is time do act? Say a slight change to your systems dns entries. Then goes dormant again.

This may not be possible but I haven't seen why not and it leaves a pretty big hole for zero day infections that this method claims to be able to catch 100%.

Re:At least one byte (1)

Anonymusing (1450747) | more than 4 years ago | (#31483512)

I would also wonder about false positives on shareware, poorly written apps, custom corporate apps, etc.

Re:At least one byte (0)

Anonymous Coward | more than 4 years ago | (#31483626)

Again, RTFA.

The article talks specifically about malware that defends itself.

That is, you start by swapping all active memory. The malware, defending itself, will attempt to stay active, and "dodge" the swap. Then, you attempt to overwrite the malware, again assuming it will defend itself and block these attempts. Once that is the case, the keyed hash becomes valid because you can detect areas where the overwrite was "blocked" from happening.

Presumably you would use the usual known detection algorithms for malware that doesn't defend itself (sandboxing et al)

Re:At least one byte (0)

Anonymous Coward | more than 4 years ago | (#31483646)

Don't say "it's" when you mean "its". You're making my eyes bleed.

I don't think the assumption is warranted (0)

Anonymous Coward | more than 4 years ago | (#31483502)

All you need is to have the checksum calculated by the verifier for the process you contaminated. Then catch the reading part. A checksum can be small enough that you do not need to swap it out and you can include in the malware. therefore, "Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies." is IMHO incorrect. You don't need to save much info, maybe 32 bytes+the code to redirect the read. Why swap that out ?

Re:At least one byte (0)

Anonymous Coward | more than 4 years ago | (#31483604)

Unless you know exactly what bytes are supposed to get read/written from any given "application", requiring a 'needle in a haystack' level of RAM analysis, you will not be able to tell which bytes are from malware and which ones are not. THAT is the problem.

Re:At least one byte (1)

Jazz-Masta (240659) | more than 4 years ago | (#31483322)

Virus scanners figured this out years ago, this is why they scan the operating memory!

The difficult part is finding out which "bytes" are bad. The problem is many elements of spy tools are often used for good too. Like VNC and all of those legitimate screen capture and key logger programs for IT.

Re:At least one byte (1)

Astrorunner (316100) | more than 4 years ago | (#31483748)

Sorry, but I think you're missing the point. Apologies if I've misunderstood your point.

I have a root kit. I have kernel level access. I know when my memory is being scanned. When another process goes to poke around my memory, I intercept that request and return back zeroes or gibberish -- anything but my code.

I in effect become invisible to your scanner. Scanning operating memory isn't good enough.

Now, since they're writing out random data to memory, they're attempting to ferret out any bad code that might be lurking out there. They don't have to identify the specific bad code, but *suspicious* code, just code that bad code attempts to hide itself by righting out to disk as random data is written over it.

Re:At least one byte (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#31483360)

It doesn't need to do even that.

They forgot that malware code can reside inside another process and it's memory space, in which case comparing and writing random bytes to free RAM is a moot point.

Dear Microsoft:

Is all of this REALLY easier than fuzz-testing and fixing your goddamned network stack, fixing IE, and teaching your users that random strangers on the Internet don't have their best interests at heart? Really, most of these exploits are freakin' buffer overflow attacks. How many billions of dollars does it take to do some bounds checking? Or to use a compiler with something like GCC's SSP?

Why Microsoft? Because we're obviously talking about Windows here. That's the only OS with a severe malware problem (MS fanboys, take note - I don't care why it has that problem, so spare us the excuses). Funny how they want the name Windows, "Designed for Windows", and Microsoft Windows plastered everywhere, except when a report comes out about malware. Then it's just "malware" infecting "computers", not "malware exclusively for Microsoft Windows". Microsoft Windows, Microsoft Outlook, and Microsoft Internet Explorer are the three biggest names in malware. Want to be a bit safer, quit using that combination.

Re:At least one byte (1)

AvitarX (172628) | more than 4 years ago | (#31483690)

Most probably do not run buffer overflows. Most are probably people running the executables stupidly.

It comes down to teaching your users that random strangers on the Internet don't have their best interests at heart, and yes I imagine that is an impossible task.

Re:At least one byte (1)

sam0737 (648914) | more than 4 years ago | (#31483486)

It doesn't need to do even that.

They forgot that malware code can reside inside another process and it's memory space, in which case comparing and writing random bytes to free RAM is a moot point.

So you mean his idea can only deal with "Computer Germs" but not "Computer Virus?"

Re:At least one byte (1)

Astrorunner (316100) | more than 4 years ago | (#31483634)

Its taking a moment to digest what the author is saying. He really doesn't mean that he can detect *any* threat at any time, but rather, in the future he *can* detect them as new threats are discovered.

If a piece of malware allows itself to be swapped out, it can then (now, or in the future when its identified as a threat) be detected without worrying about it trying to hide itself.

Then, once its swapped everything out, it can start the hunt for malware that remains in memory.

The key here to the Hunt is the writing of random data... If you overwrite the memory with random data, the app has *no choice* but to write itself out to secondary storage to hide the random data. If the memory was zeroed out, it could simply return zero for every byte, but with random data, it has to remember each byte. Since its random, it can't really compress the data to hide itself solely in memory, but there guarantee should be that it will be able to find it eventually.

He talks a bit of hype, and there's some promise to that, but it sounds like he may not be able to find zero day threats that willingly allow themselves to be swapped out.

Re:At least one byte (5, Funny)

dmgxmichael (1219692) | more than 4 years ago | (#31483232)

Not a fair comparison. Malware usually does what it's supposed to.

Re:At least one byte (5, Funny)

Chris Burke (6130) | more than 4 years ago | (#31483436)

While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

Oh see he didn't finish explaining.

Any program that wants to be resident has to occupy at least one byte of RAM. And that byte should include the Evil Bit, which all malware should set. Then your anti-virus program just checks the Evil Bit and problem solved!

Re:At least one byte (2, Informative)

alexhs (877055) | more than 4 years ago | (#31483820)

You've been modded funny, but this is exactly what this "article" is all about.

If there is no malware in RAM, the results will be the expected result. [...] Or there could be malware in RAM, and the checksum will be wrong. [...] Or malware could divert the read requests [...] . That would result in the right checksum... but a delay.

Or, there could be malware in RAM, not diverting read requests, and the checksum will be the expected result, and without a delay.

The only problem with Markus Jakobsson grand theory is that all malware are of that last kind.
Well, all malware since the memory protection era. I suppose his "product" could work for DOSes (but there is no multitasking there) Windows 3, MacOS9, AmigaOS and some embedded OSes.
And if the malware does virtualization, he is located in a memory area that his product won't be able to scan anyway.

So, simply put, it is a scam.

Re:At least one byte (1)

lgw (121541) | more than 4 years ago | (#31483442)

He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

Re:At least one byte (2, Informative)

causality (777677) | more than 4 years ago | (#31483726)

He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

I'd rather just secure my systems, thanks.

Re:At least one byte (3, Insightful)

OeLeWaPpErKe (412765) | more than 4 years ago | (#31483456)

There are several weaknesses. Note how he says something about an external verifier, which is delay-sensitive. Note how for the first 2 steps he keeps repeating "malware may of course interfere but this doesn't matter because", and then he stops considering malware interference. That's because at those points, malware interference would be fatal.

Of course, malware could simply take over the entire procedure, computing the keyed hash itself (a process which can run in a lot less memory : it doesn't actually index memory, it just generates the pseudo-random bytes directly, then it sends back the correct keyed has).

Calculating the entire hash in the processor cache will let this method outperform the checksummer by a large margin, meaning it can send the data back after any chosen interval.

(and before you say "keyed", obviously the encryptor needs to know the key)

In practice you'll also find that large amounts of ram space cannot be swapped out : BIOS, some bits of the operating system, ... There's even memory locations that aren't in the memory map, but are nevertheless backed by physical ram (in a pci card or ...), and thus won't be included in any (sane) scan. Surely one can find some place where a virus could squeeze in.

Re:At least one byte (1)

Panaflex (13191) | more than 4 years ago | (#31483826)

Oh, and lets consider that a virus writer would know the basic fundamentals of memory management, reporting back less RAM pages than expected. After all, we're depending on the OS memory allocator to provide correct number of pages, paged/nonpaged, etc..

In other words, we could alter the VM page table and not report all physical pages to the OS / "Guaranteed Virus Detection"

Another problem to consider is that often programs are compiled with buffer overflow detection, which writes a pattern of bytes before and after allocated RAM... having a virus detector overwrite these bytes without returning them back may result in inaccurate reports of buffer overflow.

It's an interesting, even a good idea - but IMHO not a provably secure scheme.

Theory and Reality (4, Insightful)

ircmaxell (1117387) | more than 4 years ago | (#31483148)

In theory, theory and reality are equivalent. In reality, they are quite different...

Seriously, how could this possibly work for ALL (including undocumented, and hereto unknown) threats? And if it does it by reading straight from RAM (through the kernel), wouldn't a rootkit be able to trivially defeat that?

Re:Theory and Reality (1, Insightful)

Anonymous Coward | more than 4 years ago | (#31483262)

Even if you whitelist, if the memory space isn't properly protected by the OS, malware could use memory owned by the OS or a common application. 7 and the most recent version of MacOS are better at this, but I think there is still room for improvement.

Re:Theory and Reality (0)

Anonymous Coward | more than 4 years ago | (#31483364)

You hit the nail on the head. Everthing goes through the kernel and a rootkit would thus be invisible. Now if you had a hardened ROM that the kernel lived in this detection method could work. However how practical is it to have a system that has a hardend ROM tied to a hardened BIOS so it would only ever read the kernel into RAM from the ROM then read in applications from FLASH.

Re:Theory and Reality (1)

notgm (1069012) | more than 4 years ago | (#31483608)

what if your external verifier was hardware based? build a little device with hardened rom and bios, give it a usb interface, or maybe even something proprietary - let the detection take place off-board.

Re:Theory and Reality (1)

dmgxmichael (1219692) | more than 4 years ago | (#31483470)

Agreed - couple this with the reality that at least half and probably more of all malware today is packed in with programs the user installed. I don't know how much BearShare and similar crap I've had to clean off relative's comptuers. This technique does nothing to stop malware of that nature - no antivirus technique known can stop the user from installing programs that do stuff other than what they advertise to do.

Re:Theory and Reality (2, Insightful)

goombah99 (560566) | more than 4 years ago | (#31483546)

His whole point was not "this is how you should do it", it was "you could do this, and because you could do this it shows that it's theoretically possible". This is a variant of what is know as a gedanken experiment-- an argument that proves or disproves some fact is true while not actually being somethign you would want to carry out. For example, you could suppose that you could measure the force field is under by running a pole from the earth to the moon and pushing slightly on it. Not that you want to do this, but it shows that measuring that force field is possible at all. Now you need to figure out an easy way to do that.

There is something that can answer your questions! (0, Troll)

spun (1352) | more than 4 years ago | (#31483584)

How COULD this work? There is an answer. You can find this answer in a foreign place, known by the mysterious and terrifying name of The Article. Here's what you do: you read it. When you read it, your questions will be answered.

Basically, I can tell from the fact that you are asking irrelevant questions that you have not read the article. And you know what? I'm not going to explain it to you. To be clear, I am not saying, "This technique will work." I am saying "You are not criticizing this technique."

Re:There is something that can answer your questio (1)

ircmaxell (1117387) | more than 4 years ago | (#31483688)

Actually, I did read the article, and I still feel my original objection is valid. They admit in the Article that the program would read Ram through the kernel. So if a rootkit was installed, wouldn't it be able to defeat that (Which was the essence of my OP)? Not to mention that swapping ALL Ram to disk and filling it with "random" information would take a non-trivial amount of time (at least seconds, possibly tens of seconds depending on size of ram and speed of hardware). And is it possible that a legitimate program would want to access said memory in that time span? So I must know (at the risk of baiting), how was I not criticizing the article/technique?

Re:Theory and Reality (1)

HungryHobo (1314109) | more than 4 years ago | (#31483588)

ya. coming up with a reliable virus detection scheme for unknown viruses is pretty much in the same area as the halting problem.

Even detecting polymorphic viruses has been proven to be NP complete.

Re:Theory and Reality (1)

n1ywb (555767) | more than 4 years ago | (#31483656)

A rootkit that is AWARE of this detection mechanism ought to be able to defeat it easily by just overwriting the computed and expected keys in the detectors memory space with a random number. No delay, the values are equal, so no problem right? Wrong. The only way to make this really work would be for the detector to have direct hardware access to the machine's RAM but be running on a different uninfected machine. That's theoretically possible, but not really practical with out of the box hardware.

Re:Theory and Reality (1)

einhverfr (238914) | more than 4 years ago | (#31483702)

I read the article and I wasn't convinced. I don't think one can guarantee malware detection. Any detection approach has false positives and/or false negatives. Typically we err on the side of false negatives, while some other approaches (host-based IDS-type approaches) err on the side of false positives.

The method addressed here does not deal with all possible attacks, but only the problem of malware interfering with the scan. Hence even with such a mechanism, all you can use it for is guaranteeing the integrity of the scan process. It doesn't tell you by itself whether a given executable is malware or not. For that you have to either look for known threats or for suspicious system changes (sha256 checksums on files changing, for example).

"All we need" is generally an indication that something is missing in the analysis and this is the case here.

Wrong from the getgo! (1)

mcrbids (148650) | more than 4 years ago | (#31483714)

Not only that, but his initial premise is already wrong! Most people conceptualize a program like an application - it's launched, loads into memory, and then does stuff. And while that's typical, it's a grave mistake to think that's the ONLY way to go!

Off the top of my head, I can think of registering malware as a callack handler for a system event. In this case, you have an infected computer without any code running at all, in a context and namespace different than running applications!

Winows just wasn't designed with a multi-user security model. Adding this after the fact is showing itself to be exponentially more difficult!

So it has to be in RAM (1)

magsol (1406749) | more than 4 years ago | (#31483174)

The hard part is actually finding it.

Does it have to be in RAM? (2, Informative)

characterZer0 (138196) | more than 4 years ago | (#31483282)

It could be in ROM.

It could be in processor cache.

It could be in the video card's memory.

Could it be in pipelined instructions waiting to be executed?

Re:Does it have to be in RAM? (1)

magsol (1406749) | more than 4 years ago | (#31483664)

I would guess, though, that in order to do something we would typically characterize as "malacious" it would have to eventually wind up in RAM (except in the case of ROM...that would really fubar the system).

Processor cache would eventually result in execution, which takes place in RAM. Video card memory would just do funky things to the display. A pipelined instruction would also eventually end up in RAM once it was executed.

Perhaps the key here is "actively executing" malware. I suppose it can lay dormant just about anywhere that can possibly store writeable information for an extended period of time, but in order to wreak havoc it'd have to eventually wind up in RAM.

Re:So it has to be in RAM (2, Funny)

dissy (172727) | more than 4 years ago | (#31483290)

The hard part is actually finding it.

That reminds me of a signature I've seen around here (Sorry, I don't remember who was using it)

cat /dev/ram | strings | grep llama
OMG, my RAM is full of llamas!

Re:So it has to be in RAM (1)

Engeekneer (1564917) | more than 4 years ago | (#31483400)

Not as hard as reading the article

In case anybody was wondering... (5, Informative)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#31483188)

He is indeed selling something [fatskunk.com] ...

Theory and hand-waving (3, Interesting)

Lord Grey (463613) | more than 4 years ago | (#31483198)

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

<sarcasm>Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.</sarcasm>

That whole bit about swapping, though.... If I write malware and hide it somewhere in execution space, do I really care if it gets swapped out? So the code that steals keystrokes or sniffs for credit card numbers doesn't get executed for short while. Big deal. At some point it will get loaded again (if written properly, that is).

Or am I missing something obvious?

Re:Theory and hand-waving (1)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#31483352)

Punting the problem to an "external verifier" may or may not be a cop-out depending on the context.

If you want it to work, with a software update, on today's general purpose x86 office boxes, your "external verifier" might as well be a magic pony that sneezes rainbows and poops out the factors of any arbitrarily large primes that you feed it. Not Happening.

On the other hand, if your target is "Paranoid embedded architectures, 2-5 years from now" you can posit pretty substantial hardware changes at only modest cost. A little widget that can temporarily halt execution on the main CPU and do whatever checking it likes across the system's RAM would add cost and complexity to an embedded design; but probably not a gigantic amount. He isn't asking for an Oracle here.

Re:Theory and hand-waving (0)

Anonymous Coward | more than 4 years ago | (#31483460)

the factors of any arbitrarily large primes.

PROtip: Primes, large or small, do not have factors. That's why they're prime.

Re:Theory and hand-waving (1)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#31483488)

That's why it's a magic pony...

Re:Theory and hand-waving (1)

JustOK (667959) | more than 4 years ago | (#31483572)

You mean other than itself and 1.

Re:Theory and hand-waving (1)

NotBornYesterday (1093817) | more than 4 years ago | (#31483746)

WHOOOOOOSH ...

Re:Theory and hand-waving (1)

taustin (171655) | more than 4 years ago | (#31483368)

This proposal isn't to detect what malware is present, or to remove it. It is onl to detect that there is some malware present, which can then lean to more thorough scanning to detect and remove. Knowing that something is there is half the battle.

Re:Theory and hand-waving (1)

TheLink (130905) | more than 4 years ago | (#31483774)

> It is only to detect that there is some malware present,

No, he claimed in the article that:

> This tells us a few interesting things. We can guarantee detection of malware.
> And that includes zero-day attacks and rootkits. We can even guarantee that we will
> detect malware that infected a device before we installed our detection program.

To me _guaranteeing_ detection of malware (especially zero-day ) is similar to solving the halting problem (without having the source code and knowing all the possible inputs).

I am not a Computer Scientist but to me getting an external verifier to figure out "is this bunch of bits malware or not" is going to be as hard or harder than getting an external verifier to figure out "does this program halt or not" given the program and its inputs. Since with the former you do NOT know its full inputs - it could download new instructions from the Internet.

A better way to defend against malware is sandboxing. Sandboxing is _analogous_ to avoiding the requirement of solving the halting problem by actually setting a time limit on how long a program can run.

Re:Theory and hand-waving (1)

Lord Grey (463613) | more than 4 years ago | (#31483776)

Detecting the malware depends on the malware trying to stay in memory. My point was that "properly written" malware wouldn't necessarily care if it is was swapped. Allow the swap, get a clean bill of health from the "external verifier," then get reloaded and continue Bad Activities. Downtime for the malware is negligible.

Re:Theory and hand-waving (1)

ircmaxell (1117387) | more than 4 years ago | (#31483416)

Well, wouldn't this require ALL other processes to "sleep" while the check is performed? Sure, Ram is fast, but writing to 4gb of DDR3 would take around 1/3 of a second (excluding the time it took to generate that data and store the hash) considering the peak transfer rate of DDR3 is around 12800 MB/s (Using the best case)... So in reality, you're looking at well over 1/3 of a second (potentially into the seconds. And that's just for writing. You need to swap everything out first. So the whole process could take several seconds to complete. Now, if the computer is doing ANYTHING (GUI is active, servers are active), they'll either cause the memory to be paged back in (And hence be detected as malware), or (if this software blocks the paging attempt) stall waiting for it to page. So the computer would have a several second "pause" where it wouldn't react to anything (and possibly lose the inputs in that timespan, since memory can't be written to)... So that means this is useless on any kind of an active computer (Server, computer being used, computer with any kind of process that runs long term, etc)?

Finally (1)

spun (1352) | more than 4 years ago | (#31483672)

A valid criticism. And if the malware is actively resisting the scan, by moving the random bits back in from secondary storage before the hash, the external verifier knows about it because it takes even longer. By design. So, unless you are running a load balanced cluster and can afford to take a server offline for a few minutes when you want to scan, yes, this is a problem with this approach.

Re:Theory and hand-waving (1)

bill_mcgonigle (4333) | more than 4 years ago | (#31483498)

Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.

It may be worth doing right. Look for malware from a hypervisor (memory, disk, network, etc.). Running this all inside the insecure machine is just asking for trouble, though, but is the best currently available. But even today there are cpu's shipping without virt support, so this can't be done for every machine yet or for a while. Still, I think many would spend the extra $50 if it worked well.

Still a needle (4, Insightful)

dmgxmichael (1219692) | more than 4 years ago | (#31483200)

A needle in a haystack wants roughly the same amount of space as a straw - doesn't make it any easier to find (indeed, that's part of the reason it's so hard to find).

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

Re:Still a needle (1)

Pojut (1027544) | more than 4 years ago | (#31483242)

Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

As with most things in life, stupidity is the leading cause of problems.

Except death. I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

Re:Still a needle (4, Insightful)

mooingyak (720677) | more than 4 years ago | (#31483422)

I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

You're sleeping with your mother? Gross.

Which one is the detector? (4, Insightful)

mangu (126918) | more than 4 years ago | (#31483202)

How about a malware that masquerades as this detector and reports the RAM checksum is OK?

Re:Which one is the detector? (3, Insightful)

evanbd (210358) | more than 4 years ago | (#31483668)

Detecting a malware detector is just as hard as detecting malware. In general, detecting software of a specific type is halting-equivalent. In practice, the goal is to take shortcuts so that your adversary has a halting-equivalent problem and you don't. At present, the malware authors are winning. If we could force them to detect the malware detectors, that would be a huge advance.

My skepticism about this is the obvious one: what if the malware just lets itself get swapped out, and relies on stealth to survive the process?

Re:Which one is the detector? (0)

Anonymous Coward | more than 4 years ago | (#31483732)

Or malware that changes the total amount of RAM appearing to be available for the operating system? Good ol' boot sector virii did that. 20 years ago.

"Guarantee" (4, Insightful)

Reason58 (775044) | more than 4 years ago | (#31483256)

You can't guarantee anything in security. Especially when a human is involved.

Re:"Guarantee" (4, Funny)

dmgxmichael (1219692) | more than 4 years ago | (#31483316)

Reminds me of Murphy's 7th law - "Should you ever idiot proof anything God will make a better idiot."

Re:"Guarantee" (0)

Anonymous Coward | more than 4 years ago | (#31483590)

Never argue with an idiot.

They'll drag you down to their level, and then win on shear experience...

Re:"Guarantee" (1)

Hatta (162192) | more than 4 years ago | (#31483336)

I guarantee there's at least one thing that can be guaranteed.

Re:"Guarantee" (1)

Reason58 (775044) | more than 4 years ago | (#31483466)

I guarantee there's at least one thing that can be guaranteed.

You would be wrong about that.

Re:"Guarantee" (1)

dmgxmichael (1219692) | more than 4 years ago | (#31483580)

I guarantee you're going to die someday.

-- Not a death threat.

Re:"Guarantee" (1)

Hatta (162192) | more than 4 years ago | (#31483594)

Can you guarantee that?

Okay (1)

somersault (912633) | more than 4 years ago | (#31483274)

And what if the malware lets itself be swapped out of RAM the same as all of the other apps?

I'd love to have an approach to malware that could always detect unwanted processes, I'm just trying to find holes here.

Underestimating your enemy (1)

Mordocai (1353301) | more than 4 years ago | (#31483280)

Even if this did work in theory, someone would think of a way around it. We'll never be completely safe from malware, no matter what security mechanisms are in place. It's like in physical security, security system companies come up with new locks, and thieves come up with new lock breakers. Unless we brainwash the entire population of the world to be nice and not try to break systems, there will never be a conclusive way to detect malware. Oh, and wouldn't his method for detecting malware be horribly intrusive?

Re:Underestimating your enemy (0)

Anonymous Coward | more than 4 years ago | (#31483616)

I can already see a number of attacks on this: Something wanting timing? Return to it a value from the clock that subtracts the swapping to memory or disk. Or just cut off execution of the program altogether, put up a screen that looks exactly like the scan was successful, and then not worry.

I know of only one solid way to remove malware from a system, and that is to boot from a CD or USB flash drive, and this is assuming the BIOS, keyboard drivers, or anything else flashable isn't compromised.

False positives? (1)

Andy Dodd (701) | more than 4 years ago | (#31483284)

Yeah you can detect that SOMETHING is there, but how do you determine whether that something is supposed to be there or not?

If you assume all "somethings" are not supposed to be there, you'll have a worse situation than UAC with users being prompted all the time and getting conditioned to click "yes".

After reading the article, it seems no different from doing an offline scan using ClamAV from a LiveCD except maybe slightly more convenient. You boot a "secure" detection mechanism in place of whatever is normally operating on the machine.

The hooks needed to do it as described (which implies a hypervisor-esque antivirus/antimalware solution) would provide malware authors new vectors with which to attack systems, potentially vectors that would allow malware to escape from the likes of an "antivirus on LiveCD" solution.

Since I actually read the article (1)

Rogerborg (306625) | more than 4 years ago | (#31483302)

I note that he seems to have missed a rather obvious possibility: there's malware in RAM, but it allows itself to be swapped out with all the other processes. Why wouldn't it? If it got loaded into RAM once, it'll get loaded again by the same vector. In fact, it has to rely on that happening, since at some point the RAM is going to be physically powered down. There's no point in trying to dig in like a tick.

So as far as I can see, his magic technique will only catch malware that attempts to protect itself. I'm not clear on why he thinks this will catch all malware, but I'm sure he'll explain further if I pay him some $$$.

So what I'm getting... (1)

O('_')O_Bush (1162487) | more than 4 years ago | (#31483304)

is that he's saying, in a round-about way, that if we know what everything on the computer is that isn't Malware, then if it's not what we know, then it must be Malware.

That and some test for checking behavior.

The problem which he doesn't seem to resolve is, "How do we know everything that isn't malware?". I mean, I see that he goes on about running this using only in kernel mode, so there should only be kernel memory in RAM, but what if the malware exists hooked in (as many seem to be that I've found) in other programs or window'ing components that don't start up in kernel mode?

Maybe I'm missing something, but it seems that he can only "guarantee" detection of malware when it's already really obvious.

Ludicrous (no, not the rapper). (1)

clone53421 (1310749) | more than 4 years ago | (#31483308)

Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong.

Ah ha! All you need is a kernel-mode algorithm that knows exactly what should be in RAM, at all times! In other words, it has to emulate all of the legitimate software you’ll use, because otherwise how would it tell the difference between legitimate software’s use of RAM and malicious software’s use of RAM?

What an idiot.

won't work (1)

SkunkPussy (85271) | more than 4 years ago | (#31483310)

either
* the malware is in the kernel, in which case it can provide a false checksum of the memory to the external verifier
or
* the malware is in userspace in which case it gets swapped out, the verifier determines there is no malware in the system, then it gets swapped back in and carries on performing its malicious activities with its user privileges

Swapping out all RAM? (1)

grub (11606) | more than 4 years ago | (#31483328)


Swapping out all RAM then overwriting and hashing?

Sounds crazy? Yes? It's coming to you in OpenBSD 4.7! or not

Ok, so you have verified there's no malware in RAM (1)

characterZer0 (138196) | more than 4 years ago | (#31483330)

If the malware is /sbin/halt, you've still got a problem.

Common mistake... (1)

Rah'Dick (976472) | more than 4 years ago | (#31483348)

Everyone who claims to solve a long-standing problem "guaranteed", does not know all the possibilities that could thwart their solution. Guaranteed.

Easy (1, Flamebait)

camperdave (969942) | more than 4 years ago | (#31483354)

If $OS=="Windows" Then print "Malware Detected";

What I don't get. (1)

LWATCDR (28044) | more than 4 years ago | (#31483370)

Is why didn't Microsoft make the OS files read only way back when?
Make the user give explicit permission to over right system files?
It wouldn't make it impossible to get malware but it sure a shooting could make getting ride of it easier.

Re:What I don't get. (1)

characterZer0 (138196) | more than 4 years ago | (#31483596)

Time to market.

Hmm (0)

Anonymous Coward | more than 4 years ago | (#31483374)

I would love to meet the genius who discovered that any program that wants to be active in RAM uses some RAM...even if it is only 1 byte.

Re:Hmm (1)

clone53421 (1310749) | more than 4 years ago | (#31483396)

Yes; In fact, I’m going to walk out on shaky ice and claim that every program that wants to be active in RAM uses more than one byte!

Enjoy your swim... (0)

Anonymous Coward | more than 4 years ago | (#31483766)

Malware detection is Bogus. (4, Informative)

bmo (77928) | more than 4 years ago | (#31483380)

How about we change things in Windows so it actually prevents infection in the first place?

1. Educate users. Microsoft does a piss-poor job of this.
2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.
3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.
4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

Really. Seriously.

No, the above won't cover every situation, but it's a pretty good start.

--
BMO

Re:Malware detection is Bogus. (1)

bill_mcgonigle (4333) | more than 4 years ago | (#31483586)

No, the above won't cover every situation, but it's a pretty good start.

You say those as if Microsoft isn't aware of the problems with their design decisions. They by-and-large don't bear the costs of their poor security but would bear additional costs if people had to learn new ways of interacting with files (support costs, engineering, etc.).

So far they're right - their market position hasn't been adversely affected by the malware crapfest they've foisted onto their users. A cookie for he who figures out how to internalize what are currently their externalities.

register (4, Interesting)

bugs2squash (1132591) | more than 4 years ago | (#31483392)

Some processors may have big enough register sets that malware could reside entirely within the CPU.

Some amazingly bad assumptions (5, Insightful)

nahdude812 (88157) | more than 4 years ago | (#31483414)

Sure, malware has to occupy memory. That doesn't mean it has to be its own memory. Buffer overflows are all about corrupting another application's memory space.

His basic argument is that if you want to scan RAM, the kernel can halt all processing except its RAM scanner, and have a go at the RAM safely. If it's particularly insidious malware, it'll try to hide itself in various ways, one of which would be to masquerade the portion of RAM it was using with something legitimate looking (maybe erase that portion of memory). But you know it did this because you can see that memory which was supposed to be free is no longer free. Except the hardware has no concept of free or occupied memory. It just has memory, and the OS keeps track of what's free and not. The OS - the same space where malware is running.

OR, the malware could simply not do this, then its behavior is no different from any legitimate program. So how do you detect it now? You still need definitions that say, "When running in memory, this virus looks like X," then look through memory for that pattern.

Besides, who's to say that the kernel space is guaranteed free of malware itself? Even if you would have successfully identified the threat in RAM, you have no guarantee that the malware hasn't corrupted the identification routine.

It's like someone came along and said, "Hey, you guys are looking for malware wrong. You have to look for it! And I mean really look for it!"

Wow (1)

Dunbal (464142) | more than 4 years ago | (#31483552)

Someone has discovered the white-list.

Please take a number and stand behind the perpetual motion people. When I'm done with them, I will explain the few finite set of cases where this method DOES work, and you can assume that in the infinite number of OTHER cases, this method does NOT work.

Idea (1)

Superdarion (1286310) | more than 4 years ago | (#31483554)

Wouldn't it be posisble for malware to intententionally slow down many random bits of ram so that the detector would think that this is just latency?

I mean, malware could divert the checksum of his own memory bytes to the hard-drive and then divert many other bytes there too just to confuse the detector.

Its easy. (1)

Kenja (541830) | more than 4 years ago | (#31483582)

1) install malware
2) report that there is malware installed

Looks a lot like Pioneer from SOSP 2005 (1)

sseshan (258488) | more than 4 years ago | (#31483736)

This looks a lot like Pioneer:

Seshadri, Arvind, Mark Luk, Elaine Shi, Adrian Perrig, Leendert van Doorn, and Pradeep Khosla.
"Pioneer: Verifying Integrity and Guaranteeing Execution of Code on Legacy Platforms."
In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), Brighton, United Kingdom, October 2005.

http://sparrow.ece.cmu.edu/~adrian/projects/pioneer.pdf [cmu.edu]

Impossible By Definition (1)

davidshewitt (1552163) | more than 4 years ago | (#31483828)

It is impossible to guarantee the detection of malware by definition. New techniques for malware to hide itself will be developed when new detection techniques are created. This is the way of security.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?