Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Tuning Linux VM swapping

michael posted more than 10 years ago | from the got-swap? dept.

Linux 324

Lank writes "Kernel developers started discussing the pros and cons of swapping to disk on the Linux Kernel mailing list. KernelTrap has coverage of the story on their homepage. Andrew Morton comments, 'My point is that decreasing the tendency of the kernel to swap stuff out is wrong. You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful.' Personally, I just try to keep my memory usage below the physical memory in my machine, but I guess that's not always possible..."

cancel ×

324 comments

Sorry! There are no comments related to the filter you selected.

fp (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017796)

fp

asdf (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017798)

**eagerly awaiting s witty FP troll**

Re:asdf (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017822)

well, so far these first few FPs are all terrible. back to troll college, the lot of yuo

the obligatory... (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017809)

Woot first post! ...god I'm lame

Easy steps, 1, 2, 3 (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017811)

1. Remove disk
2. Throw out of window
3. Dance on grave of that guy down the road who died. You know, the old guy who used to steal your football when you were young? Yeah, him. Bastard.

Ob. /. joke (2, Funny)

Anonymous Coward | more than 10 years ago | (#9017813)

First swap!

God no... (5, Interesting)

0123456 (636235) | more than 10 years ago | (#9017815)

"You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful."

I absolutely despise the way that XP swaps out applications in order to make the disk cache larger. I have 1GB of RAM on my machine precisely so I don't have to wait two minutes for it to swap my web browser back in after it's swapped out... yet if I copy a 2GB file from one drive to another, the stupid operating system will swap out all the applications it can just to make the cache larger.

Please, please, don't take Linux down the same braindead route as Microsoft has done for XP. It's utterly insane to swap out my browser so that a 2GB file can be copied two seconds faster when I then have to wait two minutes for the browser to swap back in. Or at least provide some kind of '#define STOP_VM_SWAPPING_STUPIDITY' so that I can disable it.

Re:God no... (0)

Anonymous Coward | more than 10 years ago | (#9017857)

I hate to tell you this but it actually sort of does this already. It swaps out ig chuncks of data that it hasn't seen accessed in a while to increase cache size and IMHO this is a good thing. Why waste ram that could be used for live data?

Re:God no... (3, Interesting)

0123456 (636235) | more than 10 years ago | (#9017907)

"I hate to tell you this but it actually sort of does this already."

Maybe 2.4 does, but my 2.2 system has never suffered from this problem (haven't got around to upgrading it yet). Small amounts get swapped out, but nothing noticeable in interactive use.

"Why waste ram that could be used for live data?"

Because when I want to use my web browser again after playing a game for an hour, I don't want to have to sit there for two minutes watching it slowly swap back in... interactivity is far more important to me than small performance benefits from an extra 64MB of disk cache.

Re:God no... (2)

ninewands (105734) | more than 10 years ago | (#9017899)

I agree. My box has 768 megs of RAM specifically to minimize swapping. If BloatyApp occupies it all and I start up another app to do something else, I'm prepared to tolerate the swap delay, but if BloatyApp is all I'm using and I start a file copy in a shell, I'd rather get back to what I was doing in BloatyApp than have the copy finish an immeasurable fraction sooner.

Re:God no... (0)

Anonymous Coward | more than 10 years ago | (#9017903)

I hate that too. Why do I have to wait for my apps to swap back in just because the screen saver comes on when I go take a p break. Stupid design. NT was better.

Re:God no... (2, Interesting)

Anonymous Coward | more than 10 years ago | (#9017945)

Mozilla is the application I know of that has that demonstrates that behavior under XP so I'd suggest using another browser if it bothers you that much.

I use XP extensively and it is very agressive at swapping stuff out. However, I've never had the problems with other applications besides Mozilla.

Re:God no... (3, Interesting)

1000StonedMonkeys (593519) | more than 10 years ago | (#9018035)

Java also has the same problem. Almost makes you think they do it on purpose.

Re:God no... (5, Informative)

petabyte (238821) | more than 10 years ago | (#9017952)

Actually, you can change it on the fly with /proc/sys/vm/swappiness Increasing the number will increase the agressiveness of the swapout. Mr. Morton runs with his set at 100 (the max). 0, I believe would turn swap.

My kernel has autoswappiness enabled so it figures out the number on its own. I'm running at 64 ATM on a 256 Meg system (ram donations accepted) :).

Re:God no... (0, Offtopic)

MikeHunt69 (695265) | more than 10 years ago | (#9018305)

... and how exactly will that help the OP in XP?

Re:God no... (3, Informative)

kinema (630983) | more than 10 years ago | (#9017959)

All you need to do is: "echo 0 > /proc/sys/vm/swappiness" and the VM will do it's best to keep from swapping pages to disk.

Re:God no... (4, Interesting)

Shakrai (717556) | more than 10 years ago | (#9017980)

Please, please, don't take Linux down the same braindead route as Microsoft has done for XP. It's utterly insane to swap out my browser so that a 2GB file can be copied two seconds faster when I then have to wait two minutes for the browser to swap back in

Does it really make it faster anyway? Unless parts of that 2GB file were already in the cache then how is the cache going to make it transfer any faster?

As a side note I haven't noticed Linux swapping much out in favor of the cache. My home grown samba/sql/dhcp/nat/intranet server has 768 megs of memory. As of today (43 day uptime -- Linux 2.4.25) there is only 2,528k in SWAP. 8,444k of free memory, 191,952k used for buffers, 296,004 used for cache and the rest for applications.

I wouldn't mind seeing Linux swap out programs that aren't touched in several days/weeks (like the 12 agetty processes on my monitor less machine -- yes I know I could disable them if I wanted) but I definitely don't want to see it swapping out that browser I used 5 minutes ago in favor of increasing the disk cache size. Now if I launch Quake that's a different story.

As far as the other posts about rule of thumb for swap size go -- I stopped using the 1:1 or 2:1 ratio a long time ago. I have a 256meg swap partition on my 768meg Linux box. That's pretty much as big as I go with swap spaces. Are you seriously going to setup a 768 (or worse x2) swap space? A) You'll never use it, B) If you do use it your machine will barely be useable.

As far as XP's stupidity goes look under My Computer -> Properties -> Advanced -> Performance Settings -> Advanced and make sure both options (processor scheduling and memory usage) are set to "Programs" and not "background services" or "System cache". That may (or may not -- it is Windows after all) help you a little. On the flipside of the coin I discovered that I needed to reverse the memory option on my Windows 2000 Terminal Server to prevent stupid HP print drivers from sucking up 100% of the CPU and 90% of the physical memory.

Re:God no... (2, Informative)

Elm Tree (17570) | more than 10 years ago | (#9017995)

If I recall correctly it's runtime tuneable. So us power users with 1+ gig can tune swapping down, and desktop distros can tune it up. That's my favorite part about linux, I can just
cat 0 > /proc/sys/vm/swappiness
and I have instant control as to the performance of my machine. In fact... I could even write wrappers to specific programs so that they can tune the system's swappiness to better suit them. I.E. Programs that use huge ammounts of memory, less swappy, programs with repetetive disk access more swappy... Is swappy even a word?

Re:God no... (0)

Anonymous Coward | more than 10 years ago | (#9018020)

Set the file cahe to be a fixed size. It is not XP's fault. It is yours.

There are many legitamate things to gripe about MS. Your ignorance of the OS isn't one of them.

Re:God no... (1, Interesting)

Anonymous Coward | more than 10 years ago | (#9018123)

Have you ever really looked at the results when you do make the changes to the file cache. It does NOTHING! It must be a legacy setting (just one of thousands laying around the registry).

If you can prove to me otherwise I would be indebted to you forever, cause the file caching management is one of my biggest gripes about W2K/XP.

Re:God no... (1)

nikster (462799) | more than 10 years ago | (#9018170)

i hear you.

i have the same problem with Retrospec backup software. one run and XP swaps out everything, system, applications, Explorer, etc.., so i have to wait >5 minutes before i can use the system again. This is with 1 G of physical RAM installed.

the solution would be a limit to the disk cache to a reasonable size. i can see that servers would want all RAM for caching, but desktops? probably not. there should be a limiting percentage, like 10% of RAM. 100M is plenty of disk cache for my use...

be sure to check if system properties->advanced->performance options->advanced->memory usage is set to "Programs".

hmm... did i mention this is extremely LAME?

Re:God no... (1)

Monkelectric (546685) | more than 10 years ago | (#9018196)

That's odd because the NT swapping strategy shouldn't do that (unless they've changed it for XP?).

NT is supposed to maintain *small* disk caches to avoid the situation you're talking about, where as linux has always had a less conservative policy of using pretty much all avaliable ram for disk cache and pushing things out when needed.

I would actually be pretty surprised if that was the case... the os SHOULDN'T kick programs out for disk cache except under extreme situations. For all the shit we give MS their kernel design is usually pretty damn good. Next time you do something like that check your disk cache levels and see if thats whats actually whats going on. If you are concerned O&O soft makes software to manage disk cache, and I believe nt internals has a utility to alter some of the disk cache properties.

Re:God no... (1)

ByteSlicer (735276) | more than 10 years ago | (#9018242)

If you have 1GB of RAM, as I do, you can just disable the swap file. I'm running XP for a few months like that now, seems to work fine. Or at least set the minimum and maximum swap sizes to 100MB, to reduce that rediculous 1.5GB swap file (although XP can still resize it then if it runs out of RAM).

How fast is swapping really these days? (2)

jaf (121858) | more than 10 years ago | (#9017820)

Memory access vs. disk access I mean?

Back when P90s were the norm, was RAM access about as fast as disk access is today?

Re:How fast is swapping really these days? (5, Informative)

Moderation abuser (184013) | more than 10 years ago | (#9017891)

Well, disk access speed, say 5ms. RAM access speed 10ns so RAM is approx half a million times faster than disk.

Re:How fast is swapping really these days? (0)

Anonymous Coward | more than 10 years ago | (#9017982)

you are talking latency not bandwidth, there is a huge difference.

Re:How fast is swapping really these days? (4, Insightful)

julesh (229690) | more than 10 years ago | (#9018165)

True, but in most real applications the latency of the disk is the real cost of using the system -- you don't tend to do sustained transfers from swap to CPU. Most memory accesses use a small amount from a single page and then skip to using another. Modern programs tend to have very bad locality of reference.

Re:How fast is swapping really these days? (2, Informative)

drinkypoo (153816) | more than 10 years ago | (#9018295)

Hard drives have seek time and maximum bandwidth. Memory has latency and maximum bandwidth. PC100 SDRAM (for example) has about 10ns latency. As you say, hard drives have much longer seek times than SDRAM has latency; usually between 9 and 20 ms. Hard drives typically transfer between 10 and 30 MB/sec; PC100 SDRAM which is 64 bits (8 bytes) wide has 100 MHz x 8 Bytes = 800 MB/s transfer (peak theoretical.) According to SiSoftware Sandra 2004 Pro, a Via KT133 chipset machine with a fairly fast AMD processor develops 760MB/sec peak transfer. (For comparison, my Athlon XP 2500+ system with dual channel DDR333 and a bunch of crap running in the background cranks out 2309 MB/s on top of whatever memory access is already occurring. It uses an nForce2 chipset.)

Re:How fast is swapping really these days? (0)

Anonymous Coward | more than 10 years ago | (#9017989)

> Back when P90s were the norm, was RAM access about as fast as disk access is today?

No.

Any other [stupid (*)] question ?

(*) because reading from disk imposes to wait for a physical event (the head passing over the data), which can take *dozen*of*milliseconds* to read even one byte while a 1908's 4.77 MHz could already obviously blow that score away by several orders of magnitudes.

This reminds me of an old convo I had ... (5, Interesting)

DikSeaCup (767041) | more than 10 years ago | (#9017827)

I had this conversation with a fellow sysadmin, about the time that RAM was fairly cheap and we had a budget.

She had just procured a new Sun machine with 2 GB of RAM. Mind you, disk space hadn't grown all that significantly and you could still get machines with 9 GB drives.

The original practice was to make swap 2xRAM. So when the student she had putting the machine came to her and said, "What do I make swap?" she responded "Twice the RAM."

He said, "Are you sure? That's like almost half the boot drive."

She thought about it for a second and said, "Oh, yeah. I guess just make it the same as the RAM."

So this begs the questions: What do you make your swap now? When does your rule of thumb change? And remember when you could run a "fast" linux box on a P100 with 64MB of RAM and 128MB of swap?

Re:This reminds me of an old convo I had ... (0)

Anonymous Coward | more than 10 years ago | (#9017859)

What do you make your swap now?

10x the ram, actually, though on linux each swap partition can only be 2GByte, I think.

How much swap? (1)

milgr (726027) | more than 10 years ago | (#9017929)

About 2 years ago I discussed this issue with an OS guru. He was of the mindset that you should always have Swap space = 10xmemory.

I find that Linux just isn't that good at paging. I never use a significant portion of my 2GB swap partition, and memory contention is still high sometimes. Hmm... Maybe I do need to adjust the swapability number.

Re:This reminds me of an old convo I had ... (0, Offtopic)

Mwongozi (176765) | more than 10 years ago | (#9017950)

So this begs the questions

No it doesn't. [wsu.edu]

Re:This reminds me of an old convo I had ... (0)

Anonymous Coward | more than 10 years ago | (#9018088)

No it doesn't.

Yes it does. [alt-usage-english.org]

Re:This reminds me of an old convo I had ... (0)

Anonymous Coward | more than 10 years ago | (#9018290)

Yes it does

No it doesn't [shepherd.edu]

Re:This reminds me of an old convo I had ... (0, Offtopic)

tootlemonde (579170) | more than 10 years ago | (#9018314)

Yes it does.

Nope. Still doesn't

The article cited says: Many people unaware of the technical meaning of "to beg the question" in logic use it in one of two looser senses.

Stated in one of the two looser senses, this statement begs the question of why people who are unaware of the technical meaning would use an odd phrase like "beg the question".

In idiomatic English, one would say "raises the question" and avoid the implication of ignorance masquerading as erudition.

Re:This reminds me of an old convo I had ... (1)

B5_geek (638928) | more than 10 years ago | (#9017953)

I have 1.5Gb of ram in my main box, and I set my swap sixe to 2Gb on all my drives. (hard limit so it won't grow)

I then run a defrag program that moves the swap file to the inner tracks of the HD's.

Re:This reminds me of an old convo I had ... (1)

cexshun (770970) | more than 10 years ago | (#9017965)

My normal practice is more of solid values. If ram =512 then swap =128. I only have 512MB DDR in my linux box with 128MB of swap. The ONLY time I get into swap space is when I run winex games. And my swap has still never gone above 60% full. Although with the new info from this article, I'm going to mess with my swappiness file and see what benefit tuning can give me. Although just having units from 0 to 100 are kinda vague values. I think the 2x ram rule is completely outdated.

Paging vs Swapping (1)

Jon_E (148226) | more than 10 years ago | (#9018083)

We do understand that paging is different than swapping, and that Solaris has changed the memory allocators and algorithms multiple times across releases right?

That said, you might want to look into a recent Solaris Internals book or course, and also look into the history of things like priority_paging and page coloring ..

Re:This reminds me of an old convo I had ... (1)

troon (724114) | more than 10 years ago | (#9018103)

And remember when you could run a "fast" linux box on a P100 with 64MB of RAM and 128MB of swap?

Hey - that's my web server [plus.com] you're talking about! Admittedly it has a gargantuan 96MB now, albeit with only 80MB of (RAID-1) swap, but it also does lots of other jobs.

Re:This reminds me of an old convo I had ... (4, Interesting)

drinkypoo (153816) | more than 10 years ago | (#9018158)

The reason the original practice in Sun shops was to have swap be twice the RAM is that SunOS4 swaps real ram to swap space on a 1:1 basis such that the first n bytes (where n is the number of bytes of physical memory) correspond to the first n bytes of the swap file.

When SunOS5 rolled around, this was no longer necessary, and your swap is additive, so you only need as much swap as, well, you actually need.

On my linux firewall system with 256MB real RAM, I have 512MB swap space. On my Windows system with 1GB real RAM, I have 768MB of swap space. This number is actually a hold-over from when I only had 512MB of RAM, I could probably decrease it to just about nothing now.

Amusingly enough my system has ~480MB of real RAM free, and is using 701MB of my paging file. Go windows! Like I need 480MB free all the time. Still, it is nice not to have to swap something out if I start a big application - but Windows is awful about returning from swap.

Some other more or less useless data points: My Indy (running gentoo) with 128MB has 256MB swap, which has been enough. I probably could have gotten away with 128MB but believe it or not my primary concern is whether I'll be able to compile some of the biggest C++ programs without the larger amount of swap. Certainly 128MB will not do it, even when you are booted from the gentoo installer CD and there's nothing much running.

Re:This reminds me of an old convo I had ... (5, Interesting)

13Echo (209846) | more than 10 years ago | (#9018171)

I don't normally make my swaps more than 512 MB on my Linux machines. In fact, when I had 1204 MB of RAM on my last machine, it only ever touched the swap once (when I was compiling Mozilla). The machine was so responsive with 1024 MB of RAM, it virtually never needed to use the swap with that much RAM.

Now that I have a newer machine, and RAM prices have increased (had to replace SDRAM with DDR), I only have 512 MB in my home machine. It seems to be nearly as responsive, practically never needing to touch the swap. I've only ever seen it use a few MB of the swapfile. When partitioning my Linux drives, I almost always have more than one drive in the machine. HDA1 normally gets the root partition. HDB1 is normally my swap, at the front 512 MB of the drive, followed by home on HDB2. This system makes everything snappy.

Even on my work machine, which is only a p3 450 with 256 MB of RAM, things operate quite well under Gnome 2. I have two drives in that machine as well, and the swap is on a seperate drive from the root partition. Programs can load from one drive while simultaneously swapping (if necessary) to a second drive. Even with Gnome 2 running, in addition to my browser and several other apps, only a few KB of space is being used on the swap.

I can't see most desktop Linux users needing more than 512 MB of swapfile space, assuming that they have at least 256 MB of RAM. The general rule of thumb, though, is to put the swap partition at the front of the drive for the best performance, in the event that it does need to get used.

I've really been impressed with Linux's memory management, even in the 2.2/2.4 series kernels. I've heard that 2.6 even makes some improvements as well. When I used Windows 2000, on the other hand, it INSISTED on using the swap even with a gig of RAM, even after I tweaked it for the best performance. I even used a RAID0 array, and Linux is still faster and more efficient at managing memory WITHOUT the RAID array. I was surprised that the array wasn't even really needed on Linux for fantastic disk access speeds with my 3 year old 7200 RPM drives.

Of course, the rules will be different for server application. More swap is probably a necessary thing. It's possible, however, that users of Linux (on the desktop) may not even need a swapfile with more than 512 MB of RAM.

Re:This reminds me of an old convo I had ... (0)

Anonymous Coward | more than 10 years ago | (#9018234)

My thumb rule is:

create a swap partition which can hold at least the systems memory + some kernel space

simply spoken: If the system ever crashes you can be sure there is enough swap to create a core file including the whole system memory's content.
My last Ultra's never crashed...

Problem (4, Interesting)

FreeLinux (555387) | more than 10 years ago | (#9017836)

Personally, I just try to keep my memory usage below the physical memory in my machine, but I guess that's not always possible..."

No it isn't possible. With today's RAM prices I almost always have more physical RAM than the system requires. But, due to aggressive VM swapping there are still hundreds of megs swapped out to disk when there is no need at all. This means that those applications, when their time does finally come, are slow because they must be retrieved from disk first. It's really annoying sometimes. Yet, even with excess RAM turning off swap is disasterous.

Re:Problem (5, Interesting)

Trifthen (40989) | more than 10 years ago | (#9017935)

No, turning off swap is not disastrous. We've turned it off on our production web server cluster that routinely serves 60Mb sustained traffic. We've turned it off because we have 2GB of ram in these machines, and Linux insisted on preferring buffers and cache over our running applications. Fuck that, we said. With over 1GB Of buffers and cache, we had RAM to spare; bye-bye swap.

Re:Problem (0)

Anonymous Coward | more than 10 years ago | (#9017988)

I run WinXP (Home) on a laptop with 768MB or
RAM, and Swap turned off. I haven't noticed
any problems at all, except that some system
monitoring utilities don't run properly.
(they just die upon trying to start up).

FYI: typical list of simultaneously running
programs: Wordperfect 10, Firefox, explorer,
Acrobat reader, Winamp, Xearth. (and sometimes
a long Winrar job, or DivX player with a
movie I watch when I take a break)

Re:Problem (0)

Anonymous Coward | more than 10 years ago | (#9018217)

I'm really wondering what kind of movie you can watch on a break?

Re:Problem (1)

mcx101 (724235) | more than 10 years ago | (#9018127)

Yet, even with excess RAM turning off swap is disasterous.

I find that swap partitions in Linux and FreeBSD are just a nuisance once you've got enough RAM for your apps. Swap files are preferable because you can change the size and number of the files after installation. Swap partitions are just wasting valuable space on your HDD.

I have 1Gb of RAM on my laptop and Linux, FreeBSD, Windows 98 SE and Windows XP all run fine without any swap partitions or files on my quadruple boot.

The virtual memory algorithms in Linux, *BSD and NT all prefer you to have 1.5 times the virtual memory as you have physical memory. When you have 1Gb of RAM that's a hell of a lot of space to waste, especially in a multiboot situation.

At any rate it has little affect on the total memory available in practice to Windows 98. I used to play around a lot with DJGPP [delorie.com] which is a port of gcc to DOS DPMI (DOS Protected Mode Interface). I worked out that the maximum theoretical amount of memory you could have with such a DPMI memory manager was 512Mb, which some people had reportedly achieved using OS/2's DOS emulator. So I used to think Microsoft were pretty smart to be able to support 1Gb on my laptop on Windows 98 until I saw the BSODs with the "Out of Memory" exceptions when wordpad was running!

Re:Problem (1)

drinkypoo (153816) | more than 10 years ago | (#9018211)

Turning off swap is only disastrous on windows (and older versions of Unix) which last I checked (I have not checked very recently) would allow you to entirely turn off the paging files, but will not boot if they are not present, whether you have enough memory to contain everything or not.

Re:Problem (1)

gr8_phk (621180) | more than 10 years ago | (#9018219)

I thought the point was to swap things out when more RAM was needed than available. If big-app is using all the memory and I start something else, big-app goes to disk. Why swap if not needed? That would be like windows programs that load at startup - everything loads at startup so it will be faster when you want to use it, but that causes my startup time to be like 2 minutes...

Don't swap until it's necessary seems the right thing to do. If IO isn't busy, you could send older data to disk, but you'd need a way to mark the RAM AND DISK both as valid until the RAM changes. If you need more RAM then just invalidate the RAM copy. Does the MMU support this? Sounds like a patentable idea to me....except that I just posted it to /.

Andrew Morton's patches reflect this (1)

bangular (736791) | more than 10 years ago | (#9018224)

You probably want to try Andrew Morton's kernel patches. I've got some servers running his kernel patches and it's nearly impossible to make the damn things swap. Machines with uptimes of 4 months and more, and top shows swap file usage at 0 kilobytes. His kernel patches generally give quite a performence gain. I don't know how they would react under an environment of older machines though.

Memory (1)

nuggz (69912) | more than 10 years ago | (#9017839)

Personally, I just try to keep my memory usage below the physical memory in my machine, but I guess that's not always possible..."

No, it isn't really. Unless you don't use your computer.
In some cases it makes sense to use your physical memory as disk cache rather than for unused applications.
Swap out that sshd, and give the database server more memory. Swap out that screensaver and email client, give quake more.

2Gb of RAM, 300Mb of apps running. (1)

Moderation abuser (184013) | more than 10 years ago | (#9017845)

So, what... I want my apps paged out to disk so that I can wait for them to be loaded back in when I switch over from Mozilla to Open Office?

The big issue (4, Interesting)

MrIrwin (761231) | more than 10 years ago | (#9017858)

The main cuase of memory usage on Linux is the use of many different shared libraries and not bloated apps.

I think developers could do more at a library level. For example.....dare I suggest using common sub libraries within libraries, that is people like KDE and GTK get thier heads together and say "are thier functions we include in our libraies that could just as well be linked to an underlying library?"

Re:The big issue (2, Informative)

turgid (580780) | more than 10 years ago | (#9018075)

dare I suggest using common sub libraries within libraries, that is people like KDE and GTK get thier heads together and say "are thier functions we include in our libraies that could just as well be linked to an underlying library?"

Well, you see, KDE is written in C++. GTK is C. C++ stuff does not play well with different version of the same compiler let alone different compilers or even different languages.

In theory you're only "supposed" to use either GNOME or KDE and therefore only have one set of libraries installed. In practice you often want to use applications from both systems, and end up installing them both.
The problem is political and religious as well as technical.

Re:The big issue (3, Informative)

alex_tibbles (754541) | more than 10 years ago | (#9018082)

In fact, one big problem is the way that the loader performs relocations on C++ libraries. Google it. It's why KDE apps take a few seconds to load (and used to be even worse). IIRC the main problem is that many objects (function, variable etc.) need to be copied into the address space of each application using it, so that the sharing never happens in practise.

All lies (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017860)

Your government is lying to you! [bbc.co.uk] They are lying to you and then infringing upon the 1st Amendment rights of the journalists at CBS. These "soldiers" are just as bad as the men your government told you they were trying to get rid of when they lied to you about Iraq. Now they're lying again, and American "soldiers" are abusing and torturing men in Iraq, and they're doing it in YOUR NAME.

Why arn't you asking your Government why it has lied to you and continues to lie and attempts to pervert the truth?

Re:All lies (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017917)

Because the rank and file American is apathetic towards anything outside the United State's immediate borders?

Re:All lies (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9017920)

Now they're lying again, and American "soldiers" are abusing and torturing men in Iraq, and they're doing it in YOUR NAME.


Which is much better than them being here doing it to us, so I say let them go to Iraq and beat the living shit out of the Iraqis. I for one welcome our new Iraqi beating overlords.....

But on a more serious note, you must be pretty lame to be anonymously preaching to the slashdot crowd. Really.

Re:All lies (-1, Flamebait)

Anonymous Coward | more than 10 years ago | (#9017987)

But on a more serious note, you must be pretty lame to be anonymously preaching to the slashdot crowd. Really.

The ad-homimen attack, the last refuge of the idiotic and wrong. You know I'm right and American troops are acting like evil bastards whilst your Government is lying about it, so why not just admit it? I'm sure it's painful to realise that all the bullshit fed to you about America being the greatest country on Earth, home of the fair and just was all lies, but admitting it is the first step towards healing.

Re:All lies (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9018027)

We don't have to ask, you assclown. The military itself is investigating the actions of these individuals. Why aren't you paying attention?

It doesn't work that way (4, Informative)

Pharmboy (216950) | more than 10 years ago | (#9017861)

Personally, I just try to keep my memory usage below the physical memory in my machine, but I guess that's not always possible..."

I keep my memory usage much below the total ram on the servers, but in real life, the machine still swaps. This is because even tho the machine NEVER needs more ram than is available at any given time, over a period of days, it will use more than the available ram. It caches out the old data that was used 12 hours ago.

Unless you reboot every day (as in a client machine) you will use swap on just about any machine. Using swap is not bad. Using swap for a currently running application is not so good. This isn't a bug, its a feature. Reading data from swap after it has been accessed is still faster than reading new data from the drives, especially if its a network drive.

Re:It doesn't work that way (1)

endx7 (706884) | more than 10 years ago | (#9018072)

Using swap is not bad.

No it isn't, but constantly swapping a lot things in and out is, and you'll notice a considerable slow down of your machine.

And that's when you need to consider buying more ram.

What's wrong with many resident pages? (4, Informative)

YetAnotherName (168064) | more than 10 years ago | (#9017862)

You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine...

Why not? BloatyApp, if it's that bloaty is probably an object oriented program with template instantiation (or is by Micro$oft); these programs are notoriously huge, but also have notoriously poor locality of reference. The user will get better perceived response if you can keep more of BloatyApp resident.

If there's space in memory, I don't see the point of pre-emptively ejecting as many LRU pages of BloatyApp. (Of course, I haven't RTFA, but this is /. so you're not supposed to!)

Bloaty apps? Are you kidding me? (4, Insightful)

Trifthen (40989) | more than 10 years ago | (#9017866)

Ah yes. It's all the fault of bloaty apps. Apps like database daemons and high-traffic httpd daemons. We've turned swapping off on our servers because we were sick of seeing almost a GB of cache/buffer memory, while it was swapping 500MB of shit to disk. Want a bloaty app? How about the linux Kernel? I love the thing, but Jesus Tapdancing Christ it would rather swap our starting DB process to disk, than free up the fucking buffers and cache. Is there something wrong with wanting it to give precedence to not swapping?

Re:Bloaty apps? Are you kidding me? (2, Insightful)

Anonymous Coward | more than 10 years ago | (#9017898)

Yeah there is nothing I love more than coming to an idle X console session on box I haven't touched in a while and watching it grind itself into oblivion because everything has been paged out.

Re:Bloaty apps? Are you kidding me? (1)

Moderation abuser (184013) | more than 10 years ago | (#9018050)

You could always just tune the cache down to bugger all. It's one of the kernel parameters.

Re:Bloaty apps? Are you kidding me? (1)

Trifthen (40989) | more than 10 years ago | (#9018070)

Which kernel and what parameter? We've tried sysctl settings up the wazzu, but nothing seemed to actually change how it handled swapping.

Re:Bloaty apps? Are you kidding me? (1, Insightful)

Anonymous Coward | more than 10 years ago | (#9018094)

/proc/sys/vm/swappiness

Swapless since 1997 (3, Insightful)

redelm (54142) | more than 10 years ago | (#9017893)

Ever since I've had a 32 MB machine (1997), I've not bothered to even set up a swap partition. On the rare occasions when I need swap, I'll create a swapfile. Sure it's slower, but swap is already hugely slow.

With read-only & demand code-page loading and copy-on-write even bloatware really doesn't eat memory. And bloatware has to be frequently restarted to recover the memory it leaks.

Sure, there are some jobs that needs swap -- lots of seldom used memory pages.

But not mine. I prefer to save myself the complexity and performance headaches.

VM you say? (2, Insightful)

freeze128 (544774) | more than 10 years ago | (#9017902)

At what point does VM stop meaning Virtual Machine and start meaning Virtual Memory.

Or is it just the Virtual "M"?

Re:VM you say? (1)

jepaton (662235) | more than 10 years ago | (#9018112)

Computing wouldn't be the same without multiple definitions of the same TLAs (Two Letter Acronym).

Re:VM you say? (1)

julesh (229690) | more than 10 years ago | (#9018122)

VM meant virtual memory a long time before it meant virtual machine.

Its the change in meaning of UML that I can't get my head around these days...

swapoff /dev/hda2 (1)

datastalker (775227) | more than 10 years ago | (#9017912)

Unless I'm using something that is very memory intensive (ie, VMware), I just turn my swap partition off. It allows me the freedom to run Evolution/XMMS/etc. quite quickly when I'm not also using VMware, and still run VMware at a good speed just by turning the swap partition back on when I need it.

they missed one of the biggest points! (1)

whateverdude (205077) | more than 10 years ago | (#9017916)

what about disk corruption? you swap your memory
to disk, its gets mis-written, and you read it back in and now your memory is corrupted!

Re:they missed one of the biggest points! (2, Insightful)

cloudmaster (10662) | more than 10 years ago | (#9017968)

Disk corruption is handled seperately. If you don't trust your drive controller, then you should spend some time fixing up the disk writing susbsytem. Disk writes should be guaranteed if they say they're succesful, except in the case of a power failure. At that point, swap integrity doesn't matter anyway. :)

Re:they missed one of the biggest points! (0)

Anonymous Coward | more than 10 years ago | (#9017999)

Well, i have no memory of that happ(%$nning ...

Re:they missed one of the biggest points! (1)

jdunn14 (455930) | more than 10 years ago | (#9018006)

If you have *any* problems with a swap partition corrupting your data it is time for you to replace that disk. Besides one could easily check a checksum on the re-read data. And before anyone panics about the time it takes to checksum the data, you're pulling it off the disk. That's half a million times slower than memory? One more pass over the data in memory is not going to have such a horrible impact. Still, I'd personally say toss the dieing disk. They're cheap now.

Re:they missed one of the biggest points! (1)

42forty-two42 (532340) | more than 10 years ago | (#9018135)

Don't disk drives do a CRC in hardware automatically? I thought that was how bad sectors were detected and reallocated, by how many reads were needed to get a good CRC.

Re:they missed one of the biggest points! (1)

julesh (229690) | more than 10 years ago | (#9018222)

That's half a million times slower than memory?

That half million times is only true if you're accessing a single word from the page. It'll come down to about (500,000 / (PAGE_SIZE/MEM_ACCESS_SIZE)), or about 1000 for IA32 processors [*1], if you're accessing the entire page, as you'd need to do to perform a checksum. Still, it is just a drop in the ocean...

*1 - using 4K pages and 64 bit memory accesses. What size page does Linux use for swapping? larger pages would be more efficient...

Re:they missed one of the biggest points! (0)

Anonymous Coward | more than 10 years ago | (#9018279)

That's why in a production enviroment you mirror your swap space.

Other reasons (5, Interesting)

Halo1 (136547) | more than 10 years ago | (#9017948)

Another reason to gradually and pro-actively swap things out, is that when another program later needs a lot of memory, your system doesn't come to a grinding halt because suddenly a lot of stuff has to be swapped out at once (followed by zeroing all that memory, since you don't want to have one program leaking data to another).

At least, that's the rationale I've read behind OS X's strategy of swapping things out long before all physical memory is used (and of keeping a pool of zeroed memory pages ready to fulfill most requests). Note that this does not require superfluous swap-ins if your reuse strategy is balanced properly, as the fact that something is swapped out doesn't mean that the memory which contained that data will be cleared/reused immediately (i.e., if it's needed again shortly afterwards, that page can be reactivated without having to go to disk).

Under most desktop OS'es, programs can even give some hints to the system regarding their usage of a memory region using e.g. the madvise() system call.

My vote.... (4, Informative)

tsmithnj (738472) | more than 10 years ago | (#9018081)

is to do something like AIX does, where I can use "vmtune" to customize the percentages of memory I devote (hard or soft limit) to filesystem pages or computational pages. This way I can tune for my Bloatware, tune for file copying a la XP, or tune for my DBMS, whatever suits me.... The developers could take it one step further and provide a simple, understandable (as opposed to AIX's) interface for configuration......

La, La, La (0)

malia8888 (646496) | more than 10 years ago | (#9018014)

Andrew Morton humorously replied, "I'm gonna stick my fingers in my ears and sing 'la la la' until people tell me 'I set swappiness to zero and it didn't do what I wanted it to do'."

Gotta love this guy. In articles people usually try to impress you with their brilliance. This man is a kick in the pants! I love Andrew Morton's style.

Windows already has this (2, Funny)

novakane007 (154885) | more than 10 years ago | (#9018026)

Under the performance tab you can use the slider to tune the machine for 'foreground' or 'background' apps.

RTFA - swappiness is tweakable (1)

42forty-two42 (532340) | more than 10 years ago | (#9018030)

echo 0 > /proc/sys/vm/swappiness

Re:RTFA - swappiness is tweakable (1)

whateverdude (205077) | more than 10 years ago | (#9018051)

isn't that feature only in 2.6 though?

Re:RTFA - swappiness is tweakable (1)

42forty-two42 (532340) | more than 10 years ago | (#9018167)

I don't know, but 2.6 has been stable enough for me that I don't care :)

Return of the Sticky Bit (2, Interesting)

Anonymous Coward | more than 10 years ago | (#9018039)

In the good old days, "chmod +t prog" told the kernel to leave prog in the swap partition even after it had exited. It was a way of making humongous programs like vi :-) more responsive on startup.

In modern Unices (including Linux) last I heard, the sticky bit is ignored since everything is simply demand paged.

Could not sticky bit be revived with some similar meaning? As in, "don't be too keen on paging these out?"

Situational advice? (0)

Anonymous Coward | more than 10 years ago | (#9018049)

So, for standard desktop usage (i.e. XMMS + Mozilla + Vim in KDE), will most users with normal systems (say, 256mb RAM) experience better performance with a swap drive, or with swap deactivated, as some posters here advocate?

MS Office "OSA" anyone? (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9018064)

"You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful."

What? But in MS Office, they provide this nifty tool called "Office Startup Assistance" to increase the performance of Office. They even install it by default. Isn't that helpful? :-)

Please set up a proper default (0, Redundant)

dorfsmay (566262) | more than 10 years ago | (#9018095)

Unlike say latest versions of AIX where the OS gives higher priority to the cache than to apps, and you have to go deep in the tuning manual to read that the default probably doesn't suit many situation, please adjust maxperm etc...

The amount doesn't matters, it's the stickinness (3, Interesting)

buserror (115301) | more than 10 years ago | (#9018131)

I dont mind the kernel swapping out "old" stuff to grow a huge disk cache. Really, thats OK, it makes things faster for disk hungry processes allright.

However, what I mind is the fact that the pages that are swapped out STAY there!
Why not aging the disk cache the same way the RAM pages are aged ? On an idle machine, the disk cache would gradualy decay and be replaced by the pages back from the swap, and the machine would be all responsive again.

It means that if the user leaves for lunch and a cron wants to eat all the disk, with some luck, when the user gets back, his machine is as responsive as it was when he left.

I have a laptop with 192Mb of ram, I always hate when 2/3 of the ram is "free" while it takes 10 seconds for the kmail window to move to the front. Even if the machine has been idle for hours.

I even regularly do a "swapoff -a;swapon" to claim back the cache!

The kernel's page cache is the key... (5, Insightful)

aksansai (56788) | more than 10 years ago | (#9018142)

Many people don't realize how smart modern page caches are designed to speed their system. Linux, MacOS X, Win2K+, etc. all boast aggressive page caches that make loading applications from disk more efficient.

Without a swap file, the kernel has no place to stick memory segments that are rarely used. They stay in resident memory la-la land until the process is terminated. Those segments add up over time and erode the memory available to the page cache.

Page caches are wonderful. When you load an application (like Firefox [mozilla.org] ), you're not just getting the web browser. You're firing up a large chain of shared objects/DLLs that support the widgets, I/O, and components of the application. All of these components must be read into memory anyhow for program operation, so the kernel tends to just leave it in there for future use (the page cache).

When you shutdown Firefox, you're also releasing the necessity of those libraries (provided nothing else is using them). Those libraries also remove themselves from memory. If you load another application (like Thunderbird [mozilla.org] ) that uses the same type of libraries, the kernel will not have to go to disk in order to fetch those libraries. It will instead opt for the page cache contents.

Turning off the swap file in the historic era of VM infancy was the best way to remove the hard drive bottleneck from system. The operating systems of yester-year did not have good page cache schemes that took advantage of all that unused memory. It is a little different now.

Applications are so modularized that they are broken up into a billions of smaller libraries so that code can be shared. This increases memory efficiency by keeping a shared library resident for multiple processes. These libraries are frequently accessed, more often than many people realized. Getting THOSE into memory is better than making sure my 500+ Linux applications stay resident.
$ cat /proc/meminfo
total: used: free: shared: buffers: cached:
Mem: 1055653888 1036296192 19357696 0 70488064 892309504
Swap: 542367744 235892736 306475008
Notice that on a web server with 1GB of RAM the Linux kernel is still putting things out to swap. These processes that stay asleep for long periods of time do not need to waste the memory that page cache is currently using (892309504 bytes or 753.7MB). What would be stored in that 753.7MB of memory? The database that drives the website (instead of having to seek the disk). The entire web page hierarchy used to display pages on the web site. All the scripts that are used to display dynamic content on the web site (etc. etc.)

Now, if we subtracted from the page cache the amount of memory that was stored in the swap file, we would have over 200MB less that we could keep cached in memory. That could be an entire database that the kernel would then waste needless CPU cycles to fetch from disk.

The only advantage to turning off a swap file on these modern machines would be for a machine that runs only a select few applications, and not having a lot of processes in the background doing things.

What algorithm are they going to use (3, Interesting)

dorfsmay (566262) | more than 10 years ago | (#9018143)

Hopefully they'll use something modern like ARC [python.org] that tries to keep in cache stuff that have been read at least twice, NOT LRU !!

AIX uses LRU today, so when you do a backup, the system tries to keep all filesystems in cache (well that what was read last !!), and will happily swap your apps out to disk in order to do so (with default tuning parameter).

I fondly remember the days when I was running Linux with no swap, none whatsoever...

Dumb Swapping is Computer Abuse (5, Interesting)

stuffduff (681819) | more than 10 years ago | (#9018153)

Programmers have put a lot of time and effort into the VM swapping algorithm; mostly with the intention of being prepared to have a lot of memory ready and waiting for the next thing it will be asked to do. Unfortunately that's not so much of an issue with cheap ram and disk storage and faster and faster front side buses. What we really need is more intelligent swapping, which can only come about when the VM gets a set of API hooks (would make for a great 'shared object') that would enable the system administrator (and maybe someday the end user) to assist an intelligent VM manager to establish priorities and consistently respect those priorities.

Unfortunately the current crop of best guess VM managers end up denying the end user the experience of their computer's peak performance. Coupled with the horrible state of application bloat, modern 'state of the art' hardware and software combine to give us less and less in terms of overall performance. Software developers throw more code at the cpu to add functionality with little or no concern for performance. And hardware manufacturers add more and more 'special instructions' and 'pipelining' which the majority of software is completely unable to access. If anything it's more like a bunch of dysfunctional co-dependents than an industry that is cogent as to what really needs to be going on. If the folks dealing with processors and the application software could take a page from the gamers (look at the high levels of integration between game engines and video cards) for example, and more effort put into consolidating functionality in dlls and shared libraries; we would be amazed at how truly fast these machines could perform.

Copy vs Swap? (1)

SpinyNorman (33776) | more than 10 years ago | (#9018182)

I don't know if Linux does this at all, but it seems that one useful VM strategy would be to copy to disk rather than swap to disk. That way you can continue to run bloaty app without swapping it back in, but interactivity is still good since you can resuse memory immediately without needing to swap stuff out at the point the demand occurs. Of course it'd need to be tunable and/or smart (no point copying highly volatile areas of memory for a start).

Re:Copy vs Swap? (1)

xyzzy_jp (710764) | more than 10 years ago | (#9018264)

MVS does this (At least when I last played with it pre-1996). It is called logical swap. Swap out address spaces in the background and only swap them back if there page frames were stolen.

Not amused (4, Interesting)

MrLaminar (774857) | more than 10 years ago | (#9018195)

Actually, I haven't been very impressed by the whole swapping thing under Linux lately. I'm running 2.4.22 with a 400MB swapfile.

Some apps _can_ make the system unresponsive enough to ignore keystrokes, which is *very* annoying. At other times, xmms will stop playing while the disk goes crazy... Switching from emacs to Firefox after 10 minutes usually takes an extra 5 seconds to redraw the window and load all the stuff again.

Running GNOME2 on this laptop is also quite noisy on the disk. It swaps all the time...

Swapping on servers vs. Desktops... (2, Interesting)

Jonny Royale (62364) | more than 10 years ago | (#9018232)

'My point is that decreasing the tendency of the kernel to swap stuff out is wrong. You really don't want hundreds of megabytes of BloatyApp's untouched memory floating about in the machine. Get it out on the disk, use the memory for something useful.

This point is useful, but only if free RAM is at a premium. For the most part, on servers, there will be sufficient RAM to support the on board applications, and the amount of free RAM remaining will be able to handle the variable load of a standard workday, if the server has been sized properly ahead of time.

On desktops, however, where the number and type of apps can vary much more widely, the need for free RAM is much higher. However, pushing all apps off to swap space as a default to keep as much RAM as possible free isn't necessarily an effective solution, since the OS will spend much more time performing swap operations that it necessarily should.

Perhaps a better solution might be an application where certian portions could be swapped off earlier, as they are less used, or not even loaded at all, while maintaining the core of the application in RAM, with "hooks" to the swapped out areas, to let the OS know that swap procedures are required. If the app could tell the os "I probably won't need functions c.d & e, so go ahead and put them in the swap file", it might lead to a good balance of swapping and performance.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>