Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Linux 2.4 VM Documentation

michael posted more than 11 years ago | from the virtual-documentation dept.

Linux 115

popoutman writes "Mel Gorman has announced the availability of a guide to the 2.4 kernel VM including a walkthrough of the VM code. Anyone interested in obtaining a solid understanding of the Linux 2.4 VM will certainly want to take a look at this documentation. Mel says that the effort is at least several weeks from being finished, but that he's releasing it now with the hopes of getting feedback to be sure he's on the right track. He also notes that the 2.5 VM is still too much of a moving target for him to document it just yet." See also a Kerneltrap story.

cancel ×

115 comments

Sorry! There are no comments related to the filter you selected.

FP! (-1)

Anonymous Coward | more than 11 years ago | (#5088646)

F1R$t p0$t

FP 8 MINUTES AFTER THE STORY POSTS. EAT IT, /. (-1)

Subject Line Troll (581198) | more than 11 years ago | (#5088664)


Here's a tip for the author. (2, Insightful)

gpinzone (531794) | more than 11 years ago | (#5088706)

Somewhere in your document, you ought to explain what "VM" stands for. That goes for the poster as well.

Re:Here's a tip for the author. (0)

Anonymous Coward | more than 11 years ago | (#5088776)

See the title

Code Commentary On The
Linux Virtual Memory Manager
Mel Gorman

VM = Virtual Memory, I guess

Re:Here's a tip for the author. (2)

gpinzone (531794) | more than 11 years ago | (#5089144)

Then it should be called the Linux VMM, no? I can guess at a possible meaning, too...but I shouldn't have to.

Re:Here's a tip for the author. (2, Interesting)

tprox (621523) | more than 11 years ago | (#5088781)

It stands for Virtual Machine, though I know what you mean about explaining acronyms.

Re:Here's a tip for the author. (1)

stratjakt (596332) | more than 11 years ago | (#5088850)

I skimmed the article, and it seems to be about the Virtual Memory Manager's implementation in the kernel.

I assumed it was about Virtual Machines in user land as well.

Oh well, the fact that it's an incomplete document on something that noone but kernel hackers care about is irrelevent. It's only on /. because of the word 'Linux'.

Re:Here's a tip for the author. (3, Informative)

iggymanz (596061) | more than 11 years ago | (#5088856)

heh, it can stand for a few things in the realm of computers:
1. virtual memory (using secondary storage to allow the program to address more memory locations than there is in physical RAM)
2. virtual machine (a machine implemented in software on a real machine, which *could* have different opcodes, or the same ones as the real machine)
3. VM - the operating system by IBM which allows one of their mainframes to act like many (stands for virtual machine)

( Also, there's the VM in VMS, the DEC Vax and Alpha operating system, which stands for VIrtual Memory as in #1)

Re:Here's a tip for the author. (0)

Anonymous Coward | more than 11 years ago | (#5090396)

Acronym? Do you say "Vee Em" or "vum"? If the former, it's NOT an acronym. Go read up on what they are (think NATO, AIDS etc.)

Re:Here's a tip for the author. (0)

Anonymous Coward | more than 11 years ago | (#5090666)

The target audience of this paper *knows* what VM means in the context of the Linux Kernel.

Best course feedback I ever heard (1)

smitty_one_each (243267) | more than 11 years ago | (#5091927)

"Instructors used too many TLAs without explaining what they mean".
TLA=Three Letter Acronym, BTW

Re:Here's a tip for the author. (1, Funny)

Anonymous Coward | more than 11 years ago | (#5088829)

Choose one :

VM = Value Management
VM = Variable Metric
VM = Velocity Management
VM = Velocity Meter
VM = Velocity Modulation
VM = Version Manager
VM = VIBE Magazine
VM = Video Monitor
VM = Vietnam (Socialist Republic of Vietnam)
VM = View Mail (Emacs)
VM = Virtual Machine
VM = Virtual Madness
VM = Virtual Memory
VM = Vivamos Mejor (Let Us Live Better; Guatemalan agency)
VM = Voice Mail
VM = Voice Message
VM = Voice Modulation
VM = Voltmeter

but not that one!!!

Re:Here's a tip for the author. (2, Informative)

tomhartung (71654) | more than 11 years ago | (#5088843)

Down a ways on the page we see the title of one of the documents: "Understanding the Linux Virtual Memory Manager."

As a (an ex-)Java guy I immediately thought it stood for "Virtual Machine."

Re:Here's a tip for the author. (5, Insightful)

thodi (37956) | more than 11 years ago | (#5089189)

People who don't know what it stands for would not want to read the document anyway, don't you think?

Better document... (1, Flamebait)

Anonymous Coward | more than 11 years ago | (#5088712)

Not to sound like a bitch; but if any you want to understand the concept of a good VM, you should read this [freebsd.org] which is a proven VM, and has been proven vastly superior to the new Linux VM(s).

Re:Better document... (1)

snak0rific (317445) | more than 11 years ago | (#5089629)

yes, unfortunately i can't take fbsd places where linux goes with reckless abandon. what if i want to hax on a Hitachi SuperH ? learning fbsd is useless to me. how about if i want to target a toaster? can't run fbsd on that either! being a computer science type i'm not really interested in computers per se but "computing in general" which linux is proving to be a great OS for... in general.

Re:Better document... (2, Informative)

cgh4be (182894) | more than 11 years ago | (#5089713)

You're looking for NetBSD [netbsd.org]

Platforms supported:
acorn26 acorn32 algor alpha amiga amigappc arc arm32 atari bebox cats cesfic cobalt dreamcast evbarm evbmips evbppc evbsh3 evbsh5 hp300 hp700 hpcarm hpcmips hpcsh i386 luna68k mac68k macppc mipsco mmeye mvme68k mvmeppc netwinder news68k newsmips next68k ofppc pc532 playstation2 pmax pmppc prep sandpoint sbmips sgimips shark sparc sparc64 sun2 sun3 vax x68k x86_64

Re:Better document... (2)

AtrN (87501) | more than 11 years ago | (#5089944)

Being a "computer science guy" and being interested in "computing in general" means you should not limit yourself to a single kernel. Learn, look around, be all that you can be :)

Also, since we're talking VM systems here, I see little need for a CPU with an MMU in a toaster and you'd likely be using the MMU-less Linux in that case. However there may be valid reasons for needing virtual memory in your toaster's OS but I really just want something that slightly burns bread (yes I know about the voice controlled one done at UW) which can be done very cheaply these days. Bettter be a fancy toaster!

Re:Better document... (1)

snak0rific (317445) | more than 11 years ago | (#5091831)

yes, but then it's already done for me. since there's no official "GE Toaster" port, i get to write it myself, and still have the ownij GNU userland! but it still won't be official, just a port to "GE Toasters" :/

Would have read it (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#5088726)

Why oh why do web sites need those ugly backgrounds? The article makes my eyes wet - not because of the technical stuff, but the background...

IN SOVIET RUSSIA ... (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#5088731)

VM code walks through YOU!

JVM (1, Interesting)

Anonymous Coward | more than 11 years ago | (#5088732)

If I were to spend a lot of time integrating a JVM into the kernal, would it be useful?

Kind of like FreeBSD did, fast JAVA under linux would be awesome.

Re:JVM (5, Funny)

axxackall (579006) | more than 11 years ago | (#5089434)

yeah, right. let's also integrate to the kernel Perl, Python, Tcl, Lua, Emacs (for Elisp), Guile, Hugs, OCAML, Bash, Apache (for PHP) and Gecko (I want my Mozilla to work faster too!). I wonder, why is X server still not there? And don't forget about at least two CORBA brokers: Gnorba (everyone would love faster Gnome) and OmniOrb (just for a case). Hey, let's put everything into the kernel! Ooops... It's not kernel anymore and it doesn't want to run either. What was the mistake?

Re:JVM (0)

Anonymous Coward | more than 11 years ago | (#5089504)

not so much a JVM integrated, but have it run bytecode instead of standard executables

Re:JVM (4, Funny)

CableModemSniper (556285) | more than 11 years ago | (#5090325)

OMG! You've reverse-engineered windows!

Re:JVM (0)

Anonymous Coward | more than 11 years ago | (#5090682)

no if it had all those scripting languages integrated into the kernel, windows would actually be really useful and easy to automate.

Re:JVM (5, Funny)

PetiePooo (606423) | more than 11 years ago | (#5090174)

I suspect it would be a waste of your time. Lets look at a bit of history.

There used to be a kernel-space HTTP server. It was integrated into the kernel for a specific reason: zero-copy access to the network interface memory. It was fast and relatively feature-poor. If it crashed (fortunately, a rare occurance), you got a kernel panic.
Along came a user-space, zero-copy HTTP server. It was faster and had a few more features to boot. Being a user-space program, if it crashed, you got a core-dump. It could also be run in a chroot jail, a gigantic step more secure than running in ring-0.

Two lessons can be read from this:
1. Don't integrate something with the kernel unless there is a specific advantage you hope to gain from it. Will making a JVM part of the kernel really speed it up? Are you sure?
2. Don't under-estimate the speed of a properly designed user-space Linux program. The kernel developers have done a magnificant job tuning the kernel and providing APIs for performance-critical apps.

always nice to see some documentation (2, Insightful)

mark_lybarger (199098) | more than 11 years ago | (#5088748)

but this kernel has been out now since 04-Jan-2001. i know it took them a while to shake out the vm system, but the kernel is now stale. anybody who needs to know the internals of how vm works in the 2.4 kernel already knows. the rest of us just argue on /. about why there wasn't lots of testing done in past releases or about how you don't make big changes in a supposidly stable release series.

i'm sure those who need to know though are full aware of the vm workings of the 2.6 kernel (ibm, redhat, oracle, rik, google, etc).

Linux: the big player everyone likes to root for, but noone wants to put in the lineup.

MORE documentation needed. (1)

dmelomed (148666) | more than 11 years ago | (#5088891)

This is great, however, where's the internals documentation for other subsystems of the kernel? People need documentation for JFS, ReiserFS, and other large systems. We'd see more of these in other kernels.

Re:MORE documentation needed. (1)

mark_lybarger (199098) | more than 11 years ago | (#5089080)

http://www-124.ibm.com/developerworks/oss/jfs/

here's the first one i found relevant. i don't know if it's the current implementation or not. i don't follow that stuff too much. i just save files and use whatever doesn't fsck every time the system crashes or locks up.

i like the method of releasing specs that implementations are to adhere to. and deviation from that should then be documented. W3C i believe was to accomplish a similar goal, but somehow they weren't too successfull.

Re:MORE documentation needed. (2)

inode_buddha (576844) | more than 11 years ago | (#5091773)

cd /usr/src/linux
make sgmldocs psdocs pdfdocs htmldocs mandocs

However, I also subscribe to the relevant mailing lists, and review them daily.

HTH

This should be helpful (0, Troll)

Jack Wagner (444727) | more than 11 years ago | (#5088773)

Most low level coders are going to find this very useful and I for one, even though I use Gnu/FreeBSD (lower context switch latency and better pipelining performance when under heavy paging loads, etc), am certainly glad to see the Linux camp start to come around regarding the documentation issue.

Fred Brooks in "The Mythical Man Month" states that proper documentation should be given a 20% time slice of each project if you want to have a successful product and this is certainly THE one arena where Linux is lacking, stopping them from commercial success.

As a tip for best results I suggeest using the Extreme Documentation method when writing your docs, it's saved time on the order of Olog(n) for me and a proven time saving technique.

Warmest regards,
--Jack

Re:This should be helpful (0)

Anonymous Coward | more than 11 years ago | (#5088832)

Please don't use "GNU/FreeBSD" as a label for FreeBSD, the GNU parts are replaceable, in fact, the only reason it's still around is because the GCC collection is the best opensource compiler, but by no means irreplaceable.

Re:This should be helpful (0)

Anonymous Coward | more than 11 years ago | (#5089331)

If he loads GNU versions of the BSD system tools, then he is able to call it GNU/FreeBSD. But I doubt that's what he did because he would have had to write some of his own tools in that process. Haha, he probably loaded GNU tar and GNU emacs the same way people do for Solaris, but he just trolls away with FreeBSD.

Re:This should be helpful (0)

Anonymous Coward | more than 11 years ago | (#5091219)

If I had to run FreeBSD I'd replace the userland tools with the GNU ones (GNU fileutils, sh-utils, etc) because I hate BSD's. Obviously FreeBSD's C library would stick around, but I'd make the system as GNU as I could.

Re:This should be helpful (0)

Anonymous Coward | more than 11 years ago | (#5088964)

"I use Gnu/FreeBSD (lower context switch latency and better pipelining performance when under heavy paging loads, etc)"

Translation:

I have a much bigger dick than you linux n00bs. I am 13371111, 1337 I tells you!!1111111 Fear my skillzzzz I am the 1337est of them alllll111111 I am so 133333333333333........

(Disappears up own arsehole)

TROLL - Please mod parent down (0)

Surreal_Streaker (636407) | more than 11 years ago | (#5089119)

This is a known troll. Mod's: please check this user's other posts and moderate accordingly.

Re:TROLL - Please mod parent down (1)

Jack Greenbaum (7020) | more than 11 years ago | (#5089343)

One man's troll is another man's humor. I find this supposed troll [slashdot.org] particularly amusing.

-- Jack

Re:TROLL - Please mod parent down (1)

NTT (92764) | more than 11 years ago | (#5090068)

Agreed. Anybody who puts a domain name that isn't registered in their user info is a fraud.
See it here...
http://www-whois.internic.net/cgi/whois?w hois_nic= wagnerconsultingllc.com&type=domain

Re:This should be helpful (5, Funny)

sporadek (639621) | more than 11 years ago | (#5089145)

As a tip for best results I suggeest using the Extreme Documentation method when writing your docs, it's saved time on the order of Olog(n) for me and a proven time saving technique.

Define your function "Olog", please. Surely Mr. "Wagner LLC Consulting Co. - Getting it right the first time" couldn't have meant O(log n)... :)

Re:This should be helpful (1)

uid8472 (146099) | more than 11 years ago | (#5090637)

Define your function "Olog", please. Surely Mr. "Wagner LLC Consulting Co. - Getting it right the first time" couldn't have meant O(log n)... :)

I believe the OP was in fact referring to the Olog-hai, the race of trolls enhanced by Sauron. HTH.

Re:This should be helpful (2)

be-fan (61476) | more than 11 years ago | (#5091076)

Uruk-Hai and it was Saurumon.

Mod Parent Down (1)

driverEight (598719) | more than 11 years ago | (#5089164)

Surreal_streaker is right this is a TROLL!

Re:what's Gnu/FreeBSD ? (0)

Anonymous Coward | more than 11 years ago | (#5090130)


didn't you mean BSD/FreeBSD ?

Re:This should be helpful (2)

glrotate (300695) | more than 11 years ago | (#5090310)

FreeBSD has had commercial success?

last post (-1)

Anonymous Coward | more than 11 years ago | (#5088774)

last post hehe

Is late documentation a problem with Open Source? (1)

WarSpiteX (98591) | more than 11 years ago | (#5088800)

I'm no expert on Linux/Open Source, this is purely my curiosity at work. But it seems like many of the smallest projects, or those with large teams don't get documentation well after release. Or this is a misconception of mine?

Re:Is late documentation a problem with Open Sourc (1)

cyba (25058) | more than 11 years ago | (#5089129)

It's the other way round.
The real problem is "release early (without documentation)" rule.

Re:Is late documentation a problem with Open Sourc (2)

inode_buddha (576844) | more than 11 years ago | (#5091814)

I truly think this should be modded as "Insightful", it's quite true in my experience.
The cure that I use for this is to read the code comments and subscribe to the mailing lists.

Re:Is late documentation a problem with Open Sourc (0)

Anonymous Coward | more than 11 years ago | (#5090194)

V2_OS [v2os.cx] has had plenty of documentation for a long time, but no one to code it. It might be an exception among the Open Source projects, though.

An Idiot Savant's Book (0)

Anonymous Coward | more than 11 years ago | (#5088801)

ooo kernel wapner ... need to read book....

Where I work (5, Funny)

oliverthered (187439) | more than 11 years ago | (#5088806)

We design aned document things first, work out the bugs in the high level design and then code.

Well maybe not all the time and with serveral itterations, but I only manage your credit raiting, not you kernel VM.

Good news for santa claus? (-1, Troll)

Gizzmonic (412910) | more than 11 years ago | (#5088811)

Wow, those santa clauses who work for Linux keep giving, don't they? Long past December, and here we have the shiny gift of Linux 2.4 kernal VM documentation?

I'll tell ya, I was just about done with SCSI over IP: in theory and praxis and I needed something to read on the crapper. Well, this certainly fits the bill! Thanks to all the hard workers at Linux! Good luck on the Playstation 3!

Sad news ... Stephen King dead at 55 (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#5088812)



I just heard some sad news on talk radio - Horror/Sci Fi writer Stephen King was found dead in his Maine home this morning. There weren't any more details. I'm sure everyone in the Slashdot community will miss him - even if you didn't enjoy his work, there's no denying his contributions to popular culture. Truly an American icon.

VM design (-1, Troll)

AnonymousCowheard (239159) | more than 11 years ago | (#5088840)

Correct me if you think YOU are wrong...

I think some of the purpose of having Virtual Memmory (VM) is to address the constraints of any given architecture's idle processes being in a state of inactivity and their data in the system's RAM is "swapped" unto a file on the harddrive for the system's RAM to be available to processes that are still active and will benefit from the performance all ans while the processes don't recognize any VM subsystem or may choose to interact with it and this is one crazy run-on sentance. So, what's the problem and why do they feel the need to re-design and re-implement Linux VM support? Maybe you should first understand the scope of the principle of VM: applications do not know it is happening. Then slowly appen to your implementation hooks that allow applications to reschedule the kernel's interaction with virtual memmory and you lost scope of the *purpose*.

That's how I implemented VM in one of my applications! The difference between kernel-based VM is the application is not built to recognize and interact with VM, it is just there as a condition. VM-enabled kernels ARE SUPPOSED TO RELEASE KERNEL INFORMATION that instead of having 2GB of system RAM 4GB is available: not that bullshit "2GB free swap available" stuff. Relying on applications to interact with kernel-based VM decides the portability of applications. /drinks small glass of whisky and passes out.

every developer worth his salt knows... (5, Funny)

syle (638903) | more than 11 years ago | (#5088849)

...that virtual memory works because of small, magical faeries and gnomes.

Re:every developer worth his salt knows... (2)

SpaceLifeForm (228190) | more than 11 years ago | (#5089473)

And pixie dust. [ibm.com]

Re:every developer worth his salt knows... (2, Funny)

FroMan (111520) | more than 11 years ago | (#5089475)

How would this be modded informative... I'd like to meta-meta mod that moderation to +1 Funny along with this post as +1 Funny.

Links to actual documentation (1)

braz (68726) | more than 11 years ago | (#5088881)

Congratulations to Mel Gorman for actually managing to get down to this low a level and still explain it sensibly.

Here are the links for interested readers:
http://www.kerneltrap.com/node.php?id=55 5 (kerneltrap article)

Actual documentation:
The documentation comes in two parts. The first is "Understanding theLinux Virtual Memory Manager" and it does pretty much as described. It isavailable in three formats, PDF, HTML and plain text.Understanding the Linux Virtual Memory Manager
PDF: http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/un derstand.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/u nderstand/
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/u nderstand.txtThe second part is a code commentary which is literally a guided tourthrough the code. It is intended to help decipher the more crypticsections as well as identify the code patterns that are prevalent throughthe code. I decided to have the code separate from the first document asmaintaining the code in the document would be too painfulCode Commentary on the Linux Virtual Memory Manager
PDF: http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/co de.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/c ode
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/c ode.txt

Re:Links to actual documentation (0)

Anonymous Coward | more than 11 years ago | (#5089456)

there should be a new moderation category: "karma whore" :)

Re:Links to actual documentation (0)

Anonymous Coward | more than 11 years ago | (#5091009)

But is it +1 or -1?

Re:Links to actual documentation (0)

Anonymous Coward | more than 11 years ago | (#5089942)

braz's [slashdot.org] article [slashdot.org] cleaned up, and with links for easy clicking (and AC to avoid karma-whoring)...
========

Congratulations to Mel Gorman for actually managing to get down to this low a level and still explain it sensibly.

Here are the links for interested readers:
The kerneltrap article [kerneltrap.com]

Actual documentation:
The documentation comes in two parts. The first is "Understanding the Linux Virtual Memory Manager" and it does pretty much as described. It is available in three formats, PDF, HTML and plain text.
"Understanding the Linux Virtual Memory Manager" as PDF, [csn.ul.ie] HTML, [csn.ul.ie] and Text. [csn.ul.ie]
The second part is a code commentary which is literally a guided tour through the code. It is intended to help decipher the more cryptic sections as well as identify the code patterns that are prevalent through the code. I decided to have the code separate from the first document as maintaining the code in the document would be too painful Code Commentary on the Linux Virtual Memory Manager
"VM Code Commentary" as PDF, [csn.ul.ie] HTML, [csn.ul.ie] and Text. [csn.ul.ie]

Pretty good English for a foreigner (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#5088975)

Where the heck is .ie? He has surprisingly good English for someone who wasn't born in the US.

Re:Pretty good English for a foreigner (1)

unDiWahn (599102) | more than 11 years ago | (#5089213)

Uh -- that's a joke, right? Can't really tell...
It's .ie as in Ireland, y'know, one of the many other English-Speaking countries out there... like, say, England?

Re:Pretty good English for a foreigner (2)

iggymanz (596061) | more than 11 years ago | (#5089959)

..but is it English speaking by choice? Check this [bbc.co.uk] out.

Re:Pretty good English for a foreigner (1)

hughmcg (210910) | more than 11 years ago | (#5089238)

Ireland boss....

Re:Pretty good English for a foreigner (1)

xanadu-xtroot.com (450073) | more than 11 years ago | (#5089249)

Right here [domainregistry.ie]

Re:Pretty good English for a foreigner (1, Offtopic)

lukew (528994) | more than 11 years ago | (#5090405)

Where the heck is .ie? He has surprisingly good English for someone who wasn't born in the US.

Here you go fellows, an excellent example of the [hopefully minority] view that America is the center of the universe.

Re:Pretty good English for a foreigner (0)

Anonymous Coward | more than 11 years ago | (#5091042)

English good as am beatnick useless Americaner slave in corner. Feednick good laces and shoe, da?

IN SOVIET RUSSIA... (-1, Troll)

Anonymous Coward | more than 11 years ago | (#5088987)

...kernel VM documents YOU!

VM: Does it really matter? (4, Interesting)

XaXXon (202882) | more than 11 years ago | (#5089011)

I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory..

I'm not trying to troll, and maybe someone can explain this to me, but as far as I know, VM is used when a computer runs out of RAM. You take some of what's in the RAM and toss it on to the hard drive. When you need it again, you reload it back into RAM and use it.

I've got 1.5GB of RAM in the box I'm typing this on, and the only reason I don't have more is that I have no use for it.

At my work, we've got some systems that can eat up 2GB of RAM, but we just put 4GB in the box and don't worry about it. We've found that if we let anything touch VM it slows things down way too much.

Anyways, I was hoping someone could enlighten me as to why VM has been such a big deal recently.

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5089058)

> I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory..

Because the "today's world" you describe is limited to the affluent portion of computer users/buyers. Many, many many computers exist in non-affluent parts of the real world that have 32-64Mb of RAM, and virtual memory is useful there. These machines are not ignored by the various Open Source OSs, they're deliberately targeted as being ideal platforms - still useful and usable despite being "obsolete" by Microsoft/Intel standards of What A Computer Should Have To Be Useful.

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5089093)

Because "slows things way down" is a damn sight better than "crashes applications and/or the system", especially on multiuser systems where there are certainly spikes of activity.

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5089143)

Two words: Graphic arts. I edit reproductions of oil paintings, and some of my image files are upwards of a gigabyte themselves (before layers), so I imagine my undo buffer plus whatever GIMP needs to run whatever filter it's running can take up quite the hefty chunk of memory... and let's not even start on the amount of RAM it takes to hold a full-length feature film at high resolution in memory *uncompressed*.

Re:VM: Does it really matter? (5, Informative)

GGardner (97375) | more than 11 years ago | (#5089216)

I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory..

This is a good question -- there are several features the VM system allows for in addition to overcommitting physical memory. If your system has enough memory to run with no swap/paging space set up, the VM system still provides these features without ever paging or swapping out memory.

The first is demand paging. That is, only those pages of an executable that are needed are brought into memory, and on-demand. Yes, this saves memory, but more importantly, it makes program startup much faster. Without demand paging, the whole program would need to be brought into memory at startup. Nowadays, when disks are much slower relative to CPUs, than they used to be, this makes a big difference, especially in the Unix shell-pipeline style of programming, where you run many different short-lived programs.

Secondly, the VM system allows for shared memory segments between processes. This allows for shared memory, threads and shared libraries.

Finally, the VM system implements caching of the file system, which we all know and love as a good use for all the memory we stuff into our machines now.

Re:VM: Does it really matter? (2, Informative)

Jack Greenbaum (7020) | more than 11 years ago | (#5089371)

Secondly, the VM system allows for shared memory segments between processes. This allows for shared memory, threads and shared libraries. Um. Don't you mean it enforces memory separation between processes?

-- Jack

Re:VM: Does it really matter? (5, Interesting)

GGardner (97375) | more than 11 years ago | (#5089411)

Um. Don't you mean it enforces memory separation between processes?

Good point -- this is such a given on Unix systems that I didn't word it very well. What I should have said is that it enforces memory separation and protection between most memory segments and most processes, and allows for sharing of segments when explicitly setup. This is perhaps the most important thing the VM system does.

Re:VM: Does it really matter? (2, Informative)

sjames (1099) | more than 11 years ago | (#5091919)

Another benefit is sparse allocation. When memory is allocated, it's all mapped to a zero filled page and marked copy on write. That allows programs to allocate massive arrays that are expected to be primarily filled with zeros without actually requiring that much memory. When a value is writen, a new page is allocated and mapped into the virtual space.

If desired, a file can be mmaped into the array instead to provide a simple persistant storage.

IIRC, suspend to disk is (or at least can be) also accomplished through VM.

The VM also allows an x86 system to have more than 4GB of ram. No single process can have more than a total of 4G (counting the kernel space) but using PAE, more than a total of 4GB can be allocated to multiple processes through the VM.

Although Linux doesn't do this, some OSes completely replace the filesystem with a VM. All files and directories are objects and are demand paged.

Re:VM: Does it really matter? (3, Informative)

heh2k (84254) | more than 11 years ago | (#5089519)

VM does NOT mean just paging/swapping. that is a small part of it. mac os and windows users constantly misuse the term. the vm does demand paging, copy-on-write, file/page cache, buffer cache, shared mem, mmapped files, and more. the most important of which is protection.

Re:VM: Does it really matter? (1)

uid8472 (146099) | more than 11 years ago | (#5090699)

VM does NOT mean just paging/swapping. that is a small part of it. mac os and windows users constantly misuse the term.

In all fairness, if under the old MacOS you booted with "virtual memory" off then virtual memory (in the correct sense) really was disabled.

Re:VM: Does it really matter? (1)

arjun (33278) | more than 11 years ago | (#5089624)

I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory..
three basic reasons
1. resource virtualization
2. information isolation
3. fault isolation

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5089670)

We have a 3 systems (unix/not linux) each with 16G of RAM, 6TB of fiber attached SAN storage. When doing a sql query, the rdbms likes to cache indicies in RAM (real or perceived). If the dba team does not have a handle on what data needs to be pinned in RAM and what not, then VM becomes very popular and important. Unless you have data that will never change and an audience that will never make any different choices, then this predicting is a crap-shoot at best.

When to start chucking things off into paging space ? What stuff to chuck first ? File accesses ? Computational memory ? Processes ? Which processes go first ? Who's processes are more important ? Can the VM system be tuned without a reboot ? I have really been keeping my eye on these discussions for these reasons.

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5090695)

Virtual Machine --- Not Virtual Memory.

Re:VM: Does it really matter? (2, Informative)

tstoneman (589372) | more than 11 years ago | (#5090733)

If you are asking this question, it really means you have not worked on systems that do not have virtual memory, like DOS.

Virtual Memory means virtualizing a process's access to memory such that the memory addresses it accesses isn't necessarily the same physical memory address. It allows for complete memory separation between processes, which offers a vast amount of protection so that processes can't crash other processes.

The only alternative to Virtual Memory is to use actual physical memory addresses. This was exactly the case with DOS. In DOS you would have to manage memory and make sure your process (it's single-threaded... remember?!?) didn't overwrite memory that it didn't own. Everything in memory could be accessed from every single process.

With virtual memory, two processes could access memory location 0x1000000 and it would map to different physical memory addresses. In fact, the OS will give each process a full memory address range that is completely usuable by the process without any fear of stepping over someone else's memory. This effectively separates processes and prevents a hell of a lot of corruption because of memory bugs in programs.

This really simplifies programming because whenever we get a memory address from malloc, we don't have to do anything about making sure we don't step over someone else's memory information.

What you are in fact talking about is paging, the act of having a backing store of information on hard drive. This allows you to keep track of more memory than you physically have, by swapping memory back and forth from physical memory to a hard drive. Any unused memory would be paged to disk.

Yes, today you can have gigabytes of memory, and when running Mozilla, boy you really don't need virtual memory.

But what about 10 years from now, when Microsoft Office takes up 50 GB of hard drive space, and 16 GB of memory?

Virtual memory is a way of abstracting memory use on a per process basis, and it's inclusion in most modern operating systems is one of the reasons why things are relatively stable these days, compared to the old DOS days, anyway.

Re:VM: Does it really matter? (5, Informative)

Pseudonym (62607) | more than 11 years ago | (#5091145)

I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory.

Because it's important. :-)

Some people think that virtual memory means paging to secondary memory, such as disk. It doesn't. Virtual memory systems can support this, but many OSes (e.g. QNX) support virtual memory with no disk paging. (OK, QNX does support disk paging, but only as an afterthought, so that QNX can be self-compiled. GCC takes a lot of memory.)

Virtual memory provides a virtual address space for each process. The benefits include:

  • The system can avoid memory fragmentation because contiguous virtual pages do not need to be contiguous in physical memory.
  • Processes don't step on each others' address spaces.
  • Separate instances of the same program can share program text space and shared libraries.
  • Different processes can use the same addresses for different purposes.
  • Processes can share memory between each other in a controlled manner (for example, one process may have read-only access to some region, or memory can be shared between a parent and child process via a read-only or copy-on-write mechanism).
  • Processes on different physical machines can share memory. (This can be done using the standard Unix interface. No extra kernel support is required.)
  • Processes can view files as memory (e.g. using mmap), reading in only as much of the file as is necessary.
  • Processes can allocate large regions of memory and only have the amount of those regions which they actually use backed up by physical memory (whether that is primary or secondary memory).
  • The OS has a lot more freedom in how to lay out a process in physical memory to optimise cache usage.

This is just off the top of my head.

Re:VM: Does it really matter? (2)

nathanh (1214) | more than 11 years ago | (#5091587)

I've always wondered why, in today's world of gigabytes of memory in personal computers, why such a big deal is made about virtual memory.

Because VM is not just used for swap. It's also used for memory protection, buffer caching, shared libraries, demand paging, and dozens of other performance improvements.

Re:VM: Does it really matter? (0)

Anonymous Coward | more than 11 years ago | (#5091892)

While you, like us, have a Gbyte or more on your desktop system, most of the servers I've seen have an order of magnitude less. The vast majority of our 40+ servers have 128 Mbytes or less. That's because the desktops are the cheap junk we replace every couple of years. This round, we're using Dells. On our servers, we buy quality so we can't afford to upgrade nearly as often. $3,500 IBM rack-mount servers + $500 redundant power supplies + $1,500 RAID controllers + $800 or much more worth of SCSI hard drivers + ECC RAM, etc, is expensive. Over half of our servers that run Linux have an uptime of over one year. It would be very costly to replace them with something else that reliable. So, like most places, our servers end-up on the trailing edge of technology. Also, the cost of replacing servers is very expensive. The downtime and the pain of moving software is costly. We swap-out desktops in about half a day, but it took us three months to trackdown all of the little problems when we upgraded our PostgreSQL server. Small differences are painful when you have about 400 different programs that access data on that one server. So while on your desktop you only have to worry about a few applications, when we upgrade one of our servers, many more applications and many more people (including customers directly, if it's a web server!) are affected. That's why most servers have less RAM than you think they would. You just can't afford upgrade them!

This is why Linux is so nice. It makes good use of the hardware so, in the case of our Sun servers, it has given us three years (so far) more use out of the same hardware. And, no we didn't have 1.5 Gbytes of RAM on those machines when we bought them five years ago.

Documentation and uncommunicative maintainer (4, Informative)

FeatureBug (158235) | more than 11 years ago | (#5089387)

Documenting the VM is a good idea. I hope it is accepted into the /Documentation directory in 2.4.x kernel tarballs but I'm not sure whether you'd get a reply from the maintainer, Marcelo Tosatti. I sent him a carefully written email in August 2002 documenting an invalid config in 2.4.19 which causes the build process to fail. [paraset.com] It was disappointing he never responded and apparently did not fix the bug which is still present in 2.4.20 and more recent patches.

Re:Documentation and uncommunicative maintainer (0)

Anonymous Coward | more than 11 years ago | (#5089542)

Could it be that the kernel maintainer [marcelothe...enguin.com] is buried alive in email ?

I work at the same office as Marcelo and I know for a fact that he gets so much email that it's pretty much impossible to read all of it. Now imagine a conference or a project for a client, which would mean not being able to read email for a week, ie. impossible to catch up later.

Before criticising Marcelo, it'd be good to imagine how you would handle being bombarded with hundreds of bug reports and patches every day...

I think Marcelo is doing a good job and shouldn't be flamed just because part of the job is impossible.

Re:Documentation and uncommunicative maintainer (2)

FeatureBug (158235) | more than 11 years ago | (#5089810)


Ok, I sympathise with his daily email overload but maybe he should delegate some of his workw , like dealing with kernel email? There are many other qualified people who have offered to help with maintaining 2.4.x kernels. That would be better than silently dropping email. Maintaining a relatively common Linux kernel series like 2.4.x is such an important job we should not have to rely on one overloaded person -- that's an unnecessary bottleneck. This is not a criticism of Marcelo's quality of work, which I think is very high. I am suggesting that he should delegate at least the work of dealing with kernel email.

Re:Documentation and uncommunicative maintainer (0)

Anonymous Coward | more than 11 years ago | (#5089880)

maybe he should delegate some of his work, like dealing with kernel email?
This is called the Linux Kernel mailing list and it can be reached at linux-kernel@vger.kernel.org. If you mail Marcelo personally chances are he won't have time fixing bugs (he is too busy integrating bugfixes by other people), but if you mail the mailing list chances are somebody will have time to solve the bug and send the fix to Marcelo...

You did send your bug report to the mailing list, didn't you?

Re:Documentation and uncommunicative maintainer (2)

FeatureBug (158235) | more than 11 years ago | (#5089964)


From other people's comments it sounds like Marcelo is too overloaded even to deal with all the bug fixes he receives by email, let alone the bug reports. He silently drops fixes as well as bug reports, so my point is he should delegate some of his heavy workload, such as dealing with email, to the many other qualified people who have offered to help as maintainers of 2.4.x kernels. For some reason he hasn't accepted any of those offers of help.

LKML is a very high traffic mailing list. I'm wary of adding to that flood of email. However, due to the lack of response from Marcelo I will probably email LKML.

Different VM (4, Interesting)

be-fan (61476) | more than 11 years ago | (#5091029)

I did a lot of research on VMs about a year ago, and while the different VM systems out there really are interesting they're all kind of limited by the whole "page-level protection" model we've stuck with since the '70s. There are a lot of clever algorithms in the FreeBSD and 2.4 VMs to deal with high load and paging, but the core VM stuff (mapping memory, protection, sharing etc) is largely the same, and is limited by the MMU capabilities of current CPUs. All CPUs basically seperate processes into unique memory contexts, and map (using some sort of page table or reverse page table) physical memory pages of fixed size into the contexts. This solution is sub-optimal for object oriented systems with large scale sharing (the page tables get unweildy, and certain secondary structures, like reverse maps*, grow out of control). What I'd like to see is something new, just for the hell of it being new.

Here's what I'm thinking. Ditch pages and memory contexts entirely. Instead, divy up a 64-bit virtual address space among individual processes, say 48 bits apiece. If a process wants to access memory outside it's 48-bits, it would need to access it through special pointers (which, thanks to a tagged-RAM architecture) could written to by the OS (allowing the OS to define its own protection and sharing mechanisms). Does anybody know of any existing systems that work even vaguely like this? Or of a different MMU architecture at all? I was hoping that AMD might at least include software TLB management, because there is some nifty stuff you can do with that, but it looks like Hammer will use the same VM mechanism that came out with the 386!

Re:Different VM (2)

Wesley Felter (138342) | more than 11 years ago | (#5091080)

Here's what I'm thinking. Ditch pages and memory contexts entirely. Instead, divy up a 64-bit virtual address space among individual processes, say 48 bits apiece. If a process wants to access memory outside it's 48-bits, it would need to access it through special pointers (which, thanks to a tagged-RAM architecture) could written to by the OS (allowing the OS to define its own protection and sharing mechanisms).

Look, it's AS/400!

Doing something "just for the hell of it being new" doesn't seem too popular among CPU manufacturers who are trying to actually make money.

Re:Different VM (2)

be-fan (61476) | more than 11 years ago | (#5091280)

Um, it's called 'innovation.' I mean, we could have stuck with segmentation forever (or better yet, overlays!) but aren't you glad we didn't?

PS> Hm. I knew AS/400 had a tagged RAM architecture, but I was under the impression that it was for marking crypto data. Could a scheme like the above be implemented on AS/400, and if so, where can I find some reference info about the MMU?

Re:Different VM (2)

Wesley Felter (138342) | more than 11 years ago | (#5091474)

Doing something different just to be different is not innovation. Doing something different because it's better is innovation. In this case, I don't think there's much benefit to single-address-space systems.

I knew AS/400 had a tagged RAM architecture, but I was under the impression that it was for marking crypto data. Could a scheme like the above be implemented on AS/400, and if so, where can I find some reference info about the MMU?

My impression is that OS/400 has a single address space and a single level store. The hardware is mostly undocumented AFAIK.

Re:Different VM (1)

catbutt (469582) | more than 11 years ago | (#5091651)

Doing something different just to be different is not innovation. Doing something different because it's better is innovation.


Maybe, but in my opinion people who just like to try different stuff (because it might be better, or because it interests them to try it, or just because...) tend to be the types who eventually come up with something better. While people who don't want to try anything because they worry they might be "reinventing the wheel" or that "maybe it won't be better" or whatever....probably are unlikely to ever create something useful.

Usually the best ideas are not arrived at directly, but after exploring many different ideas along the way. Most of those ideas are probably not better than what they replace, but they might be a stepping stone to a really great idea.

(sorry to go further off topic, but I just hate to see creativity squelched :) )

copulating modules (2, Funny)

ghassanm (130547) | more than 11 years ago | (#5091770)

2.1.2 Browsing the Code

Generally, related operations are clustered together in the same file and there is not much copulation between modules.

I hope the discovery channel will do a special on copulating modules some day. I've seen module coupling but no copulation yet.

Re:copulating modules (2, Funny)

Mel (21137) | more than 11 years ago | (#5091880)

> I hope the discovery channel will do a special on copulating
> modules some day. I've seen module coupling but no copulation yet

Haven't you ever heard of sexy code? Is is meant to be taken literally, isn't it?

(typo corrected in the work in progress version)

So any clues why VM goes zombie? (3, Interesting)

Jayfar (630313) | more than 11 years ago | (#5091852)

5 root 9 0 0 0 0 Z 0.0 0.0 0:18 kswapd

My kswapd went zombie after ~18days uptime; it had gone zombie after about 70 days prior to that reboot. I've found references to this occurance ...going back at least through 2.2 kernels and it still happens with 2.4.18 (not the latest, but no real reason for me to build a new kernel unless it solves this). I've never read a thread on this that arrives at a solution or even a description of what is going on. I first encountered it when I went from Mandrake 8 (2.4.8 kernel) to 8.1 (2.4.18).

Fortunately, I have plenty ram and this doesn't seem to otherwise affect anything.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>