Swap file on Ramdisk?

[Yamyam]

Is it possible to put my WinXP swapfile on a RAMdisk, or does Windows require that the RAMdisk be ready before Windows loads?

How much would this improve the (subjective) performance of my PC? I've already got the core system files cached to RAM. Total RAM is 1 gig of DDR at 148 MHz, Cas2, with an Athlon XP running at 1.7 GHz.

Thanks

 

[sechs]

Ever wonder why you have a swap file?

Maybe you should.

 

[bsobel]

The purpose of a swapfile is to provide virtual memory for your system, it is a complete waste (and will hurt performance) to take part of your physical memory and use that to provide virtual memory to the system.
Bill

 

[Yamyam]

As I understand it, the purpose of the swapfile is to decrease my RAM requirements by using the hard disk for real-time "RAM" storage (ie, virtual memory). Since I've got plenty of RAM, it doesn't seem to make sense to use slow hard-disk storage to ease the load on the physical RAM. Windows has gotten very upset with me in the past when I disabled virtual memory (my Win98 box crashed frequently until I re-enabled it), so I had hoped this would be another way that I could work around it.

The goal here is to avoid bogging the system down with hard-drive accesses when I've got plenty of physical RAM available. Is this a lost cause, or is there another approach that would work better? Please advise.

 

[Smilin]

Keep in mind if you reduce the amount of ram available to the system by allocating some of it to a virtual disk then you'll cause it to page more. NT/2000/XP have very good memory management schemes. Let the OS handle things, it will do a better job than you possibly could.

What you are REALLY trying to do is this: Make your system use 100% ram and NO disk. It's not going to happen. Put enough memory in and just let it run. Hack the registry and tell it not to page the executive...that's about the best you'll do.

 

[sechs]

I'll tell you what. On my old machine, with 640MB of RAM, I did some measuring. On a regular basis, that machine used exactly 4MB of it page file. Just 4MB. Even when playing games, it never used more than 160MB (and that was with a bunch of other crap running in the background).

If you want to make better use of your memory and decrease disk accesses, then you should tweak your memory settings, like system cache. There is only one good reason that you would ever use a RAMdisk in Windows NT, and no home user runs a large database and has 4GB of memory.

 

[bsobel]

Originally posted by: Yamyam
As I understand it, the purpose of the swapfile is to decrease my RAM requirements by using the hard disk for real-time "RAM" storage (ie, virtual memory). Since I've got plenty of RAM, it doesn't seem to make sense to use slow hard-disk storage to ease the load on the physical RAM. Windows has gotten very upset with me in the past when I disabled virtual memory (my Win98 box crashed frequently until I re-enabled it), so I had hoped this would be another way that I could work around it. The goal here is to avoid bogging the system down with hard-drive accesses when I've got plenty of physical RAM available. Is this a lost cause, or is there another approach that would work better? Please advise.

You don't understand it properly. The purpose of the swapfile is to provide 'virtual' memory to the system, available allocateable space which is used when your physical memory is used up. Taking physical memory to provide a swap file just helps make sure you run out of physcial memory earlier.

You'll just have to trust us, this is a horrible horrible idea.

Bill

 

[Yamyam]

Thanks, guys..... I think I understand now.

 

[Oogle]

You don't understand it properly. The purpose of the swapfile is to provide 'virtual' memory to the system, available allocateable space which is used when your physical memory is used up. Taking physical memory to provide a swap file just helps make sure you run out of physcial memory earlier.

You'll just have to trust us, this is a horrible horrible idea.

Bill

Could you please clarify? If your swapfile is used once your physical memory is used up, how come I always run out of swapfile space before running out of physical memory? On second thought, I probably can answer my own question. I just don't understand the answer...

Answer: "You didn't allocate more disk space for your swapfile. Allocate more and you'll be fine..."

This is still very confusing. I have 1GB of RAM and 64 MB fixed swap space. Why would I need more swap space if I didn't even come close to running out of RAM? I would love to see a truly techy OS article that explains why swapfile space is used even though you have plenty of RAM. An academic article of PhD/Masters research quality would be really convincing. I'm tired of reading all those PC web articles out there saying to "just do it" because MS knows best or because "the rule of thumb is always 2.5x RAM".

 

[Dug]

how come I always run out of swapfile space before running out of physical memory?

Because some programs are programmed to use the swapfile no matter how much physical memory you have. No one should write a program that purposely uses up you physical ram and then go to the swapfile. This is because people operate many programs at the same time. This is also the reason you don't want to limit your swap file, but instead let Windows manage it, because you never know. A good example is the game NWN. It uses a great deal of your swapfile, up to 300MB at a time. Another is Photoshop.

 

[Nothinman]

how come I always run out of swapfile space before running out of physical memory?

Because in Windows the swap file is used as a backing store for more things than you think. It's not just a repository for memory pages when physical memory is full. Windows VM subsystem was designed with the idea that a page file will always be present, if you try and run without one eventually it'll fall on it's face.

I would love to see a truly techy OS article that explains why swapfile space is used even though you have plenty of RAM

You mean 'Windows article', Linux runs just fine without swap space as long as you be carefull not to run out of memory, but even then all the kernel does is kill a few of your apps to make more room =)

I'm tired of reading all those PC web articles out there saying to "just do it" because MS knows best or because "the rule of thumb is always 2.5x RAM".

Get a book on OS internals. There's a Linux kernel book with a few very good chapters on memory management and there's an Inside Windows book that probably has the same.

No one should write a program that purposely uses up you physical ram and then go to the swapfile.

Actually noone should write a program that cares where the memory it's using came from, it should just ask for memory and let the OS do it's job from there.

This is because people operate many programs at the same time. This is also the reason you don't want to limit your swap file, but instead let Windows manage it,

When I used Windows I set my swap min and max the same, because I don't want the overhead of Windows having to dynamically resize my swapfile on the fly. If I need more swap space I'll make it bigger myself, reboot and try again.

 

[Oogle]

Because some programs are programmed to use the swapfile no matter how much physical memory you have. No one should write a program that purposely uses up you physical ram and then go to the swapfile. This is because people operate many programs at the same time. This is also the reason you don't want to limit your swap file, but instead let Windows manage it, because you never know.

So now who's really doing the memory management? Applications or Windows? It feels like a double standard. Consumers are supposed to trust Windows, but software developers aren't?

Why can't devs just say to Windows, "I need some memory, Windows. Give it to me however you want. I don't care. You figure out where to get it..."? I thought the whole idea behind virtual memory paging was to abstract the idea of a memory pool from the developer and let the OS do all the management between RAM and disk.

If developers are going to outright demand for slow disk memory, why is the concept of virtual memory used at all? I'm sorry for all the questions... but the idea that a developer optimizes their program by purposely using slow disk memory just doesn't make any sense at all.

 

[Dug]

When I used Windows I set my swap min and max the same, because I don't want the overhead of Windows having to dynamically resize my swapfile on the fly. If I need more swap space I'll make it bigger myself, reboot and try again.

That doesn't work so well with XP. In fact there is no perceivable overhead whatsover. (relative to speed of machine) Even if there was some that was noticable, it would be a lot faster than having to readjust it manually and reboot.

developers are going to outright demand for slow disk memory, why is the concept of virtual memory used at all?

Why can't devs just say to Windows, "I need some memory, Windows. Give it to me however you want. I don't care.

They can, but--
because the programmers really don't have time to add code to automatically adjust for how much memory you have and optimize your program on the fly. It's much easier to use the swap file, which is limitless (as long as its dynamically controlled). That way they don't have to rewrite code for every configuration out there (which would be very hard to do.) And even if they did, it might not be as efficient as you may think.

Let's say you did have a program that used up your main memory and didn't use a swap file. What would happen if you loaded up IE, or Word, or something else? How much memory is that program going to have to give up and send to the swap file so another program can run? It doesn't know. It would be a giant guessing game. And in the end it would yield slower results, more tech calls, etc.

Actually noone should write a program that cares where the memory it's using came from, it should just ask for memory and let the OS do it's job from there

If I was writing a program for the general public, I sure wouldn't want to do that.

You mean 'Windows article', Linux runs just fine without swap space as long as you be carefull not to run out of memory, but even then all the kernel does is kill a few of your apps to make more room =)

But nobody runs Linux without a swap partition- for the same reason Windows swap file is dynamic. You never know when you'll need it.

 

[Oogle]

Because in Windows the swap file is used as a backing store for more things than you think. It's not just a repository for memory pages when physical memory is full. Windows VM subsystem was designed with the idea that a page file will always be present, if you try and run without one eventually it'll fall on it's face.

Actually noone should write a program that cares where the memory it's using came from, it should just ask for memory and let the OS do it's job from there.

Interesting. What about memory hogging processes that aren't part of the Windows subsystem (e.g. Photoshop, Mozilla, etc.)? If what you say about memory allocation is true (which I assume as well), why doesn't Windows tell these apps to use as much RAM as available and swap out later than sooner?

... Linux runs just fine without swap space as long as you be carefull not to run out of memory, but even then all the kernel does is kill a few of your apps to make more room =)

That's an interesting quirk. Is that by design? I can see how the kernel can delegate the process reaper if it needs the memory for itself. Just curious to know who commits suicide regular when multiple user-mode processes need the memory.

Get a book on OS internals. There's a Linux kernel book with a few very good chapters on memory management and there's an Inside Windows book that probably has the same.

I guess I'll have to. I was kind of hoping for a little free knowledge, though. I do understand a little bit about memory management and virtual paging in general. I just don't know where the necessity of using slower disk memory fits in when user-mode processes have so much fast memory available.

 

[Lorne]

In lantents, The swapfile is used for overflow and idle programs.
Dont forget that the texture appature is also stored in ram and when thats full the swapfile goes into affect so the more detailed your GFX are the faster it goes, The ram on the card itself is for only whats displayed at the time, Nothing else is.

I used to use the "ConservativeSwapfileUsage=1" in 98x sys.ini all the time and Ive also wondered if XP supports the same setting.


Unless you can run perfectly stable with it off I suggest not disabling the swapfile.

 

[sechs]

I'm going to type off the top of my head a bit, here, so forgive me if I'm wrong.

As I recall, programmers in Windows can set a tag in their code that says that certain bits of code can be swapped out to disk and some shouldn't. This is such that the programmer can say what is important and should stay in memory until it is really needed for something else, and what can be dumped. Windows takes a conservative approach to physical memory, swapping items out immediately, so as to leave as much physical memory open as possible. On NT OSes, your free memory is used by stuff such as the system cache to improve overall system performance. Doing this also prevents the sudden dump to disk when you run out of physical memory, basically freezing the computer until the process is done; those who used the conservative swap "tweak" in Win9x probably ran into this at least once.

In this way, the system and applications can stay active and available as much as possible, with as little impact as possible. While this always allows some free physical memory for opening the next application, it does leave you with a lot of stuff in you page file. For this, you must mostly blame the programmers of your applications.

 

[Oogle]

Let's say you did have a program that used up your main memory and didn't use a swap file. What would happen if you loaded up IE, or Word, or something else? How much memory is that program going to have to give up and send to the swap file so another program can run? It doesn't know. It would be a giant guessing game. And in the end it would yield slower results, more tech calls, etc.

Hmm.. I was just browsing through the Win32 API in the MSDN library. The situation you described should be solved easily by telling the OS, "Hey, don't swap me out completely! I want to make sure I have this much memory in RAM available to me so I can stay speedy. Feel free to swap all my other data out..." (See VirtualLock). Instead of guessing, the program can now let the OS know ahead of time how much RAM to reserve. I am almost positive Windows developers know about this API. So there should be no excuse to let the OS guess.

However, this doesn't solve my question. My question is why does the OS force user-mode processes to use the swap space so early when the system still has plenty of physical RAM available. You're talking about situations involving taxed memory environments. I'm talking about non-taxed environments with loads of RAM free. I'm trying to figure out why the OS gives user-mode non-system processes all that swap space.

 

[Dug]

Hmm.. I was just browsing through the Win32 API in the MSDN library. The situation you described should be solved easily by telling the OS, "Hey, don't swap me out completely! I want to make sure I have this much memory in RAM available to me so I can stay speedy. Feel free to swap all my other data out..." (See VirtualLock). Instead of guessing, the program can now let the OS know ahead of time how much RAM to reserve. I am almost positive Windows developers know about this API. So there should be no excuse to let the OS guess.

That is true. I'm oversimplifying in my example. Some programs do use up all the physical ram it needs and swaps out when another program interupts or the program needs more ram than available. But this is rare. Remember developers are trying to optimize for the lowest system requirements possible for a wide customer base.
Not only that but there are very few programs that will work better with more ram and then automatically optimize itself to run with less ram on the fly. Instead what would happen is you would have a huge performance hit as soon as that happened. So programs are meant to run very well with low system memory and a large swap file right from the start. (Usually a program will be doing something while its swapping) Just like a video card will be drawing something + swapping in new textures at the same time. So the performance difference between being all in physical memory or being in physical memory and using a swap file is negligable.

I'm not saying that programs wouldn't run faster if you could put them all in memory. (In fact I'm sure they would). It's just that most aren't that efficient. The trend is changing (especially in games) to use more memory though. Until developers are sure that everyone has 512k to 1g of ram, programs won't change too much.

However, this doesn't solve my question. My question is why does the OS force user-mode processes to use the swap space so early when the system still has plenty of physical RAM available.

It's usually not the OS that's doing the forcing.

 

[bsobel]

Hmm.. I was just browsing through the Win32 API in the MSDN library. The situation you described should be solved easily by telling the OS, "Hey, don't swap me out completely! I want to make sure I have this much memory in RAM available to me so I can stay speedy. Feel free to swap all my other data out..." (See VirtualLock). Instead of guessing, the program can now let the OS know ahead of time how much RAM to reserve. I am almost positive Windows developers know about this API. So there should be no excuse to let the OS guess.

Wrong wrong wrong wrong wrong wrong. Did I mention wrong. Virtuallock generally should not be used by most applications, there is usually no reason to force the pages to be held in physical memory and not be swappable. It's the job of the memory subsystem to make these decisions, not each developer (otherwise, I promise you EVERY application would simply try to lock everything into memory).

However, this doesn't solve my question. My question is why does the OS force user-mode processes to use the swap space so early when the system still has plenty of physical RAM available. You're talking about situations involving taxed memory environments. I'm talking about non-taxed environments with loads of RAM free. I'm trying to figure out why the OS gives user-mode non-system processes all that swap space.

Your not taking the file system cache into account (which the system also manages). Additionally, a lot of benchmarking has been done but the kernel team, they make design choices based on what *generally* provides the best peformance across many scenarios. You can certainly think of some where the decisions may appear suboptimal.

On another earlier point, someone made the comment about a swapfile sometimes being required and certain applications needing one. The issue is not that the application itself (per say) 'needs one', but rather the developer used certain API's (such as unnamed memory backed files) which are internally implemented via the swap file.

Bill

 

[Yamyam]

Um, so I think we're back to my original question. Since I've got plenty of physical RAM, and I'm not running a whole lot of RAM-intensive applications, wouldn't it make sense to just override this virtual memory merry-go-round by putting the swapfile on a RAMdisk? I paid for the RAM, and I'd kinda like to use it. I get kinda upset when Windows goes off and starts splattering swapfile data all over my hard drive even though I've got a perfectly good gigabyte of empty RAM, just sitting there and depreciating.

It seems like the only difficulty (aside from getting Windows to accept the RAMdisk) would be choosing how much memory to keep as "real" RAM, and how much to allocate as swapfile. The more I read about swapfiles, the more it seems like a rather silly solution to a problem that most users of modern PC's probably don't even have to worry about. Couldn't Windows just keep a registry flag somewhere, something like "Is this application a RAM hog?", and use swapfile space ONLY for those applications until the physical RAM is, say, at least 50% full?

 

[Dug]

wouldn't it make sense to just override this virtual memory merry-go-round by putting the swapfile on a RAMdisk? I paid for the RAM, and I'd kinda like to use it. I get kinda upset when Windows goes off and starts splattering swapfile data all over my hard drive even though I've got a perfectly good gigabyte of empty RAM, just sitting there and depreciating.

It makes sense in theory, but in reality you will need a swap file. Or I should say your programs will need the swap file. It's not necessarily up to Windows, its the app. that specifies that it wants to use it. And seperating your memory for that swap file would seem impossible without it being on a seperate bus.

 

[bsobel]

It makes sense in theory, but in reality you will need a swap file. Or I should say your programs will need the swap file. It's not necessarily up to Windows, its the app.

No, it's upto the OS, the application never specifies to use the swap file, it just turns out some API's require them.

that specifies that it wants to use it. And seperating your memory for that swap file would seem impossible without it being on a seperate bus.

Huh?

Bill

 

[Oogle]

Wrong wrong wrong wrong wrong wrong. Did I mention wrong. Virtuallock generally should not be used by most applications, there is usually no reason to force the pages to be held in physical memory and not be swappable. It's the job of the memory subsystem to make these decisions, not each developer (otherwise, I promise you EVERY application would simply try to lock everything into memory).

Ack... now I'm really confused about this issue. First I hear that devs explicitly control the memory to get the best performance. Next I hear that the OS is making all the memory decisions, not the apps. Bottom line, what pushes an app to use swap space, the OS or the applications themselves? Do developers force apps like Photoshop and Mozilla to use swap space, or is that the OS's job?

On another earlier point, someone made the comment about a swapfile sometimes being required and certain applications needing one. The issue is not that the application itself (per say) 'needs one', but rather the developer used certain API's (such as unnamed memory backed files) which are internally implemented via the swap file.

Ok, this is my understanding so far...
The memory manager will try to use RAM as much as possible and not swap out UNLESS
1. The subsystem wants to use the disk as memory to perform its operations.
2. Apps tell the OS that it wants to use the disk as memory.

What I don't understand is why. Fundamentally, the disk is supposed to slowly access unwieldy amounts of hardly used non-volatile information. Using the disk as memory defeats this purpose because
1. You're dealing with a manageable amount of data
2. You're quickly accessing and modifying data all the time
3. You're dealing with volatile information

The ONLY time I see an app using the disk as memory is when a system is stressed with multiple memory-hogging apps. In other words, when 1 or more of the above conditions fail. But, how can a system be "stressed" if several memory-hogging apps combined take only 1/4 of the total system memory(1 GB)?

Feeling tired now. Sorry if it feels like I'm ranting.

 

[sechs]

Originally posted by: Yamyam
Um, so I think we're back to my original question.

I think that everyone needs to start at the top of the thread and read all the way through, because someone pushed the "silly idea" button again.

 

[opk]

It seems like our friend here has a bad opinion of Microsoft's capabilities. Not saying much for the general sense, but the memory manager in NT isn't bad. The way I understood it, all the memory manager shows the world is a big ole pool of memory. Whenever you want some of it, you ask nicely, and it doles out some. You have no idea where it actually resides, what the CAS latency of it is, or what the available bandwidth on the bus is. All you know is that you got X pages of memory. You don't even really know what the real addresses are, just the virtual ones. The memory manager takes care of all the rest for you.

The theory behind the memory manager is he'll make everyone happy, right? Well, kernel processes are more happy, but that's beside the point. In theory, the manager looks at all his available fast RAM and doles it out first. He also keeps track of the least likely to be used pages so that if things get tight they're the first to get sent out to pasture. Pasture in this case being that achingly slow hard disk space.

Correct me if I'm wrong here, but if you decrease the prime land by reserving it for a RAMdisk, don't things get tight a whole lot sooner? Doesn't that mean the memory manager has a smaller realm of good lots? He's gonna get mad about that and have to do a lot more work putting perfectly good pages out to pasture. (The fact that this particular pasture happens to be a kludgey 'fast' pasture is lost on him.) You can't directly mount (access) pages out in pasture, however fast that is, without first moving them back into real RAM. That sounds like a whole lot of work shuffling things between parts of your land. You'd have to hire lots more cowboys and dogs and fences and the like. If it's all your land to start with, why fence some of it off for the explicit purpose of making things harder on yourself?

I don't know why the NT memory manager thinks some pages smell so bad that they need to be stored outside when the inside is so sparsely populated. Barring rewriting the manager-- remember linux this is not, shouldn't we just accept that the manager might have a good reason to throw a few pages outside too soon? Can't we assume that they're pretty useless pages to begin with?

BTW: to look into the guts of this 'problem?' you can monitor it with the XP Performance Monitor. Start -> Administrative Tools -> Performance. You'll be specifically interested in monitoring Memory->Pages/sec, Memory->Page Faults/sec & Memory->Cache Faults/sec.