page file on SSD

[Nothinman]

Of course there is AWE, but that is something else and something a bit more special.

Also, AWE is physical memory only. Most people have access to more harddrive space than they have physical memory.

But the point is that with AWE a 32-bit app can indeed use more than 2G of VM without managing it's own scratch disk. PS only took that route because Mac OS' memory management was virtually non-existent and it carried over to the Windows port.

 

[Rubycon]

Not on Windows, MS specifically disallows that AFAIK. And those cheap flash drives are usually slower than a decent spindle drive so it would be worse anyway.

If a drive is readyboost ready it should be faster than a spindle drive.
Too bad they have it blocked. Of course if someone yanks the drive and the system is expecting it to be there that could be a problem!

 

[Voo]

They are not using memory pointers above 2G. They are resuing the address space (swapping data in/out). It is the reusing that Windows cannot provide.

But if you reuse the address space, you can have several values associated with one pointer. So how do you differentiate between the swapped out data at 0xdeadcafe (and what if you've swapped it out several times already?) and the data in memory? When do you swap the other one in, when not?
It's provably impossible to access more than 2^x addresses with x bit, without further data/information - which in the end is the same as using longer pointers.

 

[hanspeter]

But the point is that with AWE a 32-bit app can indeed use more than 2G of VM without managing it's own scratch disk. PS only took that route because Mac OS' memory management was virtually non-existent and it carried over to the Windows port.

The point is, most users don't have vast amount of physical memory. They have large harddrives.

But if you reuse the address space, you can have several values associated with one pointer. So how do you differentiate between the swapped out data at 0xdeadcafe (and what if you've swapped it out several times already?) and the data in memory? When do you swap the other one in, when not?
It's provably impossible to access more than 2^x addresses with x bit, without further data/information - which in the end is the same as using longer pointers.

It is not that hard to keep track of what data you have where. You just need to build a data structure for it.

Anyways, this wasn't a discussion about how easy or how hard it would be to implement. It was about a 32bit application being able to manipulate large amount of data (without large amount of physical memory/AWE).

 

[Nothinman]

The point is, most users don't have vast amount of physical memory. They have large harddrives.

Compared to when PS was designed to use that scratch drive they most certainly do have vast amounts of memory. Not enough to safely disable their pagefile except in very specific, niche circumstances, but enough to keep huge images completely in memory.

 

[hanspeter]

No, not all users have many gigabytes of physical memory. Especially in older days. PS does something that the OS cannot do. Is that so hard to accept?

 

[Nothinman]

No, not all users have many gigabytes of physical memory. Especially in older days. PS does something that the OS cannot do. Is that so hard to accept?

Memory is a lot cheaper than PS, if you paid for PS and didn't get yourself enough memory you deserve what you get. PS does something extremely retarded to work around the VM system already in place, is that so hard to except?

 

[hanspeter]

Is doesn't work around anything. Windows cannot do what PS does.

 

[Baasha]

When I first set up my current rig, I moved the pagefile.sys to my SpinPoint F3 and it is 12GB... same as my RAM.. My OS drive (2 SLC SSDs in RAID0) is 64GB (total) so I didn't want to hog up 12GB because of the pagefile.

What do you guys recommend? I move the pagefile back to the C: drive and shrink it to say 4GB? I have 12GB of RAM and want to maximize performance.

 

[deimos3428]

And because of how VM works every system is potentially memory deficient. You can try to curb it's use and sometimes control individual process' memory usage, but you can't account for everything that might happen. Leaving a pagefile there just in case is just smart.

Actually, I can and do account for everything that might happen. (That's my job. I'm a system administrator.)

As you said, a pagefile is smart if it costs you nothing. For home use where you don't really care about performance, I can see someone thinking it's no big deal. But in principle I'd argue it's really not the best practice. Let me explain why I think so.

At work I'm responsible for literally thousands of 24/7 production VMs. Everything is [over-]provisioned quite well, but occasionally there are memory leaks that crop up in the application code. We usually catch them before they hit production; sometimes we don't.

The best a pagefile might do in this scenario is give us a little more time before it too is consumed by the memory leak. It certainly doesn't fix the problem, it only delays the inevitable. That's bad in itself, but there's a far larger concern. Performance.

One of the things we provide is a rather large customer-facing service that must respond to requests in real time. The last thing I want is for one of those VMs to hit a pagefile if it's having a problem! That would immediately trash our SLAs as it continues attempting to serve requests. Several million customers could potentially notice, and there is a potential for a huge loss of revenue. Having a pagefile is definitely not cost-free.

In this scenario it is several orders of magnitude better performance-wise for a given VM to simply run out of memory and die off. That way it is taken out of the pool by the load balancer immediately, and the memory problem isn't hidden from us. Somebody pages me right away, I swear quite loudly, then get up and get to work. The problem gets resolved and documented and we roll out a new image with the fix.

This setup makes us very nimble. In fact, not only do we not have pagefiles -- we don't write to local disk at all. Live migration of a VM to another host for any sort of maintenance is a breeze.

So yeah, I've gotten in the habit of not relying on the pagefile at all, even if perhaps I could "get away" with it at times. To me it's effectively ignoring an underlying problem, and that's just a very bad habit for a sys admin. The only reason I'd enable a pagefile is if I *couldn't* fix the problem otherwise, and that has only happened once to date at home.

It was when I tried to run Civ V with only 4GB. The game only really needs about 800MB RAM, but it also wants to cache a huge amount of data in memory. For whatever reason, Windows would not free that memory cache like it should, at least not in a timely fashion. So lacking a pagefile, it would crash whenever the memory cache eventually filled up the RAM.

Ultimately the fix was to *temporarily* add a pagefile, which kept it alive but was horrible on performance. I kept it that way just long enough to get to the computer store and back with new DIMMs. Turns out it needs about 4.6GB before it runs out of things to memory cache. Memory is cheap; I'd much rather pay out some cash vs. page out some cache.

 

[john3850]

FYI, I recently had a static pagefile set to 2GB min/max (Win XP Pro). After reading this thread, I then let the system manage the file, and the size jumped to ~3300 max, (2MB min). Which is interesting, because I have 4GB RAM. No discernable difference in performance. This is on a spindle drve, btw.

FWIW, the only issue I had in the past with a small static page file (<500MB) was that the system threw a low virtual memory warning after running Doom 3. Setting it to ~1 GB made everything copacetic again.

In most games just edit the games ini. file if you wish to run a small swapfile

 

[Zebo]

snip
Obviously building an overcommited system from the start is dumb, but so is disabling the pagefile when it provides a safety net and costs you nothing.


.

Thats just not true or I'd leave it enabled. Even pro pagefile site I linked admitted there are some caching issues when a program is in the background for awhile which slow you down for putting it back into memory when accessing it again. Much of this is mitigated if one keeps page file on SSD since they are so fast but I still notice, but nothing like old days. Either way there is a penalty and one has to decide if all the other negatives you correctly mention are worth restoring that application to be instant.

 

[Nothinman]

As you said, a pagefile is smart if it costs you nothing. For home use where you don't really care about performance, I can see someone thinking it's no big deal. But in principle I'd argue it's really not the best practice. Let me explain why I think so.

But we're talking about home use and I believe the main concerns here were conserving space and SSD write cycles. All of that about VMware overcommit doesn't even remotely apply.

 

[Zebo]

When I first set up my current rig, I moved the pagefile.sys to my SpinPoint F3 and it is 12GB... same as my RAM.. My OS drive (2 SLC SSDs in RAID0) is 64GB (total) so I didn't want to hog up 12GB because of the pagefile.

What do you guys recommend? I move the pagefile back to the C: drive and shrink it to say 4GB? I have 12GB of RAM and want to maximize performance.

ALWAYS keep pagefile on SSD if you have the space since its fast. Don't believe the wearing out issues, it's FUD - Anand's done the math and it's hundreds of years.

Edit linked
http://www.anandtech.com/show/4159/ocz-vertex-3-pro-preview-the-first-sf2500-ssd/2

 

[flamenko]

ALWAYS keep pagefile on SSD of you have the space since its fast. Don't believe the wearing out issues, it's FUD - Anand's done the math and it's hundreds of years.

http://www.anandtech.com/show/4159/ocz-vertex-3-pro-preview-the-first-sf2500-ssd/2

This should be good.

You are very forthright in your point so... why are you exacly keeping pagefile on again? Lets take it one step at a time.

 

[Zebo]

I don't keep one.

 

[flamenko]

LMAO.... I just went back becaue I thought exactly as you just said. NM eheheh.

 

[Zebo]

He wants one I just suggested best place for it, not a slow mechanical HDD thats for sure. I personally disable them but it's not recommend for most. See page I linked on PP2 goes through it all.

 

[flamenko]

Ya....been through it. The abs best debates I have had over the past few years were 64 bit 3 years ago and also Pagefile. 64bit is not much of a debate any longer but Pagefile still remains to be very....lively.

 

[deimos3428]

But we're talking about home use and I believe the main concerns here were conserving space and SSD write cycles. All of that about VMware overcommit doesn't even remotely apply.

Who said anything about VMware? Not I; don't know where you got that from. (Not that it's relevant, but we actually run Debian Lenny/Squeeze under Xen.)

The point being the very same principles of system management that apply in the enterprise are equally true at home. If you want to brush them off as unimportant in the home environment be my guest, but that doesn't make ignoring them sound advice.

As for putting a pagefile on an SSD, Zebo is right that it's faster than on an HDD but it's also a rather expensive use of a limited resource. If someone insisted on having a fast pagefile for whatever reason, you could put one on a ramdisk. Personally I think that's absurd, but it's much faster and cheaper than the same space on an SSD.