Question Virtual Memory?

[meeshu]

Running Windows 10 64 bit with 32 GB RAM, and with two SSD's at 1 TB each and a HDD at 8TB.

Video application installed on the system/boot SSD. On running this application, the application takes almost all of the system RAM.

While running this application, I noticed that my HDD was frequently being used, presumably as virtual memory(?)

The system/boot SSD has 685 GB free space. The second SSD, used for installing and running some applications and games, has 865 GB free space. The HDD is used for data storage only and has 6.95 TB free space.

Why is my computer (apparently) using the storage HDD for virtual memory, and not using the system/boot SSD which has plenty of spare capacity?

 

[sdifox]

You can control that

https://www.windowscentral.com/how-change-virtual-memory-size-windows-10

Also check the video app, it may have the temp folder set to the hhd.

 

[mikeymikec]

While running this application, I noticed that my HDD was frequently being used, presumably as virtual memory(?)

That is a rather large presumption and would constitute a pretty serious bug in Windows's code for pagefile management (to be using the slowest drive in the system by orders of magnitude for paging).

For example, my Win11 setup has a boot SSD with plenty of space and a data HDD and pagefile.sys is not on the root of D:\. I'd start with that basic check. To get a dependable result, I would open a command prompt and do DIR driveletter:\ /A (where 'driveletter' is the drive letter of your HDD), which will show all files and folders on the root of the drive. The typical hidden ones I'd expect on a non-system volume are $RECYCLE.BIN and "System Volume Information".

In very odd exceptional scenarios (e.g. IIRC if a user bans all pagefile usage and Windows is crossing its legs and saying "I really wanna go though!"), IIRC a 'swapfile.sys' file gets created elsewhere, maybe in C:\Windows?

I think the more likely explanation for unexpected HDD usage is for example that Windows decided to defrag the drive in the background or virus-scan it. When my system is booted into Win11 and it's allowed to run idle and switch the screen off, it's almost invariably the case that the HDD LED starts going nuts and CPU usage is evidently going up because fan noise becomes noticeable. Windows likes to find stuff to do. Another possibility is you've set one of your user's "special folders" (e.g. Documents / Pictures / Videos) to the HDD and Windows therefore wants to index it.

 

[meeshu]

Thanks for the comments.

I've had a look at the video application preferences. There doesn't seem to be a setting for location of temporary files listed, but there's a setting for cache which was set to the boot/system SSD anyway.

The system virtual memory was set to the system/boot SSD with the second SSD and the HDD unset. Setting for virtual memory using the system/boot SSD was at ~ 15 GB, I increased that to 64GB.

While doing test processing of a video using the video application, I had maybe five instances of File Explorer open with perhaps three of them referencing files on the HDD. My internet browser was open with perhaps around a dozen tabs open referencing various websites with the browser running from the system/boot SSD. All of these open applications would have taken up some of my system RAM.

While processing the video, the HDD was working away. But on closing the video application, the HDD was no longer working away.

I had trouble closing the video application as it seemed to have difficulty in closing, so I had to use the Task Manager to close the application. After this, I noticed that except for one instance of File Explorer window, all other instances were closed! Also all of the browser tabs had lost their website content (temporarily) and I had to refresh the website contents for each instance of browser tab.

Now guessing that the system was trying to maintain the contents of those instances of File Explorer referencing the HDD, but couldn't do so due to lack of system RAM (and possibly due to lack of virtual memory availability on the system/boot SSD) while the video application was running.

Bottom line is that it seems I'll need more RAM, which I'm planning to increase to 64GB as it seems that there was insufficient RAM (32GB) to run the video application, internet browser and File Explorer instances at the same time.

 

[sdifox]

if the app is not configured to use the hdd and the virtual memory is set to ssd, I don't see how your hdd is thrashing. Is the source files from the HDD?

 

[meeshu]

The source video was indeed from the HDD.

 

[sdifox]

The source video was indeed from the HDD.

LoL yeah then it would read the video. Why not move it to SSD?

 

[mikeymikec]

Thanks for the comments.

I've had a look at the video application preferences. There doesn't seem to be a setting for location of temporary files listed, but there's a setting for cache which was set to the boot/system SSD anyway.

Therefore the most likely place that the application would be storing temp files is in the user's temp folder (C:\users\username\appdata\local\temp).

If the app was running with elevated privs then *maybe* it would be storing temp files in C:\Windows\Temp (though I think for this to be the case the program would have to be a background service e.g. third party antivirus, running with system privs or something similar), but if it's just an app I'd expect it to be the first location I mentioned.

 

[sdifox]

Therefore the most likely place that the application would be storing temp files is in the user's temp folder (C:\users\username\appdata\local\temp).

If the app was running with elevated privs then *maybe* it would be storing temp files in C:\Windows\Temp (though I think for this to be the case the program would have to be a background service e.g. third party antivirus, running with system privs or something similar), but if it's just an app I'd expect it to be the first location I mentioned.

he said in post #6 the source is in the HDD xd

 

[mikeymikec]

he said in post #6 the source is in the HDD xd

Yeah, I know, I was just providing information about temp file storage.

 

[meeshu]

Yes. The source video was from the HDD.

In hindsight it would have been preferable to place the source video (or a copy of the source video) within either of the two SSD's for faster data reads.

There were no other (significant) applications running in the background like anti-virus etc. while processing the video.

But there seemed to be a memory issue due to lack of physical RAM. As mentioned previously, I am intending to increase system RAM from 32 GB to 64 GB. Funny thing is that the minimum RAM required for running the video application is 16GB while the recommended RAM is 32 GB which I already have. But despite the 32 GB RAM, I had memory problems and slow video processing issue.

 

[mikeymikec]

What video processing is this application doing exactly?

I'm just curious because while I don't do much in the way of video transformation work, ripping DVD/BR content to H.264 / H.265 has never in my experienced caused significantly elevated RAM usage.

It's not a topic that I regard myself to be particularly knowledgeable about, but my understanding always was that video processing is first and foremost hard on the CPU (or GPU if hardware acceleration is in use).

 

[sdifox]

Video processing is not memory hungry, it's CPU or GPU bound

 

[meeshu]

The application is After Effects and I was attempting to use Content Aware to remove a small area of text from a short video of 33 minutes as a test. Video is MP4 format.

In my case the memory usage after running AE jumped from around 25~30% to about 95+%. I don't recall CPU usage for certain but I think it did fluctuate somewhat and jumped from somewhere around 10~20% to maybe 80+% at times(?)

 

[In2Photos]

After Effects is more of a video editing software rather than a processing software. I suspect that the RAM usage is greatly affected by the resolution of the file you are editing as well as the length of the file. A 33 minute video is rather large. What happens to the RAM usage if you work on a smaller file?

Also, always work off of the SSD, then move your completed work to the HDD.

 

[sdifox]

Quick Tip: Allocating RAM in After Effects

Is your computer RAM optimized for After Effects? In this post, we share a simple tip to rev up your processing speeds in AE.

www.premiumbeat.com

 

[meeshu]

Thanks for the additional comments.

With 32GB RAM installed on my system, AE allocated 26GB for itself, leaving 6GB for other applications.

I've now installed 64GB of system RAM and did another AE Content Aware Fill on the test video. The test video clip was copied to my other SSD to maximize data read rates when processing the clip with AE.

The system RAM usage jumped from around 30% and slowly increased to around 86% before I terminated the test. CPU usage was varying somewhat from around 60% to nearly 100% at times.

With 64GB RAM installed, AE allocated 58GB for itself, leaving 6GB for other applications.

The analysing part of Content Aware Fill process still seemed to be slow and remained at 0% after waiting for about 15 minutes before terminating the process.

The test video is a random video downloaded from YouTube. Test video is 1280 x 720 @ 224MB.

Shorter video should process faster and should take less resources. I'll cut a small part of the test video for further testing.

 

[sdifox]

It will take as much ram as you let it.

Hardware Recommendations

Our workstations for Adobe After Effects are tested and optimized to give you the best performance and reliability. View our hardware recommendations.

www.pugetsystems.com

 

[mikeymikec]

I would allow 8GB for the rest of the system, assuming you're not heavily using it while processing video.

 

[meeshu]

Comments noted, thank you.

Yes, AE loves heaps of RAM, as I suspected all along.

 

[sdifox]

See if this applies to your use case. Me think you need to adjust your workflow.

https://www.reddit.com/r/VideoEditing/comments/zo4kzu/how_can_i_render_a_heavy_hour_long_video_full_of