high cpu usage

[rnj]

Hi all,

My cpu usage has been high enough to significantly impact simulations and other work on the computer. This is on several computers in our organization, it could be due to a security software issue. Basically the cpu usage remains high as if programs are being intercepted or audited. No single program uses significantly more than 1-2%, a large number of (normally idle) apps are doing this, total CPU use is high. The computer slows to a crawl and simultions take more than 5x normal speed, if they run at all. Standard apps usage becomes difficult.

It's been > 4 months, but the issue remains unresolved and not fully understood by our internal IT groups or security groups. The theory is that it is due to a mcaffee auditing process, although the right combination of settings, if this is the case, has not been found.

It does not appear immediately on-boot, but after several hours or days of use, (eerily similar to a memory leak). Memory use is reasonable.

First goal is to understand what could be causing this, one example shot is below,

https://prnt.sc/i897t6

 

[TheELF]

No single program uses significantly more than 1-2%, a large number of (normally idle) apps are doing this, total CPU use is high.

Usually if stuff that normally runs at 0% (less then 1% ) starts running at 1-2% it means that CPU clocks have dropped significantly.

 

[LTC8K6]

What do CPU temps look like when the slowdown occurs?
Maybe you are getting throttling from a wonky fan?

 

[XavierMace]

Can we get a full screenshot of task manager? Also, what does your disk latency look like?

 

[VirtualLarry]

If this is Windows 10, then... well, it could just be Windows 10.

But speaking more generally, it could well be coin-mining malware, or other malware.

Or it could be an "intrusive" AV application suite, which are often barely better than the viruses that they claim to protect against, as far as system load and compatibility issues.

 

[rnj]

this is on several laptops and desktops, same basic issue. Windows 7. Also, it started happening on all of them at roughly the same time. yes it could be a security software, we seek to find the correct settings which avoid this behavior.

another shot of the task manager - cpu. The problem actually exists right now, it's at a relatively low level.
https://prnt.sc/i8t4pe

resource mon
https://prnt.sc/i8t5z1

taskman processes
https://prnt.sc/i8xyoj

 

[VirtualLarry]

I was just talking to a friend of mine on the phone, and his Windows Security Essentials, on Win7 64-bit, had quarantined a coin-miner malware, just today.

 

[XavierMace]

Oh god, these systems look to have Bit9 on them. Cleanse them with fire. You've also got an assload of processes running. While McAfee's Enterprise A/V will beat your CPU to a pulp while it's running a scan, Bit9 has a habit of being a performance roadblock 100% of the time.

That said, elaborate on your issue. You're talking about high CPU but so far your only CPU graph (which you said the probably exists on) shows 35%. Are you actually getting high CPU usage or are you just calling it high CPU usage because things are slow?

 

[Ketchup]

Hit "show processes for all users" on task manager, sort by memory usage, and show or tell us what is at the top. What AV are you using?

What is the date of your last Windows Update (and what was the name of the update)?

 

[rnj]

in the first post, the snapshot of the task manager obviously doesn't show everything but the cpu is close to 100% occupied. i barely got the screenshot off.

in the second set of images, for which i said the problem is at a 'low level', the usage goes up and down , but yes averages ~40%, (yes there are a a bunch of apps open, but these would normally sit in memory and idle. there is no activity on the part of the user. Actually a lot of memory is occupied, Not that this isn't a problem in and of itself, but it's not a major roadblock, more could always be added). These machines run resource-intensive simulations. Yes, I acknowledge computers need to be secure, (if it is security software that is the cause), but I don't find this kind of perf. satisfactory . In this particular case, simulations run at >2x the duration than if the computer was 'idling properly'.

The first step is figuring out what exactly is causing the use, it could be that adjusting particular settings could help significantly. Unfortunately none of those programs you call out are likely to be removed, this being the standard build across the organization. Prior to the start of this issue, we used to at least have certain elements of bit9 installed without much problems.

This is another desktop, with I believe again a 'low level' version of the problem. Basically the computer is supposed to be idling, but a lot of apps show ~1% cpu usage and overall cpu is decently occupied with something..

https://prnt.sc/ia41a9

 

[Ketchup]

We aren't seeing everything here. For example, in your latest image, you only have 39 processes showing in the picture, and it appears to be sorted by CPU usage, lowest being on the top. So according to your pic, there are 83 processes running that we can't see. And presumably the ones with the higher CPU usage are on that lower end.

If I look at the task manager on my laptop right now, I can count numbers in the CPU column (with 'show usage for all users selected') and get the same number showing on the CPU usage graph. At the very least, you need to monitor one of these computers for a day (give them a spare) and see when/where the usage spikes. But I still would like to see what is at the top end of these graphs you are sharing, sorted with highest CPU usage at the top.

If getting extended access to the computer is not an option, run a performance monitor through a day of work and analyze the results. You cam PM me if you have questions about this.

 

[rnj]

here is another snapshot, during which a user is running something in matlab (one logical core's worth). A bunch of other apps are using the CPU. Though it's not a concern, procexp is using a decent chunk of CPU.

https://prnt.sc/ia7e5t

 

[Ketchup]

Might want to try Windows task manager and see if it defines the 30% at the bottom of your latest pic, as process explorer isn't catching it.

 

[rnj]

what are you referring to? If the totals in the CPU column are added up, (excluding system idle process), they are dead-on matching the cpu utilization of 69.23%, which is what I thought you were looking for in your previous post, with 100-69.23 = system idle process usage

 

[Ketchup]

Thanks. I see it now. So, Matlab is running here, which means that this computer at least isn't idle. Possibly the ap got on update? Not sure what else you are looking for here? You may want to look into CPU upgrades, not sure what you has now.

 

[rnj]

matlab is running because a user was running a simulation on there the second time I snapshotted. however the whole point of this thread, in a small sense at least, is captured in that screenshot. Basically a large percentage of the CPU is spent 'working' on apps that should be idling, compromising very significantly its use as a sim. machine. Even in the previous screenshot, when no users were running anything, the cpu was being used similarly (minus the work spent on matlab, procexp). And that is a screenshot of the problem at a 'low level'. At high levels there is no point in using the machine at all.

 

[Ketchup]

Without knowing the specs of the machine, it would be difficult to get you a good answer on that one. One thing that probably should be addressed, at least in your latest screenshot, is that the user is running both Norton and McAfree products on the same machine. While not huge on resources, they can slow things down when "double" scanning comes into play.

 

[XavierMace]

OP: So far I'm not seeing anything I'd call abnormal in your screenshots. You've got an unspecified quad core processor running multiple applications that perform tasks in the background. McAfee, Bit9, Citrix (if actively connected to an ICA session), Altiris, and Dameware. Running in the background doesn't mean it's not running at all. Right now your focus seems to be that all these processes "should" be showing 0%. In a perfect world that may be true, but I'd have to say in 2018, 4 cores isn't much for a sim machine. I know in the past a lot of Matlab's stuff wasn't multithreaded, but I don't know if that's still the case.

Without knowing the specs of the machine, it would be difficult to get you a good answer on that one. One thing that probably should be addressed, at least in your latest screenshot, is that the user is running both Norton and McAfree products on the same machine. While not huge on resources, they can slow things down when "double" scanning comes into play.

I see no indication of Norton on that last screenshot. Symantec does not automatically mean Norton.

 

[rnj]

The goal is to minimize the interference by adjusting some settings. I think we should definitely try to cut down 40-50% (this case) of cpu usage on nonrelevant apps. Essentially the same issue is on all workstations regardless of core count--I just have easy access to that one because it's more on the low-end as you point out.

Also this is the issue when it's relatively mild. The cpu usage in this way slowly creeps up, eventually to 90 - 100% (image linked in first post). That these processes should be showing zero is meant to be used as an investigative tool. (Should I rather just ignore this finding?) I just ran a case right now on a computer in a separate environment which lacks the apps you call out, with > 100 processes running and the idle average is < 5%.

Yes, there are functions in matlab that are multithreaded, Matlab is ~single-threaded in that case because of the way the user wrote the code. The user actually let me know this afternoon that the computer 'seemed slow' and asked if I was running something in the background (I hadn't even been logged in since a few days past.)

Again the overall goal here is to ameliorate a situation in which, besides the computer being slow to respond, more impactfully simulations are bogged down by 50 to 300+%. While I would love to remove all the apps you call out, the only thing feasible really is adjustment of those apps' settings, assuming they have sufficiently fine-grained configurability. Perhaps this is some standard technique being employed by security software..

 

[XavierMace]

Expecting all the apps running on your computer that you aren't actively using to be using 0% CPU is unrealistic, especially if it's a slower processor. Unless that other computer you tested on is identical save for the security software, it's a moot test. If you want to minimize your unneeded CPU usage, close the mountain of apps you have running in every single screenshot. Like I said before, you've got an assload of processes running in several of the screenshots. You've got multiple browsers open, most of the MS Office suite, etc. Trying to lower your idle cpu usage without ending any processes is a lost cause.

Security software is always going to have a performance impact.

 

[VirtualLarry]

Maybe a judicious application of the tool "Process Lasso" would help here, to give your simulations all necessary CPU time, when they're running, by elevating their process / thread priorities, above the background applications? Couldn't hurt to try. I don't know if that's pay-ware, but I've heard it mentioned.

 

[XavierMace]

Maybe a judicious application of the tool "Process Lasso" would help here, to give your simulations all necessary CPU time, when they're running, by elevating their process / thread priorities, above the background applications? Couldn't hurt to try. I don't know if that's pay-ware, but I've heard it mentioned.

Eh, it might help IF he can get it past Bit9 (which he probably can't) but it's not a magic wand. He's trying to take processes that are using 1% CPU each and drop them down to 0%. He'd be far better off closing applications that aren't in use.

 

[rnj]

the other computer ( averaging < 5 % ) is significantly weaker and older hardware-wise than the case snapshotted more recently. Also regarding this, it may seem like there is an assload of processes, but none of these are close-able. They are essentially 'background' processes, as excessive as they may seem. There are only a couple real apps open in that case. Also the very first screenshot has more apps, but yet a reasonable number of user-interactive apps open, ( couple notepad, calc, outlook, chrome, spaceclaim, powerpoint, an rdp session)

Again, all these apps, even after they've been used extensively, (If they are used at all, as many of them are as described before 'background'.), remain satisfactorily close to 0% for awhile. After some time, some start creeping up in %, then more, until they act ultimately, according to the task manager, in concert to choke most of the cpu, each of them using about the same % of CPU each. Again, the majority of CPU usage is from this behaviour.

Even if all user-closeable apps were closed too much problem would remain. (Also, clearly, useability decreases..) Low-level apps like notepad, have the same behaviour. I have 10 notepads open? Well expect a significant fraction of cpu use by notepad.exes after a few hours.

If it is related to security software, I guess I need to contact each of the vendors to figure out which one could be doing it. And, potentially, refine the settings on the software.

process elevation sounds interesting. Though, in the worse variants of this problem the computer isn't really functional.

 

[Ketchup]

Sounds like you need an (or another) IT guy in your organization. As stated, you have a LOT of processes running, there are a lot of details we don't know, and I wouldn't expect you to reveal them in an online forum.

 

[rnj]

The processes largely are easily deciphered by searching online. plus, the problem here is not so much the process count, which has stayed static before and after this problem, as mentioned previously. I'm not sure what other details youd need to know, or that I havent mentioned already.

As mentioned in the second paragraph of the first post in this thread, the problem is not fully understood by Information Services, or Security