SVM Windows OS Scanning Benchmarks

[#1]

Synopsis

This article provides general information on the scanning performance of the various scanning methods available in the Software Vulnerability Manager solution, and it provides more detailed and concrete information on the performance of SVM Agents for Windows OS (Windows executable) as measured with the SVM On-Prem Agent for Windows. 

While the tests used for benchmarking the performance of SVM scans, and in particular Agent scans, have been tested with the SVM On-Prem edition, the performance should be perceived widely to be the same with the SVM Cloud Edition, as the core code functionality and execution of both solutions is identical, and the only difference may come from the larger amounts of data processing and delays in network traffic outside the control of SVM or its Agents. 

SVM Scan Benchmarks w/ SVM 7.6.1.4 

The following benchmark results have been produced with SVM On-Prem RPM build 7.6.1.4and measured against a Client system with 4GB RAM and around 100 installed applications.

Scan Methods Performance 

1. Local Host Agent (installation: csia.exe -i - L) - Microsoft Official Update Server
 

Type 1 scan :	14 sec
Type 2 scan :	81 sec
Type 3 scan :	1220 sec

2. Network Appliance Agent (installation: csia.exe -i - L -A) - [WU = Microsoft Official Update Server]

Type 1 scan :	13 sec
Type 2 scan :	97 sec
Type 3 scan :	1320 sec

3. Localhost Scan with SVM Plugin for Internet Explorer - Microsoft Official Update Server

Type 2 scan :

38 sec

4. Remote Scan with SVM Plugin for Internet Explorer - Microsoft Official Update Server

Type 1 scan:	30 sec
Type 2 scan:	224 sec

SVM Agent [7.6.1.4] CPU Performance:

The CPU utilization is seen to remain below the 20% threshold for all tested scan types.  

Max CPU utilization at some point during the active test scan has reached the following:

SCAN TYPE	SVM Scan time	SVM version	CPU (Max Peaked)
1	14 sec	7.6.1.4	17%
2	81 sec	7.6.1.4	17%
3	1220 sec	7.6.1.4	17%

Agent Benchmarks w/ SVM 7.6.1.7

Agent version: 7.6.1.7

Tool Used: Process Explorer

Date of Testing: 31-05-2019

1. Local Host Agent - (Installation: csia.exe -i - L) - Microsoft Official Update Server

Type 2 Scan	4 min 25 sec
Type 2 scan(without windows update) :	1 min 9 sec

CPU utilization (Max peaked): 22%

Average CPU utilization: 5-10%

1.a) I/O information for Local Host Agent for Windows:

Tools used: Wireshark Network analyzer 

Test Client: 192.168.10.10 [70 applications installed on it]

SVM Server: 192.168.10.2 

Client	Client Port	Server	Server Port	Packets	Packets Client to Server	Packets Server to Client	Bytes	Bytes Client to Server	Bytes  Server to Client	Bits Client to server	Bits Server to Client	Duration (Seconds)	API Calls
192.168.10.10	65305	192.168.10.2	443	39	485	725860	3396	722464	2375	505298	11.27	rules
192.168.10.10	65305	192.168.10.2	443	28	15	12	15598	14469	1129	11505	897	10.053	data
192.168.10.10	65305	192.168.10.2	443	24	6	16	20939	799	20140	22634	817937	4.09	status

2. Network Appliance Agent (NAA) - (Installation: csia.exe -i - L -A) - Microsoft Official Update Server

Type 2 scan :

4 min 50 sec

• NAA CPU utilization (Max peaked): 40%
• NAA Average CPU utilization: 5-10%

3. I/O information for Agent scans with enabled "Stop Agent Polling" setting in SVM:

Test Client	Client Port	Server	Server Port	Packets	Packets Client to Server	Packets Server to Client	Bytes	Bytes Client to Server	Bytes Server to Client	Bits Client to Server	Bits Server to Client	API calls	Duration
192.168.10.10	60806	192.168.10.2	443	708	225	483	736k	13k	722k	9142	492k	rules	11.74
192.168.10.10	60908	192.168.10.2	443	96	51	45	71k	68k	2979	54k	2370	data	10.05
192.168.10.10	60914	192.168.10.2	443	8	5	3	1382	722	660	1576	1141	status	3.66

Additional Considerations

There are few things you need to take into additional account when considering the benchmark results: 

• The more applications are installed on a system, the longer a scan can take to analyze all files
  • For instance, Win 10 scanning takes a little longer because these builds contain twice as many system files by default than a Windows 8.1 build.
• Content inspection filters, Proxies, Firewall analysis, and other security solutions can delay scans. 
• SVM On-Prem server configuration for load balancing can have a direct effect on scanning times.