Functional & Performance Testing RecommendationsΒΆ
Any new feature must also contain a test tool/PoC that demonstrates protection capabilities.
We cannot impose particular performance tests, or specific metrics regarding the performance of HVI. This has to be done on a case-by-case basis, with several factors in consideration:
- What will be protected? Endpoints, servers, VDIs?
- How many VMs/hosts will be protected?
- What is the regular workload/purpose of the protected VMs?
- What level of protection will be enabled for HVI? Will there be custom protection features?
Once these - at a minimum - questions are answered, performance can be assessed individually for each case. Generally, we recommend some tests to get a general picture about the impact of the Introspection module:
- Application startup time - measure how long it takes an application to start, with and without HVI;
- Browser performance - done in at least two ways:
- measure the time taken to open URLs, to navigate through pages, etc.;
- run a browser benchmark to assess the in-browser performance;
- Micro-benchmarks - performed for each intercepted code flow:
- process creation/termination;
- changing memory permissions;
- executing code from a data page;
- loading/unloading modules;
- Host performance - assess the overall host behavior when HVI is used (for example, the number of VMs that can be realistically used with vs. without HVI); in this regard, we recommend LoginVSI;
- Specific benchmarks to assess various other aspects of the in-VM performance, such as:
- unixbench;
- phoronix;
- General benchmarks which measure the hardware performance will usually yield an insignificant performance impact (for example, FPU performance, memory performance, etc.);
Any new feature will be subject to performance measurements before being accepted into introcore.