Functional & Performance Testing RecommendationsΒΆ

Any new feature must also contain a test tool/PoC that demonstrates protection capabilities.

We cannot impose particular performance tests, or specific metrics regarding the performance of HVI. This has to be done on a case-by-case basis, with several factors in consideration:

  • What will be protected? Endpoints, servers, VDIs?
  • How many VMs/hosts will be protected?
  • What is the regular workload/purpose of the protected VMs?
  • What level of protection will be enabled for HVI? Will there be custom protection features?

Once these - at a minimum - questions are answered, performance can be assessed individually for each case. Generally, we recommend some tests to get a general picture about the impact of the Introspection module:

  • Application startup time - measure how long it takes an application to start, with and without HVI;
  • Browser performance - done in at least two ways:
    • measure the time taken to open URLs, to navigate through pages, etc.;
    • run a browser benchmark to assess the in-browser performance;
  • Micro-benchmarks - performed for each intercepted code flow:
  • process creation/termination;
  • changing memory permissions;
  • executing code from a data page;
  • loading/unloading modules;
  • Host performance - assess the overall host behavior when HVI is used (for example, the number of VMs that can be realistically used with vs. without HVI); in this regard, we recommend LoginVSI;
  • Specific benchmarks to assess various other aspects of the in-VM performance, such as:
    • unixbench;
    • phoronix;
  • General benchmarks which measure the hardware performance will usually yield an insignificant performance impact (for example, FPU performance, memory performance, etc.);

Any new feature will be subject to performance measurements before being accepted into introcore.