Development guideline¶
Developing New Protection Features Guideline¶
- Design the new protection feature:
- Usually a small design document, justifying the need for the feature, is required;
- Make sure the feature is feasible - dwelling too deep into undocumented OS structures or intercepting very commonly accessed structures may make the feature unworthy of the performance impact or maintenance complexity;
- Implement the new protection feature in Introcore:
- Add new hooks and handlers, if required; Usually, the protection feature will require:
- activation logic (enabling the feature - finding structures in memory, placing memory hooks, etc.);
- deactivation logic (disabling the feature - removing any hook set on protected structures);
- handling logic (actually handling the events that are used to identify potential attacks - EPT violations, etc.);
- Add new global/per process protection flags, if required: one must be able to enable/disable the new protection feature via dedicated new flags inside the global Introspection options or the per-process flags;
- Add new alert type or extend existing alert types, if required; protection features will either use existing violation structures, or define new ones;
- Add new hooks and handlers, if required; Usually, the protection feature will require:
- Implement test tools for the new protection feature:
- Test as many attack scenarios as possible;
- Don’t be cheap on scenarios! For example, when testing a memory write, make sure you have as many scenarios as possible: page boundary writes, writes using various sizes (1, 2, 4, 8 bytes), writes using different instructions (MOV, ADD, XOR, MOVNTDQ, etc.), writes using implicit memory accesses (for example, by stack pushes issued by CALL instructions), etc.
- Implement performance testing for the new feature:
- If a new hook is placed, this will inevitably cause performance impact; asses this impact!
- If possible, implement a micro-benchmark for the feature. For example, if the new feature involves intercepting execution of a certain API, asses by calling the involved API and observing how much of an impact Introcore brings!
- Asses the viability of the new protection feature:
- Performance, in real-life scenarios:
- Is the performance impact obvious?
- Is any workload impacted?
- Is the overall behavior of the VM affected?
- False-positives:
- Does the new feature trigger false-positives?
- Are they triggered by natural OS behavior?
- Are they triggered by shady software?
- Actual Protection:
- Are there actual attacks that could be prevented using this feature?
- Is it feasible to employ such an attack?
- Performance, in real-life scenarios:
- Document the new feature
- Overall functionality and the use-cases (the design document, which will be part of this documentation, Development Guidelines chapter);
- The new hooks/handlers, if any (source code must be documented using Doxygen);
- The new flags;
- The new alerts;
New feature design document template¶
When designing a new feature, the following template can be used to answer the most important questions regarding that feature. Any pull-request for a new feature must include answers for all of these questions.
Question | Answer |
---|---|
What will the new feature protect? | Detail the structure(s) that will be protected by this feature. Example: The feature will protect the kernel exports directory (EAT) against reads. |
How will the protection be made? | Describe how the feature will achieve protection, technically. Example: The feature will place an EPT read hook on the kernel exports directory. |
Why is the feature required? | Motivate the decision of implementing the new feature, by explaining it’s value. Example: When exploiting a kernel vulnerability, the shellcode that gets code execution will need to locate the address of various kernel functions to work properly. This is usually achieved by directly reading and parsing the exports section of the kernel image. |
Are there any real attacks leveraging this? | List the attacks that are known to use this technique. Don’t be cheap - the more attacks there are, the faster the feature will be merged! Example: Attacks such as EternalBlue or BlueKeep contain shellcode that once they have gained execution, will try to locate the address of important kernel functions by using the described method. |
How likely is for false-positives to appear? | Justify, with concrete examples, the likelihood of false-positives. List any software that you know triggers this behavior and must be whitelisted. Example: Regular legitimate kernel code will locate needed functions via the implicit loader during image load, or via known API such as MmGetSystemRoutineAddress. It is unlikely that benign components will trigger this behavior, but security drivers (AV, DLP, etc.) might trigger this. |
What is the projected performance impact? | Justify the performance impact with numbers. Example: The kernel EAT is read on each driver load. On a system with 150 loaded drivers, 150000 EPT read violations were triggered due to the normal kernel loader operations. Additional read accesses inside the NT EAT are generated only when drivers are loaded - on average, 1000 reads per loaded driver. |
Important terms¶
To better understand the following sections (and several terms used throughout the source code), several important terms must be clarified and defined.
- Guest Virtual Address - a guest virtual address is a regular virtual address used by applications (or the kernel inside the guest). The term is used interchangeably throughout this documentation and the Introcore source-code with guest linear address (normally, the guest linear address is obtained once segmentation is applied to the guest virtual address). You will most often see these abbreviated: GVA - Guest Virtual Address, GLA - Guest Linear Address.
- Guest Physical Address - a guest physical address is obtained by translating a GLA using the in-guest page-tables. Guest physical address or GPA for short are then translated using the Extended Page Table to a host physical address.
- Host Physical Address - the actual physical address that gets accessed, once the GPA is translated using the EPT.
- Extended Page Table - the extended page table, or EPT, is the second-level address translation mechanism implemented by Intel CPUs. It is configured and controlled by the hypervisor, and it is also used by Introcore to enforce access restrictions to certain GPA pages inside the guest.
- The translation mechanism - the usual translation mechanisms look like this: GVA/GLA → (Guest page-table) → GPA → (EPT) → HPA.
- Page Table - this term is used to refer to physical pages that are used by the CPU to translate GLA to GPA. Usually, when used independently, the term refers to any level of the translation (PML5, PML4, PDP, PDP, PT), but when used in explicit sentences involving other layers of translation as well, Page Table or PT for short will refer to the last level of translation.
- Frame Number - hypervisors (especially Xen) will usually work with frame numbers. Usually, this term refers to the guest physical page number, which is basically the GPA shifted 12 positions to the right.
- #PF, #UD, #VE, etc.- throughout the Introcore source code and this documentation, x86 exceptions will usually be referred to using the Intel abbreviation, as defined in the Intel SDM.
- WORD, DWORD, QWORD, etc. - Introcore and this documentation use the Intel definition of these data types: WORD = 2 bytes, DWORD = 4 bytes, QWORD = 8 bytes.
- Hook vs. Detour - generally, Introcore defines a hook as being a memory hook placed using the EPT - for example, “page X is write-hooked” means that the GPA page X translates to is marked non-writable inside the EPT. When referring to regular API hooks, the term detour is used, and it represents regular API hooks, but which will usually trigger a VM-exit.
Overall internal architecture¶
The overall internal Introcore architecture is built around
the gGuest
structure. The gGuest
describes a guest VM, and
it contains most of its state and information. In addition to the
gGuest
, there is also the gVcpu
. This will always point
to the current VCPU, which generated the event that is currently being
handled. When an EPT violation takes place, Introcore will make
gVcpu
point to the actual VCPU that triggered the event.
The main entry point of the introspection engine is the
IntNewGuestNotification
- this API tells Introcore that a new
guest VM can be introspected. Once the guest is introspected and the
protection is activated, the main entry points inside the Introcore are
the callbacks located inside callbacks.c file. These callbacks are
called by the integrator whenever the designated event took place. There
is a separate callback for every type of event that can reported (EPT,
MSR, CR, etc.), plus the special timer callback that gets called once
every second. In addition to the callbacks, introapi.c also contains
several functions exposed via the glue interface, functions that can be
called by the integrator on demand. Example of such a function is the
agents injection API.
When the guest is shutting down, or the integrator wishes to explicitly
disable protection, the IntDisableIntro
API can be used. The
introspection will not be unloaded immediately - it may take some seconds
until it is ready to actually stop protection (because agents may need to
be removed from the guest, or RIPs still point inside code that must be
removed from the guest).
During the lifetime of the Introspection protection, alerts can be
generated. The alert types are split in two main categories:
violations, which indicate
a potential attack (for example, EPT violations, memcopy violations, etc.),
and events, which indicate
informative guest activity (for example, process creation/termination,
module load/unload, etc.). The alerts are always generated in an
asynchronous manner - they can originate only from a callback (EPT
violation, timer, etc.). In addition, during the lifetime of the
Introspection protection, the protection options can be changed via
the IntModifyDynamicOptions
API, and processes can be added to or
removed from protection (even if they are already running).
Once Introcore identifies the guest OS type and version, it will immediately place various hooks (API hooks - detours, EPT hooks, MSR hooks, etc.). The hooks have three main purposes:
- They notify about the guest activity - for example, hooks may be placed on internal process management functions to determine process creation and termination.
- They offer protection for sensitive resources - for example, an EPT write hook will be placed on the kernel image.
- Optimizations - some hooks may be placed (although this may be counter-intuitive) to accelerate Introspection processes - for example, certain instructions may be instrumented and modified to avoid performance impact.
Protection activation flow¶
The main initialization flow can be summarized in the flow chart below:
Note that arrows represent regular function calls, whereas dashed lines
represent asynchronous calls of a registered callback. For example,
the Set a CR3 write hook will place a CR3 write hook, and then
the IntNewGuestNotificationCallback
will return. At some point,
when the CR3 register is written, the CR3 write callback will be
called, the write handled, and initialization will continue.
Setting memory hooks¶
Memory hooks are placed using the EPT (Extended Page Table) functionality exposed by the hypervisor, and they are perhaps the most important mechanism in all of HVI. Most of the protection provided by Introcore relies on this system, so it’s important to gain a good grasp of what it is and how it is used.
A memory hook can be set for any access: read, write or execute. When placing hooks on regions of memory, only one access type can be specified; if one wishes to place a hook for read, write and execute accesses, he has to place three independent hooks, for each access type.
A memory hook can be placed for both physical pages or virtual pages. Normally, the physical page hooks should only be used for hooking page-tables.
Memory hooks operate with the granularity of the hardware memory page. This means that the minimum size of any hook will be a multiple of the minimum supported hardware page, which is usually 4K. Placing a write hook for a 1 byte region would inevitably monitor the entire 4K page containing that byte, so VM exits will be triggered for any write access inside that page (the registered callback, however, will be called only when that particular byte is accessed; accesses made outside that 1 byte region will be discarded). Note however that Intel added support for sub-page permissions technology, which allows the hypervisor to configure write access for 128 byte regions of memory (sub-pages). This technology, however, is not available on older CPUs (older than Ice Lake), and only works for write accesses. In addition, the CPU may still generate EPT violations for accesses outside the designated 128 bytes region, in certain scenarios. In addition, HV that choose to use large EPT pages (2M, 1G) will have to split them into 4K pages, in order to ensure good performance.
Guest physical memory hooks¶
These hooks are set using the API provided in hook_gpa.h. Such
hooks must be established directly on physical pages only when the
caller is sure that the purpose of those physical pages will not change
(for example, they will not be swapped or remapped). It makes little
sense to use GPA hooks directly on other regions of memory except for
page-tables. The IntHookGpaSetHook
function is used to place a GPA
hook. In order to remove a GPA hook, use IntHookGpaRemoveHook
.
Guest page-table hooks¶
This type of hooks is used when the caller wishes to intercept all
levels of translation for a given guest linear address. Once such a hook
is placed, a designated callback will be called every time that
translation is changed. Internally, this system uses the guest physical
memory hooks. In order to start monitoring the translation of a linear
address, use IntHookPtsSetHook
, and for removing such a hook
use IntHookPtsRemoveHook
.
Note
When placing guest page-table hooks, there are two types of EPT violations that can be generated on them: memory manager induced ones (explicit writes made by the kernel) and page-walker induced ones (when the CPU sets the A/D bits). Because the A/D bits events generated by the CPU page-walker are not used and are a significant source of performance overhead, they are discarded at the HV level, so the Introspection engines does not rely on them. Consider that any integration of HVI treats these A/D writes as “implementation specific” and don’t rely on them being notified to Introcore.
Note
The A/D write events may be generated for any page-walk at any
level, even if only a single level of page-tables doesn’t already have
them set (for example, if PML4.A == 1
, PDP.A == 1
,
PD.A == 1
, PT.A == 0
, the CPU may generate an EPT violation
for all - PML4, PDP, PD and PT, even if A bit is already set in the first three levels).
Note
When placing page-table hooks, one must be aware that it may race with the guest. In this regard, a page-table entry must not be read/accessed before the page-table itself is hooked inside EPT (this applies to the page-table hooks system, not to regular memory accesses that require translations, but one must be aware that it may happen there as well). This way, once an entry is fetched, knowing that the PT is hooked in EPT, any modifications made to it will trap to Introcore, which will naturally handle the modification.
Guest virtual memory hooks¶
This is the most common type of hook used. Internally, it makes use of
the guest page-table hooks and the guest physical memory hooks mechanism
in order to provide an abstract interface capable of monitoring linear
addresses without worrying about translation changes. When protecting
operating system or application pages, this system will be used. In
order to place a virtual memory hook for a given region,
use IntHookGvaSetHook
. In order to remove such a hook,
use IntHookGvaRemoveHook
.
Hook objects¶
The already mentioned hook systems (guest physical memory hooks, guest
page-tables hooks and guest virtual memory hooks) can be used for a
single page at a time. In order to provide a method for placing a memory
hook for larger regions of memory, the hook objects were created. A hook
object contains multiple regions (a region is defined as a
contiguous region of linear guest memory), and each region can contain
multiple pages. When operating with hook objects, you first need to
create the object using the IntHookObjectCreate
function. Once an
object is created, you can use IntHookObjectHookRegion
to hook
multiple contiguous regions of memory or IntHookObjectRemoveRegion
to remove hooks from previously protected
regions. IntHookObjectDestroy
can finally be used to destroy an
object once it’s no longer needed (note that this function will also
remove all remaining regions).
The Virtual Address Space Monitor¶
The virtual address space monitor is a special hooks system that allows
the caller to monitor an entire virtual address space against
modifications. All page-table writes will be reported via the provided
callback. As this function induces a significant performance overhead,
use it with great caution. In order to start monitoring an address
space, use the IntVasStartMonitorVaSpace
function. In order to
stop monitoring an address space, use IntVasStopMonitorVaSpace
.
Internally, this system uses guest physical pages hooks in order to monitor
the guest page tables. To simply monitor select virtual address, please use
the guest page-table hooks (IntHookPtsSetHook
and IntHookPtsRemoveHook
)
functions instead.
The Unpacker¶
HVI provides a simple module aimed towards detecting unpacked/decrypted
code in memory. The relevant functions are located in the unpacker
module. The caller can use the IntUnpWatchPage
function to start
watching a virtual page for unpack/decryption events. It can then
use IntUnpUnWatchPage
to stop watching a page for
unpack/decryption. This module uses internally the guest virtual memory
hooks, so they are set per-process. The algorithm used by this module
is a simple execute-after-write algorithm: executing code from a
previously modified page will trigger the unpack/decryption detection,
and the callback passed to the IntUnpWatchPage
will be called.
The Integrity Mechanism¶
There are structures which cannot be protected using the regular EPT
protection mechanism (please see above an explanation why). To provide
some level of protection on these structures, one can use the integrity
mechanism. This is a simple mechanism that computes an initial hash on
the designated structure, and then periodically re-calculates that hash
in guest, in order to see if it changed. Changes to the structure will
induce changes to the hash, thus triggering an integrity violation. The
caller can use the IntIntegrityAddRegion
function to add a region
to be integrity protected, and it can use IntIntegrityRemoveRegion
to remove it. The integrity checks are performed on the timer callback,
which is normally called once every second.
Setting register hooks¶
Hooks can be placed on different types of registers, which can be summarized on the table bellow (for additional info about the API please consult the Doxygen documentation):
Register Type | Set hook function | Remove hook function | Notes |
---|---|---|---|
Descriptor Table Registers | IntHookDtrSetHook |
IntHookDtrRemoveHook |
Places hooks on IDTR and/or GDTR. Note that some Hypervisors may not support this - check IG_QUERY_INFO_CLASS_DTR_SUPPORT before trying to set this type of hook |
Control registers | IntHookCrSetHook |
IntHookCrRemoveHook |
Currently, Cr3 is intercepted only for initialization purposes. Keeping the Cr3 write hook may induce a significant performance overhead. |
Model Specific Registers | IntHookMsrSetHook |
IntHookMsrRemoveHook |
The hypervisor may choose to allow read/write access only for certain MSRs. The SYSCALL/SYSENTER MSRs can be intercepted. |
Extended Control Registers | IntHookXcrSetHook |
IntHookXcrRemoveHook |
XSETBV in essence. Used for initialization purposes only. |
Setting API hooks¶
To intercept certain guest operations, VMEXIT events may not be enough. For example, if we want to know when a new process is inserted inside the guest process list. For these cases, Introcore is able to hook guest functions. This is done in a pretty standard way: the first few instructions in a hooked function are replaced with a jump to a memory zone that contains code controlled by us (the in-guest detour handler). The in-guest handler can notify Introcore about the event by issuing a hypercall. It can also do pre-processing on the event, apply filters, or change the behavior of the guest without notifying Introcore. Detours are OS-dependent and can be set in only kernel space.
Windows detours¶
Windows hook descriptors¶
Each detour is described by a hook descriptor. These can be found in the winhkhnd.h header. An exhaustive documentation of a hook descriptor can be found by consulting the Doxygen documentation for the API_HOOK_DESCRIPTOR structure. From a high-level point of view, a descriptor contains information about:
- The kernel module that contains the hooked function - this can be any kernel module, but currently hooks are only set inside ntoskrnl.exe.
- The name of the hooked function - for exported functions this name is used to find the function inside the module, for functions that are not exported this is simply an identifier that links it against the proper CAMI function pattern.
- The OS versions for which this hook is available.
- A list of handlers that will be injected inside the guest.
- A description of the arguments passed when the in-guest handler issues a hypercall.
- A list patterns that are used to find functions that are not exported.
- Introcore options for which the hook must be enabled or disabled.
- A callback that will handle the hypercall.
- Optional pre- and post-hook handlers that can be used to customize the detour before or after it is written inside the guest.
During initialization, the IntWinApiHookAll
function iterates over
all the available hook descriptors and sets the needed hooks. If setting
a critical hook fails, Introcore is stopped. Hooks are removed when
Introcore stops by the IntDetUninit
function, or when a
Detour callback requests this.
Windows Hook handlers¶
Each descriptor must have at least one hook handler. These are also found in the winhkhnd.h header. An exhaustive documentation of a hook handler can be found by consulting the Doxygen documentation for the API_HOOK_HANDLER structure. Only one handler is injected for each hook. From a high-level point of view, a hook handler contains information about:
- The OS versions for which this handler is available - this is used to select the handler that will be injected: the first handler that matches the current OS version is selected.
- The code that will be injected inside the guest - this is a small assembly stub to which the hooked function will jump to. It must do any pre-processing and filtering, then issue a hypercall, if needed.
- The type of the hypercall used - currently, three types are supported: no hypercall, INT3, VMCALL. A handler can not issue more than one hypercall.
The handlers and the changes done to the hooked function are not visible to the guest.
Windows Detour callbacks¶
This is the function that will be invoked when the in-guest handler issues a hypercall. It can inspect and analyze the guest state and handle the event in any way it sees fit. It can even change the guest state to modify the guest behavior. For example, an action can be denied by modifying the function return value. These changes must be synchronized with the in-guest handler. Guidelines outlined in Accessing guest memory and Accessing guest state must be followed - the guest state must not be trusted and validation is encouraged. These callbacks can disable or entirely remove the hook.
Pre and Post callbacks¶
The definitions in winhkhnd.h are static, but sometimes hooks must be customized with information available only after a guest is started (for example, with the offset of a field inside a structure). For this, there are two optional callbacks that can be set: pre-hook and post-hook. The pre-hook callback is called by Introcore before the function is hooked and the detour handler is written inside the guest. This callback can change the detour handler. For easier changes, each handler can expose a series of public data offsets, which identifies offsets inside the handler by name. The pre-hook callback can also instruct Introcore to skip the hook. For similar reasons, there is a possibility of supplying a post-hook callback that is invoked after the hook is written inside the guest.
Linux detours¶
Linux Hook descriptors¶
Each hook is described by a hook descriptor. These can be found in the lixapi.c file. An exhaustive documentation of a hook descriptor can be found by consulting the Doxygen documentation for the LIX_FN_DETOUR structure. From a high-level point of view, a descriptor contains information about:
- The name of the hooked function - this name is used to find the function using
kallsym
. - The name of the hijack function - this is the name of a function-call from previously specified function; used if a function-call from the previously specified function is hooked.
- Introcore options for which the hook must be enabled or disabled.
- A callback that will handle the hypercall.
During initialization, the IntLixApiHookAll
function iterates over
all the available hook descriptors and sets the needed hooks. If setting
a critical hook fails, Introcore is stopped. Hooks are removed when
Introcore stops by the IntDetUninit
function, or when a
Detour callback requests this.
Linux Hook handlers¶
Each descriptor has a corresponding LIX_GUEST_DETOUR structure that is located in guest virtual memory. The LIX_GUEST_DETOUR contains the information about:
- The name of the detoured function name.
- The name of the detour hijack function name, if any.
- The address of the handler code.
- The address where the handler must jump back when it finished the execution (the original function).
- Introcore options for which the hook must be enabled or disabled.
To add a new entry in the array of LIX_GUEST_DETOUR, the
init_detour_field(<function-name>)
/init_detour_hijack_field(<function-name>, <hijack-function-name>)
macros must be used. The detour handlers require that the name of handler-function to be exported; to export
the name of the handler-function the following macros must be used:
def_detour_asm_vars(<function-name>)
/def_detour_hijack_asm\_vars(<function-name>, <hijack-function-name>)
and def_detour_vars(<function-name>)
/def_detour_hijack_vars(<function-name>, <hijack-function-name>
).
The hook handlers are made up of a trampoline and handler function.
- The handler-function is a piece of code written in C.
- The trampoline is a piece of code written in assembly that calls the
handler-function; the trampoline could be one of the following:
def_tramp <function-name> <function-name>_jmp
: this trampoline calls only the function-handler (calls the handler-function that has the<function-name>
).def_tramp_fn <function-name> <handler-function> <function-name>_jmp
: this trampoline calls only the function-handler (calls the<handler-function>
).def_tramp_skip <function_name> <function_name>_jmp
: this trampoline is used to block a function by returning an error (calls the handler-function that has the<function-name>
).def_tramp_skip_fn <function_name> <handler-function> <function_name>_jmp
: this trampoline is used to block a function by returning an error (calls the<handler-function>
).def_tramp_ret <function-name> <number of saved params> <function-name>_jmp
: this trampoline is used for return hooks; the hypercall is generated when the function returns (calls the handler-function that has the<function-name>
).def_tramp_ret_fn <function-name> <handler-function> <number of saved params> <function-name>_jmp
: this trampoline is used for return hooks; the hypercall is generated when the function returns (calls the<handler-function>
).
Linux Detour callback¶
This is the function that will be invoked when the in-guest handler issues a hypercall. It can inspect and analyze the guest state and handle the event in any way it sees fit. It can even change the guest state to modify the guest behavior. For example, an action can be denied by modifying the function return value. These changes must be synchronized with the in-guest handler. Guidelines outlined in Accessing guest memory and Accessing guest state must be followed - the guest state must not be trusted and validation is encouraged. These callbacks can disable or entirely remove the hook.
Reading and changing arguments¶
Detour arguments are described by CAMI.
Accessing them is done with the IntDetGetArguments
function which will return a list of 64-bit
integer values. The meaning of these values is unique to each detour handler.
Argument values can be changed using IntDetPatchArgument
.
Accessing guest memory¶
Accessing guest memory is a core functionality of any introspection engine, and it can be done in several ways.
Mapping guest physical memory¶
The fastest way to access guest memory is by mapping guest physical
pages using IntPhysMemMapToHost
. These represent what guests thinks
are physical pages, so no translation is required. Usually, these
functions are used to access guest page tables (to translate guest
linear to guest physical addresses) or to access pages that have already
been translated. After a GPA has been mapped using IntPhysMemMapToHost
it must be unmapped using IntPhysMemUnmap
.
Translating guest virtual memory to guest physical memory¶
Accessing guest linear memory using IntPhysMemMap
requires the
caller to translate those guest linear addresses to guest physical
address beforehand. This can be done using IntTranslateVirtualAddress
and IntTranslateVirtualAddressEx
. The former directly returns the
guest physical address the provided guest linear address translates to,
whereas the latter returns a VA_TRANSLATION structure which contains more
information about the translated address, such as page-table entries at
every level. Both of these functions receive the Cr3 that must be
used for translation - this Cr3 indicates the virtual address space the
translation is carried in, so make sure you use the right Cr3 for
translating the provided linear address.
Mapping guest virtual memory¶
IntVirtMemMap
can be used to map guest virtual memory, while
IntVirtMemUnmap
is used to unmap a previously mapped guest virtual
page. In essence, the IntVirtMemMap
function internally does the
translation using IntTranslateVirtualAddressEx
and then
uses IntPhysMemMap
to map the resulting guest physical page. It
must be noted that not all integrators/hypervisors are capable of
mapping multiple contiguous pages - if the callers asks to map 2 pages,
for example, this may fail. However, internally, IntVirtMemMap
supports this case by using _IntVirtMemMapMultiPage
, which will
in fact read the contents of the supplied virtual pages inside an allocated Introcore
buffer. This means that mapping multiple pages will not in fact create
mappings per se, instead it will read their contents internally. The
caller must not rely on the fact that modifying the contents of multiple
mapped pages will actually result in guest memory modifications: to make
modifications to the guest memory, make sure you map a single page at a
time or use IntVirtMemWrite
or IntVirtMemSafeWrite
.
Directly accessing guest physical memory¶
Data can be read from guest physical pages without mapping and unmapping
them. The IntPhysicalMemRead
function will read data from a
single page, whereas IntPhysicalMemReadAnySize
can read data
from multiple contiguous physical pages.
IntPhysicalMemWrite
can be used to directly write physical memory
without mapping or unmapping the page. The physical page will be
mapped/unmapped internally. This function will not make any security
checks. This means that it will not check to see if the written physical
page is actually accessible in EPT. This function should be used with
care - for example, only when writing areas of memory which are known to
be protected by Introcore.
Directly accessing guest virtual memory¶
There are several ways of accessing guest virtual memory, which are summarized in the table bellow:
Function Name | Description |
---|---|
IntKernVirtMemRead |
Reads data from guest virtual memory, using the known kernel CR3. Use this for kernel addresses only! |
IntKernVirtMemWrite |
Writes data to guest virtual memory, using the known kernel CR3. Use this for kernel addresses only! |
IntKernVirtMemFetchDword |
Reads 4 bytes from a kernel address. |
IntKernVirtMemFetchQword |
Reads 8 bytes from a kernel address. |
IntKernVirtMemFetchWordSize |
Reads 4 (if guest is 32 bit) or 8 (if guest is 64 bit) bytes from a kernel address. |
IntKernVirtMemPatchDword |
Writes 4 bytes at a kernel address. |
IntKernVirtMemPatchQword |
Writes 8 bytes at a kernel address. |
IntKernVirtMemPatchWordSize |
Writes 4 (if guest is 32 bit) or 8 (if guest is 64 bit) bytes at a kernel address. |
IntVirtMemRead |
Reads data from an arbitrary virtual address space. |
IntVirtMemWrite |
Writes data to an arbitrary virtual address space. |
IntVirtMemSet |
Sets a range of guest virtual memory to a value (memset). |
IntVirtMemSafeWrite |
Writes data to an arbitrary address space, only if the written virtual memory is writable in the page tables and the provided privilege level allows it. In addition, it makes sure the written physical address is not marked non-writable in EPT. Use this when writing to arbitrary guest memory, especially if it represents guest allocated memory or guest supplied pointers! This function will fail if the CPU triggers an exception when writing to the provided address. |
IntVirtMemFetchDword |
Reads 4 bytes from an arbitrary virtual address space. |
IntVirtMemFetchQword |
Reads 8 bytes from an arbitrary virtual address space. |
IntVirtMemFetchWordSize |
Reads 4 (if guest is 32 bit) or 8 (if guest is 64 bit) bytes from an arbitrary virtual address space. |
IntVirtMemFetchString |
Reads up to a maximum number of characters (provided as an argument) or until a NULL terminator is encountered. |
IntVirtMemPatchDword |
Writes 4 bytes to an arbitrary virtual address space. |
IntVirtMemPatchQword |
Writes 8 bytes to an arbitrary virtual address space. |
IntVirtMemPatchWordSize |
Writes 4 (if guest is 32 bit) or 8 (if guest is 64 bit) bytes to an arbitrary virtual address space. |
Functions which access kernel memory (IntKernVirtMem*
)
automatically use the System CR3 (which is the Cr3 used by kernel). This
means that accessing user-mode memory with these functions will almost
always fail.
Functions which access arbitrary virtual address space (IntVirtMem*
)
all receive the Cr3 as a parameter. The operation will be carried inside the
provided address space. If the provided Cr3 is 0, the functions will operate
using the currently active Cr3 (in the context of the current process).
When writing to guest memory, make sure that you have done enough safety
checks. Generally, it is preferred to use IntVirtMemSafeWrite
which makes several validations instead of using other ways of writing the guest memory.
Accessing guest physical memory using the cache¶
Sometimes, certain guest physical pages are accessed more often and it is important to have fast access to them. To address this, Introcore contains a GPA cache (Guest Physical Address cache). This cache keeps accessed pages mapped inside Introcore space, and when a page is needed again, it can return it very quickly and with minimal performance overhead. The GPA cache API is listed bellow.
Retrieving a guest physical mapping using the cache¶
To map a guest physical page that will most likely be reused later on,
use IntGpaCacheFindAndAdd
. This function will lookup the cache to
see if the needed GPA is already mapped. If it is, it will be returned
immediately; otherwise, it will map it and return it (note that already
mapped GPAs may be evicted from the cache, if it is full). Once done
working with the mapped GPA, the caller must
use IntGpaCacheRelease
. This function will signal the GPA cache
that the page is no longer in use. Note that
calling IntGpaCacheRelease
will not unmap the GPA from the
cache, instead, it will indicate to the cache that it can safely
replace the entry, should it be required.
Fetching or patching data from/to cached addresses¶
Sometimes, the caller needs to read or write only a small amount of data
form/to cached addresses. In this case, calling
both IntGpaCacheFindAndAdd
and IntGpaCacheRelease
may
prove too complicated. To address this,
the IntGpaCacheFetchAndAdd
allows the caller to directly read
data from guest memory via the cache, whereas
IntGpaCachePatchAndAdd
allows the caller to patch guest data via
the cache. Internally, both of these functions work similarly
to IntGpaCacheFindAndAdd
/IntGpaCacheRelease
, but they make things
easier to read when, for example, 8 bytes are fetched from a guest physical page
that ought to be cached.
Note
The page translation mechanism (IntTranslateVirtualAddress
and
IntTranslateVirtualAddressEx
) use the GPA cache internally. This means
that most of the accessed page-table pages are cached internally in
Introcore, making address translation very fast (this is why IntVirtMemMap
works by translating the address first, then mapping the resulting GPA).
Note
The integrator may implement caches of their own. These are transparent to Introcore, and work independently of it.
Accessing swapped-out guest memory¶
The functions listed in the Accessing guest memory section work only if the pages are present in physical memory. If the user wishes to access a guest virtual page that is swapped out, the functions listed there will not work. To implement a solution which is capable of accessing memory that may potentially be swapped out, the swapmem mechanism was created. It allows the caller to read memory which may be swapped out, and raises a notification when all the required data is available.
Reading swapped out memory¶
The main API in this module is IntSwapMemReadData
. This function
receives several parameters (please check out the Doxygen documentation),
but the most important are the Cr3
, which indicates the virtual address
space the read will be carried in, the VirtualAddress
to be read,
the Length
to be read and a Callback
which will be called
once the entire region has been read. The function can be called on regions
of any size and it can span multiple pages - the callback will be called only
when all the data has been read. Internally, this function will inject #PF exceptions
inside the guest, in order to force it to swap-in the needed pages. The caller,
however, can instruct the function to not inject any #PF and wait for
the desired pages to be swapped in naturally (although it may take a
long time, and there is no guarantee that the callback will be called at
all). The function will optionally return a handle to the read request,
which can be used to cancel the request later on. This handle should be
invalidated (set to NULL) once the swap-in callback has been called (all
data has been read).
Canceling a request to read memory¶
Sometimes, the caller wishes to cancel a request to read swapped out
memory. This may happen, for example, if the process in whose context a
read request was initiated terminated - in this case, the caller must
remove the transaction explicitly by calling IntSwapMemRemoveTransaction
and specifying the swap handle that was returned by IntSwapMemReadData
.
Sometimes, for example when a process terminates, the caller may wish to cancel all the
pending read requests inside that process’ context - in this case, it
can use IntSwapMemRemoveTransactionsForVaSpace
, which will remove
all transaction pending for the indicated VA space.
Hiding guest memory contents¶
Introcore may inject code or data, or modify existing guest code or data. This raises a series of problems, from hiding from the guest the fact that it is introspected, to making sure the changes are not seen by integrity mechanisms used by the guest (like patch guard on Windows). We must also make sure that attackers can not modify code or data owned by Introcore, while allowing us to easily modify and use the same. All this is ensured by the memcloak module. This module exposes APIs for setting, managing, and restoring hidden memory section.
Accessing guest state¶
Accessing the guest registers state can be done using the following API:
State type | API | Contents | Notes |
---|---|---|---|
General Purpose Registers | IntGetGprs |
RAX - R15, RIP, RFLAGS, CR0, CR2, CR3, CR4 | If KPTI is enabled and the current CR3 is a user-mode CR3, it will be replaced with the kernel-mode CR3, ensuring that both user and kernel data can be accessed. |
Segment Registers | IntGetSegs |
CS, DS, ES, SS, FS, GS (base, limit, access rights) | |
General Purpose Registers + IDT & GDT | IntGetAllRegisters |
IntGetGprs + IDT & GDT |
|
The current Instruction Pointer | IntRipRead |
RIP | Internally, it uses IntGetGprs . |
Control Register 0 | IntCr0Read |
CR0 | Internally, it uses IntGetGprs . |
Control Register 3 | IntCr3Read |
CR3 | Internally, it uses IntGetGprs . |
Control Register 4 | IntCr4Read |
CR4 | Internally, it uses IntGetGprs . |
IDTR Base | IntIdtFindBase |
IDTR base | Internally, it uses IntGetAllRegisters . |
IDT entry | IntIdtGetEntry |
IDT entry | Internally, it uses IntGetAllRegisters . |
GDTR Base | IntGdtFindBase |
GDTR base | Internally, it uses IntGetAllRegisters . |
Extended Feature Enable Register | IntEferRead |
IA32_EFER | |
FS base | IntFsRead |
IA32_FS_BASE | |
GS base | IntGsRead |
IA32_GS_BASE | |
SYSENTER MSRs | IntSysenterRead |
IA32_SYSENTER_CS, IA32_SYSENTER_EIP, IA32_SYSENTER_ESP | |
SYSCALL MSRs | IntSyscallRead |
IA32_STAR, IA32_LSTAR_MSR | |
Current ring | IntGetCurrentRing |
0, 1, 2 or 3, indicating the current ring. | |
Current mode | IntGetCurrentMode |
32 or 64 bit mode (includes legacy segments in long mode). | |
The maximum size of the XSAVE area | IntGetXsaveAreaSize |
The XSAVE area size | |
XCR0 register | IntGetXcr0 |
XCR0 | |
XSAVE area | IntGetXsaveArea |
The XSAVE area | |
Current EPTP index | IntGetCurrentEptIndex |
Current EPTP index | |
Maximum accessible physical address | IntGetMaxGpfn |
Max guest physical frame number |
Most of these functions operate in the context of a VCPU, so a CPU number must be provided as an argument. Calling any of these functions on different CPU number than the current CPU will cause a significant performance overhead, as that VCPU will have to be paused.
In addition to these query functions, the API also allow the user to
modify the contents of the general purpose registers using IntSetGprs
,
and the contents of the XSAVE area, using IntSetXsaveArea
.
The latter, however, may not be present on every hypervisor.
All of these functions are wrappers over the
lower-level IntQueryGuestInfo
API. This API receives an
information class, and queries that specified state. For a complete list
of information that can be queried using this function, take a look
at IG_QUERY_INFO_CLASS
. The functions listed in this section
all use the IntQueryGuestInfo
internally.
Working with instructions¶
Most often, there will be a need to decode instructions to infer various information about the context - for example, determining the address encoded in an instruction or determining the value written by an instruction. Introcore integrates an x86 disassembly library, which provides rich information about the decoded instructions.
Directly decoding instructions¶
To work with raw instructions, you can directly call the decoder using
the NdDecode
family of functions. These functions belong to the x86
disassembly library, which can also be integrated independently in other
projects. The output of these functions is an INSTRUX
structure,
which will contain all possible information about the decoded
instruction, including, but not limited to: explicit and implicit
operands, flags access, cumulative memory access, etc.
Using the Introcore wrappers¶
Introcore also features some wrappers over the disassembly library. This wrapper is mainly meant to provide additional contextual information to the exiting functions. The main wrappers are listed in the table bellow:
Function Name | Description |
---|---|
IntDecDecodeInstruction |
Provided a GVA, it will read the instruction from the guest memory and decode it using NdDecode |
IntDecDecodeInstructionFromBuffer |
Provided a memory buffer in Introcore space, it will decode the instruction using NdDecode |
IntDecDecodeInstructionAtRip |
Provided the current RIP for the current VCPU, this function will decode the instruction pointed by it using NdDecode |
IntDecDecodeInstructionAtRipWithCache |
This function decodes an instruction from the provided GVA, and it will cache it inside the instruction cache. Subsequent attempts to decode the instruction located at an already cached GVA will return the already decoded instruction from the cache. The instruction cache will be invalidated every time the memory page containing cached instructions is written. |
Misc decoder functionalities¶
The decoder supports various other instruction specific operations, the most important of which are summarized in the table bellow:
Function Name | Description |
---|---|
IntDecDecodeAccessSize |
Given a memory operand of an instruction, this function will compute the size of the memory access, in bytes. |
IntDecGetWrittenValueFromInstruction |
Given a memory write instruction, this function will compute the value that is being written to memory. Note that only a small subset of instructions is supported: MOV, STOS, MOVS, MOVNTI, MOVZX, MOVSX, MOVSXD, XCHG, CMPXCHG, CMPXCHG8B, CMPXCHG16B, ADD, ADC, SUB, SBB, MUL, DIV, IDIV, IMUL, AND, OR, XOR, RCL, RCR, ROL, ROR, BTS, BTR, BTC, XADD, INC, DEC, NOT, NEG, LIDT, LGDT, MOVDQU, MOVAPD, MOVAPS, MOVUPD, MOVUPS, MOVSS, MOVHPS, MOVHPD, MOVLPS, MOVLPD, ADDPS. Calling this function on other instructions will return an error. |
IntDecGetAccessedMemCount |
Given an instruction, it returns the total number of memory locations accessed by the instruction. |
IntDecGetAccessedMem |
Given an instruction, it returns the GLA of every accessed memory location by the instruction (both explicit and implicit accesses). |
IntDecGetSseRegValue |
Returns the value of a SSE register. |
IntDecSetSseRegValue |
Sets the value of a SSE register. |
IntDecComputeLinearAddress |
Given a memory operand of an instruction, this function will compute the guest linear address accessed by that operand. This function take into account segmentation, base, index, scale and displacement. |
IntDecDecodeDestinationLinearAddressFromInstruction |
Given an instruction, this function will return the first GLA that is being written by this instruction. |
IntDecDecodeSourceLinearAddressFromInstruction |
Given an instruction, this function will return the first GLA that is being read by this instruction. |
The Slack Allocator¶
The Introspection engine features a slack allocator which is
capable of allocating small regions of memory inside the guest VM.
The allocated regions are heavily restricted in size, and they can be
allocated inside only the kernel.
The IntSlackAlloc
function will allocate a region of memory inside
the guest VM, and IntSlackFree
will free it. The algorithm works
differently on Windows and Linux.
Windows Slack¶
On Windows, the slack space used for allocations is the padding area between driver sections. Normally, the slack allocator allows the caller to specify the kernel module in which the allocation will be made, but it is highly recommended to only use the nt (kernel) image or hal. The caller can also specify whether the allocated space must be pageable or non-pageable. The allocator will iterate the sections of the indicated module, and it will try to find padding space at the end of an executable section that satisfies the minimum required size. If no such space is found, an error is returned.
Note that on Windows, there is a very high amount of slack space/section padding that can be used for Introspection purposes. For example, on a Windows 10 RS5 x64, there is a total of 11864 bytes of non-paged executable slack space plus an additional 9184 bytes of paged executable space, for a total of more than 20 kilobytes of slack space. Also note that the allocated slack space is protected using the memcloak (memory hiding mechanism), so reads made from within the guest on this padding regions will always return 0, even if Introcore stores code and data there. Also note that due to this (because the slack space is hidden), slack space cannot be allocated inside data sections, because they are read very often, which will induce very many EPT violations due to the memcloak set on those regions, and, therefore, a very high performance impact.
Linux Slack¶
On Linux, the slack space used for allocations is the padding area between functions from ‘.text’ section. The allocator will iterate the first two pages of the ‘.text’ section, and it will try to find padding space that satisfies the minimum required size. If no such space is found, an error is returned.
Note that on Linux, there is a low amount of slack space padding that can be used for Introspection purposes.
The Agents Injection¶
Introcore has support for agent injection inside the guest. The overall architecture and the interaction between the integrator and the agents content is described in section Agents Architecture. This section provides development insights. There are three main types of agents that can be injected: process agents, file agents and special agents. Process and file agents can be injected on demand by the integrator via the exposed API, as described in the Agents Architecture section.
Windows Agents¶
On Windows, agent injection has several components, and it is done in
several steps. The API used to inject is IntWinAgentInject
.
Normally, agents cannot be removed from the guest, but exceptions do
apply. If a file agent is injected, the file will remain inside the
guest until someone deletes it. If a process agent is injected, it will
remain active until it gracefully exits - an exception applies here in
that a killer agent can be injected, which is a regular process that
kills other agents. Special agents are injected internally by Introcore;
the two special agents are the PT filter and the #VE agent.
These can be removed on demand, unlike file agents and process
agents. However, special agents are injected only internally by
Introcore, and the integrator cannot directly control this (although it
can disable PT or #VE via the global options). The overall steps are
illustrated bellow (note that technical details of the components and
agent injection workings can be found in Doxygen):
- An instruction is hooked on the SYSCALL flow, which will divert execution to the trampoline;
- The trampoline will jump to the bootstrap code, which is allocated inside slack space by Introcore;
- The bootstrap allocates memory for the main agent driver, bootdrv; Introcore delivers the bootdrv to the designated memory area, and the bootstrap starts it;
- The bootdrv then communicates with Introcore to finish the actual injection.
As already mentioned, the more technical details can be found in the source code and the Doxygen documentation.
Linux Agents¶
On Linux, agents injection has several components, and it is done in
several steps. The API used to inject in IntLixAgentThreadInject
.
Normally, agents cannot be removed from the guest, but exceptions do apply.
If a file agent is injected, the file will remain inside the guest until
someone deletes it. If a process agent is injected, it will remain active
until it gracefully exits - an exception applies here in that a killer agent
can be injected, which is a regular process that kills other agents.
The overall steps are illustrated bellow (note that technical details of the components and
agent injection workings can be found in the Doxygen documentation):
- An instruction is hooked on the SYSCALL flow, which will generate an VM-Exit (*breakpoint*);
- The VM-Exit handler will divert the execution to the agent’s code (bootstrap), which is allocated inside the guest, by changing the value of the RIP register;
- The bootstrap allocates memory for the main agent and starts it;
- The communication between agents and Introcore is done through VMCALL instructions;
Windows introspection¶
Windows Kernel modules management¶
For kernel mode modules Introcore is interested in two things:
- The module itself.
- The optional driver object that a module can own.
Windows kernel modules¶
Kernel module loading is intercepted by hooking the
MiProcessLoaderEntry
kernel API. This will create a KERNEL_DRIVER object
and will insert it in a global list. In this case the module is considered to be
dynamically detected.
If the module is protected the protection will be activated.
Kernel module unloading is intercepted by hooking the MiUnloadSystemImage
kernel API.
This will remove any resources held by the previously created KERNEL_DRIVER
.
For kernel modules loaded before Introcore started, the global kernel
list located at PsLoadedModuleList
is iterated and every module is
imported. This is done after the entire kernel image is read. Modules
found in this way are considered to be statically detected.
If the INTRO_OPT_EVENT_MODULES
Introcore option
is set load and unload events will be sent.
Windows driver objects¶
When a module is dynamically detected and the module is known to have a
driver object, Introcore places an execution memory hook
on the module entry point. One of the parameters passed by the kernel to
the entry point will be the address of the driver object
that the module owns. Introcore simply uses this as a way of associating
a kernel module with a driver object. However, Introcore will first
detect the driver object when it is allocated, with a hook placed
on ExAllocatePoolWithTag
. The detection is done based on the
allocation tag used. This is used to patch the allocation size and
increase it to 4KB which will allow Introcore to place EPT write hooks
on the driver object. Without doing this, monitoring writes done to the
driver object using memory hooks will negatively impact the guest
performance, as many other structures that are accessed quite often can
be allocated in the same page.
For statically detected modules, driver objects cannot be detected in
this way. The scanning mechanism is based on the kernel object
namespace. The namespace is organized in directories, starting from the
root directory (“\`") which is pointed to by the kernel global
:code:`ObpRootDirectoryObject
. Each directory can contain other directories
or objects. For driver objects Introcore is interested in
the Driver
and FileSystem
directories. These can contain
either driver objects or device objects. Device objects are ignored by
Introcore. The parsing of the object namespace makes heavy use of
the swapped memory read
functionality and thus detecting some driver objects can be delayed. The
scanning starts when the services.exe process is started.
When a driver object is protected, the fast IO dispatch structure associated with it is also protected, if needed.
Driver object destruction is detected either when the kernel module
unloads, or with the ExFreePoolWithTag
kernel API hook. This will
remove any resources associated with it.
Windows Process management¶
The Windows kernel keeps a doubly linked list of processes, with the list
head saved as a global inside the kernel: PsActiveProcessHead
.
Introcore keeps a list with all the processes that exist inside the monitored guest.
Each process is represented by a WIN_PROCESS_OBJECT structure.
This list is updated in real time by using API detours.
For process creation, Introcore intercepts PspInsertProcess
, which
inserts a new EPROCESS
structure inside the kernel process list.
It is worth noting that this API can fail, and this is used by Introcore
to block malicious process creations
Process termination is detected by hooking MmCleanProcessAddressSpace
.
On some Windows versions this function can be called twice for the same process - Introcore detects and properly handles this case.
If the currently loaded policy requests this, each process creation and termination is reported.
For processes started before Introcore, the process list is iterated and
each process is imported in the Introcore process list, with the
exception of processes that have the Delete
, Exiting
, or
VmDeleted
flags set,
as those processes will soon be removed from the process list.
The in-guest process list is iterated for one additional reason: when doing a thread safeness check.
During process creation events, other Introcore modules may be initialized. For example, when services.exe is started Introcore enables agent injections (and may inject the PT filter agent, if needed), and a static scan for driver objects is started.
For protected processes, the EPROCESS
structure might be
patched to let various detour handlers check the protection policy used
by a process, allowing them to skip some useless hypercalls.
After a process is detected by Introcore, if the currently loaded
protection policies have a protection rule for that process, protection
is activated. This may trigger additional memory scans, for example
for VADs, or new
memory hooks
might be set. All the hooks set while protecting a process are removed
when the protection is removed, or when the process terminates,
including any pending swap-mem reads.
For processes that do not need VAD monitoring, the main module is loaded
at the creation, based on the SectionBase
field of the EPROCESS
.
The handling of the module is as described in DLL Management.
Introcore will need the currently active process for handling most of
the events it can receive. The current process is detected by using the
currently loaded guest CR3. However, there is no guarantee that
the Windows kernel won’t relocate the CR3 used by a process. This is
problematic because we will no longer be ale to identify hooks by CR3,
but even more so because guest virtual address memory hooks
are set in the context of a CR3. To avoid these problems, Introcore
locks the physical page used for the process CR3 using PFN locks.
For guests that have KPTI patches installed and active, processes may use
two CR3 values: one while executing user-mode code, and another for
kernel-mode. In these cases both CR3 pages are locked and the kernel CR3
value is used for identifying the current process. Since events can be
generated while the guest is in ring 3 and has a user mode CR3 loaded,
when the guest register state is queried with IntGetGprs
a check is
done for the CR3 value, and if it is a user mode CR3 for a process, the
query will return the corresponding kernel CR3 instead.
VAD management¶
Each user-mode memory range used by a process is described by a VAD
(virtual address descriptor). These structures are organized in a binary
search tree, with the root in the VadRoot
field of the
EPROCESS structure.
There are two types of VAD structures: short
(MMVAD_SHORT
), and long (MMVAD
,
which contains a MMVAD_SHORT
structure).
The short VAD portion is common for all memory ranges reserved for a process. It contains information about the start and end of the VAD, the type of region, the original protection rights used for that region, etc. For certain memory regions (for example, for mapped executable files), more information is needed, and that information is included in the long version.
A VAD is inserted inside the process VAD tree as soon as memory is reserved by the process. Introcore intercepts, using API hooks certain operations that the OS does on VAD structures or on the VAD tree:
- Inserting a new VAD in the tree - which means that memory was reserved for the process; based on the type of the VAD Introcore may decide to monitor the way the process uses this memory range.
- Removing a VAD from the tree - which means that memory was released for the process; if this VAD described a monitored memory region Introcore will stop monitoring it, as it is no longer a valid memory range.
- Changing protection for a sub-range of the range described by the VAD - based on the type of VAD, the old and new protection rights, and the Introcore policy that is currently active Introcore may start or stop monitoring the given memory range.
These operations can be triggered directly by the process which will own
the memory ranges, using Windows APIs like VirtualAlloc
,
VirtualProtect
, CreateFileMapping
, etc, or by another process,
using APIs like VirtualAllocEx
, VirtualProtectEx
, etc.
Introcore uses this information for two purposes:
- The user mode exploit protection, to know which pages should be hooked against executions.
- Loading of user mode modules inside processes - this serves both informational purposes, as well as a double agent detection mechanism.
Currently, three types of VADs are monitored:
VadNone
VadWriteWatch
VadImageMap
The first and second types are used for detecting memory ranges that need to be monitored against executions, for the exploit protection mechanism. If the range is created executable, Introcore hooks it when the VAD is created. If the range is not created executable, Introcore hooks the pages that are made executable when one of the APIs that change memory protection rights is used by the guest.
The last range is used for detecting the loading and unloading of DLLs from a process. It uses a long VAD to describe the memory range. The long part of the VAD contains information about the file that is loaded inside the guest.
Various Windows kernel versions use different kernel APIs for managing VAD structures, so Introcore hooks different APIs, based on the Windows version, but all operations are unified in a few generic APIs:
IntWinVadHandleInsertGeneric
- all VAD creations are handled here, after the creation parameters are gathered by a detour handler.IntWinVadHandleDeleteGeneric
- all VAD (or VAD sub-ranges) frees are handled here, after the deletion parameters are gathered by a detour handler.IntWinVadHandleProtectGeneric
- all protection changes for a VAD (or a VAD sub-range) are handled here, after all the parameters are gathered by a detour handler.IntWinVadHandlePageExecution
- all execution attempts from monitored memory ranges are handled here.
In some cases, simply monitoring insertions inside the VAD tree is not
enough (for example, if a process is protected after it is started). In
those cases, Introcore walks the guest VAD tree and imports all the
relevant VADs using IntWinVadImportProcessTree
. Other special
cases may require further scans of the guest VAD tree (for example, a
VAD protection change might be intercepted before a VAD creation is
intercepted).
Monitoring shared memory regions against executions might prove problematic, especially if the memory is shared between a process that is protected and one that is not protected, as any execution attempts done by the process that is not protected will generate a VMEXIT, which can slow down the guest considerably. Currently, to avoid this, Introcore will remove the protection as soon as it detects that a shared executable memory region exists between a processes that are protected and processes that are not protected.
Reading a process command line¶
Introcore can read the command line of a Windows process. This may be needed for the engines scan mechanism for reporting it in an event, or for other heuristics applied by Introcore (for example, the Chrome command line is used to detect if NaCl is enabled or not).
The command line can be obtained both for statically and dynamically detected processes (see Process management for details), but the read is not synchronous as it implies reads from memory that may be swapped out.
When a process is detected by Introcore, if it is protected with the
PROC_OPT_PROT_SCAN_CMD_LINE
option, or if it is part of the
gCmdLineProcesses
process list, a command line read attempt is started.
Note
Since this is not synchronous, a process may terminate before the command line is obtained.
The first step is to obtain the address of the PEB
. For 32-bit
processes running on 64-bit version of Windows this uses the address of
PEB32
. The address of the PEB
is obtained from the
EPROCESS
structure. From the PEB
we read the address of
the RTL_USER_PROCESS_PARAMETERS structure. Since this is user mode memory
that can be swapped out, this read is done using IntSwapMemRead
.
From RTL_USER_PROCESS_PARAMETERS
we read a UNICODE_STRING
structure that holds the command line. Again, this is memory that can be
swapped out. Once we have the UNICODE_STRING
structure the last
step is to read the actual buffer used to hold the command line.
If a process is protected with PROC_OPT_PROT_SCAN_CMD_LINE
we limit the length of the
command line to 65534 bytes, otherwise we limit it to 1024 bytes. Processes created with the
CreateProcess
API have a limit of 32767 characters
(which is exactly 65534 bytes).
Windows stores the command line as a WCHAR
array, but Introcore
converts it to a NULL terminated CHAR
array.
After this conversion, if needed, the command line is sent to the scanning engines.
Any resources held by a command line are freed when the process terminates.
DLL management¶
User mode module loading and unloading is detected based on VAD creation and destruction.
When a VAD
with the type VadImageMap
is created we try to obtain the path
of the file it maps. This is done based on the SUBSECTION
structure associated with
the newly created VAD
. Based on the subsection address, we first check the
path cache and if an entry is present, we use that. If the entry is not
present in the cache, we obtain the FILE_OBJECT
used for the file and try to
read the path from there. The path buffer may be swapped out so the read is not synchronous.
Once the path is in memory we add it to the cache and continue the handling of the DLL loading.
For 32-bit processes running on 64-bit Windows versions, one of the first things we do is figure out if the loaded DLL is actually a 64-bit DLL. This is done based on the path of the module: if it is loaded from Windows\System32 it is a 64-bit DLL.
If the module is a 32-bit module, it is added to the list of modules inside the 32-bit subsystem of the process, otherwise it is added to the 64-bit subsystem.
If needed, the MZPE headers of the modules are read and the module is
protected against writes.
This is done based on the process protection flags.
If the process is protected with the PROC_OPT_PROT_DOUBLE_AGENT
option, suspicious DLL loading is also checked at this time.
When the main module is loaded, if the INTRO_OPT_FULL_PATH
Introcore option was used, the path from
which it is loaded is checked against the path for which we protected the process. If these do not match,
the process protection is removed. This is needed because when a process is created we know only the name
of the process, not the entire path.
Unloading is handled when the VAD
that maps the module is destroyed.
Each module is represented by a WIN_PROCESS_MODULE structure.
Module events are not sent every time a module is loaded and unloaded, but every time a write or a double agent is detected for the process that owns the modules.
When checking suspicious DLL loading, the introspection engine may
decide to block the attempt by using the DLL blocking mechanism. The DLL
blocking mechanism works with the assumption that the Windows DLL Loader
will verify the result of the call to the module entry point (usually
DllMain
) to know if the DLL loaded successfully or some error
appeared. When the DllMain
function inside the DLL which must be
blocked is called by the guest, an execution EPT violation is raised,
and the DLL blocking mechanism, based on the flags given, decides how to
block the load. If winModBlockFlagUnloadAfterExec
is given, then the
engine will force the return of FALSE from DllMain
, and the loader
should unload automatically the DLL, as returning FALSE signals that an
error occurred during the load. On the other hand, if
winModBlockFlagDoNotUnload
is provided, the DLL is not unloaded, but rather, on
every execution inside any executable section of that DLL, an execution
EPT violation will be triggered, and the introspection engine will force
a RET
, not letting anything in the given module to be executed.
Initialization steps, for example, on double agent, completing the
RTL_VERIFIER_PROVIDER_DESCRIPTOR with fake values, will be done by
the provided callback when the decision to block the module was taken.
An additional flag for blocking the DLL is
winModBlockFlagKillOnError
. When this flag is given, if the
callbacks return any error to the DLL block mechanism, the mechanism
will try to kill the given process by injecting a page fault on address
0, which, to the kernel will seem like the given DLL has accessed the
address 0.
For the protected user-mode modules, such as ntdll.dll, the introspection engine will keep an Export Cache. As, in some cases, the exports of the protected DLL are needed, for example when excepting writes over some functions, and in these cases the introspection engine can’t inject a page fault in order to read them, as it would be asynchronous and the exports are needed synchronously, and moreover it would induce a high performance impact, the exports are read when the DLL loads. However, the exports should be the same for each pair of the module path and the architecture on a given system (note that the architecture is important, as the modules are very different on WoW64 versus x64), thus they can be cached and re-used if the same module, having the same path and architecture, loads in another process. However, one problem would be if the first process on which the cache is populated is either static detected or late protected, as the module Export Address Table may have been written by another DLL before we could cache the exports and reusing those exports in other processes for which the module Export Address Table may not have been written in the same manner, thus leading to either loss of protection or possible false positives, since the exception mechanism would not have some exports when some writes occur inside the process which re-uses this specific cache. For this purpose, the engine will mark those caches as dirty, and will be used only by the process in which the given module may have been altered, as the process is either static detected or late protected, which means that the introspection engine could not see the modifications over the given modules beforehand.
Path cache¶
As most user mode modules are, most of the time, loaded from the same
path, keeping every path, even if it is repeating, can be a waste of
resources. For this reason, Introcore caches the paths, reusing a
WINUM_PATH structure for a VAD
object, if the value of the
Subsection
field inside a VAD
structure is the same as the one
cached. The Subsection GVA is considered in this case as the unique
identifier of the given path. Since user mode module paths may be
utilized by Introcore in various situations, it is recommended to always
reference the path when working with it, and dereference it when not
needed anymore. The path cache will free the path if the reference count
reaches 0. Note that this is important, as, if some paths are leaked, or
are not dereferenced when needed, the guest may re-use at some point the
Subsection GVA which serves as the path unique identifier in Introcore
for a new subsection belonging to another path. Thus, failure to
dereference paths or failure to destroy objects that reference paths
when needed, will most likely lead to mismatch of paths, resulting in
loss of protection (for example, if the path of a protected process main
module is mismatched with the path of an unprotected one) or false
positives (for example, if the path of the module which does a write,
and is excepted, is mismatched with another path).
The objects that reference and keep WINUM_PATH
objects are: VAD
,
WIN_PROCESS_OBJECT (for the main module, considered the path of
the process) and WIN_PROCESS_MODULE. Those objects will dereference
the saved paths when they are destroyed.
Windows Thread safeness¶
Some operations can introduce races with the guest (for example, modifying guest code while that code is being executed). To avoid this, Introcore implements a thread safeness mechanism. It ensures that:
- No guest threads are executing code from a code region that we want to change - this includes currently active threads, based on the RIP value on each CPU, as well as threads that will resume execution from that specific code region.
- No guest threads will return inside a code region that we want to modify.
Doing these kind of operations should be avoided. Currently, Introcore does this while setting (or removing) API hooks (which is done when Introcore is loaded and unloaded), and when certain agents are injected.
The thread safeness module exports a single API: IntThrSafeCheckThreads
which does
the checks, based on options that control what zones are actually checked (detours, agents, etc).
It can also move guest pointers out of the dangerous memory zone, if needed.
PFN management¶
The Windows kernel maintains lists of physical page descriptors in the Page Frame Number database. These descriptors allow the kernel to know the state (in use, free, etc) in which a physical page is at a given moment.
The PFN Database entries are stored in memory based on their respective page frame numbers. In other words, if we know the base of the PFN database, getting an entry for a physical address is as simple as:
pfnEntry = pfnDb + pfnEntrySize * (physicalAddress >> 12)
The PFN Database is located at the kernel global MmPfnDatabase
.
The size of the entries varies based on the type of paging that is being used:
Paging | Size |
---|---|
32-bit legacy | 24 bytes |
32-bit with PAE | 28 bytes |
64-bit | 48 bytes |
The parts of the entries that are used by Introcore included in CAMI.
Introcore uses the PFN Database to lock certain memory pages in memory, ensuring the OS does not swap them out.
The module offers the possibility of locking either physical, or virtual pages. The APIs for locking virtual pages simply abstract away the translation from guest virtual to guest physical pages and handle the cases in which a page is swapped out when the lock is placed. The actual locking is done by modifying the reference count of the PFN Database entry. Note that this is supported for pages that are in only certain states (active, modified, modified no write, or stand by).
Currently, page locking is used in order to ensure that process CR3 values are not changed.
The Windows kernel might choose to change the CR3 used by a process. This poses a serious problem to Introcore because it affects memory hooks management, pending swap-in transactions, as well as event handling because the current loaded CR3 is used for detecting the currently active process.
The APIs exposed by the winpfn module are as follows:
API | Description |
---|---|
IntWinPfnIsMmPfnDatabase |
Checks if a guest virtual address is the base of the PFN Database. |
IntWinPfnDump |
Dumps all the PFN locks. |
IntWinPfnUnInit |
Deactivates the winpfn module. This will also remove any active locks, but only as a fail-safe mechanism. The owner of a lock is responsible of removing it. |
IntWinPfnLockGpa |
Locks a guest physical address by increasing its reference count. |
IntWinPfnLockGva |
Locks a guest virtual address. This will lock the guest physical address to which the supplied GVA translates to. It will also hook the page table entries used to translate the GVA in order to handle any possible translation changes. |
IntWinPfnRemoveLock |
Completely removes a lock. It can handle both GVA, and GPA locks. It will unlock the page and remove any page table hooks as well as release any other resources held by the lock. |
Power management¶
Introcore functionality is affected by the power transitions a guest makes. To handle these transitions there are two ways Introcore is notified about power transitions:
- Direct notification from the underlying hypervisor via
the
NotifyGuestPowerStateChange
glue interface API. - Intercepting the guest
NtSetSystemPowerState
API with a detour.
Based on the source of the notification, the state to which the guest transitions, and to the currently active options different actions are taken.
For notifications received directly from the hypervisor, if the guest enters the sleep state, all future events (for example, EPT violations, MSR violations, etc) will be blocked and Introcore will attempt to handle them only when the guest resumes from sleep. If the #VE filtering or the In-guest PT filtering agents are enabled they will be re-injected in the guest when it resumes from sleep. This will be done asynchronously, not necessarily as soon as the OS resumes from sleep. For other transitions Introcore will simply prepare for an unload. Note that an integrator can choose to disable Introcore when a transition happens.
However, in certain conditions, Introcore needs to know about the
transition earlier. This is why it also hooks the NtSetSystemPowerState
API.
This offers more information about the transition and allows Introcore to properly prepare
for the transition. For example, the #VE filtering or the In-guest PT filtering agents
are disabled and unloaded if the guest transitions to sleep or hibernate
or is rebooting or shutting down. This will block the guest in a loop
inside the detoured API until the agent is properly unloaded and it is
safe for the guest to continue the transition. For hibernate events
Introcore will also disable all active options and will block all
further API calls, as handling events from now on is unsafe and may
crash the guest. Note that this will not detect resumes from sleep or
hibernate.
Linux introspection¶
Linux Kernel modules management¶
Kernel module loading is intercepted by hooking the module_param_sysfs_setup
kernel API.
This will create a KERNEL_DRIVER object and will insert it in a global list. If the module is
protected the protection will be activated. In this case the module is considered to be dynamically detected.
Kernel module unloading is intercepted by hooking the module_param_sysfs_remove
kernel API.
This will remove any resources held by the previously created KERNEL_DRIVER
.
For kernel modules loaded before Introcore started, the global kernel list located at struct module *modules is iterated and every module is imported. This is done after the entire kernel image is read. Modules found in this way are considered to be statically detected.
If the INTRO_OPT_EVENT_MODULES
Introcore option
is set load and unload events
will be sent.
Linux Process management¶
The Linux kernel keeps a doubly linked list of process, saved as global
inside the kernel: struct task_struct init_task
. Introcore keeps a
list with all the processes that exist inside the monitored guest. This
list is updated in real time by using API hooks.
For process creation, Introcore intercepts wake_up_new_task
(fork
),
which starts a newly created task for the first time, and flush_old_exec
(exec
), which flush the data used by the old task and sets the information
of the newly created task. It is worth noting that this API can fail, and this is used
by Introcore to block malicious
process creations.
Process termination is detected by detouring do_exit
.
If the currently loaded policy requests this, each process creation and termination is reported.
For processes started before Introcore, the process list is iterated and each process is imported in the Introcore process list. Static detected tasks that are dying are ignored.
The in-guest process list is iterated for one additional reason: when doing a thread safeness check.
After a process is detected by Introcore, if the currently loaded protection policies has a protection rule for that process, protection is activated. This may trigger additional memory scans, for example for VMAs, or new memory hooks might be set.
All the hooks set while protecting a process are removed when the protection is removed, or when the process terminates.
MM management¶
Each user-mode memory range used by a process is described by
a VMA
(virtual memory area). These structures are organized in a
binary search tree and in a linked list, with the root in the
mmap
field of the struct mm_struct
(each
struct mm_struct
is a field of struct task_struct
).
The VMA contains information about the start and end of the memory range, the protection flags used for that region, mapped file, etc.
A VMA is inserted inside the process VMA tree as soon as memory is reserved by the process. Introcore intercepts, using API hooks certain operations that the OS does on VMA structures or on the VMA tree:
- Inserting a new VMA in the tree - which means that memory was reserved for the process; based on the flags of the VMA, Introcore may decide to monitor the way the process uses this memory range.
- Removing a VMA from the tree - which means that memory was released for the process; if this VMA described a monitored memory region Introcore will stop monitoring it, as it is no longer a valid memory range.
- Changing protection for a sub-range of the range described by the VMA - based on the flags of VMA, the old and new protection rights, and the Introcore policy that is currently active Introcore may start or stop monitoring the given memory range.
These operations can be triggered directly by the process which will own
the memory ranges, using Windows APIs like mmap
, malloc
,
mprotect
, etc.
Introcore uses this information for the user mode exploit protection to know which pages should be hooked against executions. Currently, only the non-executable VMAs are monitored.
Various Windows kernel versions use different kernel APIs for managing VMA structures, so Introcore hooks different APIs, based on the Linux version, but all operations are unified in a few generic APIs:
IntLixVmaInsert
- create a new VMA, after the creation parameters are gathered by a detour handler.IntLixVmaRemove
- delete a VMA, after the deletion parameters are gathered by a detour handler.IntLixVmaChangeProtection
- protection flags of the VMA changed, after all the parameters are gathered by a detour handler.IntLixVmaAdjust
- the VMA changes its memory range, after all the parameters are gathered by a detour handler.IntLixVmaExpandDownwards
- the VMA changes its memory range (used for stack), after all the parameters are gathered by a detour handler.IntLixVmaHandlePageExecution
- all execution attempts from monitored memory ranges are handled here.
In some cases, simply monitoring insertions inside the VAD tree is not
enough (for example, if a process is protected after it is started). In
those cases, Introcore walks the guest VMA tree and imports all the
relevant VMAs. Other special cases may require further scans of the
guest VMA tree (for example, the VMAs of the parent process, for
fork
, are duplicated for the child process).
Linux Thread safeness¶
Some operations can introduce races with the guest (for example, modifying guest code while that code is being executed). To avoid this, Introcore implements a thread safeness mechanism. It ensures that:
- No guest threads are executing code from a code region that we want to change - this includes currently active threads, based on the RIP value on each CPU, as well as threads that will resume execution from that specific code region.
- No guest threads will return inside a code region that we want to modify.
Doing these kind of operations should be avoided. Currently, Introcore does this while setting (or removing) API hooks (which is done when Introcore is loaded and unloaded), and when certain agents are injected.
The module exports a single API: IntThrSafeCheckThreads
which does
the checks, based on options that control what zones are actually checked
(detours, agents etc). It can also move guest pointers out of the dangerous
memory zone, if needed.