Handle x86 code normalization & block hashes extraction. More...

#include "codeblocks.h"
#include "crc32.h"
#include "decoder.h"
#include "utils.h"

Macros
#define	IS_REX_PREFIX(b) ((ND_PREFIX_REX_MIN <= (b)) && ((b) <= ND_PREFIX_REX_MAX))

Functions
DWORD	IntFragHandleCommon (const BYTE Buffer, size_t BufSize, IG_CS_TYPE CsType, CB_EXTRACT_LEVEL ExtractLevel, DWORD Pattern)
	Extract a pattern of instructions without using the disassembler. More...

INTSTATUS	IntFragExtractPattern (BYTE Buffer, DWORD MaxBufferSize, IG_CS_TYPE CsType, CB_EXTRACT_LEVEL ExtractLevel, DWORD PatternSize, BYTE Pattern, DWORD TotalExtracted, DWORD TotalParsed)
	Extract a pattern of code-blocks from the given code buffer. More...

INTSTATUS	IntFragExtractCodeBlocks (BYTE Buffer, DWORD MaxBufferSize, IG_CS_TYPE CsType, CB_EXTRACT_LEVEL ExtractLevel, DWORD HashesCount, DWORD *Hashes)
	Extract a block of code-block hashes from the given code buffer. More...

__pure INTSTATUS	IntFragMatchSignature (const DWORD Hashes, DWORD CodeBlocksCount, const SIG_CODEBLOCKS ExceptionSignature)
	Match a block of code-block hashes against a list of code-block exception signatures. More...

INTSTATUS	IntFragExtractCodePattern (PBYTE Buffer, DWORD StartOffset, DWORD MaxBufferSize, IG_CS_TYPE CsType, CB_EXTRACT_LEVEL ExtractLevel, DWORD PatternSize, CODE_BLOCK_PATTERN Pattern, DWORD TotalExtracted)
	Extract a pattern of code-blocks from the given code buffer. More...

static void	IntFragLogCodeBlocks (CODE_BLOCK *CodeBlock, DWORD Count, QWORD Rip, DWORD RipOffset, BOOLEAN ReturnRip, DWORD ElemLine)
	Log a block of code-blocks. More...

INTSTATUS	IntFragDumpBlocks (PBYTE Buffer, QWORD StartAddress, DWORD MaxBufferSize, IG_CS_TYPE CsType, CB_EXTRACT_LEVEL ExtractLevel, QWORD Rip, BOOLEAN ReturnRip)
	Dumps code-blocks that can then be used to generate an exception signature. More...

Variables
static CHAR	gCbLog [512]
	Used to format log lines containing code-blocks. More...

Detailed Description

Handle x86 code normalization & block hashes extraction.

This module parses a stream of x86 instructions, it decodes each instruction, it normalizes them (it converts the decoded instructions to a single-byte value representing the generic type of the instruction), and then computes hashes (CRC32) on blocks of such single-byte normalized instructions. Only a few instructions are parsed and normalized (for example, branches, returns, string operations, mov, etc.). The general operation is:

Disassemble each instruction inside the stream;
For each instruction, output a normalized CODE_INS value; other instructions will simply be ignored;
Once a list of CODE_INS is extracted from the instruction stream, find pivots, which are fixed instruction types (codeInsJmp, codeInsCall, codeInsMovMem, codeInsMovFsGs);
Starting with each pivot instruction, compute a CRC32 on a block of CODE_BLOCK_CHUNKS_COUNT values from the pattern. The extracted hashes will then be used in exception signatures. The rationale behind this is that they are specific enough because they include a significant chunk of instructions, but they are resilient to minor code changes, such as recompiling, where only the used registers are modified.

Definition in file codeblocks.c.

Macro Definition Documentation

◆ IS_REX_PREFIX

#define IS_REX_PREFIX ( b ) ((ND_PREFIX_REX_MIN <= (b)) && ((b) <= ND_PREFIX_REX_MAX))

Referenced by IntFragHandleCommon().

Function Documentation

◆ IntFragDumpBlocks()

INTSTATUS IntFragDumpBlocks	(	PBYTE	Buffer,
		QWORD	StartAddress,
		DWORD	MaxBufferSize,
		IG_CS_TYPE	CsType,
		CB_EXTRACT_LEVEL	ExtractLevel,
		QWORD	Rip,
		BOOLEAN	ReturnRip
	)

Dumps code-blocks that can then be used to generate an exception signature.

Parameters

[in]	Buffer	The code buffer to be parsed.
[in]	StartAddress	The offset to start the parsing at.
[in]	MaxBufferSize	The size of the code buffer.
[in]	CsType	Operating mode, should be IG_CS_TYPE_32B or IG_CS_TYPE_64B.
[in]	ExtractLevel	cbLevelNormal or cbLevelMedium.
[in]	Rip	The current Rip.
[in]	ReturnRip	The return Rip.

Return values

INT_STATUS_SUCCESS	On success.
INT_STATUS_INVALID_PARAMETER	If an invalid parameter is supplied.
INT_STATUS_INSUFFICIENT_RESOURCES	If a memory alloc fails.
INT_STATUS_DATA_BUFFER_TOO_SMALL	If at least CODE_BLOCK_CHUNKS_COUNT could not be extracted.

Definition at line 1293 of file codeblocks.c.

Referenced by DbgDumpCodeblocks(), and IntExceptDumpSignatures().

◆ IntFragExtractCodeBlocks()

INTSTATUS IntFragExtractCodeBlocks	(	BYTE *	Buffer,
		DWORD	MaxBufferSize,
		IG_CS_TYPE	CsType,
		CB_EXTRACT_LEVEL	ExtractLevel,
		DWORD *	HashesCount,
		DWORD *	Hashes
	)

Extract a block of code-block hashes from the given code buffer.

This function will parse the provided code buffer, and it will extract a pattern of CODE_INS values representing the relevant instructions located inside the buffer. Once the pattern has been extracted, it will parse it, and it will compute hashes on blocks of CODE_BLOCK_CHUNKS_COUNT patterns, starting with a pivot instruction, which can be a codeInsJmp, codeInsCall or mov that involves memory or fs/gs segments.

Parameters

[in]	Buffer	The code buffer to be parsed.
[in]	MaxBufferSize	The size of the code buffer.
[in]	CsType	Operating mode, should be IG_CS_TYPE_32B or IG_CS_TYPE_64B.
[in]	ExtractLevel	cbLevelNormal or cbLevelMedium.
[in,out]	HashesCount	Will add to this variable the total number of hashes extracted.
[out]	Hashes	Will contain upon successful return the extracted hashes.

Return values

INT_STATUS_SUCCESS	On success.
INT_STATUS_INVALID_PARAMETER	If an invalid parameter is supplied.
INT_STATUS_NOT_FOUND	If no hash could be extracted.

Definition at line 746 of file codeblocks.c.

Referenced by IntExceptVerifyCodeBlocksSig().

◆ IntFragExtractCodePattern()

INTSTATUS IntFragExtractCodePattern	(	PBYTE	Buffer,
		DWORD	StartOffset,
		DWORD	MaxBufferSize,
		IG_CS_TYPE	CsType,
		CB_EXTRACT_LEVEL	ExtractLevel,
		DWORD	PatternSize,
		CODE_BLOCK_PATTERN *	Pattern,
		DWORD *	TotalExtracted
	)

Extract a pattern of code-blocks from the given code buffer.

This function will parse the provided code buffer, and it will extract a pattern of CODE_INS values representing the relevant instructions located inside the buffer. The pattern can then be used to compute hashes (code-blocks). This function will use the disassembler to decode each instruction inside the Buffer, and depending on the instruction type, a CODE_INS value will be outputted inside the Pattern buffer. This function may also call the optimized IntFragHandleCommon function which will try to handle the current instruction without calling the disassembler, but if it fails, it will still rely on it.

Parameters

[in]	Buffer	The code buffer to be parsed.
[in]	StartOffset	The offset to start the parsing at.
[in]	MaxBufferSize	The size of the code buffer.
[in]	CsType	Operating mode, should be IG_CS_TYPE_32B or IG_CS_TYPE_64B.
[in]	ExtractLevel	cbLevelNormal or cbLevelMedium.
[in]	PatternSize	Maximum size of the pattern.
[out]	Pattern	The pattern of instructions located in Buffer.
[out]	TotalExtracted	Number of CODE_INS values extracted from the Buffer into Pattern.

Return values

INT_STATUS_SUCCESS	On success.
INT_STATUS_INVALID_PARAMETER	If an invalid parameter is supplied.
INT_STATUS_DATA_BUFFER_TOO_SMALL	If the buffer is too small (the last instructions cannot be parsed).

Definition at line 990 of file codeblocks.c.

Referenced by IntAlertFillCodeBlocks(), IntFragDumpBlocks(), and IntSerializeExtractCodeBlocks().

◆ IntFragExtractPattern()

INTSTATUS IntFragExtractPattern	(	BYTE *	Buffer,
		DWORD	MaxBufferSize,
		IG_CS_TYPE	CsType,
		CB_EXTRACT_LEVEL	ExtractLevel,
		DWORD	PatternSize,
		BYTE *	Pattern,
		DWORD *	TotalExtracted,
		DWORD *	TotalParsed
	)

Extract a pattern of code-blocks from the given code buffer.

This function will parse the provided code buffer, and it will extract a pattern of CODE_INS values representing the relevant instructions located inside the buffer. The pattern can then be used to compute hashes (code-blocks). This function will use the disassembler to decode each instruction inside the Buffer, and depending on the instruction type, a CODE_INS value will be outputted inside the Pattern buffer. This function may also call the optimized IntFragHandleCommon function which will try to handle the current instruction without calling the disassembler, but if it fails, it will still rely on it.

Parameters

[in]	Buffer	The code buffer to be parsed.
[in]	MaxBufferSize	The size of the code buffer.
[in]	CsType	Operating mode, should be IG_CS_TYPE_32B or IG_CS_TYPE_64B.
[in]	ExtractLevel	cbLevelNormal or cbLevelMedium.
[in]	PatternSize	Maximum size of the pattern.
[out]	Pattern	The pattern of instructions located in Buffer.
[out]	TotalExtracted	Number of CODE_INS values extracted from the Buffer into Pattern.
[in,out]	TotalParsed	Will add to this variable the total size in bytes parsed from Buffer.

Return values

INT_STATUS_SUCCESS	On success.
INT_STATUS_INVALID_PARAMETER	If an invalid parameter is supplied.
INT_STATUS_DATA_BUFFER_TOO_SMALL	If the buffer is too small (the last instructions cannot be parsed).

Definition at line 502 of file codeblocks.c.

Referenced by IntFragExtractCodeBlocks().

◆ IntFragHandleCommon()

DWORD IntFragHandleCommon	(	const BYTE *	Buffer,
		size_t	BufSize,
		IG_CS_TYPE	CsType,
		CB_EXTRACT_LEVEL	ExtractLevel,
		DWORD *	Pattern
	)

Extract a pattern of instructions without using the disassembler.

This function leverages the fact that the majority of instructions that compose code-blocks are easy to decode without a disassembler. Therefore, it handles the most common cases without decoding the instruction, by using large switch-case/if-else statements.

NOTE: The instructions that have the 0x66 prefixes is not handled yet.

Parameters

[in]	Buffer	The code buffer to be parsed.
[in]	BufSize	Buffer size, in bytes.
[in]	CsType	Operating mode, should IG_CS_TYPE_32B or IG_CS_TYPE_64B.
[in]	ExtractLevel	How aggressive the extraction should be.
[out]	Pattern	Will contain, upon return, the CODE_INS type associated with the instruction present at Buffer.

Returns: The length of the parsed instruction, or 0 if the instruction could not be parsed.

Definition at line 37 of file codeblocks.c.

Referenced by IntFragExtractPattern().

◆ IntFragLogCodeBlocks()

static void IntFragLogCodeBlocks	(	CODE_BLOCK *	CodeBlock,
		DWORD	Count,
		QWORD	Rip,
		DWORD	RipOffset,
		BOOLEAN	ReturnRip,
		DWORD	ElemLine
	)

static

Log a block of code-blocks.

Parameters

[in]	CodeBlock	The list of code-blocks.
[in]	Count	Number of code-blocks.
[in]	Rip	The Rip the code-blocks start at.
[in]	RipOffset	Rip page offset (low 12 bits of Rip).
[in]	ReturnRip	The return Rip.
[in]	ElemLine	Number of elements to dump on one line.

Definition at line 1183 of file codeblocks.c.

Referenced by IntFragDumpBlocks().

◆ IntFragMatchSignature()

__pure INTSTATUS IntFragMatchSignature	(	const DWORD *	Hashes,
		DWORD	CodeBlocksCount,
		const SIG_CODEBLOCKS *	ExceptionSignature
	)

Match a block of code-block hashes against a list of code-block exception signatures.

This function will attempt to match the code-blocks located in the Hashes variable against the code-block signature list inside ExceptionSignature.

Parameters

[in]	Hashes	The list of hashes to be matched.
[in]	CodeBlocksCount	Number of hashes in Hashes.
[in]	ExceptionSignature	the exception signature containing the hashes to match against.

Return values

INT_STATUS_SIGNATURE_MATCHED	If the Hashes block matches a signature inside ExceptionSignature.
INT_STATUS_SIGNATURE_NOT_FOUND	If no match is found.
INT_STATUS_INVALID_PARAMETER	If an invalid parameter is supplied.

Definition at line 912 of file codeblocks.c.

Referenced by IntExceptVerifyCodeBlocksSig().

Variable Documentation

◆ gCbLog

CHAR gCbLog[512]

static

Used to format log lines containing code-blocks.

Definition at line 33 of file codeblocks.c.

Referenced by IntFragLogCodeBlocks().

Macros

Functions

Variables

Detailed Description

Macro Definition Documentation

◆ IS_REX_PREFIX

Function Documentation

◆ IntFragDumpBlocks()

◆ IntFragExtractCodeBlocks()

◆ IntFragExtractCodePattern()

◆ IntFragExtractPattern()

◆ IntFragHandleCommon()

◆ IntFragLogCodeBlocks()

◆ IntFragMatchSignature()

Variable Documentation

◆ gCbLog