KASAN_ILLEGAL_ACCESS (0x1F2)<\/em> bug check. The parameters of this bug check include useful information for debugging, such as the type of memory being accessed (heap, stack, and so forth), the number of bytes being accessed, whether the access is a read or a write, along with additional metadata.<\/li>\n<\/ul>\n\n\n\nThe ASAN instrumentation therefore constitutes the KASAN verification logic: it verifies whether each memory byte that the component accesses at run time is marked as valid in the KASAN shadow, and triggers a bug check if not, to report any illegal memory access.<\/p>\n\n\n\n
Challenges with the instrumentation<\/h4>\n\n\n\n There were several challenges in getting the ASAN instrumentation to work in KASAN, especially in cases where the compiler inserts a bytecode instead of a function call.<\/p>\n\n\n\n
Using the correct calculation<\/h5>\n\n\n\n The expected calculation to get the shadow address of a regular address is the following:<\/p>\n\n\n\n
Shadow(Address) = ShadowBaseAddress + OffsetWithinAddressSpace(Address) \/ 8<\/pre>\n\n\n\nIn the context of the user-mode AddressSanitizer, the user-mode address space starts at address 0x0. Therefore, any user-mode address is equal to the offset of that address within the user-mode address space:<\/p>\n\n\n\n
OffsetWithinUserAddressSpace(UserAddress) = UserAddress \u2013 0\n\t = UserAddress\n<\/pre>\n\n\n\nFor this reason, the verification bytecodes inserted as part of the ASAN instrumentation use the following formula, where __asan_shadow_memory_dynamic_address<\/em> contains the base address of the shadow:<\/p>\n\n\n\nShadow(Address) = __asan_shadow_memory_dynamic_address + (Address \/ 8)<\/pre>\n\n\n\nThe following is an example of a generated bytecode assembly:<\/p>\n\n\n\n
mov rcx, cs:__asan_shadow_memory_dynamic_address\nshr rax, 3\nadd rcx, rax<\/pre>\n\n\n\nHere the bytecode calculates the shadow address of RAX by reading __asan_shadow_memory_dynamic_address<\/em> and adding to it RAX right-shifted by 3 (meaning divided by 8). This implements the aforementioned formula to get the shadow of an address.<\/p>\n\n\n\nIn the context of the kernel AddressSanitizer however, the kernel-mode address space starts at address 0xFFFF800000000000:<\/p>\n\n\n\n
OffsetWithinKernelAddressSpace(KernelAddress) = KernelAddress \u2013\n 0xFFFF800000000000\n\t != KernelAddress<\/pre>\n\n\n\nTherefore, reusing the simplified formula as-is in KASAN would not be correct: it would result in the bytecodes using the wrong shadow addresses when verifying memory accesses. We solved that issue in KASAN by initializing __asan_shadow_memory_dynamic_address<\/em> to a different value that is not exactly the KASAN shadow base address:<\/p>\n\n\n\n__asan_shadow_memory_dynamic_address = KasanShadowBaseAddress \u2013\n\t (0xFFFF800000000000 \/ 8)\n<\/pre>\n\n\n\nDeveloping the formula using this value gives the following:<\/p>\n\n\n\n
Shadow(KernelAddress) = __asan_shadow_memory_dynamic_address + (KernelAddress \/ 8)\n = KasanShadowBaseAddress \u2013 (0xFFFF800000000000 \/ 8) + (KernelAddress \/ 8)\n = KasanShadowBaseAddress + (KernelAddress - 0xFFFF800000000000) \/ 8\n = KasanShadowBaseAddress + OffsetWithinKernelAddressSpace(KernelAddress) \/ 8\n<\/pre>\n\n\n\nThe formula therefore falls back to the expected calculation with KASAN: the bytecodes take the base address of the KASAN shadow, add to it the offset of the address within the kernel address space divided by 8, and this results in the correct shadow address for the given kernel address.<\/p>\n\n\n\n
Using this trick, we avoided the need to make a compiler change to modify the bytecode generation for KASAN.<\/p>\n\n\n\n
Dealing with non-tracked memory<\/h5>\n\n\n\n When we described how the KASAN shadow is mapped, we did not explain why we were using a splitting mechanism with a zeroed page. The reason is simple: the verification bytecodes always want to read the shadow of the buffers they verify, and they do not have any knowledge about whether a buffer has a shadow that backs it or not. There must therefore always be a shadow mapped for every byte of kernel virtual address memory, and we achieve that thanks to the splitting mechanism, which guarantees that a shadow always exists while minimizing the memory consumption of the non-tracked regions by having their shadow point to a single zeroed physical page.<\/p>\n\n\n\n
The fact that the physical page used in the splitting mechanism is full of zeroes causes KASAN to always consider non-tracked memory as valid.<\/p>\n\n\n\n
Dealing with user-mode pointers<\/h5>\n\n\n\n The Windows kernel and its drivers are allowed to directly access user-mode memory, for example to fetch the user-mode arguments passed to a syscall. This creates an issue with the verification bytecodes, because they need to get the shadow of an address that is outside of the kernel address space and that therefore does not have a shadow.<\/p>\n\n\n\n
To deal with this case, we pass a compiler flag as part of KASAN that instructs the compiler to never use bytecodes and always prefer function calls to __asan_*()<\/em>, except<\/strong> when the compiler is certain that the accesses are to stack memory.<\/p>\n\n\n\nThis means in practice that in order to verify the accesses to local variables on the stack, the compiler generates verification bytecodes, but for any other access the compiler uses function calls to __asan_*()<\/em>. Given that these __asan_*()<\/em> functions are implemented in the KASAN runtime, we have full control over their verification logic and can make sure to exclude user-mode pointers from the verification via a simple if<\/em> condition.<\/p>\n\n\n\nUsing this trick, we again avoided the need to make a compiler change to have the instrumentation deal with user-mode pointers.<\/p>\n\n\n\n
Telling the kernel to export KASAN support<\/h4>\n\n\n\n By default, the kernel does not create the KASAN shadow, and does not export the KASAN runtime. In other words, it does not make KASAN available to drivers by default. For this to be done, the user must explicitly set the following registry key:<\/p>\n\n\n\n
HKLM\\System\\CurrentControlSet\\Control\\Session Manager\\Kernel\\KasanEnabled<\/p>\n\n\n\n
The bootloader reads this key at boot time and decides based on its value whether or not to instruct the kernel to make KASAN support available to drivers.<\/p>\n\n\n\n
With this established, the following sections contain details of how the kernel loads drivers compiled with KASAN.<\/p>\n\n\n\n
Loading kernel drivers with KASAN<\/h4>\n\n\n\n For drivers compiled with KASAN, a small code-only library called KasanLib<\/em> is linked into the final driver binary and does two things:<\/p>\n\n\n\n\nIt declares an __asan_shadow_memory_dynamic_address<\/em> global variable that remains local to the driver itself and is not exported to the kernel namespace. The verification bytecodes described earlier that get inserted into the driver use this global variable as part of their calculation of the KASAN shadow.<\/li>\n\n\n\nIt publishes a section called \u201cKASAN\u201d in the resulting PE binary of the driver. This section contains information and metadata, the format of which may change in the future and is not relevant to discuss here.<\/li>\n<\/ol>\n\n\n\nUpon loading a driver, the kernel verifies whether the driver has a \u201cKASAN\u201d section, and can take two paths:<\/p>\n\n\n\n
\nIf the driver has a \u201cKASAN\u201d section and the KasanEnabled<\/em> registry key is not set, then the kernel will refuse to load the driver. This is to prevent the system from malfunctioning; there is, after all, no way the driver is going to work, since it will try to use a shadow that wasn\u2019t created by the kernel and call a runtime that the kernel does not export.<\/li>\n\n\n\nIf the driver has a \u201cKASAN\u201d section and the KasanEnabled<\/em> registry key is set, then the kernel parses this section in order to initialize KASAN on the driver. Part of this initialization includes setting the value described earlier in the driver\u2019s __asan_shadow_memory_dynamic_address<\/em> global variable. After this initialization is complete, the \u201cKASAN\u201d section is no longer used and is discarded from memory to save up kernel memory.<\/li>\n<\/ol>\n\n\n\nFrom then on, the driver can start executing.<\/p>\n\n\n\n
How it all falls together: example of a buggy driver<\/h4>\n\n\n\n We have now exposed all the ingredients required for KASAN to work on drivers: how the shadow is created, how the instrumentation operates, how the kernel exports KASAN support, and how KASAN gets initialized on drivers when they are loaded. To give an example of how it all falls together, let\u2019s consider a hypothetical driver that we compiled with KASAN.<\/p>\n\n\n\n
We have set the KasanEnabled<\/em> registry key in our system, and the kernel has therefore created a KASAN shadow and is exporting the KASAN runtime. We proceed to load the driver in the system. The kernel sees that the PE of the driver has a \u201cKASAN\u201d section, parses it, initializes KASAN on the driver, and discards the section. The driver finally starts executing.<\/p>\n\n\n\nLet\u2019s assume that the driver contains this buggy code:<\/p>\n\n\n\n
PCHAR buffer;\nbuffer = ExAllocatePool2(POOL_FLAG_NON_PAGED, 18, WHATEVER_TAG);\nbuffer[18] = 'a';<\/pre>\n\n\n\nHere, a heap buffer of size 18 bytes is allocated. During this allocation, the KASAN runtime initialized two redzones below and above the buffer, as described earlier. Then an \u2018a\u2019 is written into the 19th<\/sup> byte of the buffer. This is, of course, an out-of-bounds write access, which is incorrect and can cause a serious security risk.<\/p>\n\n\n\nGiven that our driver was compiled with KASAN, it was subject to the ASAN instrumentation, meaning that the actual compiled code looks like the following:<\/p>\n\n\n\n
PCHAR buffer;\nbuffer = ExAllocatePool2(POOL_FLAG_NON_PAGED, 18, WHATEVER_TAG);\n__asan_store1(&buffer[18]);\nbuffer[18] = 'a';<\/pre>\n\n\n\nHere the compiler inserted a function call to __asan_store1()<\/em> and did not choose a verification bytecode because it couldn\u2019t conclude that \u201cbuffer\u201d was a pointer to stack memory (which it is not).<\/p>\n\n\n\n__asan_store1()<\/em> is part of the KASAN runtime that the kernel exported and that the driver imported. This function looks at the shadow of &buffer[18]<\/em>, sees that it is marked as invalid (because the byte at this address is part of the right redzone of the buffer), and proceeds to issue a KASAN_ILLEGAL_ACCESS bug check to halt system execution.<\/p>\n\n\n\nAs the owners of the system, we can then collect the crash dump and investigate what was the memory safety bug that KASAN detected using the actionable information provided alongside the KASAN bug check.<\/p>\n\n\n\n
Without KASAN, this bug would not be easily observed. With KASAN, however, it is immediately detected before<\/strong> the bug triggers and turns into a real security risk. As such, KASAN is able to detect whole classes of memory bugs that could otherwise remain undiscovered.<\/p>\n\n\n\nGranularity, and memory regions covered<\/h3>\n\n\n\n As can be deduced from the details we provided thus far, KASAN operates at the byte granularity. KASAN is currently able to detect illegal memory accesses on several types of memory regions:<\/p>\n\n\n\n
\nThe global variables<\/li>\n\n\n\n The kernel stacks<\/li>\n\n\n\n The pool allocators (ExAllocatePool*()<\/em>)<\/li>\n\n\n\nThe lookaside list allocators (ExAllocateFromLookasideListEx()<\/em>, etc.)<\/li>\n\n\n\nThe IO\/contiguous allocators (MmMapIoSpaceEx()<\/em>, MmAllocateContiguousNodeMemory()<\/em>, etc.)<\/li>\n<\/ul>\n\n\n\nInternally, the support for these regions is implemented using a KASAN API that is also exported by the NTOS kernel. Microsoft will continue to improve this API to expand its implementation to other scenarios.<\/p>\n\n\n\n
Thanks to the ability to detect bugs at the byte granularity and the large number of memory regions covered, KASAN exceeds the capabilities of existing bug-detection technologies such as the Special Pool, which typically operate at a coarser granularity and do not cover the kernel stacks and other regions.<\/p>\n\n\n\n
Naturally, the KASAN shadow consumes memory, and the validity checks inserted by the ASAN instrumentation consume CPU time and increase the size of the compiled binaries.<\/p>\n\n\n\n
Some effort has gone into micro-optimizing KASAN by limiting the number of instructions that the KASAN runtime emits, by making the KASAN shadow NUMA-aware, by compressing the KASAN metadata in order to reduce the binary sizes, and so forth.<\/p>\n\n\n\n
Overall, KASAN currently introduces a ~2x slowdown, which we measured using widely available benchmarking tools. As such, KASAN cannot be seen as a production feature since its performance cost is not negligible. This cost, however, is acceptable for debug, development, stress-testing, or security-related setups.<\/p>\n\n\n\n
It should be noted that this cost is higher than that of existing technologies, such as the Special Pool, and that KASAN does not have a performance impact when not explicitly enabled. In other words, KASAN does not affect the performance of Windows 11 by default.<\/p>\n\n\n\n
Microsoft generates special builds of Windows, called MegaAsan<\/em> builds, that produce fully bootable Windows disks that have KASAN enabled on the Windows kernel and on more than 95% of all kernel drivers shipped by Microsoft in Windows 11.<\/p>\n\n\n\nBy using these builds in testing, fuzzing, but also in simple desktop setups, MORSE has been able to identify and fix more than 35 memory safety bugs in various drivers and in the Windows kernel, that were not previously detectable by existing technologies.<\/p>\n\n\n\n
We also implemented KASAN support for the Xbox kernels and the drivers they load, and similarly generate builds of Xbox systems with KASAN enabled. As such, KASAN also contributes to the quality and security of the Xbox product line.<\/p>\n\n\n\n
Extending ASAN to the other ring0 domains<\/h2>\n\n\n\n We have so far discussed KASAN on Windows kernel drivers and on the Windows kernel:<\/p>\n\n\n\nFig. 8: KASAN in the operating system<\/figcaption><\/figure>\n\n\n\nHaving KASAN is a considerable step forward, because it provides precise detection of memory errors on large and critical parts of the system in a way that wasn\u2019t achievable before. Following up on our work on KASAN, we developed similar detection capabilities on the remaining parts of the system.<\/p>\n\n\n\n
Introducing SKASAN\u2026<\/h3>\n\n\n\n The Secure kernel is a different kernel, completely separated from the Windows kernel, that executes in a more privileged domain and is in charge of a number of security operations in the system. It is part of virtualization-based security on Windows.<\/p>\n\n\n\n
We developed the Secure Kernel AddressSanitizer (SKASAN), which covers the Secure kernel and a few of the modules it loads dynamically.<\/p>\n\n\n\n
SKASAN has a number of similarities with KASAN. For example, the SKASAN support for Secure Kernel modules is implemented using a \u201cSKASAN\u201d section, comparable to the \u201cKASAN\u201d section used in Windows kernel drivers. Overall, SKASAN works similarly to KASAN, but simply applied to the Secure kernel domain.<\/p>\n\n\n\n
\u2026and HASAN<\/h3>\n\n\n\n Finally, Hyper-V is the Microsoft hypervisor that plays a central role on Windows and in Azure, and it too could benefit from the capabilities that ASAN provides; we therefore developed the Hyper-V AddressSanitizer (HASAN) which is yet another ASAN implementation but tied to the Hyper-V kernel.<\/p>\n\n\n\n
Same, but different\u2026 but still same<\/h3>\n\n\n\n KASAN, SKASAN, and HASAN are built on the same logic, which is having a shadow and a compiler instrumentation, and overall have similar costs in terms of memory consumption and slowdown.<\/p>\n\n\n\n
Some inherent differences do exist, however. First, the Windows kernel, Secure kernel, and Hyper-V kernel have different allocators, and the *ASAN support for them differs accordingly. Second, the memory layout of these kernels is not the same, and this leads to drastic implementation differences; for instance, HASAN actually uses two different shadows concatenated together.<\/p>\n\n\n\n
We leave the rest of the technical differences as a reverse engineering exercise to interested readers.<\/p>\n\n\n\n
The final picture, and results<\/h2>\n\n\n\n As of November 2022, we have developed and stabilized KASAN, SKASAN, and HASAN. Combined together, these deliver precise detection of memory errors on all<\/strong> the kernel-mode components that execute on Windows 11:<\/p>\n\n\n\nFig. 9: all *ASAN implementations in the operating system<\/figcaption><\/figure>\n\n\n\nWe produce internal MegaAsan<\/em> builds with all of these *ASAN implementations enabled, and internal teams are using them in a number of fuzzing and stress-testing scenarios. As a result, we have been able to identify and fix dozens of memory bugs of various severity:<\/p>\n\n\n\nFig. 10: Types of bugs found by *ASAN<\/figcaption><\/figure>\n\n\n\nFinally, as part of our *ASAN work we have also applied numerous improvements and cleanups to various areas, such as the Windows and Hyper-V kernels, but also to the Microsoft Visual C++ (MSVC) compiler to improve the *ASAN experience on Microsoft platforms.<\/p>\n\n\n\n
Overall, these *ASAN features have the potential to eliminate whole classes of memory bugs and, going forward, will significantly contribute to ensuring the quality and security of the Microsoft products.<\/p>\n\n\n\n
<\/p>\n\n\n\n
This concludes our first blog post on kernel sanitizers. Beyond *ASAN, we have implemented several other sanitizers that specialize in uncovering other classes of bugs. We will communicate about them in future posts.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"We share technical details of our work on the AddressSanitizer (ASAN) and how it contributes to durably improving software quality and security at Microsoft.<\/p>\n","protected":false},"author":68,"featured_media":125717,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"content-type":[3663],"topic":[3687],"products":[],"threat-intelligence":[3739],"tags":[],"coauthors":[2766],"class_list":["post-125692","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","content-type-research","topic-threat-intelligence","threat-intelligence-vulnerabilities-and-exploits","review-flag-1694638265-576","review-flag-1694638265-310","review-flag-1694638265-83","review-flag-1-1694638265-354","review-flag-2-1694638266-864","review-flag-3-1694638266-241","review-flag-4-1694638266-512","review-flag-5-1694638266-171","review-flag-6-1694638266-691","review-flag-7-1694638266-851","review-flag-8-1694638266-352","review-flag-9-1694638266-118","review-flag-alway-1694638263-571","review-flag-never-1694638263-791","review-flag-new-1694638263-340"],"yoast_head":"\n
Introducing kernel sanitizers on Microsoft platforms | Microsoft Security Blog<\/title>\n \n \n \n \n \n \n \n \n \n \n \n \n\t \n\t \n\t \n \n \n \n\t \n\t \n\t \n