You have arrived here because you have identified hat one or more of your processes are crashing due to an out of memory condition on your Windows computer. If this is not correct, then return to the Start of the Performance Guide.

Process Virtual Memory Concept

Processes are not aware of physical hardware. Each process is given its own, private, virtual address space which is a finite amount of virtual memory. This allows the Windows operating system to more efficiently manage physical memory resources (RAM and disk (page file)). If a process attempts to exceed it's virtual address space, then it will crash with an out of memory exception.

Determine the Maximum amount of Virtual Address Space

Each process has a finite amount of virtual address space that it cannot exceed. The amount of virtual memory per process depends on if it was compiled as 32-bit or 64-bit. x86 is the 32-bit implementation of Windows. x64 is the current 64-bit implementation of Windows.

• x86 processes have 2 GB of virtual address space by default
• x86 processes with the /3GB operating system switch, but without large address aware have 2 GB of virtual address space.
• x86 processes with the /3GB operating system switch, with large address aware have 3 GB of virtual address space.
• x86 processes with the /3GB operating system swtich, large address aware, and the /USERVA switch have virtual address space equal to the amount specified by the /USERVA switch.
• x86 processes with large address aware and running on an x64 operating system have 4 GB of virtual address space.
• x64 processes have 8 TB of virtual address space.

Important: Use caution before implementing the /3GB operating switch because it takes away virtual memory from the Windows kernel and can result in system-wide hangs.

Dumpbin.exe (Windows SDK) can determine if a 32-bit executable (EXE) supports large address spaces with the following command:

dumpbin /headers <path to EXE>

The following WMI command will show the maximum virtual address space that a process can have on the computer. This command will work on Windows XP/2003 and greater:

wmic PATH Win32_OperatingSystem GET MaxProcessMemorySize

Sample output of this command for a Windows 7 x64 computer is:

MaxProcessMemorySize

8589934464

VMMap.exe (SysInternals) can attach to a live process and show the virtual memory usage. This can tell you the maximum amount of virtual memory the process can address and is helpful in diagnosing virtual memory fragmentation. When the virtual address space of a process does not have enough contiguous memory to accommodate a virtual memory allocation request, then it will crash. For example, a process might have 500 MB of free virtual memory, but might not have a contiguous block of 64 MB to accommodate a new Gen 0 memory block request from the .NET memory manager.

Windows - Virtual Memory Minimum Too Low

When the Windows operating system reports that it is low on "virtual memory" and mentions increasing the paging file, then it means that the System Commit Charge has reached the System Commit Limit. The paging file is commonly referred to as "virtual" RAM, hense this reference. Unfortunately, the concept of process and kernel virtual memory is unrelated to the system commit limit. If you suspect a system commit issue, then go to "PerfGuide: Out of System Committed Memory" to troubleshoot.

Troubleshooting

If the "\Process(*)\Virtual Bytes" performance counter shows more than 80% of the maximum amount of virtual address space of a process, then the process is likely close to running out of virtual memory with a few exceptions - processes like 32-bit SQL Server commonly consume nearly 100% of its virtual address space on purpose. Also, out of [virtual] memory crashes might be logged to the Application event log such as System.OutOfMemory exceptions for .NET applications. To troubleshoot this issue:

1. Is a Virtual Memory Leak: Monitor the “\Process(*)\Virtual Bytes” counter for each process over a long period of time (typically over a week) to determine if it is gradually leaking memory over time. It is normal for an application to use more memory as it is under load, but it should release the memory when it is done. Gradually, memory that is not released accumulates until the process it out of virtual memory.
2. Use VMMap.exe (SysInternals): to show the virtual memory usage of the process. It can give clues to where memory is generally being allocated. This information can greatly help developers of the application.
3. Create Several Process Memory Dumps (*.dmp): There is many tools in the industry that are able to create process memory dumps such as ADPlus.exe (part of the Debugging Tools for Windows) and DebugDiag. A process memory dump is a snapshot of the memory usage of a process. This is similar to taking a photograph of an event. If several process dumps (each with a significant change in memory usage) of a process are analyzed by a Windows debugging expert, then it may be possible to determine where memory is being allocated unnecessarily. Please contact Microsoft Support Services for assistance with analysis of process memory dumps and/or consider the “Advanced User Mode Debugging” and “Advanced .NET Debugging” workshops offered by Microsoft Premier Field Engineering (PFE).
4. Use DebugDiag: DebugDiag is a free tool from Microsoft designed to assist in troubleshooting issues such as hangs, slow performance, memory leaks or fragmentation, and crashes in any user-mode process. Use it to create several process memory dumps (*.dmp) (each with significant change in memory usage) and automatically analyze the differences between the dump files. If you need assistance with process memory dump analysis, then go to step 3.

Treating the Symptoms

If all of the above troubleshooting steps have been exhausted, then the symptoms of process out of virtual memory issues can be treated with one or more of the following:

1. Use Large Address Aware for 32-bit Application: Enable the large address aware property/header of the process to allow it the option of using more than 2 GB of virtual address space. The benefits of large address aware are shown in the “Determine the Maximum amount of Virtual Address Space” section of this document.
2. Migrate or Recompile to x64 (64-bit): x64 applications have 8 TB of private virtual address space which should be enough to accommodate the memory needs of most applications.
3. Distribute to Multiple Processes: Each process has its own virtual address space, so consider moving heavy memory consuming functions to separate processes.
4. Defragment Virtual Memory: High virtual memory fragmentation can prevent a memory allocation from finding contiguous address space. Memory managers like the Low Fragmentation Heap can help.
   1. Microsoft .NET applications are automatically defragmented by the .NET garbage collector (GC), but it can only defragment .NET managed structures. Unmanaged structures that have pointers to memory cannot be moved by the GC preventing proper defragmentation.

Note: Keep in mind that processes are not aware of physical resources, so changing the amount of RAM or page file size has no effect on process virtual address space.

More Information

Vital Signs Workshop: Microsoft Services offers an instructor led workshop called, "Vital Signs", which goes in depth into Windows architecture focused on Windows performance analysis. If you are interested, then contact your Microsoft Technical Account Manager (TAM). If you do not have a Microsoft Premier Support contract, then consider the great benefits of having one by going to our Microsoft Services Premier Support web site at: http://www.microsoft.com/microsoftservices/en/us/support_premier.aspx