locked
HyperV memory per NUMA node RRS feed

  • Question

  • have some issue with memory allocation on my Hyper-V 2012 R2 server. Server has 16GB of RAM, with 2 x 12 core CPUs. When I run Get-VMHostNumaNode, I am getting following results:

    NodeId : 0 ProcessorsAvailability : {0, 0, 0, 0...} MemoryAvailable : 6479 MemoryTotal : 8003 ComputerName : SRV NodeId : 1 ProcessorsAvailability : {0, 0, 0, 0...} MemoryAvailable : 6632 MemoryTotal : 8192 ComputerName : SRV

    This shows that available memory for both NUMA nodes is 13.1 GB (of 16GB total).
    OS should take some memory for itself but there is to big 3GB of memory difference.

    On the other hand, when I check free memory in servers task manager it show that 15.5 GB is free.

    If I try to run VMs on server, maximum memory that can be allocated is around 13GB.
    I can run 1, 2, 3 or more always same result maximum 13GB of memory combined.

    Any suggestions what it can be or in what direction I should search?






    • Edited by Iwan-Iwan Tuesday, January 26, 2016 12:13 PM
    Tuesday, January 26, 2016 9:04 AM

Answers

  • Believe the 13GB value.  Hyper-V likes to use about 2GB and then there is overhead for each virtual machine running. 

    . : | : . : | : . tim

    • Proposed as answer by Leo Han Sunday, February 7, 2016 1:18 PM
    • Marked as answer by Leo Han Tuesday, February 9, 2016 8:49 AM
    Tuesday, January 26, 2016 2:04 PM
  • The discrepancy is due to "Standby Memory". Get-VMHostNumaNode treats this memory as unusable. Task Manager treats it as available, but does mark it in one of its displays.

    I have an article that explains this and includes a PowerShell script so that you can see the range: http://www.altaro.com/hyper-v/hyper-v-powershell-quickly-retrieve-available-host-memory/


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    • Proposed as answer by Leo Han Sunday, February 7, 2016 1:18 PM
    • Marked as answer by Leo Han Tuesday, February 9, 2016 8:49 AM
    Wednesday, January 27, 2016 2:03 PM

All replies

  • Believe the 13GB value.  Hyper-V likes to use about 2GB and then there is overhead for each virtual machine running. 

    . : | : . : | : . tim

    • Proposed as answer by Leo Han Sunday, February 7, 2016 1:18 PM
    • Marked as answer by Leo Han Tuesday, February 9, 2016 8:49 AM
    Tuesday, January 26, 2016 2:04 PM
  • Hyper-V memory consumption 300 MB for the Hypervisor plus 32 MB for the first GB of RAM allocated to each virtual machine plus another 8 MB for every additional GB of RAM allocated to each virtual machine plus 512 MB for the host operating system running on the root partition. This is from official MS docs.
    Server is clean Hyper-V with no VMs running. That's what is odd.


    • Edited by Iwan-Iwan Tuesday, January 26, 2016 3:14 PM
    Tuesday, January 26, 2016 2:52 PM
  • "Hyper-V memory consumption 300 MB for the Hypervisor"

    You are talking about the micro-kernel for the hypervisor.  That does not include any of the supporting software, such as things like device drivers, management services, utilities, etc.  Remember that what you interact with is the parent partition where all this is running and the micro-kernel is handling the hardware specific sharing.  If you read anything else about Hyper-V, you will see that the general recommendation is to expect that the hypervisor, parent partition, etc. to use about 2GB.  So what you are seeing is pretty much what is expected.


    . : | : . : | : . tim

    Tuesday, January 26, 2016 8:59 PM
  • This probably looks like I am trying to convince you into something, but actually I am not :). First, you are right about this around 2GB consumption expectancy. Second, in my post I focused on precise problem, and tried not to widen it. Original problem started when I tried to start 3 VMs that allocates 8GB, 4GB and 1GB of RAM. I never succeeded to run last (1GB) machine, because every time I did, it produced error "Not enough memory in the system to start the virtual machine". Any combination of VMs memory results with same error: 1+4+8 or 9+4 or 13 GB... As you can see, 12Gb of 16GB can be used and that there is 4GB more that is not usable. Yes, some of it is used for "hypervisor, parent partition, etc.", but unlikely all 4Gb. Maybe, I would say that all of this is ok If I don't already have 2 servers with Hyper-V server, but version is 2008, where 4 VMs are running with 8GB, 4GB, 2Gb and 512MB - allocated memory is 14.5 GB, 1.5Gb for hypervisor, OS, etc..., and that's what is expected. That is story for 2008 version. For this "problematic" 2012 R2 version, as you can see, not the case. Interesting thing is that when Get-VMHostNumaNode shows that there is 13GB free, task manager shows that there is 15GB free. So my conclusion is that there is free memory, but it's not assigned to hyper-v for some, to me, unknown reason. This 1-2Gb unavailable memory is not big problem, but my concerns are if this is some percentage issue, and since we are planing to install 32GB or more, will be there all 32 GB available or, again, 80% of it? I will finish my "memoirs" with this, hoping that I was not too boring.
    Wednesday, January 27, 2016 9:43 AM
  • I have never been able to pin down the exact amount of memory available to VMs on a Hyper-V system.  What I have seen is that as I try to squeeze that last VM into what appears to be a block of memory large enough to hold it, it almost never fits and I have to shrink the VM a little to get it to fit.  I would have absolutely no fears of adding 32GB of RAM to the system and fully expect to be able to use almost all of it for VMs.  Of course, the overhead for each VM will need to be taken into consideration when calculating how many will fit, but I have not found a formula that is 100% accurate 100% of the time because everybody is running different combinations of tasks on the server and they each consume memory differently.

    . : | : . : | : . tim

    Wednesday, January 27, 2016 1:20 PM
  • The discrepancy is due to "Standby Memory". Get-VMHostNumaNode treats this memory as unusable. Task Manager treats it as available, but does mark it in one of its displays.

    I have an article that explains this and includes a PowerShell script so that you can see the range: http://www.altaro.com/hyper-v/hyper-v-powershell-quickly-retrieve-available-host-memory/


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    • Proposed as answer by Leo Han Sunday, February 7, 2016 1:18 PM
    • Marked as answer by Leo Han Tuesday, February 9, 2016 8:49 AM
    Wednesday, January 27, 2016 2:03 PM
  • Thank you Eric for your response. I think that something else is going on, because numbers at my server don't match. Well, this are numbers:

    Task manager
    In use 13.1 GB
    Available: 2.7 GB
    Memory consumption
    Free: 2422 MB
    Stand by memory: 373 MB

    Your Script:
    UpperBound: 2793 MB
    MemoryAvailable: 722 MB

    As you can see question is not about 50MB or 500MB, but about fully 2GB.

    I don't know if it is important, but for initial VM import I used 5Nine manager, that was, let's say ok, but I would not use it again. I had to use it, due to Win 7 inability to manage 2012 Hyper-V. Maybe, 5nine created some settings that I overlooked? This sounds a bit weird, but now I am thinking about anything that could make problem.

    Thursday, January 28, 2016 12:35 PM
  • I can't discount the possibility that something is really wrong on your system, but I did just discover a fairly severe bug in the script that I shared (I fixed it and updated the post).

    Whatever is going on, you should be able to look at Task Manager and look at Get-VMHostNumaNode and be able to more or less see where the discrepancy is, in one way or another. I ran the corrected script on various hosts and Task Manager's "Available" minus "Cached" is fairly close to what I'm getting for my "LowerBoundMB". On large memory hosts, I'm seeing the difference hit 6GB and sometimes more, so I don't think that a 3GB difference is all that strange. Most of that memory should be available to you on demand. I would do testing with empty VMs and see if you can start one whose memory is between Task Manager's Available and the values shown in Get-VMHostNumaNode.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Thursday, January 28, 2016 2:43 PM