none
How many available vCPU's would I have....

    Frage

  • Ok, I've read lots about vCPU's and how many per core, per physical socket, etc., and I just want to clear this up before I pull the trigger on a purchase.  I'm planning a new Hyper-V server running on dual Xeon X5690 processors.  Each processor has 6-cores, 12 threads and runs @3.46GHz.  So, my math from what I've read says I will have a safe minimum of 96 vCPU's available for whatever mix of guests I choose.  (2CPUs * 6cores * 8ratio)  Is this correct?  Does HyperThreading add to the number of vCPU's that would be available?  How about the 12 threads that each CPU runs?  In addition, each CPU will have 72GB of RAM, for a total of 144GB per server.  Will Windows 2008R2/Hyper-V see all 144GB of RAM and make that available to the mix of guests?  Thanks!

    Mittwoch, 2. Mai 2012 16:16

Antworten

  • never include hyperthreading in vCPU counts.  All you should concern youself with is cores and threads (technically).

    The conservative estimate is 1 core has 8 threads.  Therefore 1 core gives 8 vCPU.  (becuase that was the common number when most guidance was written).

    If you know the thread cound of your system is higher, then you get more vCPU.  Lower and you get less.  Some processors report the threads as Logical Processors.  It is the same, logical processor threads...

    Each VM gets a slice of a physical RAM - don't think RAM per Core as you do when running an applicatoin server on bare metal - this works totally different - all resources are subdivided.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

    Mittwoch, 2. Mai 2012 16:56
    Moderator
  • Hi!

    Please note that I'm not a Microsoft representative.

    According to the Microsoft virtualization product team blog post here: http://blogs.technet.com/b/virtualization/archive/2011/04/25/hyper-v-vm-density-vp-lp-ratio-cores-and-threads.aspx

    There is an implication that you might take hyperthreading into count when calculating your expected limit of vCPU consumption, but there are plenty of articles and experts that says that you shouldn't. I personally haven't gone deep on that one.

    There is no upper limit on how many vCPU's you are allowed to assign to virtual machines. There is of course a technical limit somewhere, but that is a Hyper-V boundary and totally irrelevant to what kind of equipment you have.

    Just keep in mind that in common virtualization scenarios, CPU is rarely the bottleneck. The 2*6*8 equation is a rule of thumb more than anything and it really comes down to what kind of virtual machines your runing and what kind of workload the hold. A combination of performance monitoring and common sense will get you very far in vCPU planning.

    Mittwoch, 2. Mai 2012 16:39

Alle Antworten

  • Hi!

    Please note that I'm not a Microsoft representative.

    According to the Microsoft virtualization product team blog post here: http://blogs.technet.com/b/virtualization/archive/2011/04/25/hyper-v-vm-density-vp-lp-ratio-cores-and-threads.aspx

    There is an implication that you might take hyperthreading into count when calculating your expected limit of vCPU consumption, but there are plenty of articles and experts that says that you shouldn't. I personally haven't gone deep on that one.

    There is no upper limit on how many vCPU's you are allowed to assign to virtual machines. There is of course a technical limit somewhere, but that is a Hyper-V boundary and totally irrelevant to what kind of equipment you have.

    Just keep in mind that in common virtualization scenarios, CPU is rarely the bottleneck. The 2*6*8 equation is a rule of thumb more than anything and it really comes down to what kind of virtual machines your runing and what kind of workload the hold. A combination of performance monitoring and common sense will get you very far in vCPU planning.

    Mittwoch, 2. Mai 2012 16:39
  • More like 4 VCpu for 1.

    But its recommend, you can always go ahead with more and see if that works for you. Do tests as always.

    See this on 17:54 to 18:25

    http://channel9.msdn.com/Events/TechEd/NewZealand/2010/SVR313

    And also, they give you a code for Powershell to asset your ratio. That could help you.

    About Hyper threading, it depends.

    The old version of Hyper threading is bad (the first version released by Intel), and the new Hyper threading is good.

    You can enable it, however, do not count them as they will double your CPU proccesors.

    See from 29:46 to 32:05

    For memory, remember to leave at least 1 Gb available to the host.

    Mittwoch, 2. Mai 2012 16:47
  • never include hyperthreading in vCPU counts.  All you should concern youself with is cores and threads (technically).

    The conservative estimate is 1 core has 8 threads.  Therefore 1 core gives 8 vCPU.  (becuase that was the common number when most guidance was written).

    If you know the thread cound of your system is higher, then you get more vCPU.  Lower and you get less.  Some processors report the threads as Logical Processors.  It is the same, logical processor threads...

    Each VM gets a slice of a physical RAM - don't think RAM per Core as you do when running an applicatoin server on bare metal - this works totally different - all resources are subdivided.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

    Mittwoch, 2. Mai 2012 16:56
    Moderator
  • while its unlikely that you use standard edition on this machine config, keep in mind that it would have a memory limit of 32gb (http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx#physical_memory_limits_windows_server_2008_r2)
    Mittwoch, 2. Mai 2012 17:00
  • I saw that link before and I see they list 8:1 as the supported ratio for vCPU to logical CPU.  Either way, 4:1 or 8:1, I will still almost double my current ESX hosts that we are migrating from.  I have two hosts, and each host has approx 25 vCPU's in use on much smaller hardware.  I know it's not exactly an apples to apples comparison (ESX to Hyper-V), but I'm pretty confident that it's close enough that CPU resources won't be my bottleneck in the future just as it isn't now. 

    Sorry, wrong link I was referring to.  Here is the link I mentioned, not yours.

    http://technet.microsoft.com/en-us/library/ee405267%28WS.10%29.aspx

    • Bearbeitet BClark22 Mittwoch, 2. Mai 2012 18:31
    Mittwoch, 2. Mai 2012 18:18
  • You are correct, I'm looking at Enterprise or Datacenter, depending on the licensing model.  I will have for starters 12 server guests on each Hyper-V host, only going to go up from there.  I remember somewhere reading the "break-even" point in regards to licensing on Enterprise or Datacenter versions, just need to find it again and do some calculations.

    Mittwoch, 2. Mai 2012 18:21
  • CPU is very rarely a bottleneck unless you get into a situation where you are over-subscribed.  Which can be done on any hypervisor.

    The 8 vCPU to 1 physical core is a very (highly) conservative number. 

    (MSFT used to talk logical cores but has since stopped as it is too confusing as the hardware vendors don't all report it ithe same and it is not equal processing).


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.
    Disclaimer: Attempting change is of your own free will.

    Mittwoch, 2. Mai 2012 20:10
    Moderator
  • Hi Brian,

    It is amazing how life runs in circles. The main article was written 3 years and 11 days ago and I find myself exactly in his shoes. I am about to purchase 2 new servers and before I do that I need to get the Vcpu to Core count correct.

    The confusing part Brian is where you say that logical processors = Vcpu however you do say that some hardware vendors do not all report it the same.

    We are about to deploy MS Skype 4 Business which is cpu intensive and the average Vcpu count of the Vm servers is about 6Vcpu hence I need a machine that can offer that count.

    Could you please verify that I am right with the math.

    1. Intel Xeon X5680 X5680@3.33Ghz

    According to CPU benchmarks - No of Cores is 6 ( 2 logical cores per physical)

    Does this mean that I have 6*2 = 12 logical processors * 2 ( dual processor) for a total of 24 logical processors or does it mean 6 * 8 ( 1core =8Vcpu) = 48Vcpu *2 ( Dual Processor) for a total of 96 Vcpu.

    The same goes for

    2. Intel Xeon E5-2650 v2Cpu benchmarks says No of Cores 8 ( 2 Logical processors per physical ) = 16 logical processors * 2 ( Dual processor) = 32 logical processors or is it 8*8( 8Vcpu per physical Core) *2 ( Dual Processor) for a total of 128 Vcpu's

    Donnerstag, 14. Mai 2015 13:48
  • This is confusing.  And that is why the calculations are only estimations.  And the calculations only work for cases where you are giving a performance guarantee.

    What I mean by that is there is 0 CPU overcommitting in these calculations.  And CPU can be safely over-committed (on all hypervisors).

    Lots of folks get hung up on vCPU - and for a long time now I have been trying to explain that there are far more important things to plan than CPU.  My guidance is this: Buy all the CPU you can afford and balance VMs as you need to.

    As I mentioned, there is only one case where this math matters, and that is when you are hosting loads and you want to give a performance SLA. (like Azure does).

    Either way, the math is straightforward. 

    Total Cores x total logical processors = Performance SLA vCPU count.

    The number of cores is easy.  You have a physical processor and it has cores.  And you may have multiple processors.  So, the total number of cores is the number of processors x the number of cores.

    The number of logical processors you must get from the processor vendor.  These are the execution threads on each core.  As each core has a number of logical processors.  It could be 8, 12, 16, 32.  It depends on the chipset.  So, the total logical processors is the number of cores x the number of logical processors per core.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.

    Donnerstag, 14. Mai 2015 15:23
    Moderator
  • Lync/Skype is a special case. The Lync team has indicated that when virtualizing on Hyper-V, Hyper-threading must be disabled and cores for all roles except SQL must be assigned at a 1:1 ratio. Not that it was part of this conversation, but they want you to kill NUMA, too.

    So, what I'm saying is, you need to run the Lync planning tool first. You need to forget all this talk about "logical" anything because you're going to disable Hyper-threading, meaning that there will be exactly one maximum active thread per physical core. However many processors/cores the planning tool says that you need is how many processors/cores that you need. Just a fair warning that they'll have low average utilization, but with Lync, "waiting for a thread scheduler" = "choppy call performance" so you'll just have to accept that Lync is not going to make good use of the CPU sharing capabilities of a hypervisor.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Donnerstag, 14. Mai 2015 16:28
  • I have not seen the recommendations, but I also don't follow these Lync recommendations fully.

    My interpretation of your description is that these are hardware recommendations that are being extrapolated to life in a VM.  Not recommendations that are specific to life in a VM.

    Logical processors are always relevant when there are VMs.  As this is the thread of the physical core where the VM CPU time happens.  It only happens on a logical processor and no place else.  So you will greatly under-utilize your system.

    All that I can say is that - those may be the product recommendations, and thus those should be what you follow.  But they don't make sense.

    What they might be trying to do is overcome the CPU scheduling behavior because the Lync Server is not very compatible with it.  I can totally see that as a possibility - things like transcoding don't virtualize well.  And, then they may make these recommendations thinking they are avoiding scheduling - but they can't.

    In this case, I think it muddies things.  And is an outlier.

    But, the point of following the product recommendations is important.  And not making assumptions.

    (back in 2005 we could not tell a software vendor we were running their server in a VM, they would blame all problems on that and focus on it.  I would hope that products are more enlightened today).


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.

    Donnerstag, 14. Mai 2015 16:45
    Moderator
  • Hi Eric, Brian

    I just ran the planning tool and It is no different than what is on TechNet ( however It is a very helpful and detailed tool as It suggests the number of vm servers etc to use...thanks for showing it to me ) with regards to the hardware requirements. However as Brian just mentioned there has to be a correlation between threads/core/logical processors for Virtualization machines cpu allocation.

    having read and reread your comments - What I have concluded is the following... my machine an Intel Xeon E5 -2650 v2 has 8 Cores with 16 threads for a total of 128vCpu which means that I have that much to spread around my Vm's without oversubscribing. Oversubscribing will be allocating more than that number. and In addition using dynamic memory allocation against fixed allocation. I have deployed MS Lync in a machine that oversubscribes the above CPU and Memory with VMware and have not had any performance Issues however this new project is for hosted lync and we plan to have a lot more users that what we have on premises.

    Thanks for all your guidance and support. 


    robinhood_montreal


    Donnerstag, 14. Mai 2015 17:04
  • All that I can say is that - those may be the product recommendations, and thus those should be what you follow.  But they don't make sense.

    This. That was almost verbatim what I said.

    But the recommendations are right from the Lync team. Our Lync service owner talked directly to people that wrote the documentation, because after I read it, I said, "that does not make sense". They did not match what the Hyper-V team said. But, we call the Lync team for support, so:

    But, the point of following the product recommendations is important. And not making assumptions.

    This. This is what we do. Our contacts said that they performed extensive testing within a Hyper-V environment and that Hyper-threading off, 1:1 vCPU to pCore, and NUMA off is necessary for proper Lync performance. That sort of trumped everything else.

    What I meant by my post regarding forgetting the logical processors is that Hyper-threading adds additional logical processors. Since we have to disable it for Lync, those go away and you're left with 1 logical processing core per 1 physical processing core. The distinction is pretty much academic at that point.

    But yes, we're ridiculously underutilizing our Lync Hyper-V hosts, at least on paper. This product requires many open lanes for threads to operate with no wait time, so it makes its own sort of sense.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Donnerstag, 14. Mai 2015 17:04
  • Yea, hyper threading is a funny thing.

    It adds execution threads, but does not increase the number of logical threads a core actually has.

    (this is where we get into logical processors on hardware vs. logical processors in operating systems - same name, different animals.)

    It does make some sense.

    Personally, outside of the flexibility of having Lync in a VM my personal leanings are to make it an exception and put it on hardware.  Give it all the raw hardware processing it wants.  None of the virtualization CPU scheduling overhead.

    I don't implement it today.  But if I did, that is where I would be trying to push it - on bare metal.  Go for the best customer experience and sort out IT processes that support that goal.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.

    Donnerstag, 14. Mai 2015 17:13
    Moderator
  • However as Brian just mentioned there has to be a correlation between threads/core/logical processors for Virtualization machines cpu allocation.

    For Lync, it is one thread:one core:one logical processor. Other things would follow what Brian is talking about because most almost no apps have problems waiting a few nanoseconds for their time slice to come back around. Lync just doesn't happen to be one of them.

    however this new project is for hosted lync

    This will change all your other assumptions. Hyper-threading is not a good thing for you and you definitely cannot escalate it all the way to thinking that your 8 physical cores are now the processing equivalent of 16. At most, you'd have 10-12ish.

    The other thing is that you may not be able to translate your experiences directly into a hosted environment. If your current environment is heavy on one-to-one calls, then your Lync hosts are probably near idle because Lync's only involvement there is to broker the connection and handle presence. The call itself is peer-to-peer. If your users start doing conference calling, or conference video calling, resource utilization will escalate in a hurry.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Donnerstag, 14. Mai 2015 17:20
  • Personally, outside of the flexibility of having Lync in a VM my personal leanings are to make it an exception and put it on hardware.  Give it all the raw hardware processing it wants.  None of the virtualization CPU scheduling overhead.

    You're absolutely right. Lync takes almost no advantage of the hardware-sharing powers granted by virtualization. We can't use Live Migration, either. But, we are using gigantic hosts so that a number of the guests can co-reside. That means that we're using about 1/5th the total rack space of a physical deployment, along with the attendant reductions in network and SAN connections, so it's not a total waste on the physical side. Our angle is really service resiliency and hardware independence. I can't Live Migrate a guest but I do have a process in place for moving VMs from dead hosts that would be substantially faster than performing any type of restore. We do lose out on maximizing hardware utilization, but it's a win pretty much everywhere else.

    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Donnerstag, 14. Mai 2015 17:59
  • Wow - That is pretty sad and disheartening to know that my $15K server can only provide me with 16cores and so for me to truly design an infrastructure for 12K users I need almost 5 servers ( especially one that can provide at least 32cores ) but I guess that is why VMware released a document debunking the reason for disabling hyper threading http://blogs.vmware.com/apps/2015/01/virtualizing-microsoft-lync-server-lets-clear-confusion.html for MS Lync/Skype.


    robinhood_montreal

    Donnerstag, 14. Mai 2015 18:33
  • Hyperthreading for hypervisors has mixed results.

    The common recommendation is to leave it on (by the Hyper-V team).  In most cases it does no harm.  And also in most cases it really provides no benefit (to VMs).

    That said, there are particular cases where it does cause problems.  And they are few.

    Now, ESX, being emulation based, can take advantage of the hyperthreading for the management OS and for emulation overhead.  Since Hyper-V does no emulation, there is really no benefit.

    My point; you can't take one and simply apply it to the other. They work differently under the hood.


    Brian Ehlert
    http://ITProctology.blogspot.com
    Learn. Apply. Repeat.

    Donnerstag, 14. Mai 2015 18:41
    Moderator
  • That and, there are precious few workloads where you can take an 8-core CPU with Hyper-threading and say, "Yay! 8 free cores!" A 25% performance boost seems to be a fairly common finding. I thought I was being pretty generous by estimating 10-12. A 100% performance boost is an unreasonable expectation.

    We also pushed the product team on where these recommendations came from, and while we didn't get any better answers than what's in that article you linked, they explicitly told us that they tested it on Hyper-V and this was what they came to. VMware's document does not debunk that statement at all.


    Eric Siron
    Altaro Hyper-V Blog
    I am an independent blog contributor, not an Altaro employee. I am solely responsible for the content of my posts.

    Donnerstag, 14. Mai 2015 19:04