Sign in
 
HomeWindows Server 2012Windows Server 2008 R2Windows Server 2003LibraryForums
Windows Server TechCenter >
> 
> 
Need help with 2008r2 hyper-v two NICs with two different networks

Unanswered Need help with 2008r2 hyper-v two NICs with two different networks

  • Wednesday, August 08, 2012 3:15 AM
     
     
    Sign In to Vote
    0

    The server I have is a Dell 2900, dual 1 gbps broadcom NICs with the TCP offload feature.

    What I am trying to do is use NIC1 as internal network traffic shared with the host OS  -  Allow management operating system to share this network adapter

    NIC2 is an external network - AKA the internet

    NIC1 works perfect, host communicates with the other devices on the network and the internet through the default gateway, as do the virtual machines.

    NIC2 does not seem to pass traffic to the virtual machines.  Static nor DHCP addresses.

    Turning on Allow management to share, adds the virtual NIC, but still no traffic, removing the virtual network and giving control back to the host OS, gets an IP on the external network no problem.

    I have searched, and most results state the recomended way to use hyper-v is with both NICs on the same network and one dedicated to the host.  The other things I have found that seem relivant is to disable the offload,  which I did, but no effect,  and to update the broadcom drivers,  which I did as well, but no change.

    Virtual Networks:

    Internal Network - Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #49 -  Allow Mangement

    External Network - Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #52

    Internal works great,  but I am not getting traffic from the External network.

    What should I be looking for to get more infomation on what is going wrong or what further tests can I run?

    Thank you very much your time,

     

All Replies

  • Wednesday, August 08, 2012 5:50 AM
    Moderator
     
     
    Sign In to Vote
    0

    Hi,

    NIC2 is an external network - AKA the internet
    >> Did you create an External Virtual Network bind to this physical network adapter?

    If above is true and NIC2 connected to another switch(not the NIC1 connected to), then it is recommended that you check whether there is some limitation in the switch which NIC2 connected to. As far as I know, some switches do not accept two or more different MAC addresses from the same switch port.

     
  • Wednesday, August 08, 2012 6:07 PM
     
     
    Sign In to Vote
    0

    Yes,  two virtual networks listed in  Virtual Networks in the Hyper-V application,  I labeled them as Internal and External  :)

    My modem has a 4 port switch as part of the modem were my server is connected,  two other ports on the modem are connected to physical switches and have no issues with multiple MAC addresses.  I will try moving my server to one of the other switches,  but I am confident I have done this already.

    Yes, some switches do have the security feature of turning off the port when multiple MAC addresses are seen, my internal switch has this.

    Any other suggestions?

    Thank you again for your time.


    • Edited by kevinds Wednesday, August 08, 2012 6:14 PM
    •  
     
  • Thursday, August 09, 2012 2:03 AM
     
     
    Sign In to Vote
    0

    You have both NICs plugged into the modem's switch?  Does your internet provider provide you with multiple IPs?  You'd need at least 2.

     
  • Thursday, August 09, 2012 6:07 AM
     
     
    Sign In to Vote
    0

    NIC1 is plugged into my internal switch

    NIC2 is plugged into the modem's switch

    Multiple IPs, yes, several.  I also can not communicate within the same subnet to the VM, so the WAN router and modem are not issues with no communication,

    If 123.94.93.88/29 is my subnet  123.94.93.90 as a seperate physical host and a VMs using 123.94.93.91 and .92  -  can not communicate with .90

    I will install another system and see if .91 and .92, both as VMs can communicate  -  this I haven't tested yet, as they would both be on the same virtual switch, I expect that to work.

    Edit:  Yes, these two machines have network traffic between them.


    • Edited by kevinds Thursday, August 09, 2012 7:32 AM
    •  
     
  • Friday, August 10, 2012 5:46 AM
    Moderator
     
     
    Sign In to Vote
    0

    Hi,

    Let’s forget NIC1 as there was only issue with NIC2.

    Please help me to collect the following information:

    1. If you have other computers, please attach them to the modem's switch which the NIC2 connected to see whether they can get IP addresses.
    2. If you connect NIC2 to internal switch which NIC1 connected, do the virtual machines connect to the External Virtual Network(bind to NIC2) get IP addresses?
    3. If you remove External Virtual Network bind to NIC2, does NIC2 get IP address properly?
    4. If you connect NIC2(when External Virtual Network were removed) to internal switch which NIC1 connected, does NIC2 get IP address properly?

     

     
  • Friday, August 10, 2012 12:57 PM
     
     
    Sign In to Vote
    0
    I am very curious as to what the answer is. I have a very similar problem except NIC1 is ethernet and NIC2 is Infiniband. The host does not seem to pass ARP to the guests. The host is 2008R2 and the two guests are 2008R2 also. The guests can ping each others NIC2 address but cannot see the greater Infiniband network.
    • Edited by lfb001 Friday, August 10, 2012 1:26 PM
    •  
     
  • Friday, August 10, 2012 2:57 PM
     
     
    Sign In to Vote
    0

    What is the actual configuration on each NIC.  It sounds like everything is working fine on 123.94.93/29 (NIC1).  What's the configuration for NIC2?  One of the issues you often run into with multiple NICs is trying to assign multiple default gateways.  If you want to be able to communicate outside the subnet that NIC2 is connected to, you need a gateway.  Do you already have a gateway defined on NIC1?

    You mention that you have seen "the recomended way to use hyper-v is with both NICs on the same network ".  I have never seen that as a recommendation; in fact, I've always heard to not do that.  If you put two NICs on the same subnet, Windows will only use one.  They really have to be on different subnets (there are some unique exceptions).  And generally when on different subnets, only one is routed.

    tim

     
  • Friday, August 10, 2012 7:11 PM
     
     
    Sign In to Vote
    0

    1. If you have other computers, please attach them to the modem's switch which the NIC2 connected to see whether they can get IP addresses.

    Yes

    2. If you connect NIC2 to internal switch which NIC1 connected, do the virtual machines connect to the External Virtual Network(bind to NIC2) get IP addresses?

    No

    3. If you remove External Virtual Network bind to NIC2, does NIC2 get IP address properly?

    Yes, I tried and documented this above, removing NIC2 from Hyper-V networking, and it will get an external IP.

    4. If you connect NIC2(when External Virtual Network were removed) to internal switch which NIC1 connected, does NIC2 get IP address properly?

    Do you mean removing same as 3 above, but on the internal physical switch instead, I haven't done this but I can confirm this works shortly.

    [quote]You mention that you have seen "the recomended way to use hyper-v is with both NICs on the same network ".  I have never seen that as a recommendation; in fact, I've always heard to not do that.  If you put two NICs on the same subnet, Windows will only use one.  They really have to be on different subnets (there are some unique exceptions).  And generally when on different subnets, only one is routed. [/quote]

    http://social.technet.microsoft.com/wiki/contents/articles/9711.hyper-v-network-design-configuration-and-prioritization-guidance.aspx

    • 1 for Management. Microsoft recommends a dedicated network adapter for Hyper-V server management.

    This would be putting both NICs on the same subnet,  managment/host OS having control,  not having a virtual NIC.  The host OS would only be using one of them.

    The above link is for high-availability, but I have seen that advise repeated many times, in summary > don't use a virtual NIC for the host-os unless absolutely required.

    http://blogs.technet.com/b/jhoward/archive/2008/06/16/how-does-basic-networking-work-in-hyper-v.aspx

    to have at least two physical NICs in a physical machine– one (or more) dedicated to the physical machine, and one (or more) for virtual machines.


    • Edited by kevinds Friday, August 10, 2012 7:16 PM Tried to add some line-breaks
    •  
     
  • Sunday, August 12, 2012 6:29 PM
     
     
    Sign In to Vote
    0

    1 for management - No, this does not mean you would be putting both NICs on the same subnet.  It does not say anything like that in that reference.  In fact, as I stated before, Windows will only recognize two NICs on the same subnet only in unique situations, such as setting MPIO, but even there is not recommended.  It IS recommended to have a NIC dedicated to host management.  Often this NIC is a management NIC, like on a management card.  If you don't have that, you can dedicate a physical NIC to the host. 

    Yes, it is recommended to have at least two physical NICs.  If you want both of them to be on the same network, make the one for exclusive use by the host.  Make the other for exclusive use by the guests.  When you make one for exclusive use by the guests, that removes the IP stack from the second NIC so that the host will not end up with two NICs on the same subnet.

    Please post the configuration of your two NICs.  If the host does have two IP addresses on the same subnet, change the subnet on one of them.  The addresses used by the guests and the host do not have to be in the same subnet.

    tim


     
  • Monday, August 13, 2012 8:36 AM
    Moderator
     
     
    Sign In to Vote
    0

    Hi,

    Based on the information you provided, it seems that the issue only with the External Virtual Network bind to NIC2, whether it connect to “internal switch” or “modem's switch”.

    So, please perform the further suggestions to see whether you can resolve the issue:

    1. Please check whether NIC2 has the same version of driver with NIC1. If not, try to upgrade it.
    2. Please try to disable TCP offload feature on NIC2.

     

     
  • Monday, August 13, 2012 3:12 PM
     
     
    Sign In to Vote
    0

    I have this same setup in my home lab w/ Comcast business class hardware as my external device. Who is your ISP and what hardware did they give you? Just curious...

     
  • Wednesday, August 15, 2012 8:10 AM
    Moderator
     
     
    Sign In to Vote
    0

    Hi,

    Have you tried the suggestion? I want to see if the information provided was helpful. Your feedback is very useful for the further research. Please feel free to let me know if you have addition questions.

     
  • Wednesday, August 15, 2012 4:08 PM
     
     
    Sign In to Vote
    0

    Yes, NIC1 and NIC2 have the same version, they updated at the same time when I updated the drivers - downloaded the driver from Broadcom's site and used their installer, as their site specifically advised not to do them manually.

    Yes, the TCP Offload had no effect either on or off.

    ISP is Shaw Cable,  have a DPC3825 modem, 12 IPs total with a /29 subnet (gateway is on the CMTS, not routed to an assigned IP)

    Tim,

    I take host management as managing Windows, RDP, SSH, Hypervisor, not an out-of-band management card if this is wrong I'll accept that, so if the virtual switch 'breaks' you can still access the host to fix it, but none of this really matters in this situation, because I have them on different subnets and the host should not have direct access to the NIC2 subnet either.

    My NICs setup is in post 1 and 5, what other infomation are you looking for, there is a lot of little settings that I have not posted from Advanced, items such as  Network Address > Not defined.

     
  • Wednesday, August 15, 2012 8:29 PM
     
     
    Sign In to Vote
    0

    It would be helpful to see exactly how IP is configured on the NICs.  Output from ipconfig /all is what I am looking for.  You don't have to include the entire output - just that part that fully defines the NICs.

    Ethernet adapter Mgmt:

       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Mgmt
       Physical Address. . . . . . . . . : 00-25-B5-00-00-6F
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
       Link-local IPv6 Address . . . . . : fe80::adac:443f:171e:cd40%31(Preferred)
       IPv4 Address. . . . . . . . . . . : 10.29.130.21(Preferred)
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 10.29.130.1
       DHCPv6 IAID . . . . . . . . . . . : 603989429
       DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-13-2F-0D-D6-00-25-B5-AB-BA-8F
       DNS Servers . . . . . . . . . . . : 171.70.168.183
                                           171.68.226.120
       NetBIOS over Tcpip. . . . . . . . : Enabled

    Provide this for NIC1 and NIC2, labeling them, and from one of the VMs.

    tim

     
  • Saturday, August 18, 2012 2:53 AM
     
     
    Sign In to Vote
    0

    From the physical machine,

    Windows IP Configuration

       Host Name . . . . . . . . . . . . : thor
       Primary Dns Suffix  . . . . . . . : ad.ad
       Node Type . . . . . . . . . . . . : Hybrid
       IP Routing Enabled. . . . . . . . : No
       WINS Proxy Enabled. . . . . . . . : No
       DNS Suffix Search List. . . . . . : ad.ad

    Ethernet adapter Internal:

       Connection-specific DNS Suffix  . :
       Description . . . . . . . . . . . : Internal Network
       Physical Address. . . . . . . . . : 00-19-B9-CD-2C-C2
       DHCP Enabled. . . . . . . . . . . : No
       Autoconfiguration Enabled . . . . : Yes
       IPv6 Address. . . . . . . . . . . : 2001:470a:84df:1f02::aaaa:15(Preferred)
       IPv6 Address. . . . . . . . . . . : 2001:470a:84df:1f02:b939:6442:2fb:2804(Preferred)
       Link-local IPv6 Address . . . . . : fe80::b939:6442:2fb:2804%18(Preferred)
       IPv4 Address. . . . . . . . . . . : 192.168.5.15(Preferred)
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 2001:470a:84df:1f02::1
                                           fe80::250:baff:fe8c:e313%18
                                           192.168.5.1
       DHCPv6 IAID . . . . . . . . . . . : 385882553
       DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-17-36-6E-47-00-19-B9-CD-2C-C2
       DNS Servers . . . . . . . . . . . : 2001:470a:84df:1f02::aaaa:13
                                           192.168.5.13
       NetBIOS over Tcpip. . . . . . . . : Enabled

    Tunnel adapter isatap.

    The tunnel adapters do not have anything,   that is the ipconfig /all  output   NIC1 and NIC2, the hardware cards do not appear in ipconfig,  this is the NIC1 virtual switch

     
  • Saturday, August 18, 2012 4:01 AM
     
     
    Sign In to Vote
    0

    And from the virtual machine

     

    Windows IP Configuration

     

            Host Name . . . . . . . . . . . . : xp-test

            Primary Dns Suffix  . . . . . . . :

            Node Type . . . . . . . . . . . . : Unknown

            IP Routing Enabled. . . . . . . . : No

            WINS Proxy Enabled. . . . . . . . : No

     

    Ethernet adapter Local Area Connection:

     

            Connection-specific DNS Suffix  . :

            Description . . . . . . . . . . . : Microsoft Virtual Machine Bus Network Adapter

            Physical Address. . . . . . . . . : 00-15-5D-B3-60-0B

            Dhcp Enabled. . . . . . . . . . . : Yes

            Autoconfiguration Enabled . . . . : Yes

            Autoconfiguration IP Address. . . : 169.254.251.34

            Subnet Mask . . . . . . . . . . . : 255.255.0.0

            Default Gateway . . . . . . . . . :

     

     
  • Saturday, August 18, 2012 3:48 PM
     
     
    Sign In to Vote
    0

    Okay, I'm having some questions about some of the things I am finding.  In your first post you say that NIC1 is an internal network and that it works fine for accessing the Internet, other physical devices on your network, and virtual machines have the same sort of access.  I am guessing your definition of 'internal' is different from Hyper-V's.  When refering to an internal network in regards to Hyper-V, that is a network that is not associated with a physical NIC on the host.  It is just a virtual switch that is used to provide communication between the host machine and the virtual machines that are on that single host.  Yes, you could set up the physical host to run some sort of routing software, but it does not appear that you have done that with this configuration.  Therefore, I have to assume that NIC1 is actually an External network as defined by Hyper-V.

    But then I see the output of ipconfig in your virtual machine, and it is showing that is not receiving a valid IP address, meaning that it is not communicating to a DHCP server.  It has what is known as an APIPA address, which is an automatic address in the 169.254/16 range that is assigned to a NIC asking for a DHCP address but unable to contact a DHCP server.  I don't know how this machine could be talking to the Internet, even though you say it is working fine in your first post.

    Then, again in your first post, you say that NIC2 (the one that is not working) is an external network connected to the internet.  But then the output above of ipconfig does not show a NIC 2, meaning that it cannot be connected to the internet.

    So, given that we appear to be using the same terms for different meanings, can we go to some pictures?  Can you post a picture of what your Network Connections window looks like?  The picture below is from 2012, but it is basically the same for 2008.  You can get to this by typing ncpa.cpl at an elevated command prompt.

    In the above picture, each line that shows status as "Enabled" is a physical NIC that has an External virtual switch defined to it. The entries preceeded by "vEthernet" (you won't see it labeled this way on your system) are the virtual NICs that the host is using to communicate through that virtual switch. Now I have defined all my virtual switches to be accessible by both the host and the guests. If I go into a guest and do an ipconfig, that is where we would see the NIC assigned to "Microsoft Virtual Machine Bus Network Adapter", as you saw in your example.  The difference we would see in my configuration, though, is that I have valid IP addresses on mine instead of the 169.254/16 address you have on yours.

    The next picture that would be helpful would be to capture the output of what you see in your Virtual Switch Manager in Hyper-V.

    Please show this information for each virtual switch you have defined. From those two pictures, and the above output from your ipconfig should help us better understand how your environment is currently configured.

    Lastly, tell us how you want your configuration to work.  Example: I want NIC1 to provide access to to the internet and all network resources for all machines, physical or virtual.  I want NIC2 to provide access to non-internet resources on a different network.

    This information should help us.

    tim

     
  • Saturday, August 18, 2012 8:42 PM
     
     
    Sign In to Vote
    0

    From network Manager,

    Virtual Networks (They are both type External in the drop down list):

    Internal Network - Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #49 -  Allow Mangement

    External Network - Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #52

    NIC1 is my internal network - 192.168.0.0/16  - which has a NAT connection to the internet through my router

    NIC2 is my external network - the internet directly.

    I need some VPC to have external IPs - direct internet IPs, and most VPCs internal IPs, the internal works great, the external network does not get traffic

    I can get some screenshots later today if they are needed.

    For less confusion, would renaming my Internal and External network,  LAN and WAN be clearer, I wasn't thinking about the Hyper-V network types when I started posting.
    • Edited by kevinds Saturday, August 18, 2012 8:43 PM
    • Edited by kevinds Sunday, August 19, 2012 8:58 AM
    •  
     
  • Monday, August 20, 2012 9:04 PM
     
     
    Sign In to Vote
    0
    "NIC2 is my external network - the internet directly.

    I need some VPC to have external IPs - direct internet IPs, and most VPCs internal IPs, the internal works great, the external network does not get traffic"

    How are you trying to get "direct internet IPs" onto NIC2?  It looks like you are trying to use DHCP.  Direct Internet IP addresses are not assigned via DHCP, unless you have paid for a block of addresses and are running your own DHCP for that block.

    Why do you think you need "direct internet IPs"?  You have to pay for address you get on the internet.  That's why most routers have the ability to use NAT (Network Address Translation), such as it happening on your 'internal' network that has access to the internet.  It is taking the 192.168 address and presenting it to the internet on your modem's Internet address, actually hiding your 192.168 addresses from others on the internet.  If you want to be able to access a specific 192.168 address from the internet (danger, danger, Will Robinson) many modems/routers provide the ability to expose it via its NAT software. 

    tim

     
  • Tuesday, August 21, 2012 4:55 AM
     
     
    Sign In to Vote
    0

    Yes,

    As noted, DPC3825 modem, 12 IPs total with a /29 subnet

    5 staticly (manual) configured IPs, the rest DHCP from the cable modem.

    cPanel for example, wants a public static IP, not a NAT'd address, another, so that I can run my NAT router/gateway from a virtual machine, instead of a seperate PC, another example - testing software using the same ports as my internal machines.  I am familiar with network technology (have passed CCNA in the past)

    When I set from my static block in a virtual machine, I am also not able to ping or connect with my NAT router in the same subnet, which would not be an ISP IP limit at all, since it is not being routed.

    Do you know what I should be looking for in 2008r2 that would be preventing this from working?  -  I beleive I have this setup correctly, but I must be missing something.

     If you want to be able to access a specific 192.168 address from the internet (danger, danger, Will Robinson)   -  I have to ask, if 'danger, danger' is exposing an address in NAT, how else does one setup a website and other related services, this I am curious,  I have a good number of rules, which was a lot of lines to type, especially with the hairpin NAT rules to go along with them.

    • Edited by kevinds Tuesday, August 21, 2012 5:18 AM
    • Edited by kevinds Tuesday, August 21, 2012 5:18 AM
    • Edited by kevinds Tuesday, August 21, 2012 10:45 AM
    • Edited by kevinds Tuesday, August 21, 2012 10:50 AM added same ports
    • Edited by kevinds Tuesday, August 21, 2012 10:56 AM some ports
    • Edited by kevinds Tuesday, August 21, 2012 10:58 AM
    •  
     
  • Tuesday, August 21, 2012 4:29 PM
     
     
    Sign In to Vote
    0

    So you have a gateway defined on the 192 network and you are getting communication to the internet through this gateway.

    Generally, you only have a single gateway through which off network traffic is routed.  So, if you have a second network, it is trying to route off-network traffic, i.e. your NIC2 traffic, through your 192 gateway.  However, you don't have a route established between your NIC2 network and your 192 network.

    This is where drawing a picture of what you are trying to do and labeling it with all the parts - DHCP, gateway, assigned addresses, DNS, etc.  Looking at that will help define where you might need to create explicit routes in order to do what you want.

    tim

     
  • Tuesday, August 21, 2012 9:07 PM
     
     
    Sign In to Vote
    0

    My routes are working.

    I am not getting any switched traffic, nevermind routed traffic, in my virtual machines in my WAN/External network.

    This is a portion of my network, closer to the modem, there are a few pieces left out, because they are not relivant to what is not working.

    The External Virtual Switch, is where my issue is,  either traffic is not going into it, or out of it, and I don't know how to fix it.

    Eventually, I want to move the NAT Gateway/Firewall to a Hyper-V host, one interface on the Internal and the other on the External switch.