VLAN testing with Cisco Catalyst 4006 not going so well

Mon, 17 Mar 2008 19:28:38 Z

Ben, I hope you can help me out with this one...

I've got RTM Server 08 up and running on a Dell PE2950 with Hyper-V and I'm trying to get VLAN support working.

The network switch is a Cisco Catlyst 4006. Rather old but it's been reliable. We've got several VLAN's tagged on one of the switch ports and I've created a virtual switch on the host with the NIC connected to that port. The NIC is an Intel GB adapter with VLAN support enabled.

I've configured the settings on the VM to use the virtual switch and entered the VLAN id for the network I want to use. The VM is running Server 08 and I've installed the virtual machine extensions.

The guest can ping himself and the host virtual switch IP (if the same VLAN is configured on the virtual switch), but cannot ping the gateway or any other device on the same subnet.

Here's the output of a ping on the guest VM...

C:\>ping x.x.x.1 (the gateway address)

Pinging x.x.x.1 with 32 bytes of data:

Reply from (local IP of vm guest): Destination host unreachable

My network staff says they can see the MAC address of both the virtual switch and the VM guest in the proper VLAN on the Catalyst, but nothing works. Even stranger is that if I ping the IP of the VM guest from my workstation (which fails every time), I can sometimes see the MAC address of the VM guest on my machine if I do an "arp -g" (I'm in the same subnet as the VM), however most of the time while I'm pinging the guest I see all 0's for the IP and a "Type" of "Invalid" in the arp cache.

On the host I have tried the Server08 drivers, the Windows update version, and Intel's own Vista/Server08 drivers.

Any advice on helping me debug this would be greatly appreciated.

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 18 Mar 2008 16:56:44 Z

50 views and no replies. I was afraid of that

Does anyone have VLAN support working with a Cisco switch? If so, if you could share the switch model and configuration on the port I would appreciate it. Thanks in advance.

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 18 Mar 2008 17:50:26 Z

Have you installed the latest drivers from Intel? I beleive that you need to install them and configure VLAN support in the physical network adapter first.

Cheers,

Ben

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 18 Mar 2008 19:35:07 Z

I’d also confirm that the IOS version you are running supports what you are attempting to accomplish. Cisco’s site lists the following requirements:

Catalyst Platform	PVLAN Supported Minimum Software Version	Isolated VLAN	PVLAN Edge (Protected Port)	Community VLAN
Catalyst 4500/4000 - CatOS	6.2(1)	Yes	Not Supported	Yes
Catalyst 4500/4000 - Cisco IOS	12.1(8a)EW	Yes	Not Supported	Yes. 12.2(20)EW onwards.

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 18 Mar 2008 20:47:50 Z

Ben,

Thanks for the reply.

I have tried the built-in Server 2008 drivers, the Windows update version, and Intel's own Vista/Server08 drivers

I had Priority and VLAN support enabled on the adapter with all versions of the driver.

With the Intel driver I was able to use thier extensions to create a virtual adapter on one of the VLAN's and it worked correctly (i.e. it grabbed and IP from DHCP and was pingable), but that's not the way we're supposted to do it with Hyper-V correct?

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 18 Mar 2008 20:49:06 Z

Ryan,

Thanks for the reply.

I'll present this information to my network group. Thanks.

VLAN testing with Cisco Catalyst 4006 not going so well

Wed, 19 Mar 2008 18:14:54 Z

I believe that with Intel adapters today this is the only way you can get VLANs to work.

Cheers,

Ben

VLAN testing with Cisco Catalyst 4006 not going so well

Wed, 19 Mar 2008 19:11:18 Z

Ben,

If that's the case can you give me a list of supported adapters?

I would really appreciate it.

VLAN testing with Cisco Catalyst 4006 not going so well

Wed, 19 Mar 2008 20:15:33 Z

Or specifically, if you can help me get the PE2950's Broadcom NIC working that would at least get me on the right track.

I just moved a wire with access to several VLAN's over to that adapter and created another virtual switch.

I can assign a VM to that virtual switch, but if I assign a VLAN I get "Error applying network adapter change", "The operation failed with error code 2147483647."

Another day in paradise...

VLAN testing with Cisco Catalyst 4006 not going so well

Thu, 20 Mar 2008 19:37:00 Z

We have identified issues with various network drivers and are working with the hardware vendors to get these addressed. At the moment we do not have a formal list of cards that VLAN support is known to work on - but as a rule of thumg you should be downloading the latest drivers from the manufactureres website.

Cheers,

Ben

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 00:35:36 Z

I posted some info on this some time ago: http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=2745014&SiteID=17

Basically it appears that you cannot do VLAN tagging within a Hyper-V virtual network (virtual switch and/or network adapter of virtual machine) if it is attached to a network adapter which is already configured to do VLAN tagging.

Or in other words, if you create a virtual network adapter within BACS and configure it with a VLAN ID (to tag packets with a VLAN ID) then there is no VLAN support inside any attached Hyper-V virtual network (applies to Broadcom NIC's - don't know about Intel).

Example 1 - VLAN support inside a Hyper-V virtual network:

--------------

Network Device: HV-VM1-NIC VLAN tagging supported
--------------

--------------

Network Device: HV-VNET1-SW1 VLAN tagging supported
--------------

||
--------------

Network Device: BCM5708C-1
--------------

Example 2 - No VLAN support inside a Hyper-V virtual network:

--------------

Network Device: HV-VNET1-SW1 VLAN tagging NOT supported
--------------

--------------

Network Device: HV-VNET1-SW1 VLAN tagging NOT supported
--------------

||
--------------

Network Device: BASP VA #1 VLAN Tagging (e.g. VLAN 100)
--------------

||
--------------

Network Device: BCM5708C-1
--------------

Example 3 - NOT TESTED, but suspected VLAN support inside a Hyper-V virtual network:

--------------

Network Device: HV-VNET1-SW1 (suspect: VLAN tagging supported)
--------------

--------------

Network Device: HV-VNET1-SW1 (suspect: VLAN tagging supported)
--------------

||
--------------

Network Device: BASP TEAM #1 (No VLAN tagging configured)

--------------

||
--------------

Network Device: BCM5708C-1
--------------

If you need any more details on my configuration, just let me know. I hope this helps a little bit.

P.S: Wouldn't be astonished if the same applied to Intel as well.

Cheers,

Chris

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 16:48:57 Z

Ben,

Thanks for the reply.

I will use the latest drivers and see if that helps. I'm beginning the think I need to try another switch

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 17:10:39 Z

chagmann,

Thanks for your reply,

I understand that a virtual adapter bound to a VLAN via the extended drivers and then bound to a virtual switch is bound to the VLAN of of the virtual adapter and will not support VLAN tagging in a VM guest. That is what I would call the "old school" method (equivilent to your example 2) and is how I get VLAN support on MSVS and VMWare Server today. I only built such an adapter to see if the VLAN support with my Cisco switch was working correctly. After I verified that I could get an IP and ping other servers in the subnet on that VLAN I removed the adapter.

So my goal is to get your "Example 1" configuration up and running. It sounds like this is what you've gotton working using the Broadcomm adapter. Could you give me the specifc adapter settings on the NIC properties the must be enabled (or disabled) to support this? I would really appreciate it (i.e. is it "just set 'VLAN and Priority' set to enabled", or is it something more?).

I'm beginning to think I'm doing everything correctly and that the issue is with my old Catalyst 4006. I'm going to try a newer model today and see if that helps

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 17:36:36 Z

Mike,

I'm gonna gather the details for you. But you certainly need to enable VLAN & Priority (or at least VLAN) on the Broadcom adapter. I also vaguely remember that the secret to make VLANs work is to configure the Broadcom adapter with a VLAN ID of 1 (tells the adapter to accept traffic with "any" VLAN tag).

Try it out, may be all you are missing. I'll get back to you with more data shortly.

P.S: I don't think that the switch model is of any importance. Basic VLAN tagging is pretty much the same across vendors (not talking about enhanced VLAN features like private VLANs, etc.) and models.

Cheers,

Chris

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 18:15:51 Z

Here is the complete config:

Broadcom Adapter:

- Properties: Broadcom Advanced Server Program Driver, Microsoft Virtual Network Switch Protocol (everything else unchecked)

Information gathered through BACS 2

- IP Address: N/A

- Driver Name: bxvbda.sys

- Driver Version: 3.7.23.0

- Driver Date: 10/18/2007

- BASP State: Active

- Offload Capabilities: LSO, CO

- Ethernet at WireSpeed: Enable

- Flow Control: Auto
- IPv4 Checksum Offload: Tx/Rx Enabled
- IPv4 Large Send Offload: Enable

- Jumbo Mtu: 1500

- Locally Administered Address: Not Present

- Priority & VLAN: Priority & VLAN enabled

- Receive Side Scaling: Enable

- Speed & Duplex: Auto

- VLAN ID: 1
- Wake Up Capabilities: Both

Adapter representing Hyper-V virtual network switch:

- Properties: Everything checked except for Microsoft Virtual Network Switch Protocol and IPv6

Hyper-V Virtual Network:

- Name: NIC-2_HV-SWITCH

- Connection type: External and bound to "Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #2" (Broadcom Adapter as show above)
- VLAN ID: 100

Network Adapter of a virtual machine:

- Network: NIC-2_HV_SWITCH

- VLAN ID: 20

Let me know whether you can make it work.

Cheers,

Chris

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 19:49:27 Z

Chris,

I removed all current virtual switches, set the VLAN ID parameter on Broadcom NIC#2 to "1", and created a Virtual Switch on that adapter. That was the reason I was getting an error when trying to set the VLAN on the guest! After that I was able to boot the VM and it grabbed an IP in the proper VLAN via DHCP. Thanks SO MUCH!

Now I'm going to do a little more expreimenting with the Intel adapter to see if I can get that working as well. I'll post more info on this thread to let everyone know how it goes.

Once again Ben, Chris and Ryan, thanks for your help!

Ryan, I don't think PVLAN support is required for virtual switches, just 802.1q VLAN tagging support, which the CatOS does support (I'm guessing that would be Community VLAN in the chart)

VLAN testing with Cisco Catalyst 4006 not going so well

Fri, 21 Mar 2008 20:42:35 Z

I now have 2 VM's up in 2 different VLAN's on the same virtual switch. I am also able to change the settings on the VM's for the VLAN while they are running and then renew DHCP on each to get a proper IP for that VLAN. So I think I'm good to go on the Broadcom adapter.

Time to give the Intel's another shot. They don't have a VLAN ID parameter so I'm not sure they're going to work.

Ben, do know if having a parameter called "VLAN ID" is required?

VLAN testing with Cisco Catalyst 4006 not going so well

Mon, 24 Mar 2008 19:45:23 Z

Well after testing with my network group we were able to figure everything out.

First off, there is NO PROBLEM with the Catalyst switch, we were able to duplicate the issue with a more current model and now that we have things working the older switch is working fine

The "real" issue was that 802.1Q tagging was not enabled on the host NIC drivers. Having "VLAN and Priority" set to "ENABLED" IS NOT ENOUGH!

In my case I was unlucky in that the NIC's I use (Broadcom BCM5708C NetXtreme II GigE, and Intel(R) PRO/1000 PT Dual Port Server Adapter) both have what I would call "non standard" names for this setting.

For the Broadcom's, changing the VLAN ID property from 0 to 1 enables VLAN tagging and everything works as expected. Thanks for the help on that setting.

The Intel's were a tougher nut to crack as the settting is not exposed through the NIC's Advanced Properties, but only through the registry . Changing the HKLM\SYSTEM\CurrentControlSet\Control\Class\{GUID}\XXXX\VlanFiltering registry key (where {GUID} is the GUID containing the network adapter configuration and XXXX is the NIC to be confgured) from 1 to 0 and rebooting corrects the issue. The easiest way to find the proper key in your system is to use the find command in regedit with enough of the network adapter name (i.e. for the Intel's, find "1000 PT").

Identifying a registry key to enable this support actually turned out to be a plus since I was able to use it to get VLAN support working in server core. So now I can use the Broadcom’s as originally planned (for host and cluster NIC’s), and the Intel’s for VM public and/or iSCSI connectivity.

Thanks to all for your help. I hope this thread will help others having the same issue.

VLAN testing with Cisco Catalyst 4006 not going so well

Tue, 25 Mar 2008 04:27:09 Z

Postscript...

Today was very productive. I've got server core with Remote Managment, MPIO, Hyper-V with VLAN support, and failover clustering all up and running with no issues.

I'm starting to like this Server 08 thing

VLAN testing with Cisco Catalyst 4006 not going so well

Wed, 09 Apr 2008 04:22:41 Z

Mike would you be willing to elabrate more. I am woring the server core, Hyper-V and trying to vlan support working. So far i have been able to one of the virtual adapter to connect but not the second one.

VLAN testing with Cisco Catalyst 4006 not going so well

Thu, 21 Aug 2008 18:09:41 Z

Thank you very much Mike! I am also running server core on a Dell 2950 with the intel Pro 1000PT. Both of your fixes resolved my issue. I found that I had to create the VlanFiltering registry key to get the Intel nic working. But one it was created, and the system rebooted, everything worked great!

Now that you have had a few months with your environment, would you bring it up on server core again or would you go with a full install?

Jason