none
Edge to Edge Comms Requirements?

    问题

  • Hi, firstly the topology: two edge servers in a pool using DNS LB and NAT. So, in more detail, each edge has 3 private IP addresses. Each IP NATs to a public IP. There is an external DNS record for each of the access edge, web conf and AV IP (so, for example: "av.contoso.com" points to the public IPs for the two AV services).

    Everything works except for one thing. What works is: federation, AV conferences, multi-way calling, pretty much everything. However, one thing intermittently fails: application sharing during a multi-way conference call where participants are coming in via the Edge. I've explored this extensively and can now reproduce the problem: it happens when all the AV traffic is going via one Edge server (AV on edge1) but when the app sharing starts, it sends a whole bunch of traffic to AV (TCP 443) on edge2. Eventually it fails with the well-known "network problems" error in the client. If, by chance, it tries to share via edge1, everything works. So, the problem happens if the sharing edge server (for want of a better term) is not the one currently being used for AV. I can "prove" this by stopping either one of the edge servers and configuring external DNS with only a single AV and web conf record. Note: both servers on their own work. 

    A logging trace on the edge servers shows something I didn't expect: it shows that when the traffic arrives at the edge server during the attempt to share a program, that edge server attempts to contact (using STUN) the public IP for the other edge server from it's natted external interface. In fact, it tries a number of things (let's assume that edge1 is the one the AV conference is already on and edge2 is the one we're trying to share a program on):

    - external-facing private ip on edge 2 --> public ip on edge 1

    - external-facing private ip on edge 2 --> public ip on edge 1

    - external-facing private ip on edge 2 --> private ip on one of the FE servers

    - external-facing private ip on edge 2 --> public ip on edge 1

    - external-facing private ip on edge 2 --> private ip on one of the FE servers

    None work. What I find odd is that if one edge wanted to talk to another, why is the natted private IP one one trying to talk to the public IP on the other? It would make more sense for them to attempt to communicate using their private IPs given that they're on the same LAN! Anyway, I have tried to find out what the edge to edge communications requirements are and can't find a single document. I have found internet to edge and edge to internal comms requirement but nothing that formally says "if you use NAT on the edge, the natted address on one edge must be able to communicate with the public IPs on all the other edges". I doubt MS would code it that way (inter-edge traffic going via a firewall seems quirky), so I'm thinking that there must be something else wrong here. Any ideas?



    2012年3月2日 10:38

全部回复

  • Hi ,

    Would you be able to share the uccp logs ?

    Thanks

    Saleesh

    2012年3月2日 14:04
  • Hi Saleesh - thanks for the reply. Sure, I have an 8 MB file copied from this morning which is a bit big to post. Since I created this uccp log, we've disabled one of the edge servers which as I say has worked around the issue. I can email over the uccp log I have, or, I can re-enable the other edge server out of hours (tomorrow) and get a new smaller log to post here? 
    2012年3月2日 14:18
  • Fine , it would be great if you could send me the logs to saleesh.nv@hotmail.com . I will compare the logs and update you the findings if any .

    Thanks

    Saleesh

    2012年3月2日 14:22
  • Hi Mark,

    Any update?

    When configuring network address translations (NAT) for the edge pool, you should use the public IP address of the Audio/Video Edge Service.

    This NAT IP address is needed to ensure communication with the remote external client, which may be located behind a firewall, and the internal client within the corporate environment, which is also behind a firewall.

    As you know, STUN and TURN produce the candidates to find the preferred route for data transmission across the A/V Edge Service.

    And Internet Connectivity Establishment (ICE) is used to traverse NATs and firewalls with audio, video, and desktop sharing data.

    If the setting is not correctly configured, audio, video, or desktop sharing for remote users will not be available.

    Moreover, provide Microsoft Lync Server 2010 Protocol Workloads Poster for reference. Hope helps.


    Noya Lau

    TechNet Community Support

    2012年3月8日 11:18
    版主
  • Hi, 

    1. Thanks for the reply.

    2. I'm working with Saleesh on this as per above.

    3. The problem with the protocol poster is that it's client --> edge focussed. However, this doesn't explain the edge to edge comms as per the OP. Also, I cannot see why one edge should be trying to contact the other via its public IP. Given that the topology tells the each edge the natted IP of each other edge, why the edge then choses to ignore that and instead push traffic (or try to) push traffic out to the external firewall just to have it come back again, is a mystery to me.

    2012年3月8日 11:37
  • Unfortunately, the idea to set the clientappsharingport to 50000 with 10000 appsharing ports didn't work.

    But, I've done some more research and.... I was right! The problem is precisely edge to edge comms. It is described perfectly in this blog: http://silbers.net/. The two fixes are either burn a hole in the client's firewall. The blog rightly - and rather diplomatically - says "While this is the easiest method, it is not always the most preferred given the number of ports required." The only other fix is to allow "hairpin" traffic between the edge servers. My own view is that MS should make this clear in their own documentation that this is an absolute requirement when using DNS LB on edge servers. Also, going right back to my OP, why does the architecture force AV traffic between edge servers via the firewall when the two edge servers are on the same LAN? It's a network-team unfriendly solution. OK, "it's how TURN works" is probably the reply, right? But let's think about it for a second. Each Edge server knows what the NAT and unnatted (public) IPs are of all other edge servers because it's part of the topology. It can't be that difficult to code something that says "if the destination IP is a public IP and it is part of our topology, and if we have a matching private IP, and if we have a route to the private IP, try that as well"? I'm sure it's not that simple because they would have already thought of it?!

    I won't mark this as answered until I've issued a change request to the network guys and tried it (though I'd understand if the change got rejected!).

    2012年3月16日 14:04