none
Reverse Proxy BadGateway Response Error when with iis arr

    General discussion

  • I know this has been asked several times. I've followed through all the suggestions but just can't get this settled. I'm at wit's end and decided it was time to ask for help. I'm fairly certain there's a certificate problem somewhere in my configuration but I'm fairly inexperienced with certificates so thought I'd ask the experts.

    I've been slowly building out my Lync 2013 deployment over the past couple of months and most everything we need is working now (chat, Audio/video, web conferencing/meetings). The only thing that is still not working is Mobile connections. When I test the lync discover on testconnectivity.microsoft.com I get the infamous error: "A Web exception occurred because an HTTP 502 - BadGateway response was received from IIS7." The logs from my mobile clients report the same error too.

    I've got a single FrontEnd server (lync @ 10.1.1.13), Lync Edge server (edge @ 10.1.1.14 & 50.197.79.106), and a Reverse Proxy (rvspxy @ 10.1.1.15 & 50.197.79.104). I know the reverse proxy is working because we're able to host lync meetings now where we couldn't before I installed that. (as an aside, why is it that Exchange needs only a single server and single IP but the design for Lync requires at least three servers and several IPs?)

    Here are my DNS entries:

    External
    Purpose Name Type Address Server
    SIP access.mydomain.com A 50.197.79.106 Edge
    Audio/Video av.mydomain.com A 50.197.79.106 Edge
    sip sip.mydomain.com A 50.197.79.106 Edge
    webconf webconf.mydomain.com A 50.197.79.106 Edge
    dialin dialin.mydomain.com A 50.197.79.104 Reverse Proxy
    lyncdiscover lyncdiscover.mydomain.com A 50.197.79.104 Reverse Proxy
    FE External web service lyncweb.mydomain.com A 50.197.79.104 Reverse Proxy
    meet meet.mydomain.com A 50.197.79.104 Reverse Proxy
    autodiscover _sip._tls.mydomain.com SRV access.mydomain.com Edge
    autodiscover _sipfederationtls._tcp.mydomain.com SRV access.mydomain.com Edge

    Internal
    Purpose Name Type Address Server
    edge server internal edge.mydomain.com A 10.1.1.14 Edge
    lync edge lyncedge.mydomain.com A 10.1.1.14 Edge
    dialin dialin.mydomain.com A 10.1.1.13 Lync
    FE Server lync.mydomain.com A 10.1.1.13 Lync
    lyncdiscover internal lyncdiscoverinternal.mydomain.com A 10.1.1.13 Lync
    meet meet.mydomain.com A 10.1.1.13 Lync
    SIP sipinternal.mydomain.com A 10.1.1.13 Lync
    lyncdiscover lyncdiscover.mydomain.com A 10.1.1.15 Reverse Proxy
    FE External web service lyncweb.mydomain.com A 50.197.79.104 Reverse Proxy
    Reverse Proxy Internal rvspxy.mydomain.com A 10.1.1.15 Reverse Proxy
    autodiscover _sipinternal._tls.mydomain.com SRV lync.mydomain.com Edge

    For the Certificates I have tried every combination that I can think of. Currently I'm using a *.mydomain.com for everything (Front Edge URLs, Edge Server, & Reverse Proxy). I know that the official recommendation is to _not_ use a wildcard cert for the reverse proxy but that was the _only_ combination that worked for users connecting to Lync Meetings from external IPs.

    I have also tried a "Lync External Cert" that I generated from an internal CA with additional SANs for all the names listed above (sip, lync, dialin, meet, admin, lyncdiscoverinternal, lyncweb, lyncdiscover). I get the same BadGateway error no matter what combination of certs I try.

    I've spent so much time on this I'm about spent. It would be so nice to get Mobility working though. Sorry for the long post, I wanted to provide as much detail as I could.

    What am I missing?

    Thanks in advance!
    Devin


    • Edited by uf97ddim Saturday, October 05, 2013 2:00 AM typo
    Saturday, October 05, 2013 1:53 AM

All replies

  • ARR is cool once it's working but a major pain to get it there at times. A few things to try:

    1. Make sure that everything is resolvable from the IIS box itself.  It sounds like mobility is the only issue so make sure lyncdiscover.domain.com is resolvable on the box.  Typically, I use host files for a lot of this stuff.

    2. Make sure this patch is on the box. http://www.microsoft.com/en-us/download/details.aspx?id=30333  Even if it looks like it is already applied, do it again.  I fought this problem for days once where this was listed as already applied but some other update "overwrote" the files.  So I applied again, rebooted and it was fixed.

    3. Make sure you don't get any certificate errors when you are visiting sites on #1.  Everything is pass through to the stuff behind.  Sounds like you have wildcards on everything (yes that is a no-no).  Typically you would have private certs on everything internal (making sure you have all names accounted for) and then go public to private via the ARR box.  (Don't forget the internal root CA if the box isn't domain joined - that happens too often).

    Thanks,

    Richard


    Richard Brynteson, Avtex, Lync MCM, Blog - www.masteringlync.com

    Sunday, October 06, 2013 4:52 PM
  • Hi,

    Please make sure the Internal CA certificate has been installed on Reverse Proxy.

    Here is another similar case for your reference:

    http://social.technet.microsoft.com/Forums/lync/en-US/20d138f8-936b-4504-b4ca-0961efaf7773/receiving-http-502-badgateway-response-error-when-testing-lync-2013-using-iis-arr?forum=lyncdeploy


    Kent Huang
    TechNet Community Support

    Monday, October 07, 2013 7:26 AM
  • Hi, Thanks to both of you for your replies. I'll look through Richard's suggestions this morning.

    I actually spent a lot of time dissecting that other case that you linked to. The trouble is, I'm such a certificate rookie, I'm not entirely sure what I'm doing. Can you please elaborate on what you mean by "Internal CA certificate"? 

    Is that the certificate that I've assigned to the FE Lync server? If so, then why do I still get the error when I assign the same certificate to every server?

    Also, by "installed" do you mean that it just needs to show up in the MMC certificates plug-in or does it need to be in the https binding in IIS for the default site - or something else?

    Thanks again!

    Monday, October 07, 2013 4:11 PM
  • Hi Richard,

    Thanks so much for your reply. Regarding point #1, when you say "IIS box" do you mean the Reverse Proxy server?

    I'll try a hosts file. Probably a stupid question but it'll be easier to just ask it.  I've read so many different blogs/forums about this my head is spinning. :-)

    Should that hosts entry for lyncdiscover direct to the internal or external interface for the reverse proxy? Also, if the reverse proxy server uses DNS should it use the internal or external servers?

    I'm guessing external, but this whole concept kind of seems like magic. I don't recall anywhere in the Reverse Proxy installation where I actually tell it where the FE server is.

    Thanks again
    Devin

    Monday, October 07, 2013 4:41 PM
  • OK, I think I figured out the Internal CA part. Even though the Reverse Proxy is domain joined, I followed the instructions on this link.

    How to Export Root Certification Authority Certificate
    http://support.microsoft.com/kb/555252

    Definitely different behavior now. With the wildcard cert on the Reverse Proxy I still get the 502 error. However, if I change the certificate for IIS on my Reverse Proxy to a non-wildcard cert, one that I generated with my internal CA, I get a "Connection is Untrusted" error when I go to https://lyncdiscover.mydomain.com from an external site

    Do I have no choice but to order a certificate for my Reverse Proxy from a public CA?

    Thanks again,
    Devin

    Monday, October 07, 2013 7:47 PM
  • The certificate on the IIS portion of your reverse proxy should be a public certificate.  That is what is going to be presented to the user connecting.  Then the certificate on the internal IIS on the front-end server for Lync is typically a certificate from your internal enterprise CA.  So from a flow perspective it is:

    Internet -> public certificate (IIS/ARR) -> private/internal certificate -> Port 4443/8080 of Lync Front-End Server

    The names on the certificate must match the entire way.  So lyncdiscover.domain.com (or wildcard on public) will map directly to the lyncdiscover.domain.com on the front-end server (port 4443).  So the IIS/ARR needs to have the Enterprise Root Certificate on it so it trusts the internal cert on the final leg.

    If you are using public certs on all servers (remember wildcard is not support on the front-end servers) than you don't have to mess with the internal CA stuff.  It just gets expensive for all of those names.

    Thanks,

    Richard


    Richard Brynteson, Avtex, Lync MCM, Blog - www.masteringlync.com

    Monday, October 07, 2013 7:54 PM
  • Hi Richard,

    Thanks for the details. I'm looking into getting a budget to get that public certificate we need.

    How does the Reverse Proxy map to a lyncdiscover.domain.com on the FE server? I thought that wasn't supposed to be entered in an internal DNS?

    Thanks again
    Devin

    Wednesday, October 09, 2013 8:32 PM
  • Hi Richard,

    I've ordered a certificate with 5 names:

    • access.domain.com (my SIP)
    • lyncdicover.domain.com
    • meet.domain.com
    • dialin.domain.com
    • lyncweb.domain.com (external web service URL)

    I've got it installed on my Reverse Proxy now. Since wildcards are a no-no I also went back through the cert installation on my Front End server and issued private certs from my internal CA. (I then installed the root CA cert on all three servers - FE, Edge, & Reverse Proxy).

    Lastly I uninstalled that patch you mentioned and then reinstalled it just to ensure it was there.

    I'm still getting a Bad Gateway error. Specifically, when I try to reach https://lyncdiscover.domain.com from the reverse proxy server I get a 502.3 error: Bad Gateway: Forwarder Connection Error (ARR)

    How exactly does the Front End server know to look for lyncdiscover.domain.com? I'm not really clear how the communication makes that hop since lyncdiscover is only in external DNS and I don't recall where in the ARR setup steps I would have told the reverse proxy where the Front End server is.

    I've spent so much time on this I was really hoping the cert purchase would take care of everything. Any suggestions on what to look at next?

    Thanks again!
    Devin



    • Edited by uf97ddim Friday, October 18, 2013 9:23 PM
    Thursday, October 17, 2013 1:00 AM