SSL Off Box Termination not Working on MOSS 2007

Mon, 19 Nov 2007 17:08:00 Z

I have a BIG-IP F5 load balancer that also handles off box SSL termination. I have a web server farm, set up accordingly:

SSL resides on the F5.

User talks with the F5 on port 443 (HTTPS. User never talks directly to MOSS web servers.

F5 talks to the MOSS web servers on regular port 80 (HTTP).

user -> HTTPS -> F5 -> HTTP -> MOSS web servers

I set up AAM using the instructions for http://blogs.msdn.com/sharepoint/archive/2007/03/06/what-every-sharepoint-administrator-needs-to-know-about-alternate-access-mappings-part-1.aspx and while this worked on an ISA reverse proxy scenario, it does NOT work for the above F5 scenario.

Symptoms: Most things work fine. However, whenever I try to create a list or library based on a list template, I get the following error:

Object reference not set to an instance of an object.   at ASP._layouts_new_aspx.OnLoad(EventArgs e)
   at System.Web.UI.Control.LoadRecursive()
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

When I go around the F5 and hit the web server(s) individually, I am able to create lists/libraries without seeing the above error.

Any help resolving this issue would be appreciated.

Thanks,

Yoshio

SSL Off Box Termination not Working on MOSS 2007

Tue, 27 Nov 2007 04:20:19 Z

Hi Yoshio -

To better understand your environment, could you include the output of the following stsadm.exe command?

stsadm.exe -o enumalternatedomains

Feel free to obfuscate the URLs that are listed, although try to keep the protocol scheme and port numbers the same, i.e. https://www.contoso.com:1234 --> https://www.foo.com:1234.

Also, is this just with custom list templates or is it also with the out-of-box list types such as Announcements, Document Library, etc.? Does the error occur when selecting the list template from the _layouts/create.aspx page or when clicking the Create button on the _layouts/new.aspx page?

- Troy Starr [MSFT]

SSL Off Box Termination not Working on MOSS 2007

Tue, 27 Nov 2007 16:15:02 Z

I am in the process of collecting the diagnostic information you requested.

I did not try creating a custom list; the error happens when trying to create a list from an out-of-the-box type, e.g. Contacts, Announcements, Discussions, etc. Strangely enough, I can create sites though. The error occurs on line 23 when clicking the Create button on the _layouts/new.aspx page.

Yoshio

SSL Off Box Termination not Working on MOSS 2007

Tue, 27 Nov 2007 17:12:15 Z

http://moss:80 --> http://moss:80 (Default) Load balanced URL

https://moss.contoso.com:80 --> https://moss.contoso.com:80 (Intranet zone)

http://portal.contoso.com:80 --> https://moss.contoso.com:80 (Intranet zone)

SSL Off Box Termination not Working on MOSS 2007

Tue, 27 Nov 2007 17:34:52 Z

FYI, this is BIG-IPs F5 support engineers response to the issue. Perhaps the detail will help troubleshoot the issue:

As per our conversation, this does not appear to be an error specific to the BigIP. I've looked through the packet traces you sent me, and have seen the following behavior. I use the following shorthand:

C = Client
S = Server
B = BigIP
Thus, C>S would be from Client to Server, and so on.
Each Line begins with the Frame Number in the dump where the packet is encountered.

To begin with, let's look at a working connection, over http port 80, direct to the server. This is detailed in dump2:

336 C>S POST /_layouts/new.aspx NTLMSSP_NEGOTIATE
368 S>C HTTP/1.1 401 Unauthorized NTLMSSP_CHALLENGE
372 C>S POST /_layouts/new.aspx NTLMSSP_AUTH + Creds
376 S>C HTTP/1.1 302 Found http://moss.ttx.com/Lists/test33/AllItems.aspx\r\n
377 C>S GET /Lists/test33/AllItems.aspx NTLMSSP_NEGOTIATE
379 S>C HTTP/1.1 401 Unauthorized NTLMSSP_CHALLENGE
381 C>S GET /Lists/test33/AllItems.aspx NTLM_AUTH + Creds
393 S>C HTTP/1.1 200 OK

The Client begins the exchange by sending a POST to new.aspx with no post data attached; it appears to be a dummy to start up the NTLM authentication. The Server responds with a 401 and the NTLM Challenge. The client then POSTs again, this time with the AUTH credentials and the full post data. Once the server receives and processes the post data, it sends a 302 Redirect to AllItems.aspx (fairly common behavior for a web-app), the two Auth again, and we're left with a 200 Ok and success.

Now, when we connect through the BigIP, the following occurs. First, the Client tries to connect to the Virtual Server on the BigIP on http port 80:

C>B POST /_layouts/new.aspx NTLMSSP_NEGOTIATE
B>C HTTP/1.1 302 Found https://moss.ttx.com/_layouts/new.aspx

The refering page specifically directs the client to POST to http. However, we're trying to do everything via https, hence the iRule designed to redirect all http requests to https. Now, in dump1, on https 443, from the Client, through the BigIP to the Server (BigIP does not alter the packets in any way):

14 C>S GET /_layouts/new.aspx NTLMSSP_NEGOTIATE
17 S>C HTTP/1.1 401 Unauthorized NTLMSSP_CHALLENGE
18 C>S GET /_layouts/new.aspx NTLMSSP_AUTH + Creds
20 S>C HTTP/1.1 200 OK Error Page

As you can see, upon receiving the 302 from the BigIP iRule, the Client switched methods from POST to GET. It starts to NTLM Auth with the server, but because it's now a GET operation, the post data the server is expecting is never sent, resulting in the application generating an Error Page.

Per the Status Code definitions in RFC 2616 at http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html, the following is stated of 302:

"10.3.3 302 Found

The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued."

It would appear that the user agent, rather than ask the user for confirmation, just defaults to a GET method upon receiving a 302 Redirect, thus complying with the RFC. So, the situation here is that we're attempting to set up an implementation that offloads the SSL to the load balancer, but the Application is still sending http links. With normal GETs this would not be a problem, as the BigIP can redirect them to https, but in this specific case, a redirect won't work because of the behavior of the browser.

The simplest solution in this case would be for the application to send it's links either relative, or hardcoded https. This would prevent any need to send redirects to the client, as the public-facing connection will stay encrypted from start to finish.

SSL Off Box Termination not Working on MOSS 2007

Wed, 28 Nov 2007 02:07:19 Z

Hi Yoshio -

There's a high probably that your AAM rules do not match your F5 BIG-IP configuration, resulting in the incorrect links and the error. Specifically, the F5 engineer's comment that their iRule was fixing up an http:// link in the HTTP 302 redirect to become an https:// link. This shouldn't be necessary if AAM's configuration matched your BIG-IP configuration.

Without seeing the output of stsadm.exe -o enumalternatedomains, it's a little tricky for me to determine what the cause is for sure, but here are some suggestions...

Ensure that your public URL for a zone is the URL that end users are typing in, i.e. https://moss.ttx.com. Note that this is not on port 80, but port 443. I suspect that you already have this properly configured.
Ensure that you've added an internal URL for the same zone is the URL of the request that BIG-IP is forwarding to the SharePoint server. I suspect that this is where the problem is. You probably already have the protocol scheme (http) and port number (80) of the internal URL properly configured. However, you may not have the hostname properly configured since BIG-IP may or may not be forwarding the original HTTP HOST header. If possible, run a packet capture between the BIG-IP and the SharePoint server. In the HTTP request, note the destination TCP port (probably 80) and the HTTP HOST header. The value of the host header should match the hostname of the internal URL and the destination TCP port should match the port number of the internal URL.

- Troy Starr [MSFT]

SSL Off Box Termination not Working on MOSS 2007

Wed, 28 Nov 2007 15:22:32 Z

SSL Off Box Termination not Working on MOSS 2007

Wed, 28 Nov 2007 16:15:57 Z

Wait, the Internal URLs have to be what MOSS expects, so they must all be http://xxx.xxx.xxx Only the Outbound URL will have https://xxx.xxx.xxx, I gather.

So just have one Zone, say Default, and have Inbound URLs http://xxx.xxx.xxx and do I need to set up an Internal URL for each server on the farm as well?

SSL Off Box Termination not Working on MOSS 2007

Wed, 28 Nov 2007 16:53:17 Z

Ok, I think I figured this out.

Create the site collection using a different DNS entry, say http://portal.contoso.com, this will be the Default zone.

Now, create a new zone, Intranet, that looks like this:

http://moss.contoso.com -> https://moss.contoso.com (port 443)

http://web1.contoso.com -> https://moss.contoso.com (port 443)

http://web2.contoso.com -> https://moss.contoso.com (port 443)

I'll write back once I can give this a try.

SSL Off Box Termination not Working on MOSS 2007

Wed, 28 Nov 2007 17:49:10 Z

Got it working. Thanks to Troy Starr for his guidance. Hopefully others will be able to find this solution via a search engine and avoid the issues I did.

SSL Off Box Termination not Working on MOSS 2007

Thu, 13 Mar 2008 16:18:15 Z

Hi Y Kurtz,

I am in the exact same configuration as you, and I can't make it work.

There is something I don't understand in what you said, it is :

Y Kurtz wrote:

Create the site collection using a different DNS entry, say http://portal.contoso.com, this will be the Default zone.

What do you mean by different DNS entry. I already have a Web Application, I need to extend it, and what do I change, the "host header" or the URL?

Hope you are still registered to this thread :-)

Thanks

Julien

SSL Off Box Termination not Working on MOSS 2007

Fri, 17 Jul 2009 08:54:47 Z

Hello Guys,

I am into same situation.. still not clear on the solution..

Hope to get more inputs on fixing the issue...

Thanks

Ara