Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 11 Next »

Generally the Developer has little control over the setup of the Server and Network. Operations configures all of the network devices, the operating system, and even the Tomcat server. However, configuration problems can cause test failures, and the only way to fix these issues is to get the configuration fixed. Generally you only have to review this material when there is a major change to the network or server setup.

IdP Initiated Login

An "IdP Initiated" test occurs when you click a URL that points to our Shibboleth server and provides the EntityID of one of our Service Provider partners:

http://localhost:8080/idp/profile/SAML2/Unsolicited/SSO?providerId=nobody.yale.edu

This URL calls a Shibboleth server that could be running under Tomcat on the local desktop or could be a test version of Shibboleth running on a VM in the machine room where you have created an SSH tunnel connecting port 8080 on your desktop to the real Server port on the real Server VM.

This URL is a Shibboleth feature, not part of the SAML standard. The Shibboleth code that responds to a "/profile/SAML2/Unsolicited/SSO" URL path does not care about the hostname, port, or protocol. It makes no assumptions about how the client connected to the server or what the network looks like.

The only checks are:

  • the providerid ("nobody.yale.edu" in this example) must be the EntityID of a Service Provider configured in some Metadata file
  • The Metadata file must have a configured default AssertionConsumerService URL
  • The hostname in the ACS URL must exist on the network.

Internally Shibboleth builds a dummy SAML Request, then triggers normal SAML processing to generate a Response.

If Consent is configured (and it will be by default in most Sandbox environments) the generated Attributes will be displayed. Then the SAML Response will be sent to the ACS URL.

This will work and will attempt to deliver the Response no matter what operating system you are running on, no matter what hostname, port number, or protocol your Tomcat uses, no matter what EntityID your Shibboleth server is configured to use for itself. In short, there are no network issues for IdP Initated Login.

Of course, the SP will probably discard the Response because it will generally not be digitally signed correctly, and the EntityID that issued it may be unrecognized by the SP, but that is for later testing.

The Unicon CAS Integration

Unicon provides a Shibboleth Login ("Authn") module that uses CAS. To configure this integration, the properties file must supply the protocol and hostname to use in the Service string sent to CAS. CAS will Redirect the Browser back to this URL after it issues the Service Ticket. Unicon has a specific property name, but Yale used a different name long before we switched to using the Unicon code. We keep the old Yale property name and use it to set the new Unicon property.

So in the Install Project, you must set a property named "cas.target.url" to be the protocol and hostname of the Shibboleth server from the point of view of the Browser. When you are testing a Sandbox running on your desktop, then it makes perfect sense to set the property in install.properties and make the value to "localhost", or more specifically:

cas.target.url=http://localhost

However, this is probably not a good value to use when Shibboleth is installed in DEV or TEST. In these cases the property is set in install-DEV or install-TEST and it will reference the "auth-dev.yale.edu" and "auth-test.yale.edu" machines. This means, however, that if you install Shibboleth into DEV or TEST normally with these parameters, then you are not going to be able to use these VMs through an SSH tunnel (where they appear to be localhost:8080) unless you use some mechanism (Redirectory, Charles Proxy, hosts table) to rename the "https://auth-dev.yale.edu/" in the CAS Redirect to actually go to the SSH Tunnel.

You know you have this problem when Shibboleth sends you to CAS, you login, and then get a

HTTP Status 500 - Error processing ShibCas authentication request

error with a root cause of:

ExternalAuthenticationException: No conversation state found in session for key (e1s1)

What has happened is that Shibboleth received your original request from what it regards as one browser session, and it sent that Browser to CAS, but now it is getting back a response from CAS that appears to be coming in from a different Browser (although it is really the same Browser connecting to Shibboleth through a different network path using a different host name).

This error can also occur when two Shibboleth VMs exist behind a load balancing front end, but the front end has not been configured to route subsequent requests for the same session (based on the "JSESSIONID" parameter) to the same VM. Then one VM sends the Browser to CAS, but the response gets routed to the other VM.

SP Initiated

At one point I thought that it would be possible to run an SP Initiated test case to a Sandbox Shibboleth running on the local desktop or through an SSH tunnel to a VM in the machine room.

However, the only proven solution is to install code on the Pre-Production Shibboleth VM and then put an entry in the hosts file to point the "auth.yale.edu" name to the IP address of the Pre-Production VIP on the F5.

The rest of this section will describe the problem with all other mechanisms.

The IdP Initiated test was simplified because the code that handles the "Unsolicited/SSO" URL does not do any checks on the network address. Technically, the "Unsolicited/SSO" is a Shibboleth programming convention and not part of the SAML standard, so checks are not required.

However, if you go to the Service Provider and press a "Login from Yale" button or select Yale from a pulldown list of universities, then the SAML Request object generated by the Service Provider and the protocol by which it is sent to Shibboleth are part of the SAML standard, and that standard requires Shibboleth to do some extra checks that make the network configuration a bit more complicated.

Furthermore, while the network path from the Browser to the Service Provider (to send the Shibboleth Response) is just normal Yale networking and is not changed by the Shibboleth testing configuration, the SAML message that the SP sends to the Browser and then expects the Browser to deliver to Shib obviously is going to be affected by which Shib VM you want to use and how the Browser sends data to it.

Shibboleth processes the SAML Request first, before it even sends you to CAS in order to login. The Request has to go through a series of checks:

  • The EntityID to which the Request is addressed has to match the EntityID configured to this Shibboleth server through the "idp.entityid" property on the Jenkins Install. In the IdP Initiated testing you can use any Shibboleth (PROD, TEST, DEV, or Sandbox) and the Response will be generated and traced so you can look at its contents. It is only when the Response finally gets to the SP that it will be rejected if it comes from the wrong Shibboleth. But with an SP Initiated login you have to send the Request to the Shibboleth selected by the SP or you don't even start to do any processing. Because SPs are almost always configured to talk only to PROD, that means the Shibboleth you send the Request to has to think it has the production Shibboleth EntityID (that is, it must have been installed with the "idp.entityID= https://auth.yale.edu/idp/shibboleth" property).
  • The Service Provider will tell the Browser to send the Request (either by POSTing a form or by Redirect on a GET) to the SSO URL that the SP has in its copy of the Yale IdP Metadata. That will be a production URL, and so the Browser will be instructed to send the Request to "https://auth.yale.edu/idp/..." Since that will not be the actual URL of the test machine, you will have to use one of the tools described in the Client Configuration document to rewrite or reroute the request to the test VM.
  • Shibboleth will validate the digital signature on the Request, but that will always work because the SP is genuine and is not itself being tested.

Now we get to the last test, which is harder and requires a longer description. The SAML protocol requires a Request to include the URL to which the Request is sent to also be data in an internal XML field of the Request, and it requires the IdP to check that this field is correct. Depending on whether the Request is sent as a POST or a Redirect, the URL for PROD Shib will be "https://auth.yale.edu/idp/profile/SAML2/POST/SSO" or "https://auth.yale.edu/idp/profile/SAML2/Redirect/SSO".

Of course, this URL will also have been included in the HTTP headers, and previously some mechanism was used to rewrite or reroute the HTTP flow to the test Shibboleth. That did not change the XML content. So now the problem is to ensure that the URL that the test Shibboleth server receives matches a URL that it reconstructs using the Host header and the HTTPServletRequest object it receives.

Of course, with open source you can download and read the Java code to see exactly how it is done. You can even find the Spring Bean statement that configures the comparison and replace it with new code that relaxes the rules. However, generally speaking if the Host header contains "https://auth.yale.edu" and Tomcat believes that the message arrived over SSL (at least as far as the proxy) then the comparison will work. So you will have a problem when you are trying to handle an SP Initiated Login and either:

  • You use Redirector to rewrite the server URL before the Host header is generated instead of using the hosts file or Charles to reroute the request while preserving the "auth.yale.edu" hostname.
  • You send the data over an SSH tunnel directly to non SSL port 8080 without any Proxy protocol or Tomcat configuration that says the data was originally SSL.

If the URL in the XML and the URL that Shibboleth generates don't match, Shibboleth writes an error message to the log containing the two strings it is comparing. You know you have to fix your client configuration when one of the two strings reported in the log contains:

  • "http://localhost:8080/" -  You have both problems, Redirector has rewritten the Host header and you have a tunnel to 8080.
  • "http://auth.yale.edu" - The host header is right, but Tomcat doesn't think this is https.
  • "https://auth.yale.edu:8080" - Tomcat knows this is https, but the host header had a port number inserted.

The Client Setup article will provide some configurations that are known to work. Others may also work, but you may take some time with trial and error trying to fix the last problems.

Background on Proxies and Tunnels

The first thing to say is that in almost no modern network is the hostname of the server that the Browser uses to get to Shibboleth ("auth.yale.edu") the actual name of the VM on which Shibboleth runs. In fact, it is almost never possible for the Browser to actually talk to Shibboleth directly. For security reasons, the network addresses and firewalls are set up to block outside access to server VMs, and a front end proxy machine provides the connection between the Browser and the Server. At Yale, that Proxy machine is called the F5.

Proxies have been part of HTTP since the early days of the World Wide Web. Originally they were used when the Internet was slow and it did not make sense for hundreds of computers at Yale to each independently download their own copies of widely used web pages. Today, you can put a proxy:

  • On the same computer as the Browser. The Charles Web Debugging Proxy is a useful tool for testing.
  • You can still find a client oriented proxy somewhere in the campus network, but this is no longer common or useful.
  • The F5 sits between the secure environment of the servers in the machine room and the outside world of untrusted requesters.
  • A front end (Nginx or Apache Web Server) can receive requests on ports 80 and 443 on the Shibboleth VM and they can forward those requests to Tomcat.

As a developer testing code, you probably only have control over the first type of Proxy, and if you chose to use it then we recommend Charles Web Debugging Proxy as the simplest tool.

However, if you run a test and the SAML Request send by the SP is being rejected because the generated string does not match the expected URL, then something is going wrong and it could be with any of the other intermediate devices or it could be a Tomcat configuration problem. So you have to understand something about how this stuff works in order to debug it.

The good news is that Tomcat fully expects that one or more proxies will exist between the Browser and the server VM, so it is quite flexible in its configuration. Shibboleth is not as flexible, but the fact that it can work at all means that if you get the proxies configured right then everything will be OK.

When the Browser generates any HTTP request, it creates a Host header. This header contains the protocol, hostname, and port number from the URL that the Browser is using to get to the application. For a request that thinks it is going to production Shibboleth, this is

Host: https://auth.yale.edu

This header can go through the network and the F5 unmodified, or it can be changed. There are other headers that proxies frequently generate to indicate processing they have done. There are standards and if you have a couple of days you might find and read through all of them.

However, there are two basic rules. First, if the Browser thinks it is talking to production Shibboleth, and the rerouting is done in the network configuration, then one solution is to leave the original Host header alone. On the other hand, if the Browser knows that the test Shibboleth is really at a different URL, then it might be a good idea to have something in the proxy path that changes the Host header so it has the auth.yale.edu value shown above. In other words, if you know it is right leave it alone, and if you know it is wrong, change it.

It is trivial for Operations to fix the header, but if they are not already doing it, this will be more trouble than it is worth. Configure the client to generate the right header in the first place.

If the header is right coming out of the client, and it is being set incorrectly in the proxy chain, then you have to get Operations to fix it.

  • No labels