Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 3 Next »

SAML Messages

Shibboleth generates a SAML message (called a "SAML Response") that it sends to eachr of our Service Providers (such as box, google, service-now, archer, and so on). There is a tiny amount of important content in the Response buried in a large amount of meaningless XML junk. If the Service Provider accepts the message, then the user logs on to that service.

In order for the SAML Response to work, it has to pass a long list of checks:

  • The EntityID (an arbitrary but unique identifying string) has to be matched in one of the Metadata files.
  • The Metadata file has to provide the correct URL to which Shibboleth sends the SAML Response message. This is called the AssertionConsumerService or ACS URL.
  • The Digital Signature generated by Shibboleth has to be validated by the Public Key or Certificate configured for Yale in the Service Provider application and the EntityID name that Shibboleth uses to identify itself also has to match. In practice, this means that a Service Provider configure with PROD Shib has to get a message from a server configured with the private key credentials and EntityID of PROD Shib, while a SP that expects a message from TEST needs a message signed with the TEST key and sent with the TEST EntityID.
  • The message will contain the EntityID and ACS URL of the Service Provider. SAML messages have an implied "eyes only" rule, so these two strings have to exactly match the EntityID and ACS URL that the Service Provider expects. Since these two values came from a Metadata file the SP sent us, this check is usually never a problem.
  • The message does not contain any information about the network address or server name of the Shibboleth server. So it is not important that Shibboleth run on any particular machine.
  • The SAML Response must contain any attributes that the Service Provider has told us are required. Generally speaking, across any change to Shibboleth the value of each attribute should not change. If the user changes their name or email address, then Shibboleth will start sending the new attributes, but the attributes should not change if Shibboleth switches to a new database or we put in a new version of Shibboleth.
  • Each SAML message contains a Subject. In most cases the Subject is ignored, but there are a few special cases that also turn out to be really important cases (Google for EliApps, Service-Now) where the Subject is the only field and it is special and must have the same value each time.

The most common Shibboleth testing occurs when we have a new Service Provider to support. They will provide us with Metadata and a list of attributes they need us to supply.

If the attributes are already defined for someone else, all we need to do is to add the Metadata and to add or modify an entry in the attribute-filter.xml to release the attributes to this EntityID. Generally we can set this up in Sandbox or DEV testing, use the IdP Initiated Login URL with this EntityID, and then capture the Response. Correct behavior can be determined by an inspection of the Response. Then the code can move to PREPROD and a real login to the application becomes possible.

Occasionally someone asks for something new, or for a new version of something we are already generating. The Law School asks, "send us the list of groups in the Law School OU that contain this user as a member." We may have previously generated a list of all group membership, but now we need to filter it for just Law School groups and that is new. So now you create the new attribute, release it to this partner, and again manually check the generated SAML message to see if the right values were generated.

Attributes have a formal unique name, a friendlyName (short nickname), and a single value or occasionally a list of values. A simple example (removing all the XML boilerplate) would be:

FirstName: John
LastName: Doe
Email: John.Doe@yale.edu
eduPersonPrincipalName: jd345@yale.edu

Different partners can as for the same attribute with different names. So "John" can be a FirstName, First_Name, givenName, and so on. So you have to generate both the right name and the right value for each item.

When a new Service Provider appears, it is convenient to initially ask them to configure the EntityID and Public Key Certificate of TEST Shibboleth. Then we can verify the attribute list, attribute names, and values and verify a login to the application. When it is all working, then we promote the changes to Production Shibboleth and ask the Service Provider to change EntityID and Certificate configuration to point to PROD.

Unfortunately, Service Providers typically have no way to go back to using TEST. In fact, our InCommon Federation has no concept of DEV or TEST Identity Providers, so the only thing they can use is our PROD Shibboleth.

Fortunately, in most cases they require a SAML Response that includes the prod EntityID and a digital signature created by the PROD private key, but they do not know and therefore cannot depend on the network address of the Shibboleth server that generated this message. This creates the need for what we call the PREPROD Shibboleth instance.

PREPROD is a separate Shibboleth server on a different VM with a different network address. Since it is not technically "In Production" we can change it any time we want to. However, it is configured with the same EntityID as real PROD, and it has a /apps/shibboleth-idp/credentials directory with the Production Shibboleth private key for signing SAML messages. So a message generated by PREPROD will be accepted by all our partner Service Providers as having come from real PROD, and they will log someone on given that message.

PREPROD is necessary for final testing where it is not good enough to simply "eyeball" the SAML XML, but you really need to go all the way to login to the Service Provider and verify that you get the right account contents. This isn't necessary for ordinary day to day testing of the configuration of Service Providers, but it is necessary if someone makes a change to Shibboleth itself or do a database like IDR that provides a lot of Shibboleth data.

IdP Initiated Login

An "IdP Initiated" test occurs when you click a URL that points to our Shibboleth server and provides the EntityID of one of our Service Provider partners:

http://localhost:8080/idp/profile/SAML2/Unsolicited/SSO?providerId=nobody.yale.edu

The hostname part ("localhost:8080") could reference a Sandbox test Shibboleth running on Tomcat on your desktop, or it could be a test version of Shibboleth running on a VM in the machine room where you have created an SSH tunnel connecting port 8080 on your desktop to the real Server port on the real Server VM.

At the end, after "providerId=" you add the EntityID of a partner. It is a Yale testing convention that the ID "nobody.yale.edu" is configured to release a very large number of attributes including most of the interesting ones you might want to check. It is the first test to make because is does so much it will point out most errors.

After generating the SAML Response XML, Shibboleth will try to send it to the AssertionConsumerService URL configured in the Metadata for Entity "nobody.yale.edu". This ACS URL is bogus (literally, it is "http://www.yale.edu/bogus") and the message goes nowhere. If you are running on DEV or the Sandbox, before the SAML is sent you will get the "Consent" page where Shibboleth lists all the attributes and values it is about to send, and many problems can be debugged from that page without ever actually sending out the SAML. Otherwise, using the SAML Trace plugin in Firefox you can trap the SAML sent to the bogus address and then inspect it for problems.

Change the EntityID at the end of the URL and you can verify that the right attributes are sent to any particular Service Provider.

SP Initiated

The hard part occurs when you have to go to the Service Provider first, and it has to send you back to Shibboleth. In this case the Service Provider will have generate a SAML Request message that Shibboleth receives. Shibboleth requires that the Request be valid. It performs essentially the same checks that we described above when the Service Provider was validating the SAML Response generated by Shibboleth. I will now copy that list of checks and note the differences.

In order for the SAML Request to work, it has to pass a shorter list of checks:

  • The EntityID in the Request has to match the EntityID configured to this Shibboleth server through the "idp.entityid" property on the Jenkins Install. Because SPs are almost always configured to talk only to PROD, that means the Shibboleth has to be installed with "idp.entityID= https://auth.yale.edu/idp/shibboleth"
  • The Service Provider will send the message to the URL it has configured for Yale. Unfortunately, this will almost always be a production URL ("https://auth.yale.edu/idp/...") and so the tester has to trap this on their desktop and reroute it to the test or preprod Shibboleth server (more on that below).
  • The Digital Signature generated by the Service Provider has to be validated by the Public Key or Certificate configured in the Shibboleth Metadata that defines that Service Provider. This will be done by the SP and requires nothing on our end.
  • In addition to the EntityID, a Request contains the SSO URL for Shibboleth configured to the SP in Yale's Metadata (typically "https://auth.yale.edu/idp/profile/SAML2/POST/SSO" or "https://auth.yale.edu/idp/profile/SAML2/Redirect/SSO"). This has to exactly match a URL that Shibboleth constructs from its environment using a utility class that gets some of its information for the Host header and some from the HTTPServeletRequest object passed by Tomcat.

So the Browser, test computer on which the Browser is running, the network path between the Browser and the test VM, and the Tomcat environment on the VM all have to do two things:

  1. They have to intercept a request for auth.yale.edu and send it to the test VM, but
  2. They have to set the Host header in the request and the HTTPServletRequest object up so that the URL Shibboleth constructs using the utility class begins with :https://auth.yale.edu/idp". More explicitly, the URL constructed cannot start with "http:" (Tomcat does not believe the request comes over SSL) and there cannot be a port number (so no port number in the Host header or added by Tomcat).

One set of problems has to be solved to send the request to the test VM in the first place. No ordinary computer inside or outside Yale can communicate with the VM that runs the Shibboleth IdP. Instead, a network address like "https://auth.yale.edu/idp" is defined to a special network front end device. At Yale this is the "F5" that stands in front of all network services. The host name "auth.yale.edu" resolves to a network address that is on the F5 device. The Certificate that proves that the server is "auth.yale.edu" is installed on the F5, not the Shibboleth server VM. So Shibboleth (and pretty much every other network function) cannot generally expect to find from the computer on which it runs the host name that everyone else in the world uses to get to it. For the Shibboleth code to even know about "auth.yale.edu", then either it has to have this name configured as one of its properties provided by its Install Project, or else it has to get the name from the control blocks that Tomcat provides with each incoming request.

When the Browser generates any HTTP request, it creates a Host header. This header contains the protocol, hostname, and port number from the URL that the Browser is using to get to the application. For a request that thinks it is going to production Shibboleth, this is

Host: https://auth.yale.edu

This header can go through the network and the F5 unmodified, or it can be changed and there are other network standards for other headers that might be generated by the F5 or any other device that receives, modifies, and forwards the request. The people who configure these network devices know all these conventions, and production Shibboleth receives data through a similar path with a similar set of header modifications.

So if you trap a Service Provider generated SAML Request and decide to forward it to a test Shibboleth server, things will work correctly provided that the Shibboleth server is configured with the same EntityID as production and that when the SAML Request arrives at the test server, passes through Tomcat, and is presented to Shibboleth that the Host header (and any other headers commonly generated by machines like the F5) convince Shibboleth that this request was originally addressed to "https://auth.yale.edu/idp".

Although there will be at any time one or more specific network configurations for accomplishing this result, there are lots of rules and tricks and software that can accomplish the same thing in various ways. The thing to remember is that if you try to login to a Service Provider with a test Shibboleth, and instead of getting a CAS login you get a message saying that the request did not meet security restrictions, and the Shibboleth log contains an error message saying that the message was addressed to "https://auth.yale.edu/idp" but Shibboleth has decided that its own network address is "http://auth.yale.edu/idp" (http not https), or "https://auth.yale.edu:8080/idp" (a port number got added), or "http://localhost:8080/idp" (you didn't fake anything out at all), then you have not set the HTTP headers and the network path right and you have to map out all the intermediate boxes and tunnels through which the request is passing to figure out what is wrong or missing.

The problem is that some of this path may not be under the control of the developer or tester, and if you try and create a path that is completely under your control you now have to learn more about the configuration of Tomcat and HTTP proxy tools, which are normally a problem for someone else.

Some successful recipes will be provided in the Testing Setup document, but if you decide to deviate from them you need to understand the problem described above and figure out new solutions.

  • No labels