Background
Shibboleth is an Identity Provider that uses the SAML standard to logon to applications. SAML is an unimaginably complicated standard, but when Shibboleth is used in production then all the details are handled for you. In testing, however, the Shibboleth you are using will not be deployed identically to production Shibboleth. You can still get 99% of your testing done even when Shibboleth is running in a "Sandbox" on your desktop. However, now you To test Shibboleth in a non-production configuration, you need to know a bit more about SAML so you know how to configure the test and what things will and won't work.The the checks that are performed on a SAML message. Almost all the important testing can be performed on Shibboleth running on your desktop, but to do the testing you have to know how to avoid tripping over one of the preliminary checks that will generate an error message before you get to the part you want to test.
The usual SAML definitions:
- The applications you use Shibboleth to login to (Box, Eliapps mail, IPTV) are called Service Providers (SP) and are also called Relying Parties (RP). We will use the term SP here.
- Shibboleth and each SP has a globally unique ID string called an EntityID. To make it globally unique, it is often a Web URL. The EntityID of production Shibboleth at Yale is "https://auth.yale.edu/idp/shibboleth", but the EntityID of Google Apps is just "google.com".
- Shibboleth at Yale and each SP exchange a file called Metadata. A Metadata file contains the EntityID, an encoded Certificate that can be used to validate digital signatures, and a URL. The Shibboleth "SSO" URL is where the SP redirects the Browser to login to Shibboleth, and the SP "ACS" URL is the network address to which Shibboleth tells the Browser to send the SAML "Response" message so the user can login.
- The SAML Response message from Shibboleth to the SP also contains Attributes of the user (like firstname, lastname, email address, etc.) that Shibboleth is configured to send to that particular SP.
In testing, the SP's don't change. They are still out there on the network with the same Metadata, EntityID, and "ACS" URL. Typically they are configured to redirect the Browser to production Shibboleth, and they will always only accept a message sent by the real production Shibboleth.
So unless you are testing using the Pre-Production Shibboleth instance, you are not actually going to login to any SP. You cannot generate a digitally signed SAML Response that any SP will regard as valid, because you don't have the secret key that only Production and Pre-Production Shibboleth share. That is usually not a problem if you get used to trapping and then reading the SAML Response. You can manually validate that the Subject field is correct and that all the Attributes are correct. There still will be a final test to actually login all the way, but if you have checked all the pieces first, that test will usually be pro forma.
However, you will not get as far as generating a SAML Response if you cannot route the request to your test Shibboleth instance, or if you have configured things so that one of the components immediately regards the request as a mistake and generates an error message. Just giving a recipe for a test configuration is not enough, because when any single instruction gets missed or is misunderstood, a specific error message is generated and you need an understanding of the process to track the message back to the cause.
So part of Client setup is to understand all the sanity checks that Shibboleth and the SPs will make, why each configuration value is needed to get past the checks, and what happens when the value is entered incorrectly.
In order for Shibboleth to send a SAML Response to an SP, the following things have to be properly configured on the Shibboleth server end:
...
They will not login anyone if they get a SAML Response from Shibboleth running on your desktop, and usually not from DEV or TEST Shibboleth either. However, you can always generate the SAML Response, capture it with the SAML Tracer plugin on Firefox, and then inspect it for correctness. That will do 99% of the testing even though you could not actually login because the digital signature generated by the test machine was not valid for production.
The problem occurs when you do not get to generating the SAML Response because something in the configuration triggers an error at the beginning of processing. To avoid those errors, you need to know a bit more about SAML with extra emphasis on the checks that Shibboleth, the SP, and the network make at each step. For example, in some cases Shibboleth will generate an error if it thinks it is running on a Tomcat that is not using SSL. Most of the time it doesn't care, but in one case it does make the check. So either you have to configure Tomcat to use SSL, or else you have to configure it to tell Shibboleth the request came in over SSL even when it didn't.
To get Shibboleth to generate a SAML Response with a login request to an SP, you have to provide Shibboleth with the EntityID of the SP. For example, in either of two different ways you tell Shibboleth to process a request for "google.com", and Shibboleth generates the SAML Response and tries to send it to the ACS URL to log you into your Eliapps mailbox. In order for this to work:
- The EntityID of the Service Provider has to be matched in one of the Metadata files configured to Shibboleth
- The Metadata file has to provide an AssertionConsumerService (ACS) URL to which the Response can be sent.
One of the Yale Shibboleth IdPs (TEST, or PROD) will have been configured to the Service Provider. In order for the SAML Response sent by Shibboleth to be accepted:
- The EntityID of the IdP (auth-test or auth) that sent the Response has to match the EntityID configured in the SP.
- The Digital Signature or the Response has to validate using the Public Certificate that the SP has configured for the Yale IdP.
- The SP EntityID and AssertionConsumerService URL in the Response XML have to match what the Service Provider believes to be its own EntityID and URL.
- A change of Shibboleth should not change the values of the Subject and of any Attribute. The user can change his name or email address, but if the old Shibboleth sent the Netid as Subject, then the new Shibboleth should not sent netid@yale.edu because that might make the SP think this is a different person. This is not important if the netid is sent as an attribute and the Subject is just a "transient" hash value.
Generally speaking, this means that an SP configured to work with TEST Shibboleth can only work with TEST, though in 99% of the cases we have an SP that is configured to only work with PROD, in which case you have to test with PREPROD.
Having said that, the version of Shib you use only matters if you are going to go all the way and try to login. A lot of testing can be done by just eyeballing the generated Attributes. If you are supposed to send firstname, lastname, email, and eduPersonPrincipalName, and if DEV or Sandbox Shibboleth generate a SAML Response with firstname, lastname, email, and EPPN in the XML, then you are pretty safe to say the case passes preliminary testing.
Occasionally someone asks for something new, or for a new version of something we are already generating. The Law School asks, "send us the list of groups in the Law School OU that contain this user as a member." We may have previously generated a list of all group membership, but now we need to filter it for just Law School groups and that is new. So now you create the new attribute, release it to this partner, and again manually check the generated SAML message to see if the right group names were generated.
Attributes have a formal unique name, a friendlyName (short nickname), and a single value or occasionally a list of values. A simple example (removing all the XML boilerplate) would be:
FirstName: John
LastName: Doe
Email: John.Doe@yale.edu
eduPersonPrincipalName: jd345@yale.edu
Different partners can ask for the same attribute with different names. So "John" can be a FirstName, First_Name, givenName, and so on. You need to check that you are providing the right value, but also that you are giving it the right name.
Before we had a PreProd
When a new Service Provider appears, it is convenient to initially ask them to configure the EntityID and Public Key Certificate of ourTEST Shibboleth. Then we can verify the attribute list, attribute names, and values and verify a login to the application. When it is all working, then we promote the changes to Production Shibboleth and ask the Service Provider to change EntityID and Certificate configuration to point to PROD. After a few days, we lose our contacts with the person at Yale coordinating the installation and the contact at the Service Provider who is making any changes.
Very few Service Providers bother to have a permanent configuration for our TEST Shibboleth. InCommon doesn't even have a concept of configuring TEST Identity Providers to anyone. So if after a few years we need to upgrade Shibboleth and test the new release, we have to do it with all of our Service Providers configured to use only auth.yale.edu and the production Shibboleth credentials. This explains the need for a PREPROD Shibboleth.
PREPROD is a Shibboleth IdP server on a VM that has the same /credentials subdirectory (and therefore the same private key) as PROD Shibboleth. If it generates a SAML Response, all of our Service Providers will accept it as genuine. It uses the production databases. It has been installed passing a property with the production EntityID. The only difference is that it is not actually at the IP address that the DNS server reports to be "auth.yale.edu" and if a message goes through the F5 to get to it, then it will use a separate (but hopefully equivalent) configuration so it is handled the same as production.
If we can arrange that on one computer in one Browser, all traffic addressed to production Shibboleth goes instead to PREPROD, and all messages back from PREPROD are sent on to the Service Provider, then we can verify that the new code running on PREPROD works before the same code is installed on PROD.
PREPROD is necessary for final testing where it is not good enough to simply "eyeball" the SAML XML, but you really need to go all the way to login to the Service Provider and verify that you get the right account contents. This isn't necessary for ordinary day to day testing of the configuration of Service Providers, but it is necessary if someone makes a change to Shibboleth itself or do a database like IDR that provides a lot of Shibboleth data.
IdP Initiated Logon
About 95% of the applications that use Shibboleth support "IdP Initiated" logon, where the Browser starts with a URL that points to Shibboleth and Shibboleth sends a SAML Response unsolicited to the Application. The starting URL is of the form:
https://auth.yale.edu/idp/profile/SAML2/Unsolicited/SSO?providerId=nobody.yale.edu
The "providerId" parmeter in this URL corresponds to what every SAML or Shibboleth document or configuration file calls the EntityID of a Service Provider. When this URL is presented to a Shibboleth server, then the handler for the /profile/SAML2/Unsolicited/SSO path searches the configured Metadata providers for an entry describing the named EntityID. From the Metadata it obtains the default AssertionConsumerService, to which the SAML Response will be sent. Then it searches the attribute-filter.xml for a policy that releases attributes to this EntityId.
If the Consent dialog has been enabled on this Shibboleth server, the user first sees a page listing the attributes that will be sent and their values. This is often good enough for initial testing.
Shibboleth generates a Response with the released Attributes and sends it to the ACS URL. If the Shibboleth IdP does not have the real production /credentials/idp.key file installed, the Service Provider will reject the digital signature. However, if you have SAML Trace enabled in your Browser you will have captured the SAML Response and can examine all the generated XML.
The final test, however, is to really login. To do this, you need production credentials, and that only happens when you test against the PREPROD VM in the machine room. So now you change the IdP initiated URL to point to that server, either with a public URL that goes through the F5, or a private URL that goes through the VPN tunnel to the server VM, or through an SSH tunnel to port 443 on the PREPROD VM.
SP Initiated Login
The user can start by going to the application. A Browser that starts with "yale.service-now.com", "yale.box.com", "mail.bulldogs.yale.edu", or the Comcast IPTV URL will in some cases get immediately sent to Shibboleth and in some cases will push a button or select Yale University from a dropdown list before being Redirected back to Shib.
Some applications that support SP Initiated also support IdP Initiated. You can test all the attributes and subject and the basic login using IdP Initiated, but when something changes that might affect it, you also have to be sure that when the Application sends a Request to Shibboleth then that Request is acceptedSo you can do most Shibboleth testing by feeding the test server a sequence of what are called "IdP Initiated" requests where you call the test Shibboleth server passing just the EntityID of some SP, and Shibboleth generates a SAML Response that you can capture with the SAML Trace tool and inspect for correctness. Of course, you will get some error message from each SP indicating the SAML Response was bad (because it was not digitally signed with the Production Shibboleth key) but you can ignore that.
If you actually want to login to the SP, then you need to be running the request on a Shibboleth that the SP is configured to respect. That will be Production or Pre-Production Shibboleth for most SPs, although during early testing with a new SP, it may be configured to use TEST Shibboleth. Now things are a bit more complicated because there are a long list of checks the SP will make on the SAML Response that you send it:
- The EntityID of the IdP (auth-test or auth) that sent the Response has to match the EntityID configured in the SP.
- The Digital Signature or the Response has to validate using the Public Certificate that the SP has configured for the Yale Shibboleth Identity Provider.
- The SP EntityID and AssertionConsumerService URL in the Response XML have to match what the Service Provider believes to be its own EntityID and URL.
- The first time the SP provides service to Yale users it is typically ready to accept any format of identifier. Therefore, we can initially provide either a simple Netid or a "scoped" value of netid@yale.edu. Once we start sending one format of Subject or Attribute, changing from one release of Shibboleth to another should not change the format of the attribute, or the SP will think that "joe@yale.edu" is a different person than "joe", forget any history and start with a new user blank slate.
Generally speaking, this means that final testing that goes all the way to a real login can only be done with the Pre-Production Shibboleth. However, any Shibboleth including the one on your desktop can generate a SAML Response with the XML names, values, and formats of Subject and Attributes, and if you compare the test and production XML you can see if everything is right.
IdP Initiated Logon
About 95% of the applications that use Shibboleth support "IdP Initiated" logon, where the Browser starts with a URL that points to Shibboleth and Shibboleth sends a SAML Response unsolicited to the Application. The starting URL is of the form:
https://auth.yale.edu/idp/profile/SAML2/Unsolicited/SSO?providerId=nobody.yale.edu
The last thing in the URL, the "providerid" parameter, is the EntityID of the SP. It has to match some configured EntityID in some Metadata file. Then Shibboleth logs you in, generates the SAML Response, and sends it to the configured AssertionConsumerServer (ACS) URL in the Metadata file.
You can use an IdP Initiated Login to generate a response for every SP, unless the SP Metadata demands signed requests:
<SPSSODescriptor AuthnRequestsSigned="true"For testing purposes, you can set this parameter temporarily to "false", do your testing with the IdP Initiated login URL above, but then deploy the original Metadata to production.
If you do an IdP Initiated login to an SP that doesn't allow them, it will reject the SAML Response and you won't actually login. However, the digital signature was also probably invalid (not from production) and there may be a lot of other defects, and you probably were not planning to go all the way to a successful login at this stage anyway. You can still capture and check the Response for content.
SP Initiated Login
Some Service Providers present a login page for users. The user has to press a big button to "Login with a Netid" or "Login from Yale", or maybe they have to select "Yale University" from a long pulldown list of schools. Then the SP sends the Browser to "https://auth.yale.edu/idp/...".
This is only important if you are really going to login to the application for final testing. Before that, use the IdP Initiated format to generate a SAML Response and manually check its content to verify that Shibboleth seems to be generating the right Subject and Attributes. So you only use SP Initiated Login for final testing after that preliminary test has completed.
You will, of course, need a list of the login pages for the applications you want to test. Go to "yale.service-now.com", "yale.box.com", "mail.bulldogs.yale.edu", or the "www.yale.edu/iptv" to get started.
There are two ways an application can send its Request to Shibboleth:
- (SAML Browser POST Protocol) writes a Form to the screen containing a pre-loaded text box containing the SAML Request, then uses JavaScript to Submit the data to https://auth.yale.edu/idp/profile/SAML2/POST/SSO
- (SAML Browser Redirect ProcolProtocol) redirects the Brower Browser to "https://auth.yale.edu/idp/profile/SAML2/Redirect/SSO?SAMLRequest=" followed by the character encoded SAML Request
So, the first thing that happens in a SP Initiated is slightly more complicated because the Application will have been configured with a specific Shibboleth URL which is almost always "https://auth.yale.edu/idp". It will have placed this URL in the Request XML, and it will have Redirected the Browser to that URL. So now you have two problems.
...
login is your Browser generates a POST or GET operation to "https://auth.yale.edu/idp
...
..." and unless you trap this URL and send it to a different location then this operation will go to the Production Shibboleth server.
If you do trap the request and send it to
...
a test
...
Shibboleth
...
server, then you have a second set of problems. Shibboleth implements a set of SAML standard checks to make sure that messages have been delivered to the right server. So generally the test Shibboleth has to actually believe that it is running on a machine named "auth.yale.edu" and that
...
There are many ways to accomplish these two things in testing. They should all be listed because there are a few things you can do on any machine and get 99% of your testing done, but the final end-to-end test of everything requires a PREPROD VM in the machine room and support from Operationsit is running https on the default port.
Network Routing
In all cases, communication to Shibboleth goes through your Browser.
...