Versions Compared

Version Old Version 17 New Version Current
Changes made by

Howard Gilbert

Howard Gilbert

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Background

Shibboleth is an Identity Provider that uses the SAML standard to logon to applications. SAML is an unimaginably complicated standard, but when Shibboleth is used in production then all the details are handled for you. In testing, however, the Shibboleth you are using will not be deployed identically to production Shibboleth. You can still get 99% of your testing done even when Shibboleth is running in a "Sandbox" on your desktop. However, now you need to know a bit more about SAML so you know how to configure the test and what things will and won't work.

The applications you use Shibboleth to login to (Box, Eliapps mail, IPTV) are called Service Providers (SP) and are also called Relying Parties (RP). We will use the term SP here.

Shibboleth and each SP has a globally unique ID string called an EntityID. To make it globally unique, it is often a Web URL. The EntityID of production Shibboleth at Yale is "https://auth.yale.edu/idp/shibboleth", but the EntityID of Google Apps is just "google.com".

Shibboleth at Yale and each SP exchange a file called Metadata. A Metadata file contains the EntityID, an encoded Certificate that can be used to validate digital signatures, and a URL. The Shibboleth "SSO" URL is where the SP redirects the Browser to login to Shibboleth, and the SP "ACS" URL is the network address to which Shibboleth tells the Browser to send the SAML "Response" message so the user can login.

The SAML Response message from Shibboleth to the SP also contains Attributes of the user (like firstname, lastname, email address, etc.) that Shibboleth is configured to send to that particular SP.

In testing, the SP's don't change. They are still out there on the network with the same Metadata, EntityID, and "ACS" URL. Typically they are configured to redirect the Browser to production Shibboleth, and they will always only accept a message sent by the real production Shibboleth.

So unless you are testing using the Pre-Production Shibboleth instance, you are not actually going to login to any SP. You cannot generate a digitally signed SAML Response that any SP will regard as valid, because you don't have the secret key that only Production and Pre-Production Shibboleth share. That is usually not a problem if you get used to trapping and then reading the SAML Response. You can manually validate that the Subject field is correct and that all the Attributes are correct. There still will be a final test to actually login all the way, but if you have checked all the pieces first, that test will usually be pro forma.

However, you will not get as far as generating a SAML Response if you cannot route the request to your test Shibboleth instance, or if you have configured things so that one of the components immediately regards the request as a mistake and generates an error message. Just giving a recipe for a test configuration is not enough, because when any single instruction gets missed or is misunderstood, a specific error message is generated and you need an understanding of the process to track the message back to the cause.

So part of Client setup is to understand all the sanity checks that Shibboleth and the SPs will make, why each configuration value is needed to get past the checks, and what happens when the value is entered incorrectly.

In order for Shibboleth to send a SAML Response to an SP, the following things have to be properly configured on the Shibboleth server end:

  • The EntityID of the Service Provider has to be matched in one of the Metadata files configure to Shibboleth
  • The Metadata file has to provide an AssertionConsumerService (ACS) URL to which the Response can be sent.

One of the Yale Shibboleth IdPs (TEST, or PROD) will have been configured to the Service Provider.  In order for the SAML Response sent by Shibboleth to be accepted:

  • The EntityID of the IdP (auth-test or auth) that sent the Response has to match the EntityID configured in the SP.
  • The Digital Signature or the Response has to validate using the Public Certificate that the SP has configured for the Yale IdP.
  • The SP EntityID and AssertionConsumerService URL in the Response XML have to match what the Service Provider believes to be its own EntityID and URL.
  • A change of Shibboleth should not change the values of the Subject and of any Attribute. The user can change his name or email address, but if the old Shibboleth sent the Netid as Subject, then the new Shibboleth should not sent netid@yale.edu because that might make the SP think this is a different person. This is not important if the netid is sent as an attribute and the Subject is just a "transient" hash value.

Generally speaking, this means that an SP configured to work with TEST Shibboleth can only work with TEST, though in 99% of the cases we have an SP that is configured to only work with PROD, in which case you have to test with PREPROD.

Having said that, the version of Shib you use only matters if you are going to go all the way and try to login. A lot of testing can be done by just eyeballing the generated Attributes. If you are supposed to send firstname, lastname, email, and eduPersonPrincipalName, and if DEV or Sandbox Shibboleth generate a SAML Response with firstname, lastname, email, and EPPN in the XML, then you are pretty safe to say the case passes preliminary testing.

Occasionally someone asks for something new, or for a new version of something we are already generating. The Law School asks, "send us the list of groups in the Law School OU that contain this user as a member." We may have previously generated a list of all group membership, but now we need to filter it for just Law School groups and that is new. So now you create the new attribute, release it to this partner, and again manually check the generated SAML message to see if the right group names were generated.

Attributes have a formal unique name, a friendlyName (short nickname), and a single value or occasionally a list of values. A simple example (removing all the XML boilerplate) would be:

FirstName: John
LastName: Doe
Email: John.Doe@yale.edu
eduPersonPrincipalName: jd345@yale.edu

Different partners can ask for the same attribute with different names. So "John" can be a FirstName, First_Name, givenName, and so on. You need to check that you are providing the right value, but also that you are giving it the right name.

Before we had a PreProd

When a new Service Provider appears, it is convenient to initially ask them to configure the EntityID and Public Key Certificate of ourTEST Shibboleth. Then we can verify the attribute list, attribute names, and values and verify a login to the application. When it is all working, then we promote the changes to Production Shibboleth and ask the Service Provider to change EntityID and Certificate configuration to point to PROD. After a few days, we lose our contacts with the person at Yale coordinating the installation and the contact at the Service Provider who is making any changes.

Very few Service Providers bother to have a permanent configuration for our TEST Shibboleth. InCommon doesn't even have a concept of configuring TEST Identity Providers to anyone. So if after a few years we need to upgrade Shibboleth and test the new release, we have to do it with all of our Service Providers configured to use only auth.yale.edu and the production Shibboleth credentials. This explains the need for a PREPROD Shibboleth.

PREPROD is a Shibboleth IdP server on a VM that has the same /credentials subdirectory (and therefore the same private key) as PROD Shibboleth. If it generates a SAML Response, all of our Service Providers will accept it as genuine. It uses the production databases. It has been installed passing a property with the production EntityID. The only difference is that it is not actually at the IP address that the DNS server reports to be "auth.yale.edu" and if a message goes through the F5 to get to it, then it will use a separate (but hopefully equivalent) configuration so it is handled the same as production.

If we can arrange that on one computer in one Browser, all traffic addressed to production Shibboleth goes instead to PREPROD, and all messages back from PREPROD are sent on to the Service Provider, then we can verify that the new code running on PREPROD works before the same code is installed on PROD.

PREPROD is necessary for final testing where it is not good enough to simply "eyeball" the SAML XML, but you really need to go all the way to login to the Service Provider and verify that you get the right account contents. This isn't necessary for ordinary day to day testing of the configuration of Service Providers, but it is necessary if someone makes a change to Shibboleth itself or do a database like IDR that provides a lot of Shibboleth data.

IdP Initiated Logon

...

Audience

If you are an application owner testing a new version of Shibboleth before it goes into production, or if you are testing a new application that has not yet been configured to the production Shibboleth server, then you want to look at the instructions for configuring a desktop system to use Pre-Production Shibboleth. See Testing Your Application before a Shibboleth Change.

This document provides instructions for a Shibboleth developer who needs to test a Sandbox Shibboleth running on his desktop under Tomcat, or to test new VMs in the machine room for a future release of Shibboleth that has not yet even replaced the DEV or TEST environments.

Background

Generally speaking, you can test proposed changed to Shibboleth using https://auth-dev.yale.edu/idp and https://auth-dev.yale.edu/idp, then do final testing on the Pre-Production machine by changing your "hosts" file.

However, a Shibboleth developer does 90% of the testing using a Sandbox Shibboleth running on the local desktop computer.

When you are making a major Version upgrade, as we did moving from Shibboleth 2 to Shibboleth 3, then it is necessary to create an entirely new set of VMs in the machine room, with new Install jobs. A certain amount of testing has to be done with these VMs to make sure that the Installation process is working before you can even change the F5 front end to point to them in DEV or TEST. During this period, the only way to access the new VMs is to use an SSH tunnel that exposes the Tomcat running on the VM as if it were a local port on your desktop computer.

In both cases, you end up effectively going to "http://localhost:8080/idp", but even if that is the real address of Shibboleth, that doesn't mean that this is the exact URL you are always going to want to use. Sometimes you need to create an alias name for "localhost" (a.k.a. IP address 127.0.0.1). Sometimes you need to hide or translate the port number. There are several tools that can help.

Why isn't it good enough to simply get to Tomcat and the Shibboleth server application? Sometimes there is a problem created by the default HTTP Host header, the fact that the Sandbox uses http instead of https, and the default Tomcat configuration. Sometimes the problem comes from SAML.

SAML Checks

First, the usual SAML definitions of terms we have to use:

  • The applications you use Shibboleth to login to (Box, Eliapps mail, IPTV) are called Service Providers (SP) and are also called Relying Parties (RP). We will use the term SP here.
  • Shibboleth and each SP has a globally unique ID string called an EntityID. To make it globally unique, it is often a Web URL. The EntityID of production Shibboleth at Yale is "https://auth.yale.edu/idp/shibboleth", but the EntityID of Google Apps is just "google.com".
  • Shibboleth at Yale and each SP exchange a file called Metadata. A Metadata file contains the EntityID, an encoded Certificate that can be used to validate digital signatures, and a URL. The Shibboleth "SSO" URL is where the SP redirects the Browser to login to Shibboleth, and the SP "ACS" URL is the network address to which Shibboleth tells the Browser to send the SAML "Response" message so the user can login.
  • The SAML Response message from Shibboleth to the SP also contains Attributes of the user (like firstname, lastname, email address, etc.) that Shibboleth is configured to send to that particular SP.

Box and Google do not know or care that you are testing a new version of Shibboleth on your desktop. They are configured to talk to production (auth.yale.edu) and their Metadata, Certificates, login URL, and other parameters are not going to change. They will not accept a SAML Response generated by your Sandbox, so either you stop your test server from sending the Response or you accept that you will get an error message from the SP saying that the login failed to work. The trick is to configure your Browser to trace and capture the SAML Response your test Shibboleth generated, so even if the SP won't accept it,  you can go back and examine the XML yourself to verify that the Subject and Attributes are correct and should be accepted eventually when you run this configuration on a real Shibboleth server.

So if a problem occurs, it is triggered by an internal Shibboleth check that occurs at the very beginning of processing before Shibboleth even fetches the attributes.

You have to provide Shibboleth with the EntityID of a Service Provider. Then

  • The EntityID of the Service Provider has to be matched in one of the Metadata files configured to Shibboleth
  • The Metadata file has to provide a default AssertionConsumerService (ACS) URL to which the Response can be sent.
  • A lookup for the hostname in the ACS URL must return an address.

For example, suppose you supply the EntityID of "google.com". Shibboleth will search the Metadata files and finds in google-metadata.xml a line:

<EntityDescriptor entityID="google.com"

Having matched the EntityID, it now looks at the ACS statement in that Metadata file:

        <AssertionConsumerService index="1" isDefault="true"
            Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
            Location="https://www.google.com/a/yale.edu/acs"/>
        <AssertionConsumerService index="1" isDefault="false"
            Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
            Location="https://www.google.com/a/gdev.yale.edu/acs"/>
        <AssertionConsumerService index="1" isDefault="false"
            Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
            Location="https://www.google.com/a/gtst.yale.edu/acs"/>

There are three ACS locations that turn out to be for the production, dev, and test instances of Eliapps. The one with isDefault="true" is production, so that will be the URL to which the login message is sent. Shibboleth will verify that there is a machine on the network with name "www.google.com".

Of course, Shibboleth has to also look up the EntityID in the attribute-filter.xml file to know what attributes to send, and it has to find the attribute definitions. Those are all steps where your configuration can make mistakes, and errors will show up when the generated SAML Response message does not contain required data or contains the wrong value.

However, if the EntityID is not found in a Metadata file, or the Metadata file does not have a single or default ACS URL, then the request is incomplete and Shibboleth will not even start to process it.

If you actually want to login to the SP, then you need to be running the request on a Shibboleth that the SP is configured to respect. That will be Production or Pre-Production Shibboleth for most SPs, although during early testing with a new SP, it may be configured to use TEST Shibboleth. Now things are a bit more complicated because there are a long list of checks the SP will make on the SAML Response that you send it:

  • The EntityID of the IdP (auth-test or auth) that sent the Response has to match the EntityID configured in the SP.
  • The Digital Signature or the Response has to validate using the Public Certificate that the SP has configured for the Yale Shibboleth Identity Provider.
  • The SP EntityID and AssertionConsumerService URL in the Response XML have to match what the Service Provider believes to be its own EntityID and URL.
  • The first time the SP provides service to Yale users it is typically ready to accept any format of identifier. Therefore, we can initially provide either a simple Netid or a "scoped" value of netid@yale.edu. Once we start sending one format of Subject or Attribute, changing from one release of Shibboleth to another should not change the format of the attribute, or the SP will think that "joe@yale.edu" is a different person than "joe", forget any history and start with a new user blank slate.

Generally speaking, this means that final testing that goes all the way to a real login can only be done with the Pre-Production Shibboleth. However, any Shibboleth including the one on your desktop can generate a SAML Response with the XML names, values, and formats of Subject and Attributes, and if you compare the test and production XML you can see if everything is right.

IdP Initiated Logon

About 95% of the applications that use Shibboleth support "IdP Initiated" logon, where the Browser starts with a URL that points to Shibboleth and Shibboleth sends a SAML Response unsolicited to the Application. The starting URL is of the form:

https://auth.yale.edu/idp/profile/SAML2/Unsolicited/SSO?providerId=nobody.yale.edu

The "providerId" parmeter in this URL corresponds to what every SAML or Shibboleth document or configuration file calls the EntityID of a Service Provider. When this URL is presented to a Shibboleth server, then the handler for the /profile/SAML2/Unsolicited/SSO path searches the configured Metadata providers for an entry describing the named EntityID. From the Metadata it obtains the default AssertionConsumerService, to which the SAML Response will be sent. Then it searches the attribute-filter.xml for a policy that releases attributes to this EntityId.

If the Consent dialog has been enabled on this Shibboleth server, the user first sees a page listing the attributes that will be sent and their values. This is often good enough for initial testing.

Shibboleth generates a Response with the released Attributes and sends it to the ACS URL. If the Shibboleth IdP does not have the real production /credentials/idp.key file installed, the Service Provider will reject the digital signature. However, if you have SAML Trace enabled in your Browser you will have captured the SAML Response and can examine all the generated XML.

The final test, however, is to really login. To do this, you need production credentials, and that only happens when you test against the PREPROD VM in the machine room. So now you change the IdP initiated URL to point to that server, either with a public URL that goes through the F5, or a private URL that goes through the VPN tunnel to the server VM, or through an SSH tunnel to port 443 on the PREPROD VM.

SP Initiated Login

The user can start by going to the application. A Browser that starts with "yale.service-now.com", "yale.box.com", "mail.bulldogs.yale.edu", or the Comcast IPTV URL will in some cases get immediately sent to Shibboleth and in some cases will push a button or select Yale University from a dropdown list before being Redirected back to Shib.

Some applications that support SP Initiated also support IdP Initiated. You can test all the attributes and subject and the basic login using IdP Initiated, but when something changes that might affect it, you also have to be sure that when the Application sends a Request to Shibboleth then that Request is accepted.

There are two ways an application can send its Request to Shibboleth:

  • (SAML Browser POST Protocol) writes a Form to the screen containing a pre-loaded text box containing the SAML Request, then uses JavaScript to Submit the data to https://auth.yale.edu/idp/profile/SAML2/POST/SSO
  • (SAML Browser Redirect Procol) redirects the Brower to "https://auth.yale.edu/idp/profile/SAML2/Redirect/SSO?SAMLRequest=" followed by the character encoded SAML Request

SP Initiated is slightly more complicated because the Application will have been configured with a specific Shibboleth URL which is almost always "https://auth.yale.edu/idp". It will have placed this URL in the Request XML, and it will have Redirected the Browser to that URL. So now you have two problems.

  • You have to trap the "https://auth.yale.edu/idp" in the Browser and send it to the test instance of Shibboleth you are using.
  • You have to convince Shibboleth that is really is the machine "auth.yale.edu" and that the request was received over "https" protocol.

There are many ways to accomplish these two things in testing. They should all be listed because there are a few things you can do on any machine and get 99% of your testing done, but the final end-to-end test of everything requires a PREPROD VM in the machine room and support from Operations.

Network Routing

In all cases, communication to Shibboleth goes through your Browser.

  1. The user clicks on a link sending the Browser to the Shibboleth IdP Initiated URL
  2. The Application sends a Redirect to the Browser, and the Browser does a GET to the Shibboleth URL
  3. The Application writes a form and JavaScript on the Browser submits the form to Shibboleth URL.

So, if you are testing "https://auth.yale.edu" then the only thing you need to do is to convince Firefox running on your computer that "auth.yale.edu" is some specific network location. It can be 127.0.0.1 ("localhost") or 130.132.35.35. The question, then, is how to get your Firefox to translate that name to an IP address of your choosing.

  1. The trivial solution is that in the IdP Initiated login you type the IP address or real hostname of the test Shibboleth server in the URL because you control it.
  2. Another simple solution is to change your /etc/hosts or \windows\system32\drivers\etc\hosts file to map "auth.yale.edu" to some IP address you choose. Just remember to change it back when you are done testing.
  3. You can install a Browser plugin ("Redirector") that matches URL patterns and then rewrites them. It can match "https://auth.yale.edu/idp/*" (note the wildcard at the end) and replace it with "http://localhost:8080/idp/$1" (where $1 is a variable that inserts the rest of the original URL after the matched string).
  4. You can reroute the URL outside the Browser using a Proxy. Charles Web Debugging Proxy is easy to use and setup, while nginx is widely used and very powerful, but requires more reading because there is so much it can do.

Adding an entry in the hosts table is simple, but all it changes is the IP address.

Redirector can be configured to change both the host name and the protocol (replacing "https" with "http") and the port number. This adds a lot of flexibility to the testing, but the URL you generate gets sent on to Shibboleth and that can cause trouble later on where Shibboleth is trying to compare its own network URL with the URL in the Request the application sent, and because Redirect changed one but not the other they do not match. That will be discussed next.

The Proxy is the most complicated configuration, but Proxies exist everywhere and so Shibboleth and Tomcat expect them to be there and have configuration parameters to take care of the problem. In production at Yale there is always the F5 proxy between the Browse and production Shibboleth, so you can certainly get away with adding a proxy between your Browser and a test Shibboleth.

Preparation

Get Firefox. You can can certainly use other Browsers in production, but Firefox is needed to test.

Firefox Add-Ons

SAML messages are typically BIN64 encoded and appear to be the contents of a Text Box in a form or else part of the query string. You can line up a set of tools to trace, cut, paste, decode, and format the XML, or you can install the Firefox SAML Trace add on, which does all of this work for you.

A lot of debugging can be done by using Shibboleth on your desktop (Sandbox) and setting a URL starting with http://localhost:8080/idp/..." in the Browser address bar. However, final testing may require access to the actual application using a normal login sequence. This may require the use of the PREPROD Shibboleth VM, because it has credentials (the signing key) identical to production Shibboleth and can produce a Response that the application will accept. PREPROD may have a public URL address through the F5, or it may have an internal URL that can be accessed from a desktop, or it may require that you SSH login to the host and tunnel port 8080 to your desktop. However that works, you need the Redirector Firefox plugin which watches for a particular URL pattern "https://auth.yale.edu/idp/*" and then substitutes a replacement for the original URL "http://localhost:8080/idp/$1" where $1 is replaced by the rest of the original URL (after the matched prefix).

If you click the Firefox Menu icon (three horizontal lines in the upper right corner of the toolbar) then Add-On is an option (which looks like a puzzle piece). Click it.

Go to the Add-Ons, Search for a new Add-On with the word "SAML". Install the SAML Tracer. Look for "Redirector" and install the Redirector Add On (it has a logo of a capital R ending in an arrow).

Now SAML Trace appears in the same menu, and Redirector installs an icon on the toolbar itself.

SAML Tracer requires no configuration. When you turn it on it traces Web activity in a new Window and will highlight, decode, and display the XML in a SAML Request or Response on demand. You turn it off by closing the Trace Window.

VM Access

If you run Shibboleth under Tomcat on your Sandbox desktop, then it is http://localhost:8080/idp. No setup is required.

If you run Shibboleth on the DEV/TEST/PREPROD VMs in the machine room and they have a public URL through the F5, then you can use that URL.

If they are new VMs not defined to the F5, then the firewall will not allow access from your desktop directly to the VM. First, you need to use the standard Cisco AnyConnect client to establish a VPN session to an area of the network from which SSH traffic is permitted to the VMs. You need to be put in an AD Group to use this type of VPN, and you have to download the Profile files for these special VPN targets. Get help from another developer if this is not set up on your machine or for your Netid. Access to the VPN requires MultiFactor Authentication even though your desktop is on campus.

When the VPN is enabled, you may have access to http port 80 and https port 443 using the native VM hostname (vm-something-01.its.yale.internal). That may be good enough for testing.

In other cases, you may need use an SSH tunnel for ports 8080 or 8443, but it is almost impossible to test or debug anything without SSH access to VMs and log files anyway. SSH requires that Operations create a login for your Netid on the VM and installed your SSH public key. In addition to the terminal session on the VM, the SSH client can be configured to "tunnel" one or more port number from your desktop computer to the VM. In the general case, tunnel ports 8080, 8443, and 443 to the same port numbers on the VM.

Only one program can use port 8080 on your computer at a time. When you test Shibboleth on your local Sandbox, it also uses 8080. Using the same local port number for both the Sandbox and the SSH tunnel will generate an error message if you accidentally run both at the same time. SSH will generate error messages that it cannot create the tunnel if you have forgotten to shut down the Sandbox Tomcat, and Tomcat will generate error messages that it cannot bind to the port if you forget forget to shut down SSH before starting the Sandbox. This is a feature, because you really don't want to spend hours trying to figure out what is wrong only to discover that you are debugging the wrong Shibboleth server.

Browser Point of View

We have already explained that all communication between Shibboleth and the application goes through the Browser. That means that the Browser has to find the Shibboleth server, and it has to find the Application. What is not clear is that some parts of Shibboleth have to be configured from the "Browser" point of view.

Suppose Shibboleth is running on a VM named vm-something-01.its.yale.internal. That name, and the local IP address, are restricted to the machine room. The F5 knows how to get to that hostname, but ordinary desktop computers cannot reach it.

The F5 creates a public virtual hostname with an IP address everyone can use. It is the F5 that has the name "auth.yale.edu" or "auth-test.yale.edu". So in normal processing, all traffic to Shibboleth goes through the F5.

The missing piece here is that nothing configured to Linux or Tomcat on the "vm-something-01" host computer will tell it what name the F5 happens to be presenting to the world. There is no way for the machine to know that everyone else in the world thinks it is "auth-test.yale.edu" unless you configure that name using the install-TEST.properties file in the Jenkins Install. If you look at that file, you might be misled because the first couple of lines read:

idp.entityID= https://auth-test.yale.edu/idp/shibboleth
idp.metadata.file=auth-test.yale.edu-metadata.xml

Yes these lines include the "auth-test.yale.edu" name, but neither of them is configured to anything Shibboleth regards as a URL or network address. The "entityID" is just a string that happens by Yale convention to also be a URL, but it could have been anything. the second the name of a file on disk that happens to start with the public hostname. There is only one property that is actually a network address of the server VM:

cas.target.url=https://auth-test.yale.edu

This is the Yale property that is used to set a parameter of the Unicon CAS-Shibboleth integration that in turn sets the Service string sent to CAS. CAS will Redirect the Browser to the URL in this string, so it must be a URL that the Browser can use to get back to the same Shibboleth VM ending up at the same Tomcat port that received the request that started the CAS login.

It may seem strange that all the Applications in the world have been given a Metadata file that contains URLs that contain public URLs for Shibboleth, but Shibboleth itself has no way to find its own public URL in any configuration parameter. The reason why it can get along without that information is that Shibboleth never does anything except in response to a network request, and when any request comes in over the network it contains an HTTP Host header. A Host header may be originally generated by the Browser, and it may be modified by the F5 on its way to the VM. The Host header contains the protocol, server DNS name, and the port (from the point of view of the Client who sent it). So the Browser generates a header of the form

Host: https://auth.yale.edu

This can flow through the F5 and arrive at the VM and be passed on to Tomcat and to Shibboleth. It tells Shibboleth a subset of the URL that the Browser started with before the F5 changed the protocol, and the VM name, and the port number, to send the request through possibly more intermediaries before it gets to Shibboleth. Sometimes Shibboleth needs to Redirect the Browser to another URL on its own server. It can use the Host header to send back the right network address, so the Browser will come back to Shibboleth again through the same path.

You need to be aware of the "cas.target.url" property and the way that the Host header is used, because during testing the Browser may start with a request to "https://auth.yale.edu" but you may trap it and send it to a special IP address or even a different host name. Neither of these is necessarily a problem, but if as a result you change the Host header then certain specific Shibboleth functions may break.

Typically if you get to the right Shibboleth test server in the first place, then all the Redirects will go back to the right machine. A configuration/test problem typically shows up if you try to change ports or protocol (http or https). So if the original URL was "https://auth.yale.edu/idp" and you try to test against "http://localhost:8080/idp" then problems with configuration or Host header mapping will show up when your Browser generates an error page and the address bar shows "https://auth.yale.edu:8080/idp" or "http://localhost:8080:8080/idp"last thing in the URL, the "providerid" parameter, is the EntityID of the SP. It has to match some configured EntityID in some Metadata file. Then Shibboleth logs you in, generates the SAML Response, and sends it to the configured AssertionConsumerServer (ACS) URL in the Metadata file.

You can use an IdP Initiated Login to generate a response for every SP, unless the SP Metadata demands signed requests:

<SPSSODescriptor AuthnRequestsSigned="true"

For testing purposes, you can set this parameter temporarily to "false", do your testing with the IdP Initiated login URL above, but then deploy the original Metadata to production.

If you do an IdP Initiated login to an SP that doesn't allow them, it will reject the SAML Response and you won't actually login. However, the digital signature was also probably invalid (not from production) and there may be a lot of other defects, and you probably were not planning to go all the way to a successful login at this stage anyway. You can still capture and check the Response for content.

It is not possible to test SP Initiated Login in this kind of preliminary testing configuration. Use IdP Initiated login to verify that the right attributes are being generated, and hold off SP Initiated testing until you get your code to Pre-Production.

Network Routing

When you are using IdP Initiated login, the only network routing problem is to make sure that the URL you use to send a request to the Shibboleth server matches the URL configured in the Shibboleth server for the CAS Service.

The CAS configuration is the "cas.target.url" parameter. When you run Shibboleth in the desktop Sandbox, it is convenient to set this property in the install.properties file you edit in your Install Project. However, if you are building new DEV and TEST server VMs in the machine room, this parameter will have been set by the install-DEV.properties or install-TEST.properties file.

So on the server you want to test, you might have:

Sandbox - cas.target.url=http://localhost:8080
TEST - cas.target.url=https://auth-test.yale.edu

What you need to do is to figure out how to run the test from your client machine using the same URL for the IdP Initiated login that was configured as the cas.target.url. In the Sandbox, all you have to do is use "http://localhost:8080" in both cases and you are done.

If you have a TEST configured VM that you are accessing through an SSH tunnel from your machine to the VM in the machine room, you have a problem. The tunnel makes the machine look like it is "localhost:8080" but the cas.target.url claims that it is https://auth-test.yale.edu.

So this document explains several tricks you can use to fix that.

Preparation

Get Firefox. You can can certainly use other Browsers in production, but Firefox is needed to test.

Firefox Add-Ons

SAML messages are typically BIN64 encoded and appear to be the contents of a Text Box in a form or else part of the query string. You can learn to manually locate and decode these strings, but the SAML Trace addon to Firefox does all the work for you.

If you click the Firefox Menu icon (three horizontal lines in the upper right corner of the toolbar) then Add-On is an option (which looks like a puzzle piece). Click it.

Go to the Add-Ons, Search for a new Add-On with the word "SAML". Install the SAML Tracer. Look for "Redirector" and install the Redirector Add On (it has a logo of a capital R ending in an arrow).

Now SAML Trace appears in the same menu, and Redirector installs an icon on the toolbar itself.

SAML Tracer requires no configuration. When you turn it on it traces Web activity in a new Window and will highlight, decode, and display the XML in a SAML Request or Response on demand. You turn it off by closing the Trace Window.

VM Access

If you run Shibboleth under Tomcat on your Sandbox desktop, then it is http://localhost:8080/idp. No setup is required.

If you run Shibboleth on the DEV/TEST/PREPROD VMs in the machine room and they have a public URL through the F5, then you can use that URL.

If they are new VMs not defined to the F5, then the firewall will not allow access from your desktop directly to the VM. First, you need to use the standard Cisco AnyConnect client to establish a VPN session to an area of the network from which SSH traffic is permitted to the VMs. You need to be put in an AD Group to use this type of VPN, and you have to download the Profile files for these special VPN targets. Get help from another developer if this is not set up on your machine or for your Netid. Access to the VPN requires MultiFactor Authentication even though your desktop is on campus.

When the VPN is enabled, you may have access to http port 80 and https port 443 using the native VM hostname (vm-something-01.its.yale.internal). That may be good enough for testing.

In other cases, you may need use an SSH tunnel for ports 8080 or 8443, but it is almost impossible to test or debug anything without SSH access to VMs and log files anyway. SSH requires that Operations create a login for your Netid on the VM and installed your SSH public key. In addition to the terminal session on the VM, the SSH client can be configured to "tunnel" one or more port number from your desktop computer to the VM. In the general case, tunnel ports 8080, 8443, and 443 to the same port numbers on the VM.

Only one program can use port 8080 on your computer at a time. When you test Shibboleth on your local Sandbox, it also uses 8080. Using the same local port number for both the Sandbox and the SSH tunnel will generate an error message if you accidentally run both at the same time. SSH will generate error messages that it cannot create the tunnel if you have forgotten to shut down the Sandbox Tomcat, and Tomcat will generate error messages that it cannot bind to the port if you forget forget to shut down SSH before starting the Sandbox. This is a feature, because you really don't want to spend hours trying to figure out what is wrong only to discover that you are debugging the wrong Shibboleth server.

Configure Redirector

Redirector runs inside your Browser. Every time the Browser is about to go to a network URL, Redirector inspects it. If the destination URL matches a pattern, Redirector replaces the string with a different string. This is essentially the same as the Find and Replace function of every Text editor. You can match text with either a Wildcard or Regular Expression. Wildcard is simpler and is perfectly adequate for Shibboleth testing.

...