General Shibboleth Configuration

Spring Framework

In the old days every application programmer had to define a configuration syntax and write code to parse that syntax and create the objects that perform operations. Earlier releases of Shibboleth worked this way. Shibboleth 3 (and CAS for that matter) are Spring Framework applications. Spring provides a general purpose configuration file system that can be used by any application. XML files define "beans", which are object instances of classes provided by the application. These objects are parameterized by values from the XML file, and they are linked together by plugging a reference to one bean into a property of another bean. For more information, go to the Spring Framework website.

...

Through Spring, the Shibboleth application is assembled by selecting optional components, configuring them with parameters, and plugging them into the application framework. For example, Yale used CAS for its SSO, and Shibboleth has a framework for configuring and adding beans that perform authentication. So we use the Unicon supplied integration between CAS and Shibboleth distributed and documented at https://github.com/Unicon/shib-cas-authn3. Remember, Shibboleth was now written to expect that Unicon would add a new component, but because it is configured in Spring as long as Shibboleth provides an Interface on which new classes can be defined, then anyone can write code and plug it into the framework.

Local Disk File Configuration

Even before it used Spring, Shibboleth allowed the non-Spring XML files to come from local hard disk, or from a network URL, or from a Subversion Source Control system. If Shibboleth were a normal application then dynamically obtaining configuration from a Source Control system would be very attractive. Unfortunately, in order to recover from major datacenter failures (as happened when the power was cut to half the servers on campus) Yale stores some of its recovery plans and checklists on cloud services. To login to cloud providers, you need Shibboleth. So Shib is a "Tier 0" application that has to come up before any database, Web, or Source Control servers. That means that everything Shib needs to come up has to be on local hard disk.

So we create a slightly more complicated system. Ultimately, every Shibboleth configuration file is managed by Source Control. Specifically, these files are checked into the yale-shibboleth?-installer project in Source Control (where ? may be replaced with a version reference). However, instead of Shibboleth linking to Source Control at startup or polling Source Control while it is running, a Jenkins Install job is run by operations to check out the current version of all the files from Source Control and then update the configuration of Shibboleth on a running (or temporarily stopped) Shibboleth server. Shibboleth itself is configured to use files on local disk, but Jenkins controls when these files are replaced.

Metadata

The files in the "conf" directory are defined by Shibboleth and are supplied by Yale. One file is metadata-providers.xml. It contains a list of elements that define files or URLs that supply the metadata information defining the applications that support Shibboleth login. While Yale has been forced to create a few metadata files, normally they are supplied by the application vendor.

...

Two of our partners (Salesforce and Cvent) regularly add new AssertionConsumerService URL elements to their existing Metadata file. This happens so frequently that we have the option of replacing these specific production Metadata files with updated copies. There has not yet been any urgency to make such changes outside a normal Release cycle, but we have the ability to respond to the special needs of these two cloud partners if "every other week" becomes an unacceptable delay.

Jenkins Runtype

The runtype parameter in the Jenkins Install job determines the specific processing that this run of the Install job will perform.

...

Runtype "salesforce" and "cvent" are proposed runtypes that change a single Metadata file for the two partners that require frequent updates.

Contents of the Primary Configuration Files

Attribute-Resolver

Normally Shibboleth has a single attribute-resolver.xml file that contains two types of elements. DataConnectors define database or LDAP queries that produce result sets with columns or LDAP User objects with properties. AttributeDefinitions then take the columns and properties returned by the queries, assign a unique identifier that can be referenced in the attribute-filter (release policy), and supply SAML syntax. So two DataConnectors could query the Yale IDR database for basic identity information, and also the Active Directory for the subset of identity information it contains. Then AttributeDefinition statements can take the "FirstName" column from IDR or the "givenName" property from AD and create various SAML Attributes all with the same value of "Howard" but with SAML name and friendlyName attributes that refer to it as "FirstName", "First Name", "givenName", "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname", or "urn:oid:2.5.4.42" (informal and formal standards based SAML names for the same thing).

...

Shibboleth documentation is not particularly clear on the algorithm, so I will try to fill in something that I believe is important to understand.

DataConnectors

There are generally three types of queries that make sense:

...

The DataConnector needs a Java DataSource to provide a pool of database connections. Java DataSource management is complicated because it has to know when a database connection must be discarded because it has timed out or because the database rebooted since it was last used. Shibboleth would prefer to leave this to the database experts. Shibboleth 2 did this by using "container managed" connections provided by Tomcat or JBoss. Shibboleth 3 can still use connections managed by the application server, but now that it is a full Spring Framework application it can use DataSources provided and managed by Spring. Either way the complex database management doesn't have to be done by Shib provided code.

AttributeDefinitions

The DataConnector provides the value. We know that "Howard" is the value of the "FirstName" column of the result set returned by the "IDRQuery" database connector or of the "givenName" property of the user object returned by the "ADQuery" LDAP connector.

...

XML and Shibboleth are case sensitive, so it is important to realize that Oracle always converts its columns to UPPERCASE. To avoid errors you should always use UPPERCASE names for the sourceAttributeID field if the query is to an Oracle database, and you should define an UPPERCASE id for a default static value in the fallback connector if the Oracle query fails. Otherwise you may spend hours trying debugging the failure of the value to show up where you expect it to be.

Undefined, Null, or Empty

When you query a database or an LDAP directory and then try to define an Attribute based on the value of a column or property, several things can go wrong:

...

                if (typeof googleEmail!="undefined" && 
                    googleEmail!=null &&
                    googleEmail.getValues().size()>0) {
                        googlemailalias = googleEmail.getValues().get(0);
                }

Suppose the Database is Down

If any exception is thrown during the query, then the Shibboleth code will attempt to execute a secondary query specified in the "failover" attribute of the DataConnector. The failover can point to a different query to a different database that might return the same value. Or it can be a Static element.

...

See the examples in the attribute-resolver-connectors.xml file.

Other Errors

In vanilla Shibboleth, the only errors that are caught during Attribute evaluation are query errors. Any other error (an exception thrown but not caught in a JavaScript block of code, or a null value in an attribute passed to a nameid when Yale changes the rules and give login privileges to people who have unexpectedly less connection to the university and therefore lack even basic identity variables) is fatal and prevents the user from logging on to any partner. In Shibboleth 2 Yale has added code to wrap other evaluations with a try-catch that discards attributes in error but preserves all the other attributes. Because attribute errors tend to occur only in new attributes not widely in use or old abandoned attributes no longer in use, this makes Shibboleth more robust against real world errors without impacting security. We may or may not be able to add this change to Shibboleth 3 where the underlying code has changed. Besides, we have been burned by most of the errors and maybe have learned to avoid them.

Attribute-Filter

The attribute-filter.xml file has a long list of rules listing the Attributes (defined in the previous section) that are released to each partner. For example

...

However, you can only release attributes defined by attribute-resolver.xml, and that does not change between releases.

Relying-Party

The relying-party.xml file has three types of definitions:

...

Specific Relying Party configurations could force encryption if we needed to do it, but we have no examples currently at Yale.

More About Metadata

SAML Metadata can have a ton of useless information. There are four things that are actually important:

...