From week to week we add new Metadata files and configure the release of attributes to new applications. Occasionally we defined new Attributes or get the same Attribute from a new database. That is the routine maintenance and it is documented in a different page. It is probably a good idea to read the Shibboleth Configuration page first before reading this page because it is more important, and because I assume you understand a little about Spring Bean configuration already.
What is left are all the other Shibboleth configuration files where I have made one change here and one change there for Yale specific reasons and the only time anyone will be interested in these files is if we upgrade to a new release of Shibboleth or if Yale requirements change. That other less important stuff is documented here.
The configuration rules and syntax of all the files are defined in the Shibboleth Wiki pages. However, while the Shibboleth documentation will tell you what was done, this page will explain why Yale did it and what we expect.
If you upgrade a Shibboleth 2 to a Shibboleth 3 the install process turns on several compatibility switches and then Shibboleth 3 reads and uses the old format configuration files. We did not choose that option. The Shibboleth 3 configuration options have improvements, and it made sense to migrate the files to the new syntax manually to get all the new features.
The CAS Integration
Starting with Shibboleth 3, Yale uses the CAS-Shibboleth integration written by Unicon. This is mostly documented in the Unicon project at https://github.com/Unicon/shib-cas-authn3
Shibboleth has many built in ways to authenticate users (Kerberos, LDAP, ...), but we want it to use our standard Yale SSO solution through CAS. In Shib 2 we used the CAS Client that passes the Netid in the RemoteUser field of the request, and then we select the "RemoteUser" authentication option for Shib. The Unicon code implements the Shibboleth authentication interface and behaves like the other Shibboleth authentication options, but then it redirects the browser to CAS and handles the ticket on return. Potentially this provides additional features, like turning the SAML version of renew=true into the actual CAS renew=true parameter, which you do not get when you use a generic CAS Client that operates as a Servlet Filter unaware of Shibboleth.
Do not get confused with the internal Shibboleth implementation of the CAS protocol. Shibboleth will replace CAS, but we at Yale do not use that option. At this time the CAS protocol support has not been removed from Shibboleth and might even be functional. If it is there we don't use it and it doesn't hurt anything.
Although the link above takes you to a Github location where you could download the Unicon source, we simply download the JAR and Shibboleth configuration files. We do not recompile the source, although you are free to do so.
There are two ways you can manage the integration using Yale project management rules. You could put the Unicon jar files into the Shibboleth artifact, which would probably be the right thing to do since they are code and not configuration text files. Currently, however, we use the second option and check them into Source Control in the resources/updateArtifact/webapps/WEB-INF/lib directory of the Install project even through the jar files are not "source". This is simpler. If we made any significant changes to the Artifact and generate a Build job, then we would move the JAR files.
Any CAS integration has to provide two parameters. You need the URL of CAS itself, and you need a URL or host name for the service string you send to CAS.
You might thing you can get the service= string from the Hostname. The problem is that Yale has an F5 front end that owns the "auth.yale.edu" DNS name which is the public name used to access Shibboleth and has to be the value in the service= string, while Hostname on the Shibboleth VM is going to return "vm-shibprd-01.yale.internal" which is not the right value.
The two CAS parameters are different for DEV, TEST, and PROD because there are three CAS servers that might be configured (secure, secure-dev, and auth-dev). In the install-XXX.properties file you will find:
# The CAS Server and the service= we pass to it. cas.server.url=https://auth-dev.yale.edu cas.target.url=https://auth-dev.yale.edu ###### TEMPORARY VALUE used for SANDBOX or SSH TUNNEL to Shib #cas.target.url=http://localhost
To understand these parameters you need to understand that the names are from the Browser's point of view, not from the point of view of either the CAS VM or the Shib VM. These URLs will be sent to the Browser with an HTTP Redirect and then the Browser will find CAS or Shib itself. In a Sandbox where you are running Shib and maybe CAS on your own desktop under Tomcat, you may be using http instead of https (because it is simpler for testing purposes) and you may be running on port 8080 instead of the standard 80 or 443. More importantly, you can configure your /etc/hosts or \windows\system32\drivers\etc\hosts file to define the hostname as the local machine. Even when using DEV or TEST, you may run an SSH tunnel to the VM, and therefore access the Shib server as if it were localhost:8080.
The cas.server.url parameter is the URL of the CAS server, and the cas.target.url is the Shibboleth server. In DEV they happen to be the same DNS name because it just happens that CAS is https://auth-dev.yale.edu/cas and DEV Shib is https://auth-dev.yale.edu/idp (although they are on different VMs behind the same F5 VIP).
The cas.server.url may need a port number, but you don't add port number to the cas.target.url. I don't know why.
Credentials
The shibboleth-idp/credentials directory of the running Shib server contains files for the private key used to digitally sign SAML messages and the Certificate that contains the public key that partners use to validate the signature. In the Shibboleth/SAML world, certificates like this are always self-signed and are simply containers for the public key. They are distributed through Metadata and there is no trust chain or Certificate Authority involved.
There are three separate pairs of files, one in each environment (DEV, TEST, and PROD). Typically a given partner only trusts one environment, and if the partner installs the PROD certificate and entityID for the PROD Yale IdP, then it will expect SAML digitally signed with the PROD private key file, which is in the shibboleth-idp/credentials subdirectory of the PROD VM.
Shibboleth as distributed generates three pairs of keys (signing, encryption, and backchannel). However, Yale encrypts no Attribute assertions and in the very unlikely case that we did we would just reuse the signing key. So for now we carry forward the Shibboleth 2 "credentials" files and populate this directory with only an "idp.key" and an "idp.crt" file. These are the same files from Shibboleth 2, since any change in the keys would require reconfiguring all our partners and our InCommon data.
The name of the files to use for signing and encryption are specified in the conf/idp.properties file. We changed this configuration file from the original distribution (with separate signing and encryption keys) to use just the one pair of keys and, therefore, pair of files.
InCommon itself provides us with a credential that we can use to validate the digital signature on their Metadata file. The "incommon.pem" file is public and is checked into Source Control and is added by the Install Project.
Shibboleth 3 adds a new key and pair of files that are used to encrypt the Cookie used to provide Client-side storage of session data. This is held in the sealer.jks and sealer.kver files. Periodically a chron job generates a new key pair which is added to the "sealer" files, because when a new key is generated the old key remains valid until session timeout guarantees there is no valid data still sealed with the old key.
Generic Spring Bean Parameters
The Yale Install Project stores parameters in the install.properties file (created from the Jenkins Install job parameters), plus the install-ALL.properties and the install-XXX.properties (where XXX is DEV, TEST, or PROD). These properties are loaded into the Maven environment and then the Ant environment of the Install Project and are available only during the execution of the Install job itself. As a result, any configuration file stored in Source Control as part of the resources/updateArtifactSubstituteVariables subdirectory of the Install Project can contain ${property.name} placeholders that will be replaced by the value of a corresponding property as the Ant script in the Install Project copies configuration files into the /apps/shibboleth-idp directory. This produces configuration files that have literal numbers and strings as values.
Shibblleth 3 fully embraces the Spring Framework, which has its own "variable substitution from parameter file" capability called Spring Execution Language. At startup, Spring replaces %{property.name} placeholders in its XML files with the corresponding values from the shibboleth-idp/conf/idp.properties file (and other *.properties files in the conf subdirectory).
Of course, Spring will also use literal values. So we have a choice to do the property substitution at Ant install time or at Spring initialization time. Both should work. When we decide to use the Spring way of doing business, the problem is to get a property value defined in one of the Install Job property files transferred to the idp.properties file used by the Spring runtime. To do this the idp.properties file is itself in the Install Project as resources/updateArtifactSubstituteVariables/conf/idp.properties, and because it is in the SubstituteVariable directory at Install time an ${foo} expressions get replaced. So in Source Control Yale adds lines to that file of the form:
property.name=${property.name}
Then as this file is copied from the SubstituteVariables section of the Install Project to the /apps/shibboleth-idp/conf directory, the "${property.name}" is replaced by a string or number from the Install job properties files. Strictly speaking, it is not necessary for the Spring property name (on the left of the =) to be the same as the Install Project property name (on the right side) but it is simpler to remember when they are. This means that you have to be sensitive to the fact that %{foo} expressions in the resources/updateArtifact directory (which are not replaced by properties) are copied unchanged to shibboleth-idp and are then replaced logically at Spring startup from the values in idp.properties that are replaced during Install.
When in doubt, look at the files after running Install in a Sandbox and compare them to the Install project to see which change and which remain the same.
DataSources
In Shibboleth 2 a Datasource (a database connection pool configuration specifying the database url, jdbc class name, userid, and password) could be configured in a DataConnector element of the attribute-resolver.xml or it could be supplied by the container (the Web Server which is to say Tomcat or JBoss). There was, however, a scary comment in the Wiki that if you use Oracle as your database (which we did) then the Shibboleth provided DataConnector implementation did not work well and they recommended using a container datasource. So that is what we did.
When we migrated to Shibboleth 3 we have initially maintained this convention. Therefore, datasources are configured in the context.xml file that is copied (with variable substitution) to the /apps/tomcat/current/conf/catalina/localhost/idp.xml file by the Install Project.
However, because Shibboleth 3 is not fully a Spring Framework project, there is an alternative. The datasource can be configured as a Spring Bean. Spring has serious industrial strength support for DataSources that is as good or better than JBoss or Tomcat. Moving the datasources to a Spring Bean configuration is planned for a future (but relatively soon) release.
Logging
Logging in Shibboleth 3 is controlled by the conf/logback.xml file. Initially it is treated as a normal configuration file. However, we may want to create a special Jenkins Install runtype to change the logging levels to get more information from a running server, provided that the logging levels can change dynamically.
Services
The conf/services.xml file configures the services and subsystems. In particular, it configures the names and locations of the XML files that configure the other parts of Shibboleth.
It is important enough that Yale removed the commented out sections that are used by services we do not enable so the file is clean.
Yale made on aesthetic change. Normally there is one attribute-resolver.xml file containing both DataConnectors (queries) and AttributeDefinitons. We broke the file into two separate tiles because the two types of attribute configurations are different and because we can assign a different xmlns default to the different types of XML elements if they are in separate files.
Spring <bean> elements creating Yale configured Spring Bean objects have to go somewhere. The two obvious places to put them are global.xml and services.xml. We put them at the end of the services.xml file. The current use is to create "predicates" that return true or false based on whether the application the user is logging into is in a list of EntityIDs.
Relying Party
In Shibboleth 2 the relying-party.xml file also included MetadataProviders. In Shib 3 there is a separate file for metadata-providers.
Our Shibboleth 3 relying-party.xml file configures the same defaults that were configured with a different XML syntax in our Shib 2 configuration file. There is one really important statement:
<bean parent="SAML2.SSO" p:postAuthenticationFlows="%{idp.postauthenticationflows}"
p:encryptAssertions="false" p:encryptNameIDs="false" p:encryptAttributes="false" />
Our exclusive use of Shibboleth is Single SignOn (SSO), so this is pretty much the only statement that gets used. The parameters tell Shibboleth that by default (unless changed by Metadata) we don't encrypt the data in the SAML statements. That follows common practice where vendors use SSL/TLS to achieve privacy and encrypting attributes makes debugging problems impossible.
The postAuthenticationFlows parameter currently has one possible interesting value. If it is set to "attribute-release" then this triggers the Consent dialog, where the user is given a list of attributes that are gooing to be released and has a chance to approve them. Currently we turn on Consent for DEV (because it makes debugging easier) and for Anonymous Relying Parties (where we get a SAML Request from an EntityID for which we have no Metadata).
In the near future:
- We may want to turn off Anonymous Relying Parties. We need to confirm that Metadata has been generated for every party we currently support in production. Scanning Shib 3 logs will give us a chance to look for any remaining legitimate Anonymous RPs and configure them properly. Currently we turn Consent on for Anonymous.
Metadata Providers
The metadata-providers.xml file contains almost exactly the same content as Shibboleth 2. The only change is that Refresh periods have to be expressed as "PT5M" (which means check every 5 minutes) where as in Shibboleth 2 the same thing could be expressed as an integer number of milliseconds.
Access Control (to Status Page)
Currently the only form of Access Control is by IP address. We modified the conf/access-control.xml file to allow IP address ranges to be inserted from a property file value
p:allowedRanges="#{ {'127.0.0.1/32', '::1/128', %{allow.status}} }"
So if the properties files have a line of the form:
allow.status='172.28.187.0/24','172.17.40.0/24'
then a subnet in the DevOps area of ITS and a subnet in the SPS_DC VPN are able to display the status page. This will be expanded on request.
Logo and Footer
In a few messages we replace a dummy logo with a Yale ITS logo. The image is in webapp/images/YaleITS.png and it is used because of some lines in messages/error-messages.properties:
idp.logo = /images/YaleITS.png
idp.logo.alt-text = YaleITS
idp.message = An unidentified error occurred.
idp.footer = Yale University Shibboleth Identity Provider