General Shibboleth Configuration

...

Spring Framework

In the old days every application programmer had to define a configuration syntax and write code to parse that syntax and create the objects that perform operations. Earlier releases of Shibboleth worked this way. Shibboleth 3 (and CAS for that matter) are Spring Framework applications. Spring provides a generic general purpose configuration syntax file system that can be used by almost any application. It is not reasonable here to try and describe Spring configuration syntax in general. Go to the Spring website for that information.

Every real Spring file contains a <beans> element. In that file there are a sequence of either <bean> elements or else aliases for a particular kind of standard bean. For example, a List bean based on a Java list of objects is a <util:list> element which is translated into into a particular type of <bean>. Every bean has an id= name by which it is referenced.

Each <bean> is an instance of a Java class, and it is configured by providing values for named properties and occasionally a set of constructor arguments. When many beans will all be created with many common arguments, then an incomplete (abstract) <bean> definition is provided with the common arguments, and then other <bean> elements complete the configuration by referencing the partially configured bean with parent= and then add the missing properties to complete the configuration.

Shibboleth has to work with standard SAML configuration files, and they are ordinary XML files with no Spring elements. In general, Spring calls a text, properties, or xml file a "Resource" and the resource can be defined by a path to a disk file, or a URL to a file on a Web server. The Shibboleth configuration files that are standard XML files are a Resource that has to be read and understood by some Bean, which is different from a Spring XML file that contains a <beans> element.

In the Shibboleth conf subdirectory, the attribute-filter.xml and attribute-resolver.xml files are ordinary XML resources while the other files (services.xml, relying-party.xml, metadata-providers.xml) are Spring beans files. Of course, internally the elements in the non-Spring XML files are turned into objects, but that is done by Shibboleth processing and not just standard Spring.

Most Shibboleth configuration files can be sourced from local disk, from a Web server, or from a Subversion repository. However, Shibboleth is a "Tier 0" application that has to come up after any type of power or datacenter failure before anything else comes up, so it cannot depend on external servers. We solve this problem with a Jenkins Install job that checks files out from Subversion but then manually deposits them on the Shibboleth server in the conf subdirectory. We manage the configuration files with Source Control, but they have to be manually transferred to local disk under operations control. Once stored they are available locally when the Shib server boots.

The files in the conf directory are defined by Shibboleth and Yale. There is a separate metadata directory that defines our individual Relying Parties (google.com, yale.service-now.com, salesforce.com, etc.). However, in order for a metadata file to be recognized at all, there has to be an entry pointing to it in the metadata-providers.xml file.

The metadata-provides file defines sources of metadata. Each source is read into memory when Shibboleth starts up, and based on the configuration of the element in metadata-providers it can be "polled" in regular time intervals to see if a new copy of the file is available.

The InCommon federation curates submitted Metadata for hundreds of potential partners. We define a metadata provider for InCommon that reads the latest version of InCommon metadata from the URL they provide. This is cached to a local copy of the file on disk so that after a datacenter failure Shibboleth can come up using the cached copy of the InCommon metadata in the event that network connectivity to the InCommon server has not be restored. There is special logic in the Install job to not delete the cached InCommon metadata file when all the other metadata files are being deleted and replaced.

However, InCommon is the only dynamic metadata file, and we only trust InCommon as a dynamically updated source because the contents are curated. For the other metadata, particularly for critical business functions, each metadata file is managed as a local disk file updated when necessary by the Jenkins Install process.

Shibboleth processes a metadata-provider as a unit. If it is a file on disk, then the file is read and a single XML syntax prevents Shibboleth from using anything in the file. A metadata file can contain a single EntityDescriptor or a list of them. For safety, we maintain separate files and therefore separate metadata-provider definitions for each cloud partner or application. That way we know that a change given to us by Community Force cannot interfere with our configuration of Comcast IPTV, because they are in separate files and a syntax error in either is localized to that one file.

Shibboleth starts with the EntityID of the partner. For example, one of the Comcast application entities is entityID="https://university-dev.ccp.xcal.tv:443/openam". Shib then goes to the list of metadata providers in the xml file and searches the in memory list of elements constructed from that metadata provider for a matching
<EntityDescriptor entityID="https://university-dev.ccp.xcal.tv:443/openam" ...>
It stops when it finds the first matching EntityDescriptor in any metadata provider. Therefore, if the same entityID is in more than one metadata provider source, Shibboleth will use the source that comes first in the list of metadata providers. We use this to provide a maintenance strategy based on a few local-file metadata provider elements that are configured to regularly check for file changes:

The "emergency-override" dynamic file comes first in the search. Metadata XML files define "beans", which are object instances of classes provided by the application. These objects are parameterized by values from the XML file, and they are linked together by plugging a reference to one bean into a property of another bean. For more information, go to the Spring Framework website.

However, Shibboleth 2 already had a configuration syntax where XML files were parsed to create objects, and SAML has standardized XML configuration files that have nothing to do with Spring. So Shibboleth 3 is a hybrid system where some configuration is done with Spring beans and some use SAML or Shib 2 syntax. You can tell the difference by looking at the first XML element. If it is a <beans> element, then this is a Spring bean configuration file in the universal Spring syntax.

In general, each Spring created object is defined by a <bean> element inside the <beans> container. Each <bean> is given the name of the Java class that creates the right type of object, and it has a unique id= by which this bean can be located or referenced by other code. Spring contains some utility classes that make common generic objects easier to define with a specialized element. For example, a Java List object can be created with the Spring <util:list> element which is translated internally to a bean.

Each bean is configured with parameters that can be provided by adding p:parametername="value" elements to the <bean> element or by adding property or constructor elements to the content of the <bean> element.

Through Spring, the Shibboleth application is assembled by selecting optional components, configuring them with parameters, and plugging them into the application framework. For example, Yale used CAS for its SSO, and Shibboleth has a framework for configuring and adding beans that perform authentication. So we use the Unicon supplied integration between CAS and Shibboleth distributed and documented at https://github.com/Unicon/shib-cas-authn3. Remember, Shibboleth was now written to expect that Unicon would add a new component, but because it is configured in Spring as long as Shibboleth provides an Interface on which new classes can be defined, then anyone can write code and plug it into the framework.

Local Disk File Configuration

Even before it used Spring, Shibboleth allowed the non-Spring XML files to come from local hard disk, or from a network URL, or from a Subversion Source Control system. If Shibboleth were a normal application then dynamically obtaining configuration from a Source Control system would be very attractive. Unfortunately, in order to recover from major datacenter failures (as happened when the power was cut to half the servers on campus) Yale stores some of its recovery plans and checklists on cloud services. To login to cloud providers, you need Shibboleth. So Shib is a "Tier 0" application that has to come up before any database, Web, or Source Control servers. That means that everything Shib needs to come up has to be on local hard disk.

So we create a slightly more complicated system. Ultimately, every Shibboleth configuration file is managed by Source Control. Specifically, these files are checked into the yale-shibboleth?-installer project in Source Control (where ? may be replaced with a version reference). However, instead of Shibboleth linking to Source Control at startup or polling Source Control while it is running, a Jenkins Install job is run by operations to check out the current version of all the files from Source Control and then update the configuration of Shibboleth on a running (or temporarily stopped) Shibboleth server. Shibboleth itself is configured to use files on local disk, but Jenkins controls when these files are replaced.

Metadata

The files in the "conf" directory are defined by Shibboleth and are supplied by Yale. One file is metadata-providers.xml. It contains a list of elements that define files or URLs that supply the metadata information defining the applications that support Shibboleth login. While Yale has been forced to create a few metadata files, normally they are supplied by the application vendor.

Each metadata provider element in the metadata-providers file points to a file name in the "metadata" subdirectory of Shib. Optionally it can also point to a URL that Shib can check at configured intervals to look for updates. Yale only uses the URL update facility for the curated InCommon federation aggregated metadata, and we put that metadata source near the end of the list. Because Shibboleth scans all the metadata provider elements in the order they are defined, and it stops when it finds metadata for the entity name it is looking for, Yale configures all the individual metadata files we store on local disk to come first in the search and then configures InCommon. That way if we have a specific metadata file for a partner that is also defined in InCommon, the file we created and store on local disk will be found first and our parameters will be used to talk to that partner.

A metadata provider file can define one metadata for one entity, or it can contain as many entities as you want. Yale could have combined all its local disk metadata into one file with one metadata provider element in the metadata-providers file. That seems simpler, but there is a problem. If an XML file has a syntax error, then the entire file is ignored. So if we combine all our metadata in one big file, then a single missing "/" makes the entire file unreadable. It seems safer even if it makes the configuration file more complicated to separate each metadata configuration for each partner in a separate file, so mistakes are localized to just the one partner with the problem.

We define a number of metadata providers that initially point to empty files on disk. We take advantage of Shibboleth's ability to reload local disk files when they change, and the convention that metadata is taken from the first file that defines a particular entityid. We use this to address a problem with the unreasonably change-adverse IT administration at Yale.

The "emergency-override" dynamic file comes first in the search. Metadata placed in this file with an EntityID that matches an existing Metadata entry in a later file will logically replace the previous version of production Metadata for any partner. After each new Shibboleth Release, When we have a regularly scheduled formal Release of new Shibboleth configuration (on alternate Thursdays) this file is empty.   Metadata can be placed in it when we have an incident because an existing partner metadata has failed (typically because it has expired or the Certificate and key used by the partner has changed unexpectedly). This provides a safer form of "emergency" fix because only the one Metadata element is logically replaced instead of the entire Shibboleth configuration.The "additions" dynamic file comes last in the search. Every existing Metadata file will have already been searched, and all During the two week period, or when it is too later to schedule a regular update through the CAB committee, a runtype=emergency Jenkins Install of Shibboleth modifies just this one file. So if one partner has a problem (typically because a key/certificate changed and we did not know about it in advance) we can go to the Emergency CAB and get approval to put the updated metadata in the emergency-override file, change just that one file on the disk of the running Shib, and fix the problem with that one metadata file. In the next alternate-Thursday full release the changed metadata will be in its normal file and this file will be empty again.

The "additions" dynamic file comes last in the search. Every existing Metadata file will have already been searched, and all existing EntityID values will have matched, so you do not get to this file unless you have a new EntityID that doesn't match any existing on one (including all the InCommon entities). This file can only define new Meatadata Metadata for new entities. This becomes a relatively safe Standard Change because anything that doesn't ahve to be approved because anything put into this file cannot adversely affect existing configured services. A Of course, a new partner may need more than just Metadata. They may also need attributes released to them. Fortunately, Shibboleth allows the function of the attribute-filter.xml file to be broken up into multiple files. Existing partners are configured in attribute-filter, and an empty file named "additional-attribute-filter.xml" is deployed with every Shibboleth Release. Therefore, if a new partner has to be defined to production and cannot wait for the every-other-Thursday Release cycle, the Metadata for that partner can be placed in the metadata/additions.xml file and the attributes to be released can be put in the additional-attribute-filter.xml file. A Jenkins install of runtype=additions replaces both of these originally empty files with the data for the newly defined partner while guaranteeing by their search order that they cannot interfere with existing services. When the next regularly scheduled Shibboleth Release is ready, the changes move from the additions files to the normal Shibboleth configuration and the additions files are empty again.

...

Runtype "config" does not stop JBoss or the running Shibboleth server. Instead, it replaces the full set of configuration files. The running Shibboleth process checks the timestamps on these files, and when it sees they have changed it loads a complete new configuration (although in practice only one or two configuration files will actually have new contents). The 3/15/2015 update to Shibboleth added three new categories of runtype:Shibboleth completely reconfigures itself.

Runtype "additionsupdate" modifies only the "additions.xml" Metadata file and the "additional-attribute-filter" file. This can be used to add new Service Providers to production between the every-other-Thursday full Release cycles. Shibboleth isolates these files and appears to guarantee that this type of configuration cannot possibly interfere with existing production services.

Runtype "emergency" will change the "emergency-override.xml" file and allows us to define new Metadata for an existing production partner without affecting anything else. It may require permission from the ECAB, but it is a less dangerous change than a full configuration update. Note that the old Metadata for the partner remains in place, but is not used because the override Metadata is found first in the search order. Before the next Release cycle (the next runtype=install or runtype=config), the old production Metadata should be replaced with the new override data and the emergency-override.xml should be emptied.

Runtype "salesforce" and "cvent" are proposed runtypes that change a single Metadata file for the two partners that require frequent updates.

Contents of the Primary Configuration Files

Attribute-Resolver

Shibboleth normally has a single attribute-resolver.xml file, but Yale changed this.

The attribute-provider file has two types of elements. DataConnectors define database or LDAP queries that produce result sets with columns or LDAP User objects with properties. AttributeDefinitions then embed a value of one column or property returned by one of these queries in a SAML attribute statement with a name, friendlyName, and format.

Normally there is one file with the two different types of elements, but that produces more complicated XML because everything has to be decorated xml namespace prefixes. It just seems cleaner to use the capability provided in Shibboleth 3 to create two files, where attribute-resolver-connectors has the DataConnectors and attribute-resolver-definitions has all the AttributeDefinitions. If you don't like this, you can combine the two back together again and then add back all the xml namespace prefixes that can be defaulted if the files are separate.

DataConnectors

There are generally three types of queries that make sense:

• A database query can return exactly one row. Then you can think of the row as a user object, and the column names become the properties of the object.
• A database query can return more than one row but only one column. Then you have a "multivalued" property for one user.
• An LDAP query returns the User object from the directory. LDAP User objects have properties some of which are single valued and some of which are multivalued.

Each column in the result set or property in the LDAP object becomes an Attribute of sorts, but this attribute has no SAML formatting information and no single identifier. To identify it you must both reference the id= of the DataConnector and the column or property name. To change that, you must use an AttributeDefinition.

Building a working pool of database connections is a complicated process. Shibboleth 2 generally relied on the application server (Tomcat or JBoss) capability to create and manage DataSource objects. Shibboleth 3 can also use datasources provided by the server, but it can also use datasources created by Spring support for databases. At this time we are sticking with Tomcat managed datasources, but we may move to Spring when we have a chance.

AttributeDefinitions

A query creates an attribute object for each database result set column or each property of the LDAP user object returned. These attribute objects are not directly useful because they are attached to the object created to represent the query result. You use them to create derived attribute objects that can stand on their own. To do this you configure an AttributeDefinition element with a sourceAttributeID that names the column/property and a Dependency that references the id of the query.

<resolver:AttributeDefinition id="idrFirstName" xsi:type="ad:Simple" sourceAttributeID="FirstName">
    <resolver:Dependency ref="IDRQuery" />
</resolver:AttributeDefinition>

This element creates a new attribute with an id of "idrFirstName" that is the value of the FirstName column of single row returned by the IDRQuery database query (defined elsewhere in the file). This converts the essentially unnamed attribute generated for that column in the query into an independent attribute with its own name.

In most cases an AttributeDefinition will also have one or more AttributeEncoder elements that tell Shibboleth how to produce SAML representing this attribute when it is sent to a Service Provider. Typically the Encoder element specifies a friendlyName= like "FirstName" or "GivenName" and an (unfriendly) name= like "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname" or "urn:oid:2.5.4.42". An attribute that has an Encoder can be released by the attribute-filter.xml file and will then appear the a SAML Response.

However, in a few cases the AttributeDefinition will not have a Encoder. This attribute cannot be released and cannot generate SAML on its own, but it can be used to create a second attribute. This is most useful when several attributes have to be combined to produce a single new attribute. Attributes can be used to generate variables in a JavaScript generated attributes, or they can define variables inserted into a SQL statement in a secondary query, or they can be combined to form a composite text string in a Template attributes. JavaScript seems to be the easiest case to explain.

An AttributeDefinition can have any number of Dependency statements, but only one can reference a DataConnector (that is a Query) because there is only the one sourceAttributeID. That sourceAttributeID selects a column or property in the single query. In a Script definition, the sourceAttributeID also names the JavaScript pre-defined variable passed to the JavaScript that holds the value of this column or property. However, the script may require the value of other columns from the same or other queries, and the only way to get them is to create named (id=name) attributes with another AttributeDefinition. All the other Dependency statements (that don't reference a DataConnector, reference the id of another AttributeDefinition. The JavaScript block gets a variable with that id name and the value of that attribute.

An AttributeDefinition referenced by the Dependency statement in a JavaScript block can also have AttributeEncoder statements and be released as SAML in a Response. It can do both jobs simultaneously. An AttributeDefinition that doesn't have an Encoder can only be referenced by a Dependency statement in the definition of another attribute.

XML and Shibboleth are case sensitive, so it is important to realize that Oracle always converts its columns to UPPERCASE. To avoid errors you should always use UPPERCASE names for the sourceAttributeID field if the query is to an Oracle database, and you should define an UPPERCASE id for a default static value in the fallback connector if the Oracle query fails. Otherwise you may spend hours trying debugging the failure of the value to show up where you expect it to be.

When designing this file, you need to consider certain possibilities:

• The database query can return no rows.
• The database query can return a SQL NULL value for a column (unless you use NVL in Oracle or ISNULL in SQL Server to replace the NULL with a default value).
• An LDAP query can return no User object
• An LDAP query can return a User object, but in that object the property you are looking for may not be present.
• An LDAP query can return a User object, and the property may be present, but it may have no values in the list of values.

First, a Subject cannot be NULL. Therefore, you cannot define a Subject (a NameID) on any column or property that can be NULL or missing for anyone (student, faculty, staff, alumnus, contractor, ...).

Then you have to write JavaScript code very carefully.

A JavaScript variable is "undefined" if the variable name has never been used and is not in the name table. This can occur indirectly if an LDAP query returns a user object that does not have the property whose name you specified in the sourceAttributeID (although this may vary from version to version of Shibboleth and Java). There is a special way to check JavaScript variables to see if they are undefined and it is not "if (name==null)" because that throws an exception if "name" is undefined.

A JavaScript variable can be null. However, when the JavaScript variable is associated with an attribute then it points to an Attribute object. The object contains an array of values (because any Attribute could be multi-valued). So it is also possible that the array of values is empty. Each of these possibilities is a different test with different syntax, and you have to do all of them before you can just use a JavaScript attribute and assume it has a value.

Because it is easy to screw up, run any block of JavaScript in a try-catch, and if you catch an error then it is a reasonable practice to assume that some variable you are using is missing or has no value and generate the appropriate result based on that assumption.compares the contents of the deployed running Shibboleth configuration to the contents of each configuration file from Source Control and the installer project. If the files are the same, nothing happens. If they are different, then the new contents from Source Control replaces the file on local disk on the Shibboleth server. Since only changed files are reloaded by Shibboleth, this can change only a Metadata file, or only the Attribute Resolver configuration, or only the attribute-filter.

Runtype "additions" modifies only the "additions.xml" Metadata file and the "additional-attribute-filter" file. This can be used to add new Service Providers to production between the every-other-Thursday full Release cycles. Shibboleth isolates these files and appears to guarantee that this type of configuration cannot possibly interfere with existing production services.

Runtype "emergency" will change the "emergency-override.xml" file and allows us to define new Metadata for an existing production partner without affecting anything else. It may require permission from the ECAB, but it is a less dangerous change than a full configuration update. Note that the old Metadata for the partner remains in place, but is not used because the override Metadata is found first in the search order. Before the next Release cycle (the next runtype=install or runtype=config), the old production Metadata should be replaced with the new override data and the emergency-override.xml should be emptied.

Runtype "salesforce" and "cvent" are proposed runtypes that change a single Metadata file for the two partners that require frequent updates.

Contents of the Primary Configuration Files

Attribute-Resolver

Normally Shibboleth has a single attribute-resolver.xml file that contains two types of elements. DataConnectors define database or LDAP queries that produce result sets with columns or LDAP User objects with properties. AttributeDefinitions then take the columns and properties returned by the queries, assign a unique identifier that can be referenced in the attribute-filter (release policy), and supply SAML syntax. So two DataConnectors could query the Yale IDR database for basic identity information, and also the Active Directory for the subset of identity information it contains. Then AttributeDefinition statements can take the "FirstName" column from IDR or the "givenName" property from AD and create various SAML Attributes all with the same value of "Howard" but with SAML name and friendlyName attributes that refer to it as "FirstName", "First Name", "givenName", "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname", or "urn:oid:2.5.4.42" (informal and formal standards based SAML names for the same thing).

Shibboleth 3 allows the attribute-resolver to be configured with more than one file if you add additional elements to a list in conf/services.xml. Yale uses this to separate all the DataConnector elements which have one syntax and one natural default xml namespace from the AttributeDefinition elements that have a different syntax and a different natural default namespace.

Shibboleth documentation is not particularly clear on the algorithm, so I will try to fill in something that I believe is important to understand.

DataConnectors

There are generally three types of queries that make sense:

• A database query can return exactly one row. Then you can think of the row as a user object, and the column names become the properties of the object.
• A database query can return more than one row but only one column. Then you have a "multivalued" property for one user.
• An LDAP query returns the User object from the directory. LDAP User objects have properties some of which are single valued and some of which are multivalued.

Each column in the result set or property in the LDAP object becomes a Shibboleth IdPAttribute object, but this object has no SAML formatting information and no global name. To find one of these objects, you have to specify the ID of the DataConnector that ran the query and then the column or property name. You need to create one or more AttributeDefinition elements to convert this into a real Shibboleth Attribute with a real ID that you can release as SAML to a partner.

The DataConnector needs a Java DataSource to provide a pool of database connections. Java DataSource management is complicated because it has to know when a database connection must be discarded because it has timed out or because the database rebooted since it was last used. Shibboleth would prefer to leave this to the database experts. Shibboleth 2 did this by using "container managed" connections provided by Tomcat or JBoss. Shibboleth 3 can still use connections managed by the application server, but now that it is a full Spring Framework application it can use DataSources provided and managed by Spring. Either way the complex database management doesn't have to be done by Shib provided code.

AttributeDefinitions

The DataConnector provides the value. We know that "Howard" is the value of the "FirstName" column of the result set returned by the "IDRQuery" database connector or of the "givenName" property of the user object returned by the "ADQuery" LDAP connector.

However, to use this value in any real Shibboleth logic, it needs a single unique "id=" name. That ID can be referenced in the attribute-filter to release the attribute to an application, or it can be used in a Dependency statement to create a variable in a JavaScript block of code, or to create a variable that can be inserted into the WHEN clause of another Database query.

All you need is an AttributeDefinition statement that has a Dependency element that references the ID of the query (IDRQuery or ADQuery in the previous examples) and which specifies the column name or attribute property name as the value of sourceAttributeID=.

<resolver:AttributeDefinition id="idrFirstName" xsi:type="ad:Simple" sourceAttributeID="FirstName">
    <resolver:Dependency ref="IDRQuery" />
</resolver:AttributeDefinition>

This AttributeDefinition gives a single unique ID "idrFirstName" to the value of the "FirstName" column returned by the "IDRQuery" DataConnector.

With this AttributeDefinition, you can reference "idrFirstName" in other elements. They can create a variable named "idrFirstName" in a block of JavaScript code, or they could create "$idrFirstName" which can be added to the template for another database query. It would be a really bad idea, but you could create a NameId definition that uses this value as input to a hash that generates the value of the Subject of a SAML Response (except that Subject has to be unique and lots of people have the same FirstName, but this would make sense if the attribute was the Yale UPI number).

About the only thing this AttributeDefinition cannot do as shown is create a SAML Attribute that can be released to a partner and sent in a SAML Response. To do that you need to add one additional element, a SAML Encoder that specifies a friendlyName= like "FirstName" or "GivenName" and an (unfriendly) name= like "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname" or "urn:oid:2.5.4.42".

<resolver:AttributeDefinition id="idrFirstName" xsi:type="ad:Simple" sourceAttributeID="FirstName">
    <resolver:Dependency ref="IDRQuery" />
    <AttributeEncoder xsi:type="enc:SAML2String" name="urn:oid:2.5.4.42"
            friendlyName="givenName" /> 
</resolver:AttributeDefinition>

Now "idrFirstName" can be released by the attribute-file.xml file and can be sent in a SAML 2 Response. Of course, now the "idrFirstName" field is bound to the LDAP name and friendlyName conventions, which some partners expect. Other partners want another value. If you are going to do this, then you might change the id from "idrFirstName" to "idrFirstNameWithLDAPSyntax".

So an alternative is to create a pure AttributeDefinition using the first example code (without the AttributeEncoder), and then derive additional attributes with Encoders and more specific IDs based on the original:

<resolver:AttributeDefinition id="idrFirstNameWithLDAPSyntax" xsi:type="ad:Simple" sourceAttributeID="idrFirstName">
    <resolver:Dependency ref="idrFirstName" />
    <AttributeEncoder xsi:type="enc:SAML2String" name="urn:oid:2.5.4.42"
            friendlyName="givenName" /> 
</resolver:AttributeDefinition>

This form of AttributeDefinition references a previously defined Attribute (by setting both sourceAttributeID and the Dependency ref to the ID of a previous AttributeDefinition) and then gives it a AttributeEncoder format and a new more specific ID to use in the attribute-filter so you release to a particular partner a value and name+friendlyName format.

XML and Shibboleth are case sensitive, so it is important to realize that Oracle always converts its columns to UPPERCASE. To avoid errors you should always use UPPERCASE names for the sourceAttributeID field if the query is to an Oracle database, and you should define an UPPERCASE id for a default static value in the fallback connector if the Oracle query fails. Otherwise you may spend hours trying debugging the failure of the value to show up where you expect it to be.

Undefined, Null, or Empty

When you query a database or an LDAP directory and then try to define an Attribute based on the value of a column or property, several things can go wrong:

• The database query can return no rows from the query.
• The database query can return a SQL NULL value for a column (unless you use NVL in Oracle or ISNULL in SQL Server to replace the NULL with a default value).
• An LDAP query can return no User object
• An LDAP query can return a User object, but in that object the property you are looking for may not be present.
• An LDAP query can return a User object, and the property may be present, but it may have no values in the list of values.

First the good news. If all you do is to create an AttributeDefinition with an Encoder and to release it as an ordinary Attribute in a SAML Response, then you don't have to think about it. Shibboleth takes care of all these cases and does the right thing.

However, if you want to use this attribute in the SAML NameID server to generate a Subject, then you have a problem because the value that goes into any Subject calculation can never be NULL or missing. So it is your job to make sure that none of these things can happen (nor can duplicate values be returned for different identities) when this attribute is input to a custom element in the saml-nameid.xml file

Then you can run into problems when this attribute is referenced as a Dependency in an AttributeDefinition of type Script. Each Dependency creates a JavaScript variable that can be used in the block of JavaScript code that calculates a value for the new Attribute. However, this variable reflects the peculiar status of the column or property from which it was derived.

A JavaScript variable name can be "undefined" if it has never been used and assigned a value.

A JavaScript variable can be null if it has been assigned the value null.

A JavaScript variable can be an array of length 0 or a collection that contains no objects.

You can encounter all three conditions in a JavaScript variable created by a Dependency element in a type "Script" Attribute definition when different results are returned from a database or LDAP query. The only safe thing to do is to check all three in that specific order:

                if (typeof googleEmail!="undefined" && 
                    googleEmail!=null &&
                    googleEmail.getValues().size()>0) {
                        googlemailalias = googleEmail.getValues().get(0);
                }

Suppose the Database is Down

If any exception is thrown during the query, then the Shibboleth code will attempt to execute a secondary query specified in the "failover" attribute of the DataConnector. The failover can point to a different query to a different database that might return the same value. Or it can be a Static element.

A Static DataConnector defines one or more property names and values. It is not necessary to define a default value for every property that you could have obtained from the correct execution of the real query, provided that a null or undefined value is acceptable for the other properties.Given the previous warning about NULL and undefined and empty, you should think twice before leaving column/property names without an explicit default value (0, 1, -1, "", "undefined", etc.). However, it is not an error to omit them if you choose. The Static DataConnector cannot throw an exception.

At Yale:

Every Query must have a Failover DataConnector, which may itself have a Failover, and the chain must end with a Static Connector.

 See the examples in the attribute-resolver-connectors.xml file.

In vanilla Shibboleth, the only errors that are caught during Attribute evaluation are query errors. Any other error (during JavaScript evaluation, or because of Shibboleth bugs) is fatal and prevents the user from logging on to any partner. Yale has added code to wrap other evaluations with a try-catch that discards attributes in error but preserves all the other attributes. Because attribute errors tend to occur only in new attributes not widely in use or old abandoned attributes no longer in use, this makes Shibboleth more robust against real world errors without impacting security. We will try to interest the Shibboleth maintainers in making this fix standard.

...