Things we cannot change:
Shibboleth as distributed is designed to have large configuration files that each define all the attributes, all the attribute release policies, all the relying parties, and so on. You can break them up into smaller files, but in some cases everything gets loaded as a single operation from as many files as you have. Only with Metadata file can you achieve full isolation.
There are no federations or partners outside Yale who have a concept of a TEST IdP. You cannot define one to InCommon, and most of our partners only know about Production Shibboleth. Some partners will create one configuration while testing the SSO and then will routinely replace it with a second configuration when they go to production, but not everyone expects to do this.
If we are going to move more and more services into the cloud, then we will increasingly encounter changes that occur outside our control. We cannot schedule when a partner makes a change, and yet some changes have to occur both here and at the partner site. Some changes happen unannounced.
Because of these things, our current attempt to fit Shibboleth into the standard process model produces a bad fit.
- When the login to the https://yale.example.com site breaks down, the problem appears to be with the "example" service. Since Shibboleth provides login services to 100 services, many of them mission critical, a change to the Shibboleth CI is regarded as higher risk than it would be if we could instead isolate the problem to just "the Shibboleth login to example.com" component.
- DEV and TEST are a good place to test changes we make to Shibboleth itself. They are often useless as a way to test changes to the configuration of example.com that have to be made because of something example.com did given that example.com doesn't know about or use DEV and TEST Shib. Shibboleth could support and we need to rearrange our files, processes, and definitions to support, a TEST-configuration sandboxed section running safely inside the Production Shibboleth process.
I propose to refactor the Shibboleth maintenance process to address these two problems.
Process Refactor of Shibboleth
Break the Shibboleth configuration files and maintenance (Jenkins) functions into
- A Production Service Operation part containing the current stable partner configurations from the last Change/Release
- A trailing Service Transition section where new partner configuration files can be added so that Production Shibboleth can provide Production Identity information to new services that are under development and may not yet have final stable configuration requirements. Because Production configurations come first and will be found before the Service Transition section is checked, anything placed in this section of the configuration cannot interfere with stable production services and therefore these files are under developer control and are not regarded as Change because they do not affect existing CIs.
- A leading Problem part containing specific single element modifications added by by Problem Tasks during Root Cause determination to find and eliminate service disruptions caused by unplanned external events. If a disrupted service is restored this way, it is regarded as a temporary Workaround. The permanent solution is to convert the workaround into a Change to the Production files and to schedule the Change through the normal Change/Release process. Typically an Install done during any Release empties out the Problem section of the configuration files which ensures that Changes are properly created, documented, and approved.
Problem modifications are only applied to already malfunctioning services, and they are documented as a Problem Task attached to a Problem Ticket. Since they do not affect Production services and are not permanent, they are not a Change and do not require CAB approval. Problem modifications must be first installed in DEV and TEST, but the purpose of this phase is to verify that they install correctly and do not disrupt anything else, not to try and test if the modification fixes the original problem (because that cannot be determined in most cases before we get to PROD).
When a partner service transitions to production, or at least when its requirements become stable, then its configuration should transition from the "Service Transition" section to Service Operation, and at that point a Change is created to document the transition.
Partner Refactor of Shibboleth
A less urgent but eventually useful step would be to segregate out all configuration elements that apply to a single partner. Currently we only do this for Metadata files, which are probably the most important partner specific configuration elements. We could also break out custom attribute definitions into a file and the attribute release policy for the partner into another file.
Once this is done, there is a physical reality to the claim that there is a Shibboleth CI which is the service itself, and then there is a "Shibboleth login for Salesforce" CI that represents the specific files that the Shibboleth service uses to login to Salesforce. It is not clear whether we want to add all these separate CI's to ServiceNow, but it improves the process documentation if we are clear about exactly what a Change affects.