Versions Compared

Version Old Version 28 New Version 29
Changes made by

Howard Gilbert

Howard Gilbert

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Any given release of the CAS Server can be downloaded as a zip file from Apereo or it can be checked out from the Git server used by CAS developers as documented at the Apereo Web site. Yale modifies this file, adds additional subdirectories representing the Yale customizations, and then checks it into Yale's Subversion server where the Yale standard Build process begins. Depending on context the CAS "project" could mean the original zip file or the modified file stored in SVN.The CAS Project is structured as a "parent Maven project with subprojects"The release source is a Maven project, specifically a "parent" project with subdirectories that contain Maven subprojects. This is a common Maven way to package multiple projects that have to be compiled and built in a particular order.

The outer directory contains the parent or master POM pom.xml that establishes defines parameters shared by all the subprojects and . It also contains a list of <module> statements that identify the subprojects to be built in the order in which they should run.

...

Although you can modify the cas-server-webapp project directly, Apereo recommends this results in a directory with a mixture of Apereo files and Yale files, and the next time you get a new CAS release from Apereo you have to sift through them to find out which ones have been changed by Apereo.

Apereo recommends using the WAR Overlay feature of Maven. This simplifies the case where the only changes you are making to CAS is to replace specific JSP and HTML pages with your own wording, or CSS files with your own look and feel, or Spring XML beans configured differently. In a WAR Overlay project you reference a "template" WAR containing a starting set of files (the WAR created by cas-server-webapp in this case) and then you include in your WAR Overlay project only the things you want to change or add. Maven copies the template WAR file to the output artifact, but it replaces any file in the template that has the same name as a file in the Overlay project, and it adds any file in the Overlay project that doesn't exist in the template. Therefore the Overlay project only contains Yale changes and doesn't have to include all the unchanged Apereo vanilla files.

If you use any of the optional modules (ehcache, ldap, oauth, etc.) then you configure them in the Overlay project in modified Spring XML files and include them as dependencies in the Overlay project POM. Since you cannot include a JAR library until after it has been created, the Overlay project should be the last project executed, except possibly for projects that generate Javadoc or other documentation.

The Yale Overlay project is in a directory called cas-server-yale-webapp. However, Eclipse frequently pulls out an internal original name for the project and so it often displays as cas-server-war.

At Yale, when you run the Jenkins Build job for CAS, the result of that job is to create a cas.war file and store it in the Yale Artifactory server with a version number and some hierarchical naming. An example is Version 1.1.1 of the Yale CAS Server stored as https://repository.its.yale.edu/maven2/libs-releases-local/edu/yale/its/cas/cas-server-war/1.1.1/cas-server-war-1.1.1.war. However, you cannot build the WAR until you build everything it depends on. The Jenkins Build job runs the parent Maven project in the top level directory, and it creates the WAR file by first building everything it depends on.

The Parent Project

Whenever Yale decides to upgrade from one release of CAS to another, a major part of the work is to adapt the old Yale parent project (the pom.xml file in the root of the CAS Project directory) to the new release. This is not a case where you are simply modifying some Apereo file with Yale changes. The parent project reflects the thing you are building.

Apereo is building a template version of CAS that doesn't do anything particularly interesting but at least creates vanilla versions of all the JAR and WAR files. The most important thing is that these files conform to the rules for Open Source projects distributed publicly. They have to have the right copyright statements and the right license boilerplate.

Yale is building a version of CAS that has to satisfy Yale internal needs. It doesn't care about copyright statements or licenses because we are not giving this code to anyone else. We can strip out all the list of developers and contributors not because we don't appreciate their contributions but because it is not particularly relevant to doing the Yale customizations.

These are really two quite different objectives and they have a fairly large impact on the structure of the parent POM. As a result, you have to merge elements from the old Yale parent and the new Apereo parent. This requires a tour of the Yale version of the parent POM.

  • Version - The Apereo parent will have the version number of the CAS release (say 3.5.2.1). The Yale parent will have the version that Yale internally assigns to its CAS artifact (1.1.1-SNAPSHOT for example). The general rule is that Apereo projects and artifacts retain their old version number unless they are changed, but if Yale has to make a single line of modification to a project then it has become a Yale project and be renumber to the Yale development cycle. Fortunately, CAS started at JASIG with 3.x so we can number anything 1.x without conflict.
  • SCM - defines the Yale Subversion server where Yale maintains its production source and not the Apereo Git server used to develop new releases.
  • Repositories - defines Yale's Artifactory, but merged with other sources for artifacts from the Apereo pom.
  • pluginManagement - Maven is a modular system with extensible functions provided by plugin modules. When a project requires special configuration or processing, it can add parameters to the configuration of a particular plugin in this section of the POM file. For the most part Yale copies this section from the Apereo POM file, but specific parameters have to be changed as a secondary result of Yale decisions. For example, because we choose to run CAS under Java 1.7 instead of 1.6, the version of the aspectj-maven-plugin has to be increased from "1.4" as distributed by Apereo to "1.5". When you run the Yale Jenkins Build job operation "Perform Release", which takes version "1.1.1-SNAPSHOT" source, creates a "1.1.1" version tag and artifact, and then leaves SVN set to develop "1.1.2-SNAPSHOT", then running unit tests in the middle of the operation is not helpful and is disabled at Yale by adding <arguments>-Dmaven.test.skip=true</arguments> to the configuraton of the maven-release-plugin.
  • dependencyMangement - All the Apereo code is coordinated to use the same release of every dependency JAR file. However, Apereo does not bother to specify explict Versions of secondary dependencies (that is, JAR files required by a library that CAS uses). However, when Yale adds its own code to the mix, we may depend on a particular version of a library where the default behavior of Maven previously selected a different verision of the same library. For example, in CAS 3.5.2.1, cas-server-core depends on a library that itself depends on commons.io and from the chain of pom files Maven selects commons.io Version 2.0. Yale code requires Version 2.4. When two different subprojects independently resolve to different versions of the same dependency, the parent POM has to decide which wins. Basically this second should contain the latest version selected by either Apereo or Yale (and of course nobody should use a library whose developers are dumb enough not to maintain reasonable backward compatibility).
  • modules - Yale wants to use vanilla code distributed by Apereo. If we have no changes to distributed CAS code, then there is no reason to recompile it. So initially Yale comments out all the modules that Apereo wrote and adds in uncommented module statements for the Yale developed projects. Therefore, the Build process only compiles and builds the Yale changes. However, if it is necessary to make a change to Apereo code then a changed module has to be uncommented so it can be rebuilt. Also, when you check out code into a new Eclipse directory you may uncomment the module statement for an important project and then tell Eclipse to generate an Eclipse project to build that module, but after you have your Eclipse project you revert back to the version of the POM where the module is commented out. Letting Eclipse configure the project and compile the code once gives you all the Eclipse IDE stuff for classes and methods built by that module and it lets you trace into the module source during debugging.

 

Yale adds several new Yale specific subprojects to the vanilla Apereo Maven source project directory tree. Some of the projects create JAR files that add specific functions to CAS. The final project builds the actual WAR file that is deployed to JBoss to run the CAS Server at Yale. No changes are made to the Apereo code unless there is a bug that needs to be fixed immediately.

It is a bit strange to check a large block of vendor source into a Yale Subversion directory that is supposed to contain Yale customizations. If there is a better way to handle this code, we can certainly adopt it. However, the Apereo CAS project uses Git as a source control system, while the Yale project management conventions still use Subversion. So to get Apereo code into the Yale Subversion system you have to migrate it across source management systems and probably lose the history data during the migration. Since we don't actually change the vanilla code, this is not a problem. If you want to track the history of a vanilla Apereo source file, use a Git client and get the history from it. The Yale Subversion directory is simply a copy of the most convenient Eclipse workspace project that one can use to do CAS development based on a particular release of CAS from Apereo.

A copy of the CAS Server source with the additional Yale projects is always checked into Yale Subversion as https://svn.its.yale.edu/repos/cas. If you are making changes to the current release, then everything you need is there. If you intend to upgrade to a new release of the Apereo code, then you obtain a new copy of the source distribution for that release (either download the zip file or check it out from Git) and then copy the Yale added subdirectories from the old release project to the new release project and change the pom.xml file in the new parent to execute the projects you want to build. Copying the Yale files from one project to another will lose Yale history, but Yale doesn't change CAS code except every four years or so. If you have been running a version of the code for 4 years without modifications, then the day to day history of the old development project is no longer important.

Some might suggest that the Apereo code be kept in one project tree and the Yale code be kept in a different tree. However, the original Apereo project was designed to work with a single parent and subdirectories, and we decided not to try and change that design. Besides, if all the source from Apereo and Yale is available in your Eclipse project workspace, then all the tools to search for references to a field or during debugging to step into a subroutine automatically work correctly. It is not hard to remember not to change non-Yale code without a good reason, and then to document if you make such a change so people know that this is an issue if we move to a new release of the Apereo base.

The ReleaseNotes.html file in in the root of the cas-server project will identify the current state of new directories and any modified files. This should be a descriptive match for the physical code change history in SVN.. Yale creates a second WAR project called cas-server-yale-webapp. Instead of copying all the files from the Apereo project, the WAR Overlay project contains only the files that Yale has changed or added. Generally the WAR Overlay includes:

  • Yale Java source for Spring Beans that are slightly modified versions of standard Apereo beans.
  • Yale "branded" look and feel (CSS, JSP, HTML).
  • A pom.xml file with additional dependencies beyond just the cas-server-core that was included in the template WAR built by cas-server-webapp. For example, Yale depends on cas-server-support-ldap and cas-server-integration-ehcache. We don't modify them, but since they are optional modules they were not included in the template WAR. Adding them to the dependencies in the Overlay project POM adds them to our WEB-INF/lib.

The WAR Overlay project references the template WAR, and at the end of processing the files that Yale added or changed are "overlayed" on top of the template to build the cas.war file that is actually deployed in production. Modified files are replaced. New files are added.

At Yale, when you run the Jenkins Build job for CAS, the result of that job is to create a cas.war file and store it in the Yale Artifactory server with a version number and some hierarchical naming. For example, Version 1.1.1 of the Yale CAS Server is stored as https://repository.its.yale.edu/maven2/libs-releases-local/edu/yale/its/cas/cas-server-war/1.1.1/cas-server-war-1.1.1.war.

One or Two Projects?

If you were never ever going to change any Apereo source, then it would make sense to have two separate projects in your Eclipse workspace. One project would be the absolutely vanilla CAS source you downloaded from Apereo, and the other project would be the additional Yale modules and the WAR Overlay project. The Yale project would depend on the Apereo artifacts, but the only reason for having the source is so you can see methods that you step into while debugging your own code.

However, Apereo code can have problems. Sometimes it is a bug. Sometimes it is just a difference of opinion between the Yale way of doing things and the Apereo way. Sometimes you want to add in specific support for features that Apereo is working on but has not yet finished.

  • Apereo defaults all registered services to be able to Proxy, but it seems better if the default is that they cannot Proxy unless you explicitly authorize it. This involves changing the default value for allowedToProxy in AbstractRegisteredService.java from "true" to "false".
  • AbstractTicket is a Serializable class, but Apereo source forgot to give it a VersionUID. As a result, every time you deploy a new version of cas-server-core you have to "cold start" CAS by wiping out all the current tickets. You should only have to do that if you actually have changed the Ticket classes, not just because you recompiled the library to change something that has nothing to do with Tickets.
  • Yale does not use Single Sign-Out, but code in TicketGrantingTicketImpl added to support that option has a very small but non-zero chance of throwing a ConcurrentAccessException and possibly screwing CAS up. Since we don't need it, Yale might just comment out the statement that causes the problem.

Each of these is a one line change, and only the last is important. If CAS was not open source we would probably just live with the old code, but we have the chance to fix it.

Once you start to consider changing Apereo code, then it is inconvenient to maintain two separate projects. For one thing, you need two separate Jenkins build jobs, one for each project. Then you have release numbering issues.

The alternative is to have a single project that combines both the Apereo source for a CAS release (example: 3.5.2.1) and the Yale code that has been built and tested for that CAS release (separately designated Version 1.1.x). Most of the time the Apereo source just sits there and is not used and is not compiled because it is not changed. If you need to make a modification, it is immediately available.

The single project may be confusing, unless you have read this document. There is a large block of source checked into Subversion that appears in your Eclipse workspace, except that none of it is actually used. Why is it there at all? If the Apereo source is not going to be changed, shouldn't it be separated out for clarity?

There are advantages to the Two Project approach, and advantages to the One Project approach. Currently we use the single project structure, but if you feel really strongly about it you can separate the code in some future release.

The Parent pom.xml

The top level directory is the "parent" project. Typically it is named "cas-server" although this is simply a choice you make when you check the source out from SVN. The only thing in the parent project is its pom.xml file, and a few README type files.

However, one of the more complicated problems when Yale migrates from one release of CAS to another is to reconcile our changes to the parent pom.xml with any Apereo changes. This is not the same type of code migration that occurs if you have modifications to some HTML or Java source file.

All the CAS code, configuration files, and HTML do the same thing at Yale that they do everywhere else and that Apereo designed them to do. So our version is fairly close to the original.

However, the top level pom.xml file is not just about compiling the source and building the template WAR file (which is the part we need). Apereo is in the business of maintaining an Open Source Project that is distributed widely on the Internet. A large part of their concern is to make sure that all the legal boilerplate is properly maintained. Do the files all have a proper copyright notice? Are all the licenses to the dependency files properly listed? Maven has some impressive support for doing all this open source release management, but unless that is something you specifically have learned, setting up the legal boilerplate is a daunting task.

Yale is a CAS customer. We may have our own files, but unless and until we contribute them back to Apereo we do not care if a file that nobody sees outside Yale has a proper open source copyright notice and license declaration. If we try to use the Apereo top level pom.xml file as distributed, then every time we try to cut a Yale release we get error messages for all the boilerplate that is missing from our own files. So we remove all that stuff and create our own top level pom.xml.

Given that we have to make substantial changes, we might accept that this pom.xml is a Yale file and no longer a modified version of the original Apereo project. Then for sanity, and because our release process requires it, we change it to reflect the rest of the Yale environment:

  • Version - There are two version numbers, one for unmodified Apereo code corresponding to a CAS release (3.5.2.1) and one representing the internal Yale Release of the CAS artifact that we put into production (1.1.x or 1.1.x-SNAPSHOT). The Yale top level project and all the Yale source use the Yale version ID. If we modify an Apereo project (say we make a change to cas-server-core) then we also change the version number of that JAR file from the Apereo to the corresponding Yale ID numbering.
  • SCM - defines the Yale Subversion server where Yale maintains its production source.
  • Repositories - defines Yale's Artifactory, but merged with other sources for artifacts from the Apereo pom.
  • pluginManagement - Maven is a modular system of extensible functions implemented by optional "plugin" modules. The first step for Yale was to ditch the "notice" and "license" plugins that mange the legal boilerplate. Then we make a few parameter changes to a few other plugins. For example, when you are done working on Version 1.1.1-SNAPSHOT and want to create the official 1.1.1 Yale version of CAS and then reset the project to begin work on 1.1.2-SNAPSHOT, you call the Yale Jenkins Build job "Perform Maven Release" operation, which in turn uses the maven-release-plugin. At Yale we find that running the JUnit tests during this process is not only slow but a bad idea, so we change the configuration of that plugin in the parent pom.
  • dependencyMangement - In any official CAS Release, all of the code is coordinated so all modules and options use the same versions of all the same libraries that will be distributed in the WAR and made available at runtime. Unfortunately, when Yale develops its own code it may need a later version of the same library. For example, Apereo is happy with commons.io Version 2.0, but Yale needs Version 2.4. Apache is smart enough to make the 2.4 library downward compatible with programs written for 2.0, but swapping a newer better version of a library is tricky. The first step is that our top level pom.xml declares the 2.4 version to be the one we want so that all our projects use the same version. This also ensures that the 2.4 version of the commons-io JAR will be merged into the WAR during the Overlay processing.
  • However, there is a final processing step that has to be done in the WAR Overlay project and is related to but not directly specified in the parent pom.xml file. Because we are using some unmodified CAS 3.5.2.1 modules (particularly cas-server-webapp) and then we merge in new libraries from the WAR Overlay project, the resulting cas.war file would have both commons-io-2.0.jar and commons-io-2.4.jar in WEB-INF/lib (2.0 from vanilla cas-server-webapp and 2.4 from our project). So we have to add an exclude statement in the configuration of maven-war-plugin in the WAR Overlay project to remove the unwanted commons-io-2.0.jar file). Generally speaking, we remove any file that the build process would normally include that is bad for JBoss or where there are multiple versions of the same code and we only want to keep the latest version.
  • modules - The CAS Project (that is the source checked into SVN that also is found in the Eclipse workspace) contains all the projects distributed by Apereo in a CAS release. It is just that we do not use most of them and we try to avoid changing them, so we don't need to compile or build them. That is accomplished by deleting (or commenting out) all the <module> statements in the Apereo parent pom.xml that reference unmodified Apereo source projects, then adding <module> statements for all the Yale projects. Even though the source and the project are physically present as subdirectories, if the parent pom.xml doesn't refer to a subdirectory in a <module> statement then Maven ignores that directory during the build. On the other hand, if we decide to make a change to an Apereo project then we add back in or uncomment the <module> statement, but with Yale changes we also alter the version ID in the pom.xml for that subdirectory from Apereo "3.5.2.1" to Yale "1.1.x-SNAPSHOT".
  • Parameters - At the end of the POM there are parameters that specify the version numbers of dependency libraries and Java.

Subversion

Because CAS is maintained by Apereo in Git, and because of the way we structure and maintain the project, if you really wanted to track the history of CAS modules properly Git would be a much better choice. However, the Yale Build process is based on Subversion and that is what we have to maintain.

So in any given cycle of CAS development, you begin by obtaining a copy of the current CAS release from Apereo. You then replace the top level pom.xml with a new one created by merging any changes made by Apereo since the last release with the prior Yale top level pom.xml (and this has to be done manually by someone who understands Maven). Then you merge in the Yale added subdirectories.

This becomes a new project in the /cas area of Yale Subversion. Because it is nominally new, you start without history. If you want to see the history for Apereo code, use Git. If you want to see the actual history of Yale modules, find the Subversion directory of previous CAS development projects. Generally there are several years of stable operation between CAS development projects, and if the thing has been working for years you a probably no longer interested in the "ancient history" of a module.

The ReleaseNotes.html file in in the root of the cas-server project will identify the current state of new directories and any modified files. This should be a descriptive match for the physical code change history in SVN.

The Jenkins Build and Install Jobs

In order to put CAS into production, you have to conform to the Yale Jenkins process.

There is a Jenkins Build job. The developer normally runs it to compile the current SVN trunk to produce a Version 1.1.x-SNAPSHOT of the cas.war artifact which is stored in the Artifactory server. The convention of a Jenkins Build job is that it checks out a copy of the entire Project file from SVN to a Jenkins managed server and then it does a "mvn clean install" on the top level project (the parent pom.xml). Maven will in turn do a clean install on each of the subprojects reference by <module> statements to create updated artifacts, of which the only important one is the cas.war file.

There is a Jenkins Install job. It also checks out the installer project from SVN. That project also has a Maven POM, but the Yale convention is that it runs an Ant build.xml script to download a copy of the cas.war file created in a previous Build step and copy it over to the JBoss deploy directory. As files are copied some edits are made "on the fly" to insert parameter values for the names of userids or password used to access databases, AD, or to configure special options in Spring.

The job of the developer is to commit changes to SVN that make the Build and Install job work properly to create the CAS instance running in DEV, TEST, and PROD. In order to make those changes, you need a desktop Sandbox environment that can not only compile, test, and debug CAS but can also prototype the Jenkins Build and Install process (without using Jenkins of course).

Development

CAS runs in production on Red Hat Enterprise Linux, but you can do development with Eclipse and JBoss running on Windows or any other OS. If you plan to work on SPNEGO, however, you should use a Linux development machine or VM because SPNEGO behavior depends on the native GSSAPI stack in the OS and that is different in Windows from the version of the stack in Linux.

...