Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 11 Next »

CAS source and configuration files are stored in the Yale Subversion repository at https://svn.its.yale.edu/repos/cas. If you are updating the current release, everything you need will be there.

If you are upgrading to a new release of CAS, you need to obtain a copy of that release from JASIG. CAS releases are distributed as a zip file, but they are also checked into Git. Unless you are contributing updates back to JASIG, there is no need for Git and the Zip file will do fine. Then you need to compare the directories (and a few individual files) in the new release source with the directories in the SVN repository. Mostly you will add Yale subdirectories from the SVN to the JASIG source following instructions to be provided below.

The ReleaseNotes.html file in SVN will identify the JASIG files that have been changed. You will have to manually merge changes to the parent POM file in the top level directory, but otherwise the JASIG code is unmodified and the Yale code is in completely new project directories. So except for the one POM file there are typically no changes to reconcile from release to release.

So why have we checked in all that JASIG source if we don't change it? JASIG uses Git, and Yale has standardized on SVN, so the only way to clearly document what JASIG source we are using it is check it in to our SVN as if it was our code. Then you have it all for debugging. It may also have to be recompiled if we decide to change the release number of a dependent library file that JASIG uses, but where we need a different release number.

Yale and JASIG use Eclipse as the IDE. The most current version of Eclipse works best. Start with the J2EE package of Eclipse, and add SVN and Maven support.

Check Out and Build the Project

Open the SVN Repository Exploring "perspective" (page) of Eclipse and define the repository url. Right click the trunk of the current CAS source project and select the simple Check out menu option. Do not run the Check Out As wizard or try at this point to make it a Java project. Simple check out creates an Eclipse "project" directory in the workspace but at this point it has no attributes or nature or build process.

Now return to the J2EE perspective, right click the new project directory, and choose Import - Maven - Existing Maven Projects. The Eclipse support for Maven will search the directory for its POM and then find POM files in the subdirectories.It will then display a list of the subdirectories that contain Maven projects listed in the parent POM. Select the checkbox for all of them and push the OK button. 

At this point Eclipse will do a lot of work. It is running through all the POM files understanding what they say and is downloading from the internet into the local Maven repository all the JAR files that any of the CAS projects identify as a dependency. It creates in the Eclipse workspace a "shadow project" for each subdirectory project. This shadow is an Eclipse project with .project, .classpath, and .settings files and folders with the correct options and settings to compile Java source for the corresponding JAR or WAR that the Maven POM is going to build.

There are several types of error messages generated. Some JSP and HTML files are fragments that are not regarded as well formed, but that is because CAS will assemble a page from several fragment pieces which only become well formed when they are completely assembled. There will also be a Maven error because the Eclipse support for Maven (really the Eclipse interactive simulation of Maven batch function) does not simulate the Aspectj maven plugin. This is not a real problem because we will not be using Eclipse simulation to run CAS under JBoss, but will instead run a full Maven batch command where all the standard Maven plugins are available to build real JAR and WAR files. So ignore the errors or click OK.

When Eclipse is done with the import, it will have created shadow project directories named cas-server-core, cas-server-documentation, and so on. There is now a discrepancy between the Eclipse view of the workspace and the reality you would see if you opened the workspace directory using the operating system file explorer. From the Eclipse project view, each new project appears to have the Java source files and resource data files for that project. However, Eclipse also shows what appears to be a second copy of the same files in the various src/main subdirectories of each project subdirectory of the parent "cas-server" project. These are two views of the same files, and really the only copy of the real file is in the subdirectory of cas-server. The Java view of packages and source files in the shadow directories such as "cas-server-support-ldap" are a logical view of what Eclipse found in the project POM. This means that if you do certain global operations like Search - File you may find the same file returned twice by the matching process, once in its physical location under cas-server and once in its logical project location under one of the shadow projects. You can edit the file in either parent "folder" and any changes you make apply to both views because there is really only one file.

Interactive and Batch Modes

Eclipse wants to make things easier than they can really be. It wants to extract information from the Maven POM but then to compile the source and build the "WAR" itself. However, it doesn't really want to build a WAR file and then copy it over to the Tomcat or JBoss server. So it has some special logic to "hack" the configuration of Tomcat and JBoss servers so that they run with an imaginary WAR that doesn't really exist in their directories but is being simulated by Eclipse.

This works fine for Hello World and maybe the next five Web applications you write, but pretty soon you want to do something that Eclipse is not ready to simulate. Alternately, you may want to use some Maven feature that Eclipse cannot duplicate.

At this point what you really want to do is to run not the integrated simulation of Maven and JBoss function built into Eclipse, but rather you want to run Real Maven and Real JBoss. You can do that in Eclipse, but for sanity you have to realize explicitly that you are rejecting the "simplified" environment that Eclipse tried to create for you. You also have to understand what Eclipse just tried to do, what it accomplished, and how to do the rest yourself.

Eclipse reads the Maven POM file and creates an Eclipse project that tries to do the same thing. The Eclipse compiler can use roughly the same options to compile the same source library to the same target directory of class files. The Eclipse Maven support can turn the dependencies in the POM file into an Eclipse .classpath file that gets the right Java library files from the local Maven (.m2) repository directory. However, Maven has support for additional operations controlled by "plugin" elements of the POM file, and Eclipse does not support these extra steps.

What you can hope for is an Eclipse project where Eclipse has a .classpath built from the dependencies in the POM to find the right version of the right JAR files from the local Maven repository (by default in the .m2 subdirectory of you home directory). Thanks to this work, the source compiles without any missing includes and you get the benefits of autocomplete and autocorrect from the Eclipse editor. That is about as much as you need. It isn't important that Eclipse cannot build the real WAR file because you are going to use a real Maven batch run to do that.

Eclipse recompiles each Java source file when you save it. Since the Eclipse project was created from the Maven POM, it compiles the same source directories that Maven uses and it puts the class files in the same place Maven puts files when it compiles them. By default, Maven only recompiles a program if the source is newer than the class file. So normally, the Eclipse compiler always "wins" the race and all the source files in the project will have been compiled by Eclipse. If there is strange behavior and you believe it is due to differences in the Eclipse and Maven compile options, you can recompile everything by adding "clean" to the Maven goals.

Running Maven Jobs Under Eclipse

At any time you can right click a POM file and choose "Run as ..." and select a goal (clean, compile, install). Rather than taking all the default configuration parameters, or entering them each time you run Maven, you can build a Run Configuration. Basically, this is a simple job with one step that runs Maven. The Run Configuration panel allows you to select every possible option, including the Java Runtime you want to use to run Maven, and a specific Maven installation so you can run some things under Maven 2 and some under Maven 3, plus the standard Maven options to not run tests or to print verbose messages. Since the Jenkins jobs each run one Maven POM, in the Sandbox you can build two Maven Run Configurations that duplicate the function of each Jenkins job.

First, lets review what the Jenkins jobs do.

  • The "Trunk Build" job checks the source project out of subversion. In this case, it is a "parent" project with subdirectories that all get checked out. It then runs a "mvn install" on the parent. The parent POM contains "module" XML elements indicating which subdirectory projects are to be run. Each subdirectory project generates a JAR or WAR artifact. The Jenkins Trunk Build installs these artifacts on the Artifactory server replacing any prior artifact with the same name and version number.
  • The Jenkins Install job checks the source of the Install project out of SVN. By Yale convention, each Install project is an Ant build.xml script and a package of parameters in an install.properties file created from the Parameters entered or defaulted by the person running the Jenkins job. Minimally the Install job downloads the artifact from Artifactory and copies it to the JBoss deploy directory, although typically the artifact will be unzipped, variables will be located and replaced by the values in the properties file, and then the file will be rezipped and deployed.

In the Sandbox you already have a copy of the CAS source files checked out in your Eclipse project in your workspace, and you can also check out a copy of the Installer project. So there is no need to access SVN. Similarly, in the Sandbox we do not want to mess up Artifactory, so the local Maven repository (which defaults to a subdirectory tree under the .m2 directory in your HOME folder) holds the generated artifacts.

With these changes, a Build project compiles the source (in the Eclipse workspace) and generates artifacts in the .m2 Maven local repostory. The Install project runs the Ant script with a special Sandbox version of the install.properties to copy the artifact from the .m2 local repository to the Sandbox JBoss Deploy directory (with property edits).

There is one last step. The Jenkins install job stops and starts JBoss. In the Sandbox you will manually start and stop the JBoss server using the JBoss AS Eclipse plugin. This plugin generates icons in the Eclipse toolbar to Start, Stop, and Debug JBoss under Eclipse. You Stop JBoss before running the Install job and then start it (typically using the Debug version of start) after the Install completes.

Now to the mechanics of defining a Run Configuration. You can view them by selecting the Run - Run Configurations menu option. Normally you think of using Run configurations to run an application, but once you install M2E there is also a Maven Build section. Here you can choose a particular Maven runtime directory, a set of goals ("clean install" for example), and some options to run Maven in a project directory. The recommendation is that on your sandbox machine you create two Maven Run Configurations. The configuration labelled "CAS Build" runs a Maven "install" or "clean install" goal on the parent POM of the CAS source project. The configuration labelled "CAS Install" runs Maven on the POM of the Installer project (and it has to run Maven 2 because this is still the standard for Installer jobs).

CAS Source Project Structure

In Maven, a POM with a packaging type of "pom" contains only documentation, parameters, and dependency configuration info. It is used to configure other projects, but generates no artifact. The parent "cas-server" directory is a "pom" project. All the subdirectories point to this parent project, so when the parent sets the Log4J version to 1.2.15 then all the subprojects inherit that version number from the parent and every part of CAS uses the same Log4J library to compile and at runtime.

A "jar" Maven project compiles Java source and combines the resulting *.class files and any resource data files into a zip file with an extension of ".jar". This is an artifact that is stored in the repository and is used by other projects. The most important CAS subproject is "cas-server-core", which compiles most of the CAS source and puts it in a very large JAR file. The other CAS subprojects compile optional JAR files that you may or may not need depending on which configuration options you use. Any JAR file created by one of these subprojects will end up in the WEB-INF/lib directory of the CAS WAR file.

A type "war" Maven project builds a WAR file from three sources:

  1. It can have some Java source in src/main/java, which is compiled and the resulting class files are stored in the WEB-INF/classes directory of the WAR.
  2. The files in the src/main/webapp resource directory contain the HTML, JSP, XML, CSS, and other Web data files.They are simply copied to the WAR.
  3. Any JAR files that are listed as a runtime dependency not provided by the container in the WAR project POM are copied from the Maven repository to the WEB-INF/lib directory.

Then the resulting files are zipped up to create a WAR artifact which is stored in the Maven repository under the version number specified in the POM.

The "cas-server-webapp" project is a standard JASIG project that builds a WAR file.

WAR Overlay

Maven has a second mechanism for building WAR files. This is called a "WAR Overlay". It is triggered when the POM indicates that the project builds a WAR artifact, but the project also contains a WAR artifact as its first dependency. A WAR cannot contain a second WAR file, so when Maven sees this it decides that the current project is intended to update the contents of the WAR-dependency rather than building a new WAR from scratch.

A WAR Overlay project can have Java source in src/main/java, in which case the source will be compiled and will be added to or replace identically named *.class files in the old WAR-dependency.

Mostly, the WAR Overlay replaces or adds files from src/main/webapp. This allows Yale to customize the CSS files or create special wording in the JSP and HTML pages.

JAR files listed as a runtime dependency of the Web Overlay project are added to the WEB-INF/lib directory along with all the other JAR files in the original WAR. In this case, however, replacing doesn't make any sense. A JAR file with the same name has the same Version number, so it will be a copy of the exact same file that is already there. Because WAR Overlay only replaces identically named files, you really do not want to have an artifact with one Version number in the WAR Overlay project and a different Version number in the old WAR because they cannot be merged and the result will be garbage.

The JASIG recommends using the WAR Overlay mechanism, and the Yale CAS customizations follow that rule. In the JASIG distributed source the WAR Overlay project subdirectory is named cas-server-uber-webapp, but Yale's version of this project is named cas-server-yale-webapp. This project produces the artifact named edu.yale.its.cas.cas-server-war with the version number that we track in Jenkins and Subversion as the actual CAS Server executable.

CAS has traditionally been created using the "WAR Overlay" technique of Maven. First, the cas-server-webapp directory builds a conventional WAR from its own files and dependencies. This is a generic starting point WAR file with all the basic JAR files and XML configuration. Then a second type "war" project is run that (in the standard JASIG CAS distribution) is called cas-server-uber-webapp. What makes it different from a standard type "war" Maven project is that the first dependency in this project is the cas-server-webapp.war file build by the previous step.

A "WAR Overlay" project depends on an initial vanilla prototype WAR file as its starting point. Everything in that WAR will be copied to a new WAR file with a new name and version, unless a replacement file with the same name is found in the WAR Overlay. When Yale or any other CAS customer is building their own configuration of CAS, the WAR Overlay project directory provides a convenient place to store the Yale configuration changes that replace the prototype files of the same name in the previously run generic cas-server-webapp type "war" project.

If you made no changes at all to JASIG code, the WAR Overlay project is actually the only thing you would need to check out and edit. Yale would never need to locally compile any CAS source, because the JASIG repository version of the cas-server-webapp.war artifact for that release version number would be perfectly good input to the WAR Overlay process. However, at this time we include all the JASIG source and recompile the JASIG code every time we build the CAS source trunk. If that is a small waste of time, it prepares the structure so we can fix JASIG bugs if they become a problem.

Source Build

CAS is distributed as a cas-server directory with a bunch of subdirectories and a POM file. The subdirectories are JAR and WAR projects. The POM file contains a certain amount of boilerplate for JASIG and the CAS project, plus two things that are important. The parameters and dependency management section ensure that all elements of CAS are compiled or built with the same version of library modules (quartz for timer scheduling, hibernate for object to database mapping, etc.). Then a list of <module> statements indicate which subdirectory projects are to be run, and the order in which they are run, when Maven processes the parent POM. The possible modules include:

  • cas-server-core - This is the single really big project that contains most of CAS Server code and build the big JAR library artifact. It is built first and all the other projects depend on it and the classes in the JAR file.
  • cas-server-webapp - This is a dummy WAR file that contains the vanilla version of the JSP, CSS, HTML, and the XML files that configure Spring and everything else. The dependencies in this project load the basic required JAR libraries into WEB-INF/lib. This WAR is not intended to install or run on its own. It is the input to the second step WAR overlay where additional library jar files will be added and some of these initial files will be replaced with Yale customizations.
  • Additional optional extension subprojects of CAS supporting specific features. For example, the cas-server-support-ldap module is needed if you are going to authenticate Netid passwords to AD using LDAP protocol (you can authenticate to AD using Kerberos, in which case you don't need this module). A JASIG standard distribution includes all possible subprojects and the parent cas-server POM file builds all of them. To save development time, you can comment out the subproject <module> statement in the parent POM of modules you don't use. Disk space is cheap, and deleting the subdirectory of modules you don't use may be confusing. 
  • Yale CAS extension JAR projects. For example, the cas-server-expired-password-change module is the Yale version of what CAS subsequently added as a feature called "LPPE". Yale continues to use its code because the Yale Netid system involves databases and services beyond simply the Active Directory component and support
  • cas-server-uber-webapp - This is the "sample" version of the local customization WAR Overlay project. This directory may be renamed to match the artifact name that the POM builds, which is also the artifact name spedified in the Jenkins Install job. However it is named and wherever it is located, Yale customization is (unless it is impossible) stored in the WAR Overlay project directory. If we need Java code, we put this source here and it is compiled and ends up in WEB-INF/classes. We replace some standard JSP and CSS files to get the Yale look and feel and wording. The XML files that configure Spring and therefore CAS behavior are here. This project must always be the last module executed. The cas-server-webapp project is a dependency, so it is the base WAR file that will be updated. This project replaces JSP, CSS, and XML files in the base WAR with identically named files from this project. It adds new files and library jars (from the dependency list of this project). Details will be described below, but this is where all the action happens.

Generally you run the vanilla JASIG cas-server project POM once on your sandbox machine and a target of "install". This compiles all the source files, builds all the JAR and WAR files, and stores them in the local Maven repository on the machine. There are other ways to get the artifacts from the Yale Artifactory server or from JASIG, but compiling them is the easiest way and you probably need the project source anyway. After that, you technically only have to run the WAR Overlay project unless you make a change to any JASIG projects. However, in practice you may find it necessary to change the Version numbers in the parameters and dependency management section of the parent POM if you discover that the JBoss server you deploy to wants a different version of any standard library. Then you want to recompile everything to maintain sanity.

The WAR Overlay project can contain new dependency statements that include JAR files not used by JASIG CAS, but it should never try to introduce a different version number of a library that CAS has already introduced. The Overlay mechanism replaces identically named files, but it will not notice that quartz-1.6.1.jar in WEB-INF/lib from the cas-server-webapp prototype project is a different version of the same library as quartz-1.6.2.jar in the WAR Overlay project.

There are now two ways to organize the projects. You can put everything in the cas-server directory, starting with the JASIG subdirectories and then adding the Yale subdirectories. Then adjust the parent POM in cas-server to build what you want to build each time you run the Build job under Eclipse. Alternately, you can separate the entire JASIG source tree from the Yale project directories. You can also build the JASIG artifacts first with one job and then build the Yale artifacts with a second job, or you can create a single job that builds both at once. There are advantages and disadvantages to either approach, and on a sandbox feel free to use whichever structure seems most comfortable.

It is a common Maven convention to make the parent POM a parent directory and the <module> POMs listed in the parent subdirectories of the parent. However, the child projects reference the parent by the artifact name and version from the local Maven repository, and the parent can reference modules by path expressions. So any arrangement of directories can be made to work. However, remember that the Jenkins "trunk build" job reads Maven source projects checked into Subversion, compiles them, and stores them as artifacts in the Yale Artifactory server. So if at the end of development you end up checking in two directory trees to the Yale SVN repository, you are going to need to Jenkins build jobs.

The POM in a vanilla JASIG component subproject will have a Version number that corresponds to CAS releases (3.5.2). The POM in a Yale customization subcomponent will have a Version number that corresponds to the Yale CAS installation "Release" number for Jenkins tracking (1.1.0-SNAPSHOT). The artifact name and version number in the WAR Overlay project must match the artifact name and version number of the Jenkins Install job parameters.

The Installer

Yale has a convention that there is a separate Install job. This job reads a Maven project from SVN that contains parameters and an Ant script. Maven runs Ant, and Ant downloads the artifact from Artifactory (in this case, the WAR file produced by the WAR Overlay project) and copies it with some variable substitution to the deploy directory of JBoss.

The Installer is driven by an install.properties file. This file contains passwords used to access AD and databases, so it is not checked into SVN. Get a copy of this file saved by the last person who worked on CAS saved in some more secure or private file area and add it to the project directory after checking the Installer project out of SVN.

Standalone.xml

In theory, the developer of a project does not have passwords for the production databases, so runtime parameters are provided from a source outside the WAR. A common Yale convention was to build XML files defining each database as a JBoss Datasource and creating a service that stuffed a group of parameters into the JBoss JNDI space where they could be read at runtime using the built in java.naming classes. The Install job used to build these XML files.

However, the JBoss EAP 6.1 rewrite has changed the JBoss best practice. Now both Datasources and JNDI values are stored in the main "subsystem" configuration file. In the sandbox, this is the standalone.xml file in the /config subdirectory of the JBoss server. You need to get a copy of an appropriate configuration file for your JBoss and install it yourself. Production services will be responsible for providing the file in DEV, TEST, and PROD, but you need to notify them if you add something during sandbox development so they can also add it to the versions they manage.

To Work on Current Release

Check out from SVN the current version of the cas-server source. (If the Yale source is in a separate directory check it out too, but it has probably been added as subdirectories of the cas-server directory.) Also check out the Installer project directory. Get a copy of install.properties for the Installer project and make sure the jboss.deploy.dir points to the location where you put JBoss on your sandbox.

If it is at all possible, make changes only to the WAR Overlay project subdirectory.

Sometimes you need behavior that is just slightly different from an existing JASIG version of some bean. The design of CAS is to have standard interfaces, classes that implement these interfaces, and configuration done with Spring XML files that select the appropriate implementation class using a <bean> element. If the class you want to change is configured this way to Spring (search for its class name in XML files), then you can often make a copy of the JASIG source in the WAR Overlay src/main/java tree, changing the org.jasig package name to edu.yale and so on. Then change the behavior of the source you just copied. Now go to the XML configuration that references the original class by name and change the fully qualified classname to match the new class you just created. Typically the interfaces don't change between releases, so you may not have to update your modified source when you migrate to a new release.

What cannot be changed in the WAR Overlay project? If a class or interface is referenced by import in Java source, then the Java mechanism to search for a class by name (the "ClassLoader") typically searches first for a class of the desired name that came from the same source as the class that is looking for it. JBoss regards the WEB-INF/lib and the WEB-INF/classes as two separate sources. Although WEB-INF/classes is said to be "searched first", when some code in cas-server-core is looking for a class name it will find the version of that name that is also in cas-server-core and will not find something with the same name in WEB-INF/classes. So in general you cannot simply copy any org.jasig source to the WAR Overlay source directory, compile it, and then assume that the version you just compiled will override the original code because it is in the "classes" directory and "classes is searched before lib". Think really hard if you need to change something here, but then if you really need to change it you need to make a modification to the original JASIG source and then copy and reconcile that change to each subsequent release.

  • No labels