Content Comparison

...

1. The AD Daily Updater, a Java program driven by the Oracle ACS1 database, which populated a select set of "biographical" fields (firstname, lastname, job title) in the AD for non-private people. None of these fields were important from a systems system point of view, except for UPN which is an identifier to login to services.
2. The Mail Relay configuration, which maps Yale Email Aliases (ending where all the mail ends in "@yale.edu") to the native email addresses of Yale systems. Originally this meant "Netid@connecta "native" Email system suffix that distinguishes the specific mail system for that account  (ending in "@connect.yale.edu" for Exchange accounts and "Netid@pantheonO365 or "@bulldogs.yale.edu" for IMAP. Current Exchange has moved to O365 and IMAP has been replaced by Eliapps.Eliapps). 
3. The Exchange mail routing fields in AD (MailNickName and ProxyAddresses). When mail was sent Once the Mail Relays sent mail to Exchange, it delivered mail addressed to various Email Aliases using data stored in two fields of the AD. A separate Exchange field management function in the AD Daily Updater maintained these two Exchange Mail Routing fields/O365, each piece of mail had to be delivered to a specific Inbox. Microsoft mail configuration is stored in the AD, and these two fields determine all the different email addresses that might be delivered to the same Inbox.

This new project continues to support the biographical fields of AD, but now gets the data from IIQ instead of the old Oracle database. It updates prepares for the logic of function 3) to reflect differences between the original on premise Exchange and the O365 Azure AD requirements, and merges the function 2) code in by populating the MailNickname, ProxyAddresses, and now Target Address field of an AD object for Eliapps and legacy departmental Email Aliasesretirement of the Mail Relay system by populating the Email Routing fields in AD with data for both O365 and Eliapps instead of just O365.

We have to plan ahead for expected changes that do not currently have a schedule:

• The on premise Exchange system serves no useful purpose. It should be decommissioned. This would involve removing certain Exchange related fields from the on premise AD (fields that begin with the "msExch" prefix. The old AD Daily Updater code depends on these fields, but the new replacement code will be written to work if they are null. The one exception is that Private users are excluded from the Outlook directory with two of these fields and that function still needs to be supported.
• The Mail Relay machines will at some point be decommissioned and the Exchanged Online function in the Azure Cloud will take over routing all mail addressed to an "@yale.edu" Alias. Since Excahnge Online is configured by Azure AD in essentially the same way that on premise Exchange was configured by on premise AD, we have to add AD Email Aliases to the ProxyAddresses list and the native Email address (MAILBOX) of the aliases to the TargetAddress field for existing User objects or new Contact objects to be created for the purpose.

Currently mail addressed to an "ALIAS_NAME@yale.edu" Email address goes to the Mail Relay machines. They look up the ALIAS_NAME in configuration data and generate a native Email address that in practice sends the mail to O365 or Eliapps (although there are a few legacy entries that may go to deparmental standalone mail servers). If a Primary Email Alias points to a native Email Address ending in "@bulldogs.yale.edu" then the mail goes to Eliapps.

However, when the Mail Relays go away and are replaced by Exchange Online, then what was once a two tiered processing becomes a single step, and Exchange gets first dibs on the mail. Because Exchange Online puts its configuration data in the Azure/Local AD, some rules about the way AD has to work can conflict with specific settings in the Email Alias table. Fortunately, the current configurations that will become invalid are also in violation of Yale policy. This will not, however, avoid the requirement to communicate the change to affected users.

There are a set of Yale and Exchange rules:

• It is an AD rule that the UserPrincipalName must be unique.
• It is a Yale rule that for Users with a mail account, the Primary Email Alias (first.last@yale.edu) also be the UserPrincipalName and its prefix is the MailNickName (first.last). Users marked "Private" and Users without mail get a UPN of "Netid@yale.edu". These are the only two possible values.
• It is a Yale rule that everyone with an ITS mail account (O365 or Eliapps) must have a Mail Alias that points to that account. (Everyone with any Mail Alias is supposed to have a Primary Alias, but that rule has been broken and the data would have to be cleaned up to enforce it.)system exists only to support system tools that use it. The old AD Updater depended on it and is one of the things that had to change before we could decomission something that performs no real business service.
• The Mail Relay machines will at some point be decommissioned and the Exchanged Online function in the Azure Cloud will take over routing all mail addressed to an "@yale.edu" Alias. Since Exchange Online is configured by Azure AD in essentially the same way that on premise Exchange was configured by on premise AD, we have to add ProxyAddresses to some AD object (User or Contact) for Email Aliases that point to Eliapps (and a few legacy departmental servers).

The Mail Relays translate any given "@yale.edu" Email address to some system specific address. They are even handed and treat O365 and Eliapps equally. Exchange Online, however, is mostly a mail delivery system that will forward mail to Eliapps or some other external mail system only in the event that it cannot be delivered to any O365 mailbox. When a user has both O365 and Eliapps accounts, the "bias" that Exchange Online has toward O365 will result in some mail being delivered to the O365 Inbox that would previously have been delivered to Eliapps. There is a structural feature of the Microsoft system and we cannot work around it. Other than this limitation, we design the new code so that every piece of mail will be delivered by the new system to the same location where the old system sent it.

There are a set of Yale and Exchange rules. We can't change the Microsoft rules, and we are not prepared to change Yale rules established long ago. Enumerating the rules explains exactly how and why certain mail routing will change.

• It is an AD rule that every User must have a unique UserPrincipalName. The UPN can be used to login to Windows and must be used to login to Azure.
• It is a Yale rule that for Users with a mail account, the Primary Email Alias (first.last@yale.edu) is also the UserPrincipalName. The ALIAS_NAME (before the "@") is the MailNickName (first.last). For Users marked Private the ALIAS_NAME is the Netid, and for old accounts after the date where they are supposed to be deleted, the ALIAS_NAME is also set to Netid to free the first.last for new people.
• It is a Yale rule that everyone with an ITS mail account (O365 or Eliapps) must have a Mail Alias that points to that account. As a result, the Email Alias table provides a complete list of Yale mail accounts in both systems.
• It is an Exchange Online rule that for Users with an O365 mail account, the UPN has to be a deliverable Email address to that account. With Mail Relays, the UPN can point to either O365 or Eliapps, but if Exchange Online does all the mail routing, any mail addressed to the Primary Email Alias (which is also the UPN) will go to O365 if there is an O365 mailbox even though the Email Alias table points to Eliapps.
• It is an Exchange Online rule that the UPN has to same Email address cannot be in the two ProxyAddress list of a mail User object (that is, the UPN has to be a deliverable mail address to anyone with an O365 mail account).
• It is an Exchange Online rule that the same Email address cannot be in two ProxyAddress lists of two Azure AD objects.

Yale policy is to give users either O365 or Eliapps mail but not both, but exceptions are allowed. Yale policy is that when a user has both O365 and Eliapps mail, the O365 account is Primary. There are a small number of case where that is not true, but when you combined all the rules we will no longer be able to allow exceptions. Exchange Online will force the O365 account to be Primary even if the Email Alias table says that Eliapps is supposed to be Primary.

Except when one of these structural rules forces a change, the objective of this design is to deliver mail with the same Email address to the same mailbox it was delivered to with the old Updater and the old Mail Relays. That is, changes may be forced by structural differences but they will not happen by coding accidents.

It should generally be possible to accomplish exactly the same mail delivery for all addresses, but this may require a slightly complicated rearrangement of Aliases and the AD. For example, you can rename AD objects so that an O365 Mail account that previously belonged to a personal Netid now appears to belong to a dependent Netid, and point the old alias to it. This frees the personal Netid User object to point to Eliapps, while the O365 account, now on the dependent Netid, continues to get mail through all the old addresses. However, this is a very delicate operation and will not be done for any but the most important users.

Taking all of these requirements into consideration, data from the Alias table will be arranged by a database view to create one of four situations:

...

• lists of two Azure AD objects.

Yale policy is to give users either O365 or Eliapps mail but not both, but exceptions are allowed. Yale policy is that when a user has both O365 and Eliapps mail, the O365 account is Primary. So if we followed our own rules there would be no problem. Unfortunately, either because accounts were "grandfathered" in from previous systems or because someone important asked for an exception, there are a handful of cases where the Primary Email Alias points to Eliapps but that address will begin to deliver outside mail to O365 after we decommission the Mail Relays.

This is not an AD Updater problem. It is a Mail System restriction that we cannot program around and will be ignored by the coding. We will do what we can do. There are ways to change the Email Aliases with Dependent Netids that can work around the problem, but that again is an Email System trick and not part of this project. However, understanding how mail delivery is done and how it changes is important to understanding the design, and this particular glitch is a useful example to motivate a description of the environment to which this code must be designed.

The big design feature that derives from this explanation, is that when the Alias table is read in by IIQ (is "aggregated" in IIQ terms) it is presented as one of four different configurations of object data by a database view that coverts the flat raw table into a usable set of database "rows" (which become program objects):

1. Unchanged User object - Netids with Primary O365 accounts and no Eliapps account (most employees, including faculty). All the Aliases to the O365 account are grouped and presented as an object keyed by the MAILBOX ("Netid@connect.yale.edu"). By extracting the Netid from the MAILBOX name, IIQ correlates this to an Identity and therefore an AD User object. All the ALIAS_NAMES for that MAILBOX become ProxyAddresses. This should produce the exact same result (except for the order of the aliases in the list) as the current AD Daily Updater, so we expect no change to the AD for this type of account.
2. User object with Eliapps routing - Netids with a Primary Eliapps account and no O365 Mail (although non-mail O365 licenses) will have the mail routing data in the Alias Table turned into fields of the AD User object. The MAILBOX column value from the Primary Alias row is used to find the Secondary Aliases of the same account, and it also becomes the TargetAddress. The matching column values that share that MAILBOX will become ProxyAddresses-mail O365 licenses). Adopting the exact same logic as the previous case, except that the MAILBOX is now (AliasName@bulldogs.yale.edu), this populates the ProxyAddresses list with aliases to the Eliapps account, and sets the TargetAddress to the MAILBOX value. Initially this does nothing, but when the Mail Relays are decommissioned, it routes Eliapps mail to Google. Because Netid is not in the MAILBOX, the Netid field in the Alias Table has to be used to find the right Identity to which this data should be attached.
3. Unchanged User object and Contact with Eliapps Routing - Users with a Primary O365 account and a Secondary Eliapps account will find their User object unchanged, but a new Contact object will be created with the same TargetAddress and ProxyAddresses values described in the previous case, but now applied to the new Contact. The O365 alias data will generate the same data as in Case 1 above. The Eliapps aliases will generate a second row with the Case 2 data, but because it is a secondary Email account, IIQ has to create a second Contact object in AD to hold the Eliapps data since the User object is busy holding the O365 data.
4. Routing "Junk" Contact objects - Entries in the Email Alias table that were not processed to generate a ProxyAddress in one of the previous three steps are grouped by unique MAILBOX value. Each MAILBOX value creates a "junk" Contact whose ProxyAddresses are the list of ALIAS_NAMEs that had that MAILBOX value. Note that this method of collection may combine Aliases owned by different Netids if they point to the same MAILBOX, but this is OK.

...

Any user can have Junk, so while each mail user Netid is either Case 1, 2, or 3, they can also have any number of Type 4 entries. Since Type 4 is grouped by MAILBOX and not owner, several Netids can share a Type 4correlated to a Contact identified by the MAILBOX. It is not connected in any way to the Netid that owns the Alias.

Users with an O365 or Eliapps account can delete it, and users with one type of account can get the other. So you can transition between Type 1 and 3, or 2 and 3. This is a delicate process because IIQ cannot directly move a field like ProxyAddresses from a User object to a Contact object in a single transaction (as would be possible with database SQL). This is, however, a manual process and is outside the scope of the current designbecomes a function of the Mail System people. This code does not provide transition services but will provide documentation about how to make the transition correctly.

There are 555 Primary Email Aliases in the Alias table where the MAILBOX is "first.last@connect.yale.edu" instead of "netid@connect.yale.edu". These have to be fixed for the new code to work.

...