...

Yale intends to decommission the Mail Relay machines and replace their function with the Exchange Online component of Azure/O365. The Mail Relays are configured with data from the Email Alias system. They receive mail with a destination address ending “@yale.edu”, process the mail through SPAM/security scans, then forward the mail to the computer or mail system associated with the MAILBOX field of the Email Alias.

If we replace the Mail Relays with Exchange Online, we have to create configuration for Exchange (in Azure AD) that provides to Exchange a mapping from the the Email Alias table ALIAS_NAME field to the MAILBOX field. This means creating some AD object where the ALIAS_NAME is in the ProxyAddresses property list and the MAILBOX is the TargetAddress property.

No changes are required to the configuration of email addresses for mailboxes already managed by Exchange, including personal O365 mail accounts, distribution lists, shared mailboxes, and mail addresses in any mail domain managed by or aliased to Exchange Online. By definition, Exchange Online must already be configured to deliver such mail to the right mailbox, and that existing mapping will work the same for mail sent directly to Exchange Online as it does when the mail passes through the Mail Relays and then goes to Exchange Online.

The most common required configuration is for personal Eliapps Email accounts belonging to students. They have a Primary Email Alias where the MAILBOX is ALIAS_NAME@bulldogs.yale.edu. It is a Yale practice to put students enrolled in any class into an Exchange Distribution List for that class. We therefore needed an AD object with the TargetAddress equal to the Eliapps MAILBOX to add to the Distribution List Group. About a year ago this TargetAddress property was added to the AD User object for current active student Netids, and the Primary ALIAS_NAME was added to the ProxyAddresses list of the User object.

However, for the purpose of Canvas class distribution lists, it was not necessary to maintain these User objects across name changes. Since then, 52 students have changed their name and their Email Alias, and for this project we should begin to reflect Alias changes to the AD User object.

The changes made last year for Students highlights an important point about the Mail Relay retirement project. We can add correct mail routing information to the AD at any time before the Mail Relays are removed. When we do that, Exchange Online now knows to send mail from an O365 user directly to an Eliapps mailbox without going through the Relays. We can test a configuration change by sending mail from O365 to a reconfigured alias to verify it arrives correctly. After Exchange has been configured with every external alias, then at any time after that we can change “@yale.edu” to go to Exchange instead of the Mail Relays as the final step.

We did not add secondary Email aliases to the ProxyAddresses list. The most common source of secondary aliases is that they are the old name of a student after a name change. In addition to the 52 student who changed their name in the last year, there were more students who had already changed their name before the AD User objects were populated with Eliapps student account information. As long as the old alias remained in the Alias Table, the Mail Relays would forward mail for that old name to the Eliapps account, and if we are going to replace the Relays then we have to duplicate that function by adding all the ALIAS_NAME values (Primary and Secondary) to the ProxyAddresses list of the student’s User object.

Every Email Alias in the table has to be handled. Once you remove the Exchange Online MAILBOX values and the personal student Eliapps accounts that are already in the AD, there are a few common patterns that fall through the cracks:

There are Eliapps accounts belonging to students who were not taking classes last year but who still have active mail. Remember, last year we were looking for anyone who could go into a Canvas class distribution list. Now we have to replace the Mail Relay routing function, and there are about 1000 more Eliapps student mail accounts that need to be inserted into the AD User object of those Netids.

There are Eliapps accounts belonging to people (mostly in ITS) who have both O365 and Eliapps mail. The O365 mail account occupies their User object, so we cannot put the Eliapps mail routing in that object as we do for student Eliapps users, so we need to create a Contact object for these accounts.

There are Eliapps accounts for mailing lists and departmental shared mailboxes. These are “@elilists.yale.edu”, Yale student groups like “yale.spizzwinks@bulldogs.yale.edu”, and research projects like “thechildlab@bulldogs.yale.edu”. They also get Contacts.

In the end, for every row in the Email Alias Table (DIR_ONLINE_INFO in ACS1) one of two things must be true:

• The MAILBOX is a special object like a Distribution List of Shared Mailbox configured to Exchange Online.

• There is an object (User or Contact) for which the ALIAS_NAME is an entry in the ProxyAddresses list of the object and the MAILBOX is the TargetAddress.

After we make the various changes to AD User and Contact objects, a program will run through the table and verify that every Alias maps the same way through AD as it did through the Aliases (and therefore Mail Relays).

We cannot preserve the behavior of a handful of personal aliases where a user has created mailboxes with the same name in both O365 and Eliapps and has set the Primary Alias to point to Eliapps. With the Mail Relays, mail sent by O365 users goes to the O365 mailbox and all other mail goes to Eliapps. When the Relays are retired, all mail will go to O365. The old behavior cannot be duplicated, but this configuration violates Yale policy so we will simply inform the affected users before the changeIf the MAILBOX ends in “@connect.yale.edu” the mail is forwarded to Exchange Online, while a MAILBOX ending in “@bulldogs.yale.edu” goes to Google. Other values have to be looked up in a table that corresponds to the MX records we configure in DNS.

If we remove the Mail Relays, all “@yale.edu” mail will go directly to Exchange Online. Exchange will deliver mail to its own mail accounts first. Then for every address it does not match to an internal destination, it will search tables built from Azure AD to decide where to forward that piece of mail.

The configuration of external mail aliases in Exchange Online is done through two properties in the Azure AD. The TargetAddress property of a Azure AD object contains the MAILBOX value from a row of the Email Alias table. The ProxyAddresses property contains a list of Email Addresses that are to be forwarded on to the account in the TargetAddress.

The Azure AD object that contains these two properties can be a User or a Contact. Exchange doesn’t care, but it makes sense to us to reuse existing User objects of students whose Primary Email Alias points to an Eliapps account and who have no O365 account. In this case the ProxyAddresses list contains the Primary Alias name and any Secondary Alias names that resolve to the same MAILBOX value. Everything else generates a Contact.

These Contacts are strictly for internal routing configuration purposes. They are not published in any directory and are used only by Exchange Online to forward mail. While we could create one Contact object from each row of the Email Alias table that is not represented by a ProxyAddress/TargetAddress pair in a User object, we have decided to group all the Alias Table entries that resolve to the same MAILBOX value as entries in the list of ProxyAddresses of a single Contact object named for the common MAILBOX.

Except for the aliases that resolve to servicenow@bulldogs.yale.edu. There is a limit of 400 entries in the ProxyAddresses list of an object, and currently 111 aliases resolve to that MAILBOX. While there is clearly some room for expansion, this policy could lead at any time to someone creating 300 new aliases for servicenow, and then any code we wrote would break. So these aliases have to be handled specially, as would any new system that wants to create a similar arrangement.

Exchange Online requires that entries in any ProxyAddresses list must be unique. This process generates them from the ALIAS_NAME column of the Email Alias table which is constrained to be unique.

Because the Azure AD already has to be properly configured for O365 personal and shared mailbox accounts (or Exchange would not be able to deliver mail to these destinations), there is no need to make any changes for Email Alias Table entries that resolve to the “@connect.yale.edu” mail domain or any other domain that is an alias of Exchange itself.

For a year, we have been populating the TargetAddress field of the User object of 15000 current active or new students with an Eliapps mail account. There are approximately 1000 additional User objects for what appear to be former students who still have active mail whose User objects will now be populated.

Previously it was unnecessary to update the TargetAddress when a student changes their name. Technically this is still not necessary (because the old name remains an alias of the Eliapps account) but it seems to be better if this field is updated when the MAILBOX of the Primary Email Alias is changed. In addition, we were not previously adding Secondary Aliases to the ProxyAddresses list but will add them now. It may be unlikely that anyone will receive mail under an old name changed three years ago, but as long as the old name remains in the Alias Table, this project must configure Exchange Online to deliver mail to all currently valid Aliases no matter how unlikely it is that they are actually in use.

Therefore, there is a simple test to verify that the configuration required by this project is complete. Run through all the rows in the Email Alias Table (DIR_ONLINE_INFO). If the MAILBOX is one of the Exchange Online domains, ignore it. Otherwise, look for a ProxyAddresses entry matching the ALIAS_NAME value in some AD User or Contact object. The TargetAddress property of that object must match the MAILBOX value of the Alias.

Contacts are created for Aliases that are not Exchange accounts and are not Primary Aliases of Eliapps students. It is not necessary to say what these accounts are, but in case you want to know, they are mostly:

• Eliapps accounts of people who also have an O365 mail account. These are mostly ITS people.

• Elilists (a type of mailing list managed by Eliapps instead of Exchange)

• Student groups (example: yale.spizzwinks@bulldogs.yale.edu)

• Departmental shared mailboxes (example: thechildlab@bulldogs.yale.edu)

• The 111 aliases of servicenow@bulldogs.yale.edu

• Mail for specific department servers (invest.yale.edu, aya.yale.edu)

• Everything else. Sometimes this is recognizable junk left over from machines that have not existed since 1995 (gopher.cis.yale.edu) but it is not necessary to clean this up and that is not part of this project.

We cannot preserve the behavior of a handful of personal aliases where a user has created mailboxes with the same name in both O365 and Eliapps and has set the Primary Alias to point to Eliapps. With the Mail Relays, mail sent by O365 users goes to the O365 mailbox and all other mail goes to Eliapps. When the Relays are retired, all mail will go to O365. The old behavior cannot be duplicated, but this configuration violates Yale policy so we will simply inform the affected users before the change.

Transition

After these changes are made to the ProxyAddresses and TargetAddress of existing User objects and new Contacts are created but before the Mail Relays are retired, the only change in behavior is that mail sent from an O365 mail account will now go directly from Exchange Online to Google or any other external service referenced by a TargetAddress. Previously the mail would go from Exchange Online to the Mail Relays and then be forwarded to Google or other services. The generated AD entries must be correct for this mail to be delivered properly.

Then later on we turn off the Mail Relays. Now the generated AD configuration must be complete for the mail to be delivered properly.

Fine Tuning

This project has to refine the list of email domains that are inherently associated with Exchange Online. We know that “@expectwithme.org” is actually an O365 alias, but while ayalists.yale.edu is in Exchange, elilists.yale.edu is in Google, and “lists.architecture.yale.edu” is uncertain. This is not just an IAM problem, because Linux Systems has to fix the MX records for each of these domains.

There are data errors in the Alias Table that will generate unnecessary Contacts. It is not necessary to fix them at this time, but if we don’t do it now, there will be no better time in the future.

Source of Problems (History)

Originally Yale did not think of Email as a single topic. There was the problem of Student Mail, addressed by the Instructional Support group of ITS. Then there was Employee mail decided by the business focused groups. Once a system was selected, there was some involvement by Unix or Windows groups for technical supportdo it now, there will be no better time in the future.

Source of Problems (History)

Originally Yale did not think of Email as a single topic.

There was the problem of Student Mail, addressed by the Instructional Support group of ITS. Then there was Employee mail decided by the business focused groups. Once a system was selected, there was some involvement by Unix or Windows groups for technical support.

However, SMTP is a service that you could run on any mainframe, Unix, or Windows NT computer. At one point, main departments ran their own mail servers. The original design of the Email Alias system allowed a user to create an Alias that pointed to the computer under there desk. (So many of these machines were compromised by spammers that this became a violation of Yale security policy, but it was too late to change the Alias system).

IAM became involved for three reasons.

• Historically, there was a time when we charged back mail accounts to departments, and IAM inherited the systems that were created for that purpose even after we stopped charging for mail accounts.

• IAM creates basic resources like a Netid and AD object for new students, employees, and sponsored identities. Students and employees automatically get mail accounts, and it was efficient for IAM to create these “birthright mail” accounts when a new identity is created.

• What people think of as the Email Address is also a unique login string (a “principal” name) for the non-mail functions like OneDrive, Google Drive, and Box. IAM has to manage it as a login name and, among other things, lock and unlock it along with the Netid.

...

A student named Johnathan Doe is initially assigned a Primary Alias of “johnathan.doe” and an Eliapps account name of “johnathon.doe@yale.edu” which means the Alias MAILBOX value is “johnathon.doe@bulldogs.yale.edu”. Then he changes his name to “John” and requests his Email get “fixed”.

...

If this is done correctly, then both the new Primary “john.doe” and the secondary old “johnathan.doe” Alias Table entries have a MAILBOX of “john.doe@bulldogs.yale.edu”. Then we know that these are two aliases of the same account. However, because both the new and old names are in the Eliapps User Directory, mail does get delivered correctly if someone is sloppy and does not change the MAILBOX on the old now Secondary Alias.

If Primary ALIAS_NAME “john.doe” has MAILBOX “john.doe@bulldogs.yale.edu” and Secondary ALIAS_NAME “jonathan.doe“ has MAILBOX “jonathan.doe@yale.edu”, there is no way in the current system to plausibly realize that they are the same mail account. Google does not like us to programmatically do a lot of queries to the User Directory, so for a large number of users we cannot verify that the two MAILBOX values point to the same Eliapps account. We will put the Primary account in the AD User object of this Netid, but if the old name has a different MAILBOX value, it is going to look like a second mail account and it will generate a Contact.

The mail is delivered properly, but the AD has an unnecessary object caused by poor procedures or a lack of specific tools to perform common operations in a standard manner. So frequently enough we get:

Google makes it unreasonably inefficient to query them and determine that “john.doe@yale.edu” and “johanthan.doe@yale.edu” are two names for the same Eliapps account, and we do not maintain a local table that remembers that we renamed the account. So from what information we have locally on which we can build code, there is no way to know that these are two entries for the same account. That means that the Secondary Alias will generate a Contact object. This is not a problem because this user probably only wants the current Primary alias listed in the directory and the only need for the secondary alias to to make sure that any mail for old addresses also gets delivered.

Technical Information

The following can be read by anyone, but it is directed to technical staff who have to maintain this stuff in the future. You should decide if you want to know more about how mail is delivered.

...