Executive Summary

Yale intends to decommission the Mail Relay machines and replace their function with the Exchange Online component of Azure/O365. The Mail Relays are configured with data from the Email Alias system. They receive mail with a destination address ending “@yale.edu”, process the mail through SPAM/security scans, then forward the mail to the computer or mail system associated with the MAILBOX field of the Email Alias. If the MAILBOX ends in “@connect.yale.edu” the mail is forwarded to Exchange Online, while a MAILBOX ending in “@bulldogs.yale.edu” goes to Google. Other values have to be looked up in a table that corresponds to the MX records we configure in DNS.

If we remove the Mail Relays, all “@yale.edu” mail will go directly to Exchange Online. Exchange will deliver mail to its own mail accounts first. Then for every address it does not match to an internal destination, it will search tables built from Azure AD to decide where to forward that piece of mail.

The configuration of external mail aliases in Exchange Online is done through two properties in the Azure AD. The TargetAddress property of a Azure AD object contains the MAILBOX value from a row of the Email Alias table. The ProxyAddresses property contains a list of Email Addresses that are to be forwarded on to the account in the TargetAddress.

The Azure AD object that contains these two properties can be a User or a Contact. Exchange doesn’t care, but it makes sense to us to reuse existing User objects of students whose Primary Email Alias points to an Eliapps account and who have no O365 account. In this case the ProxyAddresses list contains the Primary Alias name and any Secondary Alias names that resolve to the same MAILBOX value. Everything else generates a Contact.

These Contacts are strictly for internal routing configuration purposes. They are not published in any directory and are used only by Exchange Online to forward mail. While we could create one Contact object from each row of the Email Alias table that is not represented by a ProxyAddress/TargetAddress pair in a User object, we have decided to group all the Alias Table entries that resolve to the same MAILBOX value as entries in the list of ProxyAddresses of a single Contact object named for the common MAILBOX.

Except for the aliases that resolve to servicenow@bulldogs.yale.edu. There is a limit of 400 entries in the ProxyAddresses list of an object, and currently 111 aliases resolve to that MAILBOX. While there is clearly some room for expansion, this policy could lead at any time to someone creating 300 new aliases for servicenow, and then any code we wrote would break. So these aliases have to be handled specially, as would any new system that wants to create a similar arrangement.

Exchange Online requires that entries in any ProxyAddresses list must be unique. This process generates them from the ALIAS_NAME column of the Email Alias table which is constrained to be unique.

Because the Azure AD already has to be properly configured for O365 personal and shared mailbox accounts (or Exchange would not be able to deliver mail to these destinations), there is no need to make any changes for Email Alias Table entries that resolve to the “@connect.yale.edu” mail domain or any other domain that is an alias of Exchange itself.

For a year, we have been populating the TargetAddress field of the User object of 15000 current active or new students with an Eliapps mail account. There are approximately 1000 additional User objects for what appear to be former students who still have active mail whose User objects will now be populated.

Previously it was unnecessary to update the TargetAddress when a student changes their name. Technically this is still not necessary (because the old name remains an alias of the Eliapps account) but it seems to be better if this field is updated when the MAILBOX of the Primary Email Alias is changed. In addition, we were not previously adding Secondary Aliases to the ProxyAddresses list but will add them now. It may be unlikely that anyone will receive mail under an old name changed three years ago, but as long as the old name remains in the Alias Table, this project must configure Exchange Online to deliver mail to all currently valid Aliases no matter how unlikely it is that they are actually in use.

Therefore, there is a simple test to verify that the configuration required by this project is complete. Run through all the rows in the Email Alias Table (DIR_ONLINE_INFO). If the MAILBOX is one of the Exchange Online domains, ignore it. Otherwise, look for a ProxyAddresses entry matching the ALIAS_NAME value in some AD User or Contact object. The TargetAddress property of that object must match the MAILBOX value of the Alias.

Contacts are created for Aliases that are not Exchange accounts and are not Primary Aliases of Eliapps students. It is not necessary to say what these accounts are, but in case you want to know, they are mostly:

• Eliapps accounts of people who also have an O365 mail account. These are mostly ITS people.

• Elilists (a type of mailing list managed by Eliapps instead of Exchange)

• Student groups (example: yale.spizzwinks@bulldogs.yale.edu)

• Departmental shared mailboxes (example: thechildlab@bulldogs.yale.edu)

• The 111 aliases of servicenow@bulldogs.yale.edu

• Mail for specific department servers (invest.yale.edu, aya.yale.edu)

• Everything else. Sometimes this is recognizable junk left over from machines that have not existed since 1995 (gopher.cis.yale.edu) but it is not necessary to clean this up and that is not part of this project.

We cannot preserve the behavior of a handful of personal aliases where a user has created mailboxes with the same name in both O365 and Eliapps and has set the Primary Alias to point to Eliapps. With the Mail Relays, mail sent by O365 users goes to the O365 mailbox and all other mail goes to Eliapps. When the Relays are retired, all mail will go to O365. The old behavior cannot be duplicated, but this configuration violates Yale policy so we will simply inform the affected users before the change.

Transition

After these changes are made to the ProxyAddresses and TargetAddress of existing User objects and new Contacts are created but before the Mail Relays are retired, the only change in behavior is that mail sent from an O365 mail account will now go directly from Exchange Online to Google or any other external service referenced by a TargetAddress. Previously the mail would go from Exchange Online to the Mail Relays and then be forwarded to Google or other services. The generated AD entries must be correct for this mail to be delivered properly.

Then later on we turn off the Mail Relays. Now the generated AD configuration must be complete for the mail to be delivered properly.

Fine Tuning

This project has to refine the list of email domains that are inherently associated with Exchange Online. We know that “@expectwithme.org” is actually an O365 alias, but while ayalists.yale.edu is in Exchange, elilists.yale.edu is in Google, and “lists.architecture.yale.edu” is uncertain. This is not just an IAM problem, because Linux Systems has to fix the MX records for each of these domains.

There are data errors in the Alias Table that will generate unnecessary Contacts. It is not necessary to fix them at this time, but if we don’t do it now, there will be no better time in the future.

Source of Problems (History)

Originally Yale did not think of Email as a single topic.

There was the problem of Student Mail, addressed by the Instructional Support group of ITS. Then there was Employee mail decided by the business focused groups. Once a system was selected, there was some involvement by Unix or Windows groups for technical support.

However, SMTP is a service that you could run on any mainframe, Unix, or Windows NT computer. At one point, main departments ran their own mail servers. The original design of the Email Alias system allowed a user to create an Alias that pointed to the computer under there desk. (So many of these machines were compromised by spammers that this became a violation of Yale security policy, but it was too late to change the Alias system).

IAM became involved for three reasons.

• Historically, there was a time when we charged back mail accounts to departments, and IAM inherited the systems that were created for that purpose even after we stopped charging for mail accounts.

• IAM creates basic resources like a Netid and AD object for new students, employees, and sponsored identities. Students and employees automatically get mail accounts, and it was efficient for IAM to create these “birthright mail” accounts when a new identity is created.

• What people think of as the Email Address is also a unique login string (a “principal” name) for the non-mail functions like OneDrive, Google Drive, and Box. IAM has to manage it as a login name and, among other things, lock and unlock it along with the Netid.

However, other groups make changes to Email accounts. The Help Desk changes the Aliases and the Eliapps account when a user changes their name. The current Email support group creates O365 and Eliapps accounts for non-birthright users, and helps to change accounts when someone changes their Yale association.

IAM is involved with Mail, but we did not set the original rules (remember, Instructional Support decided how Eliapps accounts would be named), nor are we involved in all updates. We own the DIR_ONLINE_INFO table of Mail Aliases even though the Help Desk changes the aliases in it, and we know enough about the rest of the mail system to implement this change.

Example of Problems

A student named Johnathan Doe is initially assigned a Primary Alias of “johnathan.doe” and an Eliapps account name of “johnathon.doe@yale.edu” which means the Alias MAILBOX value is “johnathon.doe@bulldogs.yale.edu”. Then he changes his name to “John” and requests his Email get “fixed”.

A new Alias of “john.doe” is created initially as a secondary alias, but it is then promoted to Primary and the old johnathan.doe becomes a Secondary Alias. The same thing happens to the Eliapps account, which is renamed “john.doe@yale.edu” but has an alternate Email address of the old “johnathan.doe@yale.edu”.

If this is done correctly, then both the new Primary “john.doe” and the secondary old “johnathan.doe” Alias Table entries have a MAILBOX of “john.doe@bulldogs.yale.edu”. Then we know that these are two aliases of the same account. However, because both the new and old names are in the Eliapps User Directory, mail does get delivered correctly if someone is sloppy. So frequently enough we get:

Google makes it unreasonably inefficient to query them and determine that “john.doe@yale.edu” and “johanthan.doe@yale.edu” are two names for the same Eliapps account, and we do not maintain a local table that remembers that we renamed the account. So from what information we have locally on which we can build code, there is no way to know that these are two entries for the same account. That means that the Secondary Alias will generate a Contact object. This is not a problem because this user probably only wants the current Primary alias listed in the directory and the only need for the secondary alias to to make sure that any mail for old addresses also gets delivered.

Technical Information

The following can be read by anyone, but it is directed to technical staff who have to maintain this stuff in the future. You should decide if you want to know more about how mail is delivered.

Testing the Change

There is a TEST instance of O365 (“yalelab.onmicrosoft.com”) and Eliapps (“gtst.yale.edu”).

There has never been a functional TEST version of the Mail Relays, but we don’t have to test the thing we are getting rid of. We only need to test what is left.

Unfortunately, there has never been a TEST philosophy for Email Aliases. There is a TEST DIR_ONLINE_INFO table in ACS2, but once or twice a year the DBAs copy the PROD table from ACS1 to ACS2. We can temporarily create TEST aliases that map firstname.lastname@yale.net to connect.yale.net.

Fortunately, all the changes we have to make for this project go in the AD, and we do have a functioning TEST AD. What is not clear, however, is if when I go to TEST something, the TargetAddress I put in the TEST AD should be a pointer to a TEST O365 or Eliapps account, or should it be a pointer to the PROD account. We are testing if the mail gets delivered correctly, and the only way to be sure is to see if someone receives it. Users do not have TEST mail accounts, and if they did they would not know how to login to them. We have learned that TEST systems frequently have to point to PROD Netids and mail accounts when users are involved.

The Alias Table

Every hour a program reads the Email Alias Table (SMART.DIR_ONLINE_INFO in ACS1) and generates configuration files for the Mail Relays. That table contains a lot of housekeeping data (when last changed, who last changed, and there is a history table), but the important fields for mail routing are:

ALIAS_NAME - The part of the Email Address in front of the “@yale.edu”. Because of the MX records configured into DNS, any mail system outside Yale believes that “@yale.edu” as a mail destination goes through the mail relays. You should note, however, that both Exchange Online and Google G Suite are each configured to believe that they are the mail system named “@yale.edu”.

MAILBOX - The “native” mail address of an account in some Yale Mail system or server that should receive mail addressed to this “@yale.edu” alias. Originally this was the name of a mailbox on a server computer (connect.yale.edu was the DNS name of the first Windows NT machine that ran Exchange and maybe PC Mail before it), but today it is most commonly the name of a mail account in either O365 or Google. The most commonly followed Yale convention is to use “netid@connect.yale.edu” for a personal O365 account or “ALIAS_NAME@bulldogs.yale.edu” for Eliapps accounts (with the difference caused by assigning different groups to make that decision in the past). We ensure that the MAILBOX value is a valid Email Address, but discourage anyone from using it normally because it will deliver mail directly bypassing the Relays and any SPAM or security filters.

TYPE and NETID - The type can be Primary (value “person”) or Secondary (value “alias”). Each alias is owned by a Netid. A Netid can own one Primary alias, and that is the personal email account of that Netid. A Netid can own any number of Secondary Aliases, and an alias owned by one Netid can point to an email account owned by a different netid. It is fairly common for a person to own Secondary email aliases pointing to email accounts owned by one of his Dependent Netids, although this is a bad practice and the alias should be owned by the Dependent Netid. Occasionally one Netid will own an alias pointing to the personal mail account of another individual.

ToDo Eventually: Some IAM tools display and operate only on the collection of aliases owned by the same Netid, but to migrate the Mail Relay function to Exchange, all Aliases pointing to a personal mailbox have to be listed in the ProxyAddresses list of the AD User object that owns the O365 mailbox or that points to the Eliapps account. Netid is significant for Primary Aliases, but Secondary aliases should be managed by the MAILBOX they point to and not the Netid that owns them. This will be a change to ODMT.

Shared Mailboxes: A shared mailbox has one or more configured account names. If any mail goes through Exchange addressed to one of these account names, it will be delivered to the mailbox. However, while the Mail Relays are running, if the shared mailbox is to have an “@yale.edu” email address, it needs a Email Alias for that name with a MAILBOX value that points to any domain associated with Exchange (typically “connect.yale.edu”). When the Mail Relays go away, the need for this alias goes away too. If all “@yale.edu” mail goes directly to Exchange, the Exchange configuration of the shared mailbox is enough to deliver the mail. However, for now it is a current bad practice to make the shared mailbox alias a Secondary owned by the Netid that generated the ticket to set the mailbox up, and then half of a better practice to create a Dependent Netid for the shared mailbox with a ProxyAddresses entry for the various names of the shared mailbox account. If this was done right, the Dependent Netid would also own the alias. Unfortunately, leaving the alias owned by the person causes trouble when eventually the owner leaves Yale without bothering to transfer ownership of the alias and Dependent Netid to someone else. It gets worse after this person is partially deprovisioned, which cleans up his personal mailbox but leaves the alias and dependent netid of a still active shared mailbox in a zombie state. Getting rid of the Mail Relays means we no longer need either the Aliases or the ProxyAddresses or the Dependent Netid. We still should know who is currently responsible for the Shared Mailbox, but that is an administrative problem and not a mail routing problem.

Right the First Time: If mail accounts (Birthright and Manual, O365 and Eliapps) are generated with correct AD ProxyAddresses and TargetAddress values, and if they are updated when aliases change or an Eliapps account is renamed, then it should not be necessary to run the “AD Daily Updater” function. It was created to allow sloppy tools to do only part of the work and then go back once a day to fix the mistakes. We can do better than that.

The Mail Relays

Someone sends mail to “john.doe@yale.edu” or “gravitywaveastronomy@yale.edu”. The sending system does a DNS search for an MX type record associated with the name “yale.edu” and gets back the following:

yale.edu        MX preference = 10, mail exchanger = oolong.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = chamomile.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = jasmine.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = earlgrey.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = darjeeling.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = chai.mail.yale.edu
yale.edu        MX preference = 10, mail exchanger = rosehip.mail.yale.edu

This is a list of the Mail Relay machines (informally known as the “teas”). The mail is sent to one of these machines.

The Mail Relay computer programs are internally configured with files generated from the Alias Table. The program that generates these configuration files looks at each ALIAS_NAME and MAILBOX value. When an incoming Email address ends in “@yale.edu”, then the part in front of “@” must match an ALIAS_NAME in the Alias Table or the mail bounces as undeliverable. If an ALIAS_NAME is matched, then in that row the email domain (the part after “@”) in the MAILBOX value tells the Mail Relay where to forward the mail. The association of mail domain suffix values to the DNS name of one or more server machines is also published as MX records in the DNS system (although the Mail Relays may already have a copy of this data in memory). Looking up the “connect.yale.edu” suffix finds:

connect.yale.edu        MX preference = 10, mail exchanger = connect-yale-edu.mail.protection.outlook.com

while the MX records for “bulldogs.yale.edu” are:

bulldogs.yale.edu       MX preference = 20, mail exchanger = alt1.aspmx.l.google.com
bulldogs.yale.edu       MX preference = 30, mail exchanger = aspmx3.googlemail.com
bulldogs.yale.edu       MX preference = 30, mail exchanger = aspmx5.googlemail.com
bulldogs.yale.edu       MX preference = 20, mail exchanger = alt2.aspmx.l.google.com
bulldogs.yale.edu       MX preference = 10, mail exchanger = aspmx.l.google.com
bulldogs.yale.edu       MX preference = 30, mail exchanger = aspmx4.googlemail.com
bulldogs.yale.edu       MX preference = 30, mail exchanger = aspmx2.googlemail.com

However, the MAILBOX value of an entry in the Alias Table can contain a suffix that is not an MX record. In this case, the suffix is the name of a computer that runs some type of mail server software. Then the relay does an ordinary DNS lookup for the machine and forwards the mail to that computer. The problem of poorly managed mail server software (which could get compromised or abused by SPAM senders) was mostly addressed a decade ago, and we discourage these configurations, but this option was never removed from the Alias system.

Exchange Online

If we replace the Mail Relays with Exchange Online, then the previous step doesn’t happen. We change the original MX record for “yale.edu” to point to “connect-yale-edu.mail.protection.outlook.com” and we take the teas and dump them in Boston harbor (we can have a party to celebrate the success of the project, but dressing up as native Americans is no longer appropriate because it is “cultural appropriation”).

Exchange Online will behave the same whether mail is sent to it directly or comes through the Mail Relays. So any mail that Exchange already processes correctly will work the same and we do not need to make changes.

The largest block of mail that will be processed differently are the student Eliapps accounts. Previously they would be sent from the Mail Relay to Google directly. Now the mail goes to Exchange. Because Exchange believes that it is “@yale.edu”, it initially expects that mail addressed to a user named “john.doe@yale.edu” is a personal O365 mailbox or a group object named “gravitywaveastronomy@yale.edu” is a shared mailbox or distribution list. However, it will find that the Azure AD User object for “john.doe@yale.edu” has no O365 account and will find no configured DL named “gravitywaveastronomy@yale.edu”.

It has always been possible to add configurations to Exchange and AD for mail accounts that belong to a university or company but which are hosted on another mail system. Originally Microsoft designed this when each department in a company had its own departmental mail server, but it works with any kind of server.

We already have the information we need in the Alias Table. Just as the current system uses the Alias Table to generate configuration for the Mail Relays, we can generate configuration in AD that Exchange will recognize as a pointer to a Yale University mail account on an external server. There has to be an AD object. The AD object cannot be a User with an O365 account (or the mail would be delivered to that mailbox). The object must be have a TargetAddress property.

1. The value of TargetAddress is essentially the MAILBOX column value in the Alias Table.

2. The AD Object can be a User or a Contact. Since every student with an Eliapps account already has an AD User object (and Yale Policy is that Eliapps mail users don’t also have an O365 mailbox), we have adopted the simplifying rule to put a TargetAddress in the existing User object for these students, but create a Contact for shared mailbox groups and everyone else.

3. The ALIAS_NAME must match one entry in the ProxyAddresses list property for the AD Object.

If we do this, then Exchange Online has the information it needs to forward mail for every Alias in the Alias Table that does not have an “@connect.yale.edu” MAILBOX, and therefore it can replace the existing Mail Relay function.

Exchange does not care how many objects you scatter into the AD, but it would certainly be a bad idea to create extra objects for the secondary Aliases of a student Eliapps account. So the student’s AD User object should contain ProxyAddresses for both the Primary Alias name and all secondary Alias names that point to the same “bulldogs” MAILBOX account.

We propose to also group Contact objects by MAILBOX value. More that one Alias can point to the same MAILBOX value, but if we group by MAILBOX value, we can then generate one Contact object for that value and add a ProxyAddresses entry for every ALIAS_NAME in the table with that MAILBOX.

Alias names change if a person chooses to change their First or Last name or if they enable and disable privacy. This is an IAM function.

Currently the old AD Daily Updater generates ProxyAddresses from Primary and Secondary Aliases of Exchange (@connect.yale.edu) MAILBOX values, although the old code gets confused unless the Netid owning the Alias is also the netid in the “netid@connect.yale.edu” value (which isn’t always true). The replacement code in IIQ (which has been written and tested but has not gone into production) fixes the incorrect assumptions by ignoring alias ownership, and it also generates the correct ProxyAddresses entries for primary and secondary Eliapps accounts (@bulldogs.yale.edu).

However, to address the “Canvas Contact” problem we have created ProxyAddresses and TargetAddress fields in the AD and Azure AD User objects of current students with Eliapps accounts. This is done only when the mailbox is created, so if a student generates Secondary Aliases or changes the Primary Alias, the ProxyAddresses list would have to be changed manually.

Work to Do: Populate the ProxyAddresses list of student Eliapps users with any Secondary Aliases that have a MAILBOX that points to their Eliapps account.

Work to Do: There may be Eliapps accounts that are the Primary Email Alias of people who are not current students and were not configured as part of the student class distribution list fix. Their Primary and Secondary aliases should be configured in their User objects if possible, or in a Contact if they also have an O365 account occupying the User object.

Work to Do: All remaining Email Aliases (excluding aliases pointing to some sort of Exchange account or a Primary Eliapps account) get turned into Contacts. We do not create or look for User objects for them. A new Contact is created in some new OU, and the MAILBOX becomes its TargetAddress and all the ALIAS_NAMEs with that MAILBOX value become entries in its ProxyAddresses list.

Decision Point: The personal O365 and Eliapps accounts are plausibly related to IAM, although we are not in the business of creating Secondary Aliases. It may be decided that it is useful to continue to audit the Alias Table and the ProxyAddresses list to see if somehow personal Alias changes slipped in without generating the required ProxyAddress changes. However, the creation and modification of shared mailboxes and departmental accounts is not an IAM function and because these objects have no information in Identity tables, we are not in a position to audit them properly. We should consider this a one time conversion of Alias information into AD objects and properties that configure Exchange Online. Going forward, all scripts, tools, and procedures used to generate departmental accounts and create their aliases should also manage their Contacts with the mail routing information.

Cleanup: If tools are created or converted to manage non-personal mail accounts, then we should consider establishing some proper rules and converting existing entries to match these rules. In particular, the practice of creating an Email Alias and a Dependent Netid and assigning ownership of the Alias to the person who owns the dependent Netid causes serious problems. People leave the university without transferring both the alias and the Dependent Netid to someone else. Then when their personal mail gets deprovisioned, the Secondary Alias for the group account remains attached to the inactive AD User object. The correct thing to do is the assign ownership of the Alias to the Dependent Netid that represents the shared mail account. The account can then stand alone, and if the owner fails to transfer the Dependent Netid to someone else, well the Dependent Netid is still active even if the former owner is inactive. This fix is not technically part of the Mail Relay project, but if we are fixing all the tools for this type of account we should fix this at the same time.