Migrating Notes users from IBM cloud to on premises

With the demise of the IBM Smartcloud per july 2020, there are suddenly a lot of requests to migrate Notes users from the IBM Smartcloud back to an on premises Domino server or a partner hosted Domino server. The process from on-prem to cloud is pretty well documented, but the process to get your mailfiles back to on prem not so much. I recently migrated a customer with about 200 mailfiles back to on prem. The customer was a hybrid Notes user, so they still had on prem Domino servers as well. These are my lessons learned.

Even a relatively small number as 200 mailfiles is simply too much to do by hand (at least for someone like me who hates repetitive work). I therefore created a tool that does most of these repetitive tasks for me. It proved to be a huge help in this migration process. These are the steps I took during the migration:

I imported a list of all users from the Domino directory that had their mailfile registered on a cloud server to my tool and let the tool calculate a filepath that made sense. In the cloud, you have pathnames like data3/200093906/200857102.nsf. On prem, a filepath like mail/mdejong.nsf would make a lot more sense. My tool just grabs the shortname from the person document as that’s supposed to be unique, so you get a filepath mail/<shortname>.nsf.

Next you want to know how much data we’re actually talking about, so I built a function that collects from each mailfile how many documents are in the cloud mailfile and what the mail file size is. Obviously you need a server on-prem/in parner-cloud (for this customer the latter) which can consume the size of all these mail files. For this customer I knew beforehand how much data it would be about, so we built the destination server before I started the migration.

Next you need to replicate the mail files from the cloud to the on prem server. This is where it starts to get interesting :-). When asking around in the community I heard that it’s not possible to do a server-to-server replication. Luckily this turned out to not be true. You need to take some steps though to get this server-to-server replication working:

• Your new server needs to have the same certifier as the other on-prem servers. Handiest to simply have it in your on-prem domain
• Add your new server explicitly in the ACL of all cloud mail files to have editor access.
  To do this, in the Smartcloud interface go to manager your organization => IBM Smartcloud Notes => Users => Batch Request and add an entry for your server, make it editor and press “Process”.
• Create a replica stub of each mail file on your local server. This is where my tool came in handy again as I created an agent to do this for me. Some C API code needed here to accomplish this part.
• Check if the batch process you created earlier was done. If so, you can start the replication with the cloud servers. I created a function to send the console command for selected mail files.
• You can also go crazy and just create a connection document for all cloud servers where you have users registered and let your mail files replicate on a schedule. Obviously I created an agent to create those connections documents.

It takes a while before all mail files have replicated, depending on the amount and size of the mail files, but I was actually surprised how easy this part went. An agent in my tool checks the size and number of documents of the cloud and on-prem replica’s to give an idea of where you are in the process and if everything replicates properly.

At this point I had a bit of a fright as some mail files had quite a few less documents as the replica on the cloud servers and one mail file even had a whopping 18000 documents missing! It turned out that 2 types of documents don’t properly replicate:

• Documents in the trash can. Some replicate. Most don’t.
• Stubs of the documents that were archived in an archive database. The missing 18000 documents turned out to be of this type

So luckily the diff between document counts is not a big problem, but it is something to be aware of.

When all mail files have replicated it’s time to move over the users. I created 2 buttons to:

• Change the person documents of selected users to the new mail file and replicate this between mail server and admin server (which also had the smtp listener on it). This code also changed the ACL of the mailfiles to list the new server as admin server, add any wanted mail file manager groups and reduce the access of the cloud servers to this on prem mail file to editor.
• Send each user a specially prepared “personal” email with a button they needed to press to add a connection document to the new mailserver, change all their location documents which had the cloud mailserver listed, do a last replication between cloud and on-prem and send back a mail to my tool to tell me if all went well (the “button pressed” column in above screenshot is the result from that).

This almost concluded the Notes migration of the users. What was left was the mobile migration and this was a bit more tricky. The password that users use for their cloud mailfile is not synched to on-prem, so users need to use their on-prem HTTP password to log in to their web- and mobile mail. If they didn’t use this password for anything else recently, chances are pretty slim that they’ll remember it. I therefore created an agent to reset the password of selected users to a previously defined default with the flag to change it on first login. The users got an extensive instruction with screenshots of every step to change their password, remove the mobile profile (in the case of iOS/Apple Mail users) or reset the Verse user and add a profile/account for the new mail server. This step turned out difficult for users with many calls to the local helpdesk as a result. Be prepared to handle the extra calls. I kept track of all users who migrated their Traveler account using an agent which fired a command to the Traveler server for the active users and reading and processing the output. That’s the “Mobile enabled” column in the tool.

When this was finally done, the last step was to disable the connection documents to the cloud servers and remove the cloud entries from the ACL. That finalised a successful migration.

A couple more things I learnt along the way.

• It’s a pita to full text index a lot of mail files through the admin client, so I automated that as well
• Even though I added the new server with the batch process to the ACL of all mail files, I didn’t see that server in the ACL of the mail file. However, it did work (where without it, it didn’t). Confusing, weird, just ignore.
• If you change the mail file template of a mail file on prem before you reduced the access of the cloud servers, the cloud servers will simply replace it again. Apparently there’s a design task there.
• It might seem smart to always update your person records on your admin server and work from there. If you do that while changing the person records to the new mail server though, this is not the way to go as it creates a short time where the smtp listener server will try to send incoming mail to the new mail server, while that server still thinks the mail file is in the cloud and sends it back creating a loop. Therefore that’s one change you want to do on the new mail server and then replicate to the admin server.
• Even though the cloud mail files have replication of unread marks set to “all servers”, I had a couple of mail files where the unread marks did not want to replicate. I have not found the cause nor a solution for that.