Determining why records appear in your employee.error file in the TDI Assemblyline

Your TDI solution directory contains a couple of files which all start with employee. These files tell you which records were added, updated, deleted, skipped or produced an error. The last file is usually the most interesting one. Often employee records give an error because a user was deleted at an earlier stage and recreated with a different GUID. Sometimes users are created with the same login ID as a previous user, which was long deleted and there are more relatively obvious reasons like that why an employee record would create an error. However, sometimes you are just completely in the dark of why a record gives an error. That’s when this little tip helps you to determine the problem.

Inside your TDI solution directory, you can find the file etc/log4j.properties. The first line in this file that’s not a comment is:

log4j.rootCategory=INFO, Default

Change this to:

log4j.rootCategory=DEBUG, Default

Now instead of the default sync_all_dns.sh/.bat file run the file collect_dns.sh/.bat. This will produce a list of users in the file collect.dns. Now just leave the problematic entries in this file and remove the rest (yes, you can skip filling this file and put the problematic entries straight in this file if you know the right format). Then start populate_from_dn_file.sh/.bat. This script will run the assemblyline over just those entries that are in your collect.dns file. After this procedure the logs/ibmdi.log file will contain detailed information on why these particular records fail.

I needed this myself recently when I was creating a new Connections 6.5. demo environment and I copied the assemblyline from another installation. As the database where the profiles had to be written was different, I did setup the profiles_tdi.properties file separately, but except for the database connection, everything was exactly the same as in the other installation. At least that’s what I thought. The result was very different though. Where the assemblyline in the other installation would work fine, this one wouldn’t import a single record. Following the above procedure, I found the following line in my ibmdi.log:

Validation failed for field sourceUrl. Value is ldap://<cut>:389/OU=e-office Users,DC=intra,DC=e-office,DC=com?(| (&(cn=*)(mail=*)(sn=*)(givenName=*)(displayName=*)(sAMAccountName=*)(UserPrincipalName=*)(memberOf:1.2.840.113556.1.4.1941:=CN=<cut>,OU=Groups,OU=e-office Users,DC=intra,DC=e-office,DC=com)(!(UserAccountControl:1.2.840.113556.1.4.803:=2)))). See the validation specification file - validate_dbrepos_fields.properties.

So there was a problem with the LDAP filter which was too long to be stored in the profiles database. The very tiny detail where my profiles_tdi.properties file differed from the original was this line:

sync_store_source_url=true

In my file it was set to true (the default). In the other assemblyline, it was set to false. So I had the choice between setting this property to false or fixing the LDAP filter. Your filter can be 256 characters. Mine was 343 characters. I opted in the end for reducing the filter as many rules were irrelevant if you also use a group where users should be a member of before they get access to Connections. I wouldn’t have spotted this so soon though if it hadn’t shown in the log file.

Thanks to Christoph Stoettner for this tip and helping me out. Don’t forget to change your log4j.properties file back when you’re done. Otherwise you’ll get huge ibmdi.log files!