This is Stage 4 of our Chemical Data Migration best practices whitepaper. Learn how to plan your project and assess your data in Stage 1, make sure your data is clean and compliant with business rules in Stage 2, execute the de facto migration in Stage 3, and ensure adoption and long term success with post-migration validation and support in Stage 4. 

After the steps outlined in the previous chapter, successful completion of the migration project is within arm’s reach. However, to ensure the long term success and adoption of the new system - including the new chemical database, as well as any new applications and processes - at the organization, you have to consider onboarding, continued support and change management as well. It is also worth thinking about similar future projects and how the lessons learned from the current project can help improve them.
In this final section, we are going to review what organizations can do to make the change stick and to maximize the value of the new system.

Change management

Onboarding

The first step to take in order to avoid or minimize post-migration issues, is having an onboarding plan. In an ideal case, the onboarding plan and its timeline is created in the initial project planning phase involving the key users of the future, new system.
As a fundamental part, the onboarding phase has to include training sessions to help the team understand both the new systems and the corresponding new processes.

To ensure that users with different backgrounds and responsibilities all have the knowledge and skills necessary to adapt to the changes, we recommend organizing training at different levels and for different audiences.

A basic training is usually recommended for everyone who has to work with the new system in some capacity including end users, developers, data engineers and managers. Following the basic training, further, more advanced or bespoke training sessions can be organized catering to the specific needs of different user groups, such as power users, administrators or integrators. If a migration partner was also involved in the project, it is recommended to work with them during the training as well to maximize the shared information.
Custom training for key users or “train the trainer” workshops can also help and make the transition smoother for the team since people are usually more open to learn from their own colleagues whom they already know and who are more familiar with the team’s workflows and processes.
Besides training, comprehensive documentation with detailed how-to guides, visual or video tutorials, FAQs, troubleshooting tips explaining system usage, business rules and differences between the old and the new system is another key component of successful onboarding.

Continued support

After completing the necessary training and onboarding activities, users can all start working with the new system and realizing the value it provides. However, even the best training and onboarding plan are not enough to ensure that users can adapt to the new software and incorporate it into their daily work routine. As people have a natural resistance to change, it is crucial to address the team’s doubts and concerns and by providing a clear explanation of the benefits of the migration and the new system.

For the sake of a smoother transition, it is worth appointing a dedicated point of contact who can articulate the advantages of the new system, answer the team’s questions and concerns and help them get the support they need.
At the same time, long term support and adaptation can also be supported by having key users who got thorough, advanced level training during their onboarding and who get regular updates on the latest improvements as the system evolves. Later, these colleagues can serve as internal champions, educate other members of the team, share the latest updates with them, as well as any tips and tricks that could make working with the new system more efficient or could help the users solve problems where they did not have a solution in the past. This approach can be combined with other forms of sharing information in the organization, such as using internal forums and channels where a team or department can share their own processes, best practices and success stories and where other teams can go for inspiration and advice.

Data consolidation

In an ideal world, the entire data consolidation and cleaning happens before importing the legacy records in the new production system, and by the final validation and testing round, all data transformation steps have been completed. In real life migration projects, however, it is not always so simple. For various reasons, such as missing records or insufficient data cleaning, we suggest continuing testing the migrated data after the official go-live as well. Since it is not just the UAT team anymore who performs the testing, but the much larger, full user base, there is an increased chance of finding previously uncovered errors.
Migrating missed records is a relatively simple act if the original data source is still available.

Before adding those records to the database, it is still recommended to run a duplicate search and check if it is the chemical structure itself that is missing or only certain earlier batches of it.

Performing additional data cleaning in the post-migration phase can be a more complex exercise depending on how many records and how many and what kind of attributes are affected. If a relatively small subset of the records have to be cleaned and only a few, non-structural attributes have to be modified, then it might be possible to perform data cleaning and the corresponding transformation steps directly in the structure database. In the other cases however, when a larger part of the whole database is affected and especially when the chemical structures themselves have to be modified, it might become necessary to export the data set in question, modify it with the help of some external tools and then reload it.

Project evaluation

Post-migration evaluations give an opportunity to the migration team to assess what went well during the project and what did not, how did their initial project plan, timeline and risk mitigation strategy work, and what should be improved in similar future projects. As the outcome of the evaluation, the team should prepare a comprehensive documentation to explain the whole migration process, including requirements, business rules, step-by-step guides, troubleshooting tips, and a description of the data curation process along with its outcome. The documentation should be shared with the migration team and every other stakeholder, and it should be made easily accessible so it can be used for future reference.

Continuous improvement

Even after officially closing a data migration project, it is worth organizing regular check-ins and follow-up meetings with the user base of a new system to make sure that they are all adapting well to the changes. These check-in meetings should also be used for gathering feedback from the different user types and for discussing how the different long-term issues are addressed. Combining user feedback and the results of a continuous performance monitoring could reveal the most high-impact improvement areas.

While post-migration issues are not unheard of, there are a lot of preventive measures that can help reduce the number of incidents or the negative impact they have on the organization. Training, documentation, designated support contacts and regular improvements are all contributing to the long term success of a migration project. Besides these measures, the post-migration phase is also time to enjoy the benefits of all the upfront work and planning you did and the benefits of involving the team from the beginning of the migration project because it enables a smoother transition and increases the team’s sense of ownership and commitment to the project’s success.

Conclusion - General Best Practices

Chemical data migration means much more than the actual export and import of legacy data. Handle it as any other project:

• • define goals and success metrics;
  • identify stakeholders and the necessary resources, including technical, material and human resources, and make sure that they are available by the time when the migration project starts;
  • set up a timeline and a communication plan;
  • identify potential risks and have a corresponding mitigation plan.

1. Allocate sufficient time and resources for chemical data analysis to understand legacy data and its architecture including the level of chemical complexity captured, uniqueness rules, data hierarchy and existing error types.
2. Understanding legacy data in conjunction with the organization’s business needs is a fundamental step in the overall process and should be the preliminary focus. Work together with internal experts who are familiar with your legacy systems, data and business logic and who have the cheminformatics knowledge to define the new business rules. Get the help of external experts, if necessary, to properly define all rules according to corporate business needs.
3. Have well-defined and well-documented business rules and chemical structure representation definitions in place before you add the first legacy or new molecule to the new production system. Chemical business rules have to define what kind of information is needed to describe molecules unambiguously, which compounds are considered duplicates and what should happen with them, how should stereoisomers, tautomers and isotopologues be stored, and what are the guidelines for drawing chemical structure and functional groups.
4. When available, use industry standard, structure-based representations to describe and store chemical features of molecules (such as stereochemical notations or salts and solvates).
5. Combine automated structure checking and standardization workflows with internal drawing conventions to maximize the quality and accuracy of chemical data.
6. Make sure that you have documentation of chemical structure checking, structure fixing and standardization rules, as well as of internal drawing guidelines, and that you have in-house experts trained to be familiar with those conventions.
7. After migrating the legacy data, allocate enough time and resources for data validation and testing.
8. Plan your team’s onboarding process in the initial project planning phase and involve the future users as well.
9. Train key users on the new system in the onboarding period, and make sure that they get regular updates on recent improvements later as well, so that they can educate their colleagues and become the internal champions of the new system.
10. Have a designated single point of contact who can address the users’ questions and concerns and can articulate the benefits of the migration project. Promote internal forums where teams and departments can share their processes, difficulties, best practices and success stories with each other.
11. Even after closing the project, have regular check-ins and follow-up meetings with the team of users to ensure that they are adapting well to the changes. Continue gathering their feedback, monitoring the performance of the new system and make sure that long-term issues are addressed properly and that improvements are added based on the ongoing feedback.
12. And finally, perhaps most importantly, chemical database management should be a continuous effort. We recommend that organizations have data product managers and data governance processes to maintain the integrity of their chemical data.

If you are interested in the previous steps of the chemical data migration process, learn about our best practices for Project Planning and Data Assessment, Data Curation, or Data Migration proper in our series.