The software was able to automatically evaluate 13 out of 19 samples (69%) in the initial test set. 6 samples were flagged for manual analysis, with 4 false negatives due to issues now fixed in the software. The blind test on 10 additional samples found the software could automatically evaluate 7 out of 10 (70%) when filtering problematic samples, or as high as 90% with improved filtering. The document evaluates the ability of automated NMR processing and structure verification software to analyze samples without human intervention.
Validating Automated Structure Confirmation Using NMR Prediction in a Blind Study
1. Looking closer at the failed samples provides some insight to the nature of the failure. ID #3 failed specifically because of an unassigned 2D peak appearing at 17.8 ppm and 2.2 ppm. It was determined that this peak was due to the aromatic methyl group in the structure. The reason for a lack of consistency between the predicted and experimental chemical shifts in this case were due to a slow rotation around the N-CO bond. As a result, this rotamer produces an experimental spectrum that looks like a mixture. The software was unable to accurately predict this mixture of forms based on the experimental conditions, and as a result, the predicted spectrum did not match the experimental and the sample was flagged for manual analysis. Figure 2. ID #3 was a false negative as the software was unable to assign the methyl group (highlighted in blue) due to slow rotation around the N-CO bond. The presence of this rotamer resulted in an inconsistency between the experimental and predicted chemical shifts. The software also flagged the sample ID #7 to be considered for closer inspection. In this particular case, the issue with this spectrum was determined to be based on the presence of a mixture. Based on some of the spectral features in both the 1 H and HSQC–DEPT spectra, it is believed that some of the product had converted to an alcohol resulting in a mixture of both the brominated and hydroxylated products. As a result, the software correctly identified this spectrum as not being consistent with the proposed structure and it was flagged for manual analysis. Figure 3. ID #7 was flagged as an ambiguous result. Spectral features in the 1 H and HSQC–DEPT dataset suggest a mixture between the two compounds shown above. The final sample in the blind test set (ID #10) was also flagged by the software. In this particular case, the software was unable to identify two protons in the experimental spectrum because the software correctly set a dark region over a large water peak. Unfortunately, the creation of this dark region resulted in the exclusion of an important multiplet in the experimental spectrum that was in close proximity to the water peak. Because of this dark region, the software was unable to confirm a match between the spectrum and structure. Table 1. The results of the 19 Aldrich datasets. For this dataset, 13 of the 19 datasets (69%) were automatically evaluated by the software. The software was unable to confirm the proposed structure in Figure 1 (ID #8). Upon closer inspection it was observed that the software failed because the experimental peak located at 4.74 ppm corresponding to atom #11 in the proposed structure had an integration value that was too low to assign correctly. As a result, the software flagged this result as ambiguous. It was later observed that the low integration value could be due to enol formation. A longer relaxation delay may have more adequately prepared the experimental spectrum for automatic evaluation by the software. Figure 1. ID #8 was a false negative because the integration value for the multiplet at 4.74 ppm was too low. A second compound, ID #11, was rejected by the software as well. Upon closer inspection of the experimental data, it was determined that the purity of this compound was not sufficient and that the sample contained several different components. In these two particular examples, the software did a good job of flagging the two problem spectra that required a closer look. Results of the Blind Test After the previous results and settings had been agreed upon, a blind test set of 10 compounds was run through the system in the exact same fashion. No changes to the processing or verification parameters were made. The results of this test are shown in table 2. Table 2. The results of the 10 blind Aldrich datasets. For this dataset, 7 of the 10 datasets (70%) were automatically evaluated by the software. Ryan Sasaki, Brent Lefebvre, Antony J. Williams, and Sergey Golotvin Advanced Chemistry Development, Inc. Toronto, ON, Canada 110 Yonge Street, 14 th floor, Toronto Ontario, Canada M5C 1T4 Tel: (416) 368-3435 Fax: (416) 368-5596 Toll Free: 1-800-304-3988 Email: info@acdlabs.com Validating Automated Structure Confirmation in a Blind Study Figure 6 . ID #10 failed due to the close proximity of an important multiplet to an intense water peak in the experimental spectrum. CONCLUSIONS The goal of this study was to evaluate a fully automated NMR processing and structure verification workflow for a blind test set of compounds. The processing and evaluation settings for a typical group of samples was set up using a pilot set of 19 compounds. Once these settings were adjusted, they were used to automatically process and evaluate a blind set of 10 compounds that were prepared under the same conditions. The results revealed that this completely automated system could reduce the interpretation workload of a spectroscopist by up to 90% if problems with rotomers and impurities are filtered out before the NMR Verification step, up to 70% when these problem samples are left in. This study highlighted several examples where datasets were flagged by the software for closer inspection by a spectroscopist. These particular examples illustrate the software’s discrimination ability that help reduce the risk of false positives. The results of this blind study suggested that a fully automated processing and interpretation system can perform sufficiently in an industrial environment. ACKNOWLEDGEMENTS The authors would like to acknowledge Dr. Timothy D. Spitzer and Randy D. Rutkowske of GlaxoSmithKline for providing us with NMR data for the compounds in this study. REFERENCES 1. Automated Structure Verification Based on 1 H NMR Prediction, Sergey S. Golotvin, Eugene Vodopianov, Brent A. Lefebvre, Antony J. Williams, and Timothy D. Spitzer . Magn. Reson. Chem. 2006; 44: 524-538. 2. Automated Evaluation of a Chemical Structure with Only 1D 1 H and 2D 1 H– 13 C HSQC, Sergey S. Golotvin, Eugene Vodopianov, Rostislav Pol, Brent A. Lefebvre, Antony J. Williams, and Timothy D. Spitzer . ENC Poster 2006. INTRODUCTION In previous work, we have presented several findings on the automated evaluation of chemical structures using 1 H, 13 C, and 2D NMR verification algorithms. 1–2 These studies have shown that these systems have performed extremely well through numerous challenges. The current study focuses not only on the performance of the verification algorithms but also on the automated preparation of experimental data through a blind test. This study was designed to prove that such a system would hold up in an industrial environment without any human intervention. This study consisted of two distinct sets of structures and spectra. The first contained 19 spectra sets (each dataset contained 1D 1 H and 2D HSQC spectra) that were provided ahead of time for adjustment of processing settings and options. This step was necessary to identify the best software settings based on the instrument and data collection practices for the laboratory where the samples were prepared and run. Once the first set was run through the system and results of the verification procedure obtained, the second, blind test, was performed on 10 distinct datasets (with chemical structures) that were not available to the software or the software operators in advance. The details and results of these two tests are presented here, along with a comprehensive look at the structures that could not be confirmed. Setting Up Ideal Processing and Evaluation Parameters In order to have a system that can run without human intervention, automated processing and structure verification procedures (macros) must be created in the software to perform these tasks. The raw 1D and 2D NMR datasets for 19 Aldrich compounds were first evaluated using ACD/Labs’ standard macros. These settings proved to be non-sufficient as the datasets contained several abnormally broad water peaks and low signal-to-noise ratios. These macros were then modified to exclude these water peaks and set more stringent peak picking guidelines to combat the S/N issues. The second attempt was improved but had some issues with the referencing in one of the 2D datasets. In addition, the 1D spectra were not well-resolved, resulting in an inaccurate evaluation of some multiplets. These issues were rectified by decreasing the line broadening setting in the software by a factor of 10. Following this modification, the settings were then deemed to be sufficient. Results of the First Test An explanation of the combined verification algorithms used to evaluate spectrum-to-structure matches have been previously reported. 2 Following the modification of ACD/Labs’ standard macros explained in the previous section, the raw data of the 19 Aldrich compounds were fully processed and evaluated automatically. The results revealed that the software was able to successfully evaluate 13 of the 19 datasets provided. In other words, for this particular dataset, 69% of the samples were automatically evaluated by software without any human intervention. The remaining 6 samples would require manual analysis by an NMR Spectroscopist as the software had flagged them as being either inconsistent or incorrect (Table 1). Of these 6 samples, it was concluded that 4 of the false negatives were a result of algorithm errors that have been fixed in Version 10 of the software (ID# 6, 12, 13, and 15). The other two ambiguous results require a closer look to be explained.