Identifying Differences in the Performance of Machine Learning Models for Off-Targets Trained on Publicly Available and Proprietary Data Sets

Author(s)
Aljoša Smajić, Iris Rami, Sergey Sosnin, Gerhard F. Ecker
Abstract

Each year, publicly available databases are updated with new compounds from different research institutions. Positive experimental outcomes are more likely to be reported; therefore, they account for a considerable fraction of these entries. Established publicly available databases such as ChEMBL allow researchers to use information without constrictions and create predictive tools for a broad spectrum of applications in the field of toxicology. Therefore, we investigated the distribution of positive and nonpositive entries within ChEMBL for a set of off-targets and its impact on the performance of classification models when applied to pharmaceutical industry data sets. Results indicate that models trained on publicly available data tend to overpredict positives, and models based on industry data sets predict negatives more often than those built using publicly available data sets. This is strengthened even further by the visualization of the prediction space for a set of 10,000 compounds, which makes it possible to identify regions in the chemical space where predictions converge. Finally, we highlight the utilization of these models for consensus modeling for potential adverse events prediction.

Organisation(s)
Department of Pharmaceutical Sciences
Journal
Chemical Research in Toxicology
Volume
36
Pages
1300-1312
No. of pages
13
ISSN
0893-228X
DOI
https://doi.org/10.1021/acs.chemrestox.3c00042
Publication date
07-2023
Peer reviewed
Yes
Austrian Fields of Science 2012
301207 Pharmaceutical chemistry
ASJC Scopus subject areas
Toxicology
Portal url
https://ucrisportal.univie.ac.at/en/publications/identifying-differences-in-the-performance-of-machine-learning-models-for-offtargets-trained-on-publicly-available-and-proprietary-data-sets(bd647375-91d5-42f3-a8dc-d8443d928d11).html