Towards exploring continual learning for toxicologic pathology in pharmaceutical drug discovery

Deep learning has shown recent promise in advancing toxicologic pathology image analysis. However, there exist notable challenges towards implementing these models in real-world scenarios, where datasets are sequentially acquired over extended periods across weeks or months in an Investigational New Drug study. A particular issue that arises in this process is the need for continuous adaptation to new data classes relevant to new tissue types being scanned, or new animal models being incorporated, and so on. Data retention is often hindered by privacy concerns, legal constraints, and storage limitations. Furthermore, existing deep networks are prone to catastrophic forgetting when trained on new tasks, resulting in a substantial loss of previously acquired knowledge. Therefore, there is an urgent need for algorithms that are resilient to forgetting and capable of generalizing to new data without the necessity of retaining large volumes of past examples. To address these challenges, we introduce a novel replay methodology that leverages generative models, augmented by a knowledge regularization approach utilizing attention embeddings from prior tasks. Our method integrates attention-based regularization, which prioritizes the relative spatial importance of features, with generative latent replay. This synergistic approach enables the model to retain and reinforce critical information from previous tasks while adapting to new data. We empirically demonstrate the superior continual learning performance of our method in non-stationary data environments, as evidenced by its application to a representative toxicologic pathology image analysis task.