Patrick Kofod Mogensen (1), Christopher Rackauckas (2,3), Joakim Nyberg (4), Vijay Ivaturi (2), and Andreas Noack (1)
(1) Julia Computing (2) Center for Translational Medicine, University of Maryland School of Pharmacy, Baltimore, MD, USA (3) Massachusetts Institute of Technology (4) Pharmetheus, Uppsala, Sweden
Introduction: Modern computers have CPUs with multiple cores but often software has been written for single core execution. Hence, much of the compute potential on laptops as well as compute clusters is never realized because the software or programming environment either makes concurrent execution difficult or simply impossible. Concurrent execution can be achieved with either multiprocess or multithreaded parallism and efficient utilization of a compute cluster of several independent compute servers connected by a fast compute network, usually requires a combination of both paradigms. However, on single compute systems sharing the same main memory such as laptops, desktops, and large single node servers common in scientific computing, it is often simpler to use the multithreading paradigm since it avoids the (implicit or explicit) handling of network memory transfer.
Pumas [1] is a recent JuliaPro [2] package for pharmacological modeling and simulation with good support for maximum likelihood estimation of PKPD models. The Julia programming language added new native multithreading functionalities with version 1.3 and support for multithreaded estimation in Pumas was added in the newly released (alpha) version 0.10 alongside the existing support for parallelized model simulation.
Objectives: Examine the speed of the newly added multithreaded maximum likelihood estimation in the Pumas package on a PKPD model for hepatitis C virus (HCV) kinetics defined by five ordinary differential equations.
Methods: The evaluation was based on the HCV model in [3]. 24 subjects were simulated for four dosing intervals of 24 hours each and 11 samples were taken in each interval. The ODEs were solved with a hybrid integrator, AutoVern7(Rodas5()), from the DifferentialEquations.jl package [4]. The inner optimization for the computation of the empirical Bayes estimates used a Newton trust region solver and the outer optimization used a quasi Newton (BFGS) solver both from Optim.jl [5]. All gradients were computed with automatic differentiation through the ForwardDiff.jl package [6]. The dataset was then estimated with Pumas’s FOCE method with 1, 2, 4, and 8 threads on a 40 core Intel Xeon Silver 4114 (SkyLake) Processor.
Results: The single-threaded run completed in 303 seconds. The runs with 2, 4, and 8 threads completed in 171, 109, and 72.5 seconds respectively. This corresponds to parallel efficiencies of 89%, 69%, and 52%.
Conclusions: Quasi Newton algorithms are not trivially parallelizable but for models with sufficient work at each iteration of the optimization distribed over a sufficient number of subjects, the new multithreaded optimization functionality in the Pumas package is able to achieve significant speedups over serial execution. Composing multithreaded model fitting over subject with multiprocess fitting over (resampled) datasets is an interesting future direction that might make bootstrapping feasible for models that are currently prohibitly costly.
References:
[1] https://pumas.ai
[2] https://juliacomputing.com/products/juliapro
[3] Nyberg J. et al. Methods and software tools for design evaluation in population pharmacokinetics–pharmacodynamics studies. Br J Clin Pharmacol. 2015 Jan; 79(1): 6–17.
[4] Rackauckas, C. & Nie, Q., (2017). DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia. Journal of Open Research Software. 5(1), p.15. DOI: http://doi.org/10.5334/jors.151
[5] Mogensen PK, Riseth AN. Optim: A mathematical optimization package for Julia. Journal of Open Source Software. 2018 Apr 5;3(24).
[6] https://github.com/JuliaDiff/ForwardDiff.jl
Reference: PAGE () Abstr 9579 [www.page-meeting.org/?abstract=9579]
Poster: Methodology - Estimation Methods