Sebastian Ueckert (1), Joerg Hipp (2), Julian Tillmann (2), Hanna Silber Baumann (2), Eric Hollander (3), Christopher Chatham (2), Xing Chen (2), and Maria Kjellsson (1)
(1) Department of Pharmaceutical Biosciences, Uppsala University, (2) Roche Pharma Research & Early Development, Roche Innovation Center Basel, (3) Department of Psychiatry, Albert Einstein College of Medicine, Montefiore Medical Center, NY, USA
Objectives: Clinical scales consist of several questions or items that collectively measure the multi-faceted clinical nature of neuropsychiatric conditions. These instruments are essential for the clinical understanding of these conditions. Often however, the methods used to assess the structure of clinical scales is variable and sample sizes are limited, leading to inconsistent recommendations on how to analyze and interpret clinical scores. In addition, it is unknown whether the properties of the scale vary in different subpopulations (e.g. at different ages or cognitive ability). A good understanding of clinical scales is essential to measure treatment effects in clinical trials. Here we applied multidimensional item response theory (MIRT) and clustering approaches to identify in an exploratory and data-driven manner the dimensionality of the Repetitive Behavior Scales-Revised (RBS-R, 43 items, [1]), a commonly used parent-reported clinical scale in Autism Spectrum Disorder (ASD).
Methods:
Data: This work used a sample of 4217 participants with ASD with cross-sectional RBS-R scores from four different data sources. From the database, all participant with both age and IQ covariate values were included in the analysis, resulting in a total of 3430 item-level RBS-R assessment entries. The data was split into five groups: i) children with typical IQ (age: 5-12; IQ>70; N=1860), ii) teenagers with typical IQ (age: 12-18; N=605), iii) adults with typical IQ (age: >18; N=282), iv) children with mild intellectual disability (IQ<70; N=448), and v) teenagers with mild intellectual disability (N=208).
Workflow: The workflow for this project consisted of three parts. Part 1 determined the structure of the RBS-R in the largest subpopulation (group i), part 2 investigated deviations in structure for smaller subpopulations, and part 3 studied the impact of these findings for the sensitivity to detect change in simulated clinical trials.
Part 1: Exploratory MIRT models from one to 15 dimensions were evaluated using ten-fold cross-validation [2]. The model with the best predictive performance was used to extract the measurement directions of the scale and perform a hierarchical clustering of the items.
Different partitions of the resulting tree were then evaluated using cross-validation, and the best partitioning was selected to determine the primary constructs of measurement.
Part 2: A penalized likelihood approach was employed to evaluate the deviations in the smaller groups (ii to v). The MIRT model from the largest group served as a prior during estimation, and the penalty factor was selected through cross-validation. The resulting model in each of the subpopulations was compared to the final model from part 1.
Part 3: The impact of the findings from part 1 and 2 were evaluated using simulations from the final models. In scenario one, an analysis using all or only a subset of items was compared. In scenario two, a pooled and a subpopulation-specific analysis was evaluated.
Results: Our analysis identified three main measurement constructs for the RBS-R: self-injurious behaviors, lower-order, and higher-order repetitive behavior. Consistent with previous literature, most items appeared to primarily measure one of the three constructs and only some measured more than one construct. The constructs identified in the reference sample (children with typical IQ) remained broadly similar across different subpopulations, with only some differences in the strength of association with the items.
Simulations showed a consistently higher power to detect a drug effect when only the relevant subset of items (e.g. using only self-injurious behaviors) and not all items were used for the analysis. Furthermore, there were no large differences between using subpopulation-specific models compared to using the reference model.
Conclusions: The methodology presented here provides a deeper understanding of the structure of a composite scale and allows to evaluate the scale’s behavior in a clinical trial.
References:
[1] Bodfish JW, Symons FJ, Parker DE, Lewis MH. Varieties of repetitive behavior in autism: Comparisons to mental retardation. J Autism Dev Disord. 2000 Jun;30(3):237–43.
[2] Reckase M. Multidimensional Item Response Theory. 2009 edition. Dordrecht ; New York: Springer; 2009.
Reference: PAGE () Abstr 9448 [www.page-meeting.org/?abstract=9448]
Poster: Methodology - New Modelling Approaches