Effect of Measurement Scales on Results of Item Response Theory Models and Multivariate Statistical Techniques
DOI:
https://doi.org/10.26713/jims.v11i1.951Keywords:
Item response theory, Factor model, Scale points, DimensionalityAbstract
The study investigates the effects of response scales of items on results of item response theory models and multivariate techniques. A total of sixty-four datasets have been simulated under various conditions such as item response format, number of dimensions underlying response scales, and sample size using R package mirt command: simdata(a,d,N,itemtype). Two main statistical techniques -- Item Response Theory (IRT) models and Factor Analysis -- are employed. We find that there is a direct relationship between parameters of IRT and those of factor models, particularly item discrimination and factor loadings. The results also show that the overall fitness of the item response model increases with increasing scale points for higher dimensionality and sample size 150 and higher. The fitness deteriorates over increasing scale points for small sample sizes for unidimensional model. Again, the number of influential indicators on factors increases with increasing scale-points which improves the fitness of the model. The study suggests that a five-point response scale gives most reasonable results among various scales examined. IRT analysis is recommended as a preliminary process to ascertain the observed features of items. The study also finds a sample size of 150 as adequate for a most plausible factor solution, under various conditions.Downloads
References
N. Balakrishnan, Handbook of the Logistic Distribution, Taylor and Francis, New York (1991).
I. H. Bernstein and G. Teng, Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization, Psychological Bulletin 105 (1989), 467 – 477.
R. P. Chalmers, mirt: A multidimensional item response theory package for the R environment, Journal of Statistical Software 48(6) (2012), 1 – 29, DOI: 10.18637/jss.v048.i06.
A. L. Comrey and H. B. Lee, A First Course in Factor Analysis, 2nd ed., Lawrence Erlbaum Associates, New Jersey (1992).
A. Cyr and A. Davies, Item response theory and latent variable modeling for surveys with complex sampling design: The case of the National Longitudinal Survey of Children and Youth in Canada, in Conference of the Federal Committee on Statistical Methodology, Office of Management and Budget, Arlington, VA (2005).
R. J. de Ayala, The Theory and Practice of Item Response Theory, The Guilford Press, New York (2009).
J. de Leeuw, Models and methods for the analysis of correlation coefficients. Journal of Econometrics 22 (1983), 113 – 137, DOI: 10.1016/0304-4076(83)90096-9.
L. R. Fabrigar and D. T. Wegener, Exploratory Factor Analysis, Oxford University Press, New York (2012).
P. J. Ferrando and U. Lorenzo-Seva, Unrestricted item factor analysis and some relations with item response theory (Tech. Rep.), Department of Psychology, Universitat Rovira i Virgili, Tarragona, retrieved from http://psico.fcep.urv.es/utilitats/factor/ (2013).
R. Gorsuch, Factor Analysis, W. B. Saunders Company, Philadelphia (1974).
J. Jacoby and M. S. Matell, Three-point Likert scales are good enough, Journal of Marketing Research 8 (1971), 495 – 500, DOI: 10.1177/002224377100800414.
R. A. Johnson and D. W. Wichern, Applied Multivariate Statistical Analysis, 6th ed., Pearson Education, New York (2007).
C. R. Kothari, Research Methodology: Methods and Techniques, 3rd ed., New Age International, New Delhi, India (2004).
R. C. MacCallum, M. W. Browne and H. M. Sugawara, Power analysis and determination of sample size for covariance structure modeling, Psychological Modeling 1(2) (1996), 130 – 149, DOI: 10.1037/1082-989X.1.2.130.
W. S. Martin, The effects of scaling on the correlation coefficient: A test of validity, Journal of Marketing Research 10(3) (1973), 316 – 318.
P. D. Mehta, M. C. Neale and B. R. Flay, Squeezing interval change from ordinal panel data: Latent growth curves with ordinal outcomes, Psychological Methods 9(3) (2004), 301 – 333, DOI: 10.1037/1082-989X.9.3.301.
E. Muraki, A generalised partial credit model: An application of an EM algorithm. Applied Psychological Measurement 16 (1992), 159 – 176.
P. Osteen, An introduction to using multidimensional item response theory to assess latent factor structures, Journal of the Society for Social Work and Research 1(2) (2010), 66 – 82, DOI: 10.4135/9781412985413.
R. Ostini and M. L. Nering, Polytomous Item Response Theory Models, Sage Publications, California (2006).
R Core Team, R: A Language and Environment for Statistical Computing, Computer software manual, Vienna, Austria, retrieved from https://www.R-project.org/ (2017).
M. D. Reckase, Multidimensional Item Response Theory, Springer, New York (2009).
B. B. Reeve, An Introduction to Modern Measurement Theory, National Cancer Institute, 1 – 67 (2002).
S. P. Reise, K. F. Cook and T. M. Moore, Evaluating the impact of multidimensionality on unidimensional item response theory model parameters, in: S. P. Reise and D. A. Revicki (eds.), Handbook of Item Response Theory: Applications to Typical Performance Assessment, pp. 13 – 40, Taylor and Francis, New York (2015).
S. P. Reise and Y. Yu, Parameter recovery in the graded response model using MULTILOG, Journal of Educational Measurement 27(2) (1990), 133 – 144, DOI: 10.1111/j.1745-3984.1990.tb00738.x.
A. C. Rencher, Methods of Multivariate Analysis, 2nd ed., John Wiley & Sons, New York (2002).
W. Revelle, psych: Procedures for Psychological, Psychometric, and Personality Research, Computer software manual, Evanston, Illinois, retrieved from https://CRAN.R-project.org/package=psych (R package version 1.7.8) (2017).
C. A. Stone, Recovery of marginal maximum likelihood estimates in the two-parameter logistic response model: An evaluation of MULTILOG, Applied Psychological Measurement 16 (1992), 1 – 16, DOI: 10.1177/014662169201600101.
B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, 6th ed., Pearson Education, New Jersey (2013).
Y. Takane and J. de Leeuw, On the relationship between item response theory and factor analysis of discretized variables, Psychometrika 52(3) (1987), 393 – 408, DOI: 10.1007/BF02294363.
C. van der Eijk and J. Rose, Risky business: Factor analysis of survey data - Assessing the probability of dimensionalisation, PLoS ONE 10(3) (2015), e0118900, DOI: 10.1371/journal.pone.0118900.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a CCAL that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
