Browsing by Author "Frank, Barrett"

Now showing 1 - 2 of 2

A classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism
(2019-01) Eaton, Philip; Johnson, Keith; Frank, Barrett; Willoughby, Shannon D.
For proper assessment selection understanding the statistical similarities amongst assessments that measure the same, or very similar, topics is imperative. This study seeks to extend the comparative analysis between the brief electricity and magnetism assessment (BEMA) and the conceptual survey of electricity and magnetism (CSEM) presented by Pollock. This is accomplished by using large samples (NBEMA ¼ 5368 and NCSEM ¼ 9905) within classical test theory (CTT) and item response theory (IRT) frameworks. For the IRT comparison, after consideration of the conceptual content addressed in each assessment, it was assumed that each of these assessments are measuring the same student latent ability (θ), specifically a student’s ability to do introductory electricity and magnetism. Via a CTT and IRT analysis it was found that both assessments are essentially equal in overall difficulty. Classical item analysis applied to 7 questions used by both assessments revealed that each assessment functions slightly differently internally. The test information curves found from IRT show that the CSEM has superior information compared to the BEMA in estimating student latent abilities for the entire range of typical latent abilities achieved by students on each assessment, θ ≈ −2 to θ ≈ 3. Information in this case is interpreted as how well a student’s latent ability was estimated by an assessment as a function of latent ability. When the circuits questions are removed from the BEMA the majority of the information is lost in the θ ≈ 0 to θ ≈ 2 range. This means the circuits questions on the BEMA are information heavy for higher ability scores. So, special considerations should be made as to which assessment a study uses depending on the specific questions a researcher is attempting to answer.
Comparing exploratory factor models of the Brief Electricity and Magnetism Assessment and the Conceptual Survey of Electricity and Magnetism
(2019-10) Eaton, Philip; Frank, Barrett; Johnson, Keith; Willoughby, Shannon D.
While numerous studies have analyzed the conceptions probed by the Force Concept Inventory (FCI), assessments dedicated to electricity and magnetism lack similar analyses. This paper investigated the conceptions explored by the Brief Electricity and Magnetism Assessment (BEMA) and the Conceptual Survey of Electricity and Magnetism (CSEM) using exploratory factor analysis techniques. Exploratory factor analysis (EFA) was performed on both assessments using 5368 and 4941 postinstruction student responses for the BEMA and CSEM, respectively. A 6-factor EFA generated model was found for the CSEM, and was fit against another sample of 4964 student responses using confirmatory factor analysis to supply evidence for the possible generalizability of the model. The 5-factor EFA generated model for the BEMA could not be fit against another sample when trying to check for generalizability. The EFA generated factor models for the BEMA and CSEM were then compared and found to be similar in conceptual content, with the exception of one or two factors. Thus, from a factor analysis perspective, the BEMA and CSEM were found to be similar in conceptual content as revealed by student responses. With a better understanding of these electricity and magnetism assessments, future research into this domain of physics will then be able to make stronger conclusions based on students’ results within these assessments.