Improving experimental methods: exploring procedural mechanisms affecting participant behaviors

Abstract

Research with human participants involves a complex combination of procedural elements in order to establish internal, external and measurement validity. Examining the accuracy of research equipment and methods that elicit similar behaviors as the general public is difficult. This research used driving as a model to address elements in the procedures that participants experience to elicit realistic behaviors. An instrumented vehicle (IV) and driving simulator (SIM) measured experimental behaviors for average approach speed (in the 20m before the legal stop line); lateral distance from curb at 20m; lateral distance from curb at legal stop line (0m) and the stopping location (distance before or after 0m); and, compared with measured general driving public behaviors at stop-controlled intersections. The linear mixed effect analyses combined two experiments. In both, surveys were administered to gather driver's trait anxiety, driving anxiety and social desirability scores. Experiment One drivers (36% female) were grouped as Novice (5, 16-17 year olds who just obtained driving license), Young (4, 16-17 year olds who obtained license over a year ago) and Adult (5, 30-55 year olds licensed near age 16). Experiment Two drivers (47 SIM, 44 IV; 35% female) were College age (18-21 year olds licensed near age 16) and exposed to 1 of 16 different combinations (one of those treatments matched Experiment One's procedure) of procedural changes for: researcher attire (casual or formal), researcher proximity (control room, front or rear passenger seat), mode of instruction delivery (spoken, read or video) and hypothesis statement (none or explicit). At the end of Experiment Two, participants' understanding of the experiment was coded into three debriefing variables. Absolute behavioral validity of the IV to public behavior was achieved in one treatment (formal, front seat, spoken and no hypothesis) and including the debriefing variables in the model; no SIM combination achieved this. Trait anxiety scores appeared to explain behaviors in the IV or SIM and improved result interpretation as interactions with other independent variables. For improved research methods, it is recommended that coded debriefing variables, specific procedural elements, and trait anxiety scores be included and used to explain interactions or differences in participant behaviors.