THE IMPACT OF IB INTERNAL ASSESSMENTS ON SCIENCE PRACTICES AND SCIENCE IDENTITY IN A HIGH SCHOOL SCIENCE CLASS by Michelle Marie Housenga A professional paper submitted in partial fulfillment of the requirements for the degree of Master of Science in Science Education MONTANA STATE UNIVERSITY Bozeman, Montana July 2021 ©COPYRIGHT by Michelle Marie Housenga 2021 All Rights Reserved ii DEDICATION To John, Thijs, and Lizzy for sharing me with this project for the last three years. Special appreciation to my colleagues, friends, and family that answered questions, checked my data, and proofread very rough drafts. Finally, to the students that make going to work every day something I look forward to. iii TABLE OF CONTENTS 1. INTRODUCTION AND BACKGROUND ....................................................................1 Context of the Study ........................................................................................................1 Focus Question.................................................................................................................3 2. CONCEPTUAL FRAMEWORK ....................................................................................4 International Baccalaureate Internal Assessment ............................................................4 Science Practices ..............................................................................................................5 Students’ Confidence in Science Classes ........................................................................9 Science Identity ................................................................................................................9 3. METHODOLOGY ........................................................................................................10 Demographics ................................................................................................................11 Treatment .......................................................................................................................11 Data Collection and Analysis Strategies ........................................................................13 4. DATA AND ANALYSIS ..............................................................................................16 Quantitative Results .......................................................................................................17 Qualitative Results .........................................................................................................25 5. INTERPRETATIONAND CONCLUSION ..................................................................27 6. VALUE ..........................................................................................................................30 Value of the Study and Consideration for Future Research ..........................................30 Impact of Action Research on the Author .....................................................................30 REFERENCES CITED ......................................................................................................32 APPENDICES ...................................................................................................................35 APPENDIX A: Institutional Review Board Approval ..........................................36 APPENDIX B: Science Practices Assessments .....................................................38 APPENDIX C: Science Practices Confidence Survey ..........................................53 APPENDIX D: Science Identity Survey................................................................55 APPENDIX E: Post IA Survey ..............................................................................57 iv LIST OF TABLES Table Page 1. Connection Between the Elements of the IB IA and the Science Practices ......................................................................................................8 2. Data Triangulation Matrix .................................................................................14 3. Averages of Student Responses to the Science Practices Assessments ...........................................................................................................17 4. Averages of Student Responses to the Science Practices Confidence Survey .................................................................................................19 5. Averages of Student Responses to the Science Identity Survey ........................21 v LIST OF FIGURES Figure Page 1. The Three Spheres of Activity for Scientists and Engineers ...............................7 2. Percent Change Between Pre-Assessment and Post-Assessment in Science Practice Assessments .............................................................................................18 3. Percent Change Between the Pre-Survey and Post-Survey on Science Practices Confidence ................................................................................20 4. Percent Change Between the Pre-Survey and Post-Survey on Science Identity .................................................................................................22 5. Science Practices Assessment Totals for Fall 2020 and Spring 2021 ...............23 6. Science Practices Confidence Survey Totals for Fall 2020 and Spring 2021....24 7. Science Identity Survey Totals for Fall 2020 and Spring 2021 .........................24 vi ABSTRACT process of completing the International Baccalaureate (IB) Internal Assessment for an IB science class pushes students to hone their mastery of the science practices. The science practices included in the Next Generation Science Standards provide skills that assist students in understanding the processes of scientific discovery and knowledge. This study examines the impact of completing the IB Internal Assessment on students’ mastery of the science practices, their confidence in the science practices and their science identity. The data used in this study was collected from pre- assessments and post-assessments of the science practices, pre-survey and post- survey of student confidence in the science practices, pre-survey and post- survey of student science identity, student feedback, and anecdotal observations made by the teacher. The results of the study indicate that student mastery increases in two of the science practices and student science identity increases in a couple of areas. 1 INTRODUCTION AND BACKGROUND Context of the Study Many students are fearful of the expectations in International Baccalaureate (IB) classes, specifically the internal assessment (IA). IB internal assessments in science consist of students designing, completing, and evaluating an individual investigation. There is often much discussion among the students about the value of the IA. This is one of the reasons to focus on the Next Generation Science Standards (NGSS) science and engineering practices. When learning about three-dimensional learning found in the Next Generation Science Standards (NGSS), I was not sure I wanted another initiative aimed at restructuring how and what I teach. However, after digging in deeper, I found myself sold on using the science practices as a vehicle for teaching content. Every year I found students lacking the skills necessary to succeed in a science class and not because they struggled with reading or math. Despite many years of yearlong science classes, students had not picked up the skills to succeed in science. The science and engineering practices are a way to prepare students to succeed in science regardless of the content. I wondered if that the intense individual investigation of the internal assessment would help students master the science practices. Washburn High School is a comprehensive public high school located in Minneapolis, Minnesota. As of the beginning of the 2020-2021 school year, 1701 students were enrolled. (Minneapolis Public Schools, 2020). The student body make-up includes 63.8% white, 21.9% black, 8.8% Hispanic, 4.1% Asian, and 1.3% native 2 American. Thirty-five percent qualify for free and reduced lunch assistance. Almost ten percent of the students have an Individualized Education Plan (IEP). Minneapolis Minnesota has a population of 429,606 according to a 2019 census with a median household income of $62,583 (QuickFacts Minneapolis City, Minnesota, 2021). Consistently, I have observed students struggle with writing explanations to their scientific questions. Over the fifteen years that I have been teaching, I have seen students shut down when asked to write explanations about their investigations or when given data to interpret and identify evidence. I have taught a range of classes from an introduction to high school science class for freshman to advanced academic classes for seniors to semester long science electives. These are necessary skills for success in advanced science courses. As an IB environmental systems and societies (IB ESS) teacher of eight years, watching students lack confidence to complete a formal lab report at the appropriate level has been very frustrating. The purpose of this study is to show that the IA can improve students’ science and engineering practices with emphasis on the science practices. The immediate reason to improve student mastery of science practices is to provide long-term value in successfully completing the internal assessment in an IB science class. The long-term reason to help students improve these practices is to ensure that they become scientifically literate citizens. If they are unable to recognize the components of a good scientific investigation and explanation, they may be persuaded to believe and support biased nonscientific articles. Also, many post-secondary institutions expect incoming students to be able to 3 write and evaluate scientific investigations. By preparing students to do these tasks successfully, not only will they perform better in my classes, but also in the future. Adding to an already full load of content can be overwhelming. Many students protest and argue that the IA is not a good investment of their time, skills, and attention. Hopefully, the data from this investigation will provide yet another benefit of the IA. The National Research Council developed the science and engineering practices alongside NGSS. The eight science practices include: 1. Asking questions 2. Developing and Using Models 3. Planning and Carrying Out Investigations 4. Analyzing and Interpreting Data 5. Using Mathematics and Computational Thinking 6. Constructing Explanations 7. Engaging in Argument from Evidence 8. Obtaining, Evaluating, and Communicating Information In this action research project, I would like to focus on the impact of the IA on student mastery of the science practices, confidence in the science practices, and science identity. The engineering practices are important as well, but for this study I am focusing on just the science components. My main questions have to deal with what will help students to perform better at the science and engineering skills. This is a big task focusing on just the science side of the science and engineering practices is a way to simplify things a bit. Focus Question The focus question for this study was, How will the completion of the IB internal assessment impact students’ mastery and confidence of use of the science practices in a high school IB environmental systems and societies class? 4 Sub-questions included the following: 1. How will the completion of the IB internal assessment impact students’ confidence in understanding and using the science practices? 2. How will the completion of the IB internal assessment impact students’ science identity? I am hoping that through exposure to science practices students will feel more confident in their science skills and more likely to view themselves as scientists. 5 CONCEPTUAL FRAMEWORK To understand the basis for this study, it is important to understand the International Baccalaureate Programme and its emphases. Another element of this study is the connection between the internal assessment and the NGSS science practices. Finally, background information about science identity will be given. International Baccalaureate Internal Assessment International Baccalaureate (IB) is an education system that focuses on shaping well-rounded learners equipped with the elements of their learner profile: Inquirers, Knowledgeable, Thinkers, Communicators, Principled, Open-minded, Caring, Risk- takers, Balanced, and Reflective (International Baccalaureate Organization [IBO], 2015). IB was started to accommodate international schools around the world in having a standard curriculum. There is a Primary Years Programme for ages 3-12 years, a Middle Years Programme for ages 12 – 16 years, and a Diploma Programme for ages 16-19 year. “An IB education provides students distinct advantages as they enter a world where asking the right questions is as important as discovering answers” (About IB, 2021). IB uses two assessments for determining Diploma Programme scores. One is the external assessment, which resembles a cumulative final exam. The second one is the internal assessment (IA). Each content area has slightly different elements. For IB ESS, the IA is a student-driven individual scientific investigation within the context of an environmental issue. According to the course guide, the IA “enables students to demonstrate the application of their skills and knowledge, and to pursue their personal 6 interests, without the time limitations and other constraints that are associated with written examinations” (IB Organization, 2015, p. 77). The elements included in the IA are as follows: Context, Planning, Results, Analysis, and Conclusion, Discussion and Evaluation, Application, and Communication. The IA needs to be 1500-2250 words in length and should be entirely student driven. Teachers are allowed to support their students through answering questions and giving feedback on one rough draft. Ten hours minimum of class time should be used to instruct students on the elements of the IA and helping them understand the IA rubric as determined by IB. Science Practices The science and engineering practices were developed by the National Research Council of the National Academies in consultation with professional scientists. “Engaging in the practices of science helps students understand how scientific knowledge develops; such direct involvement gives them an appreciation of the wide range of approaches that are used to investigate, model, and explain the world” (National Research Council [NRC], 2012, p. 42). This definitely provides the purpose to focusing on the science and engineering practices. These practices are a combination of knowledge and skill. “…we conclude that acquiring these practices supports a better understanding on how scientific knowledge is produced and how engineering solutions are developed. Such understanding will help students become more critical consumers of scientific information” (NRC, 2012, p. 41). One of the goals of this action research is to engage students and encourage future science study, both of which can be accomplished through the actual doing of science (NRC, 2012). In this way, the purpose of the science practices 7 is sense-making (Scharwz et al., 2017). Sense-making involves actively engaging learners with the natural world in order to figure out how the world works. The science and engineering practices encompass the ways that science is actually done. The eight practices can be broken into three distinct areas of scientific work: investigating, evaluating, and developing explanations and solutions as seen in Figure 1. By focusing on these real science practices, educators will equip students to be successful scientists. Figure 1. The three spheres of activity for scientists and engineers (NRC, 2012, p. 45). The National Science Teaching Association (NSTA) released a position statement about moving from more teacher-driven instruction to the three-dimensional model (Cross Cutting Concepts, Science and Engineering Practices, and the Disciplinary Core Concepts). This position of using the science and engineering practices allows educators to move away from the scientific method with the misconception that there is a set of programmed steps that scientists follow. The “science practices are not just the new 8 version of the scientific method. Practices are used in a purposeful way to build and revise knowledge about how and why the world works” (Shwarz et al., 2017, p. 357). In this action research, the IB IA will be used to assist in teaching and evaluating student mastery of the science practices. The components of the IA align with the science practices (Table 1). 9 Table 1. Connection between the elements of the IB IA and the science practices. Element of the IB IA Science Practice How that science practice is Context Asking Questions Students develop a focused coherent research question. Obtaining, Evaluating, and Students research and Communicating provide background Information information about their environmental issue. Planning Planning and Carrying Out Students design an Investigations investigation that collects sufficient and relevant data. Constructing and Using Students may include a Models model of their investigation and/or understand the elements of a model to design their investigation. Results, Analysis, and Analyzing and Interpreting Students arrange their data Conclusion Data in tables and graphs as well as identify trends and patterns. Using Mathematical and Students process their data Computational Thinking using statistical methods. Discussion and Evaluation Constructing Explanations Students explain the results of their investigation in the context of their environmental issue. Application Engaging in Argument Students justify a solution from Evidence or application for the environmental issue as a result of their investigation. Communication Obtaining, Evaluating, and Students need to concisely Communicating and cohesively Information communicate all of these elements. 10 The connections between the science practices and the IB IA components line up in a way that all science practices are incorporated within the assignment. This is the strong connection between the two. Students’ Confidence in Science Classes Students taking advanced academic science classes showed decreases in their confidence of doing well in college science courses (Conger et al., 2021). The study commented that the lower level of confidence led the students to learn more than in a non-advanced academic class. Science Identity One of the concerns that science teachers have is student confidence and identity as a scientist in order to increase engagement and mastery. “Identity can be defined as the composition of self-views that emerge from participation in certain activities and self- categorification in terms of membership in particular communities or roles” (Vincent- Ruz, 2018). One of the contributing factors to that identity includes belonging to a particular community. Using formative assessment in the classroom can improve the community of the classroom and hopefully impact students’ science identity. Another factor that leads to a stronger science identity is a match between science taught in school and the process of being a scientist (Archer et al., 2010). Science identity is composed of three main elements: recognition, competence, and performance (Carlone & Johnson, 2007). In order to measure a student’s science identity, each component needs to be taken into consideration. Another study done on 11 science identity adjusted the categories to add fascination and value and combined performance and competency (Vincent-Ruiz, 2007). That study used a survey with questions in each category. Competency is the ability to participate in science and having the scientific knowledge to complete investigations of the natural world. Fascination is the positive interest a student has towards science. Value includes the students’ understanding of the importance of science for their personal lives as well as society. Recognition is the idea of a person seeing themselves or having others see them as a person and as a scientist. All of these elements combine to create the science identity. The IB Programme is a rigorous course of study that requires students to drive their own academic achievement. The IA is an example of that self-directed requirement through an individual investigation. The NGSS science practices are woven throughout the IA process. Confidence in the science skills is impacted by a variety of factors. Science identity is a construct that can show students’ self-reflected interest, value, recognition, and competency in science. 12 METHODOLOGY In this study, the focus will be on the mastery of the science practices, student confidence in the science practices, and student science identity. These will be measured in a variety of ways. Treatment for the action research included instructing and supporting students with their required IA. Surveys on science practices confidence and science identity were given before and after the IA process. During the IA process, science practices were assessed for mastery and teacher anecdotal observations were collected. After the IA process, the surveys were given again and science practices were assessed. A student survey was given at the end of the process reflecting on the IA. Demographics The treatment was used with three sections of IB Environmental Systems and Societies (IB ESS). This consisted of 85 IB students. The IB students were all seniors except for two juniors. IB ESS is a transdisciplinary advanced academic science elective focusing on the role in society on causing and solving environmental problems. Of the 85 students in this course, six students were exited English Language Learners (ELL), one has an Individualized Education Plans (IEP), and nine have 504 accommodations. The racial identity is as follows: four students that identify as black, five that identify as Hispanic, and one student that identifies as Somali. The remaining 75 students identify as white. The research methodology for this project received an exemption by Montana State University's Institutional Review Board and compliance for work with human subjects was maintained (Appendix A). During the 2020-2021 school year, students were 13 in distance learning the entire time of the treatment. Classes met in a block schedule on Monday, Tuesday, Thursday, and Friday with an asynchronous learning day on Wednesdays. On block schedule days, three classes met for two hours each. Students have a total of six different classes. Treatment Due to the extreme changes in instruction methods during the COVID-19 pandemic, my initial treatment of focused formative assessments around the science practices as a way to increase mastery in the science practices needed to change. First of all, I was only teaching one section of regular Biology to sophomores, which was my originally intended study subject. Secondly, due to distance learning student attendance was more sporadic and unpredictable. Frustration around the IA being still in place made it a good treatment with three classes of students due to the larger sample size. The treatment is the process of completing the IA. The IA is typically introduced in November with a study of the rubric and having students score a high-quality example IA. Each step of the IA is broken down and suggested due dates and class time are given in order to scaffold this large assignment. Students are given a set of questions to help them reflect on which environmental issues they would like to explore more. They determine their data collection with most students doing their own experiment, using secondary data, or using surveys. Typically, they are required to have their research question, variables identified, and method of data collection determined before winter break at the end of December. After winter break, several class periods are spent working with data and representing data. In mid-February, students submit a rough draft 14 of their IA for teacher feedback. Teachers are allowed to offer feedback, but not edits. Feedback often includes referring back to the rubric and encouraging students to add to their IA. A final draft is submitted in early March. IB requires that teachers provide at least ten hours of in class instruction and time for the IA. This is easily exceeded every year. 15 Data Collection and Analysis Strategies Qualitative and quantitative data collection was used in this action research. Data was collected through student work, surveys and student feedback. This qualitative data helped understand why students may or may not feel confident in certain science practices. Quantitative data was collected through a rubric on a scientific investigation at the beginning of the study that includes aspects of each of the eight science practices. Data on science identity was collected through observation, survey, and student interviews. Students also completed a Likert confidence test before and after treatment. Table 2 shows the research questions and the data measurement tools. 16 Table 2. Data Triangulation Matrix. Research Question: How will the completion of the IB internal assessment impact students’ mastery and confidence of use of the science practices? Sub- Question Data Source 1 Data Source 2 Data Source 3 How will the Science Practices Free response from Anecdotal teacher completion of the Assessment students about their observations and IB internal mastery of the student check-ins assessment impact science practices students’ mastery of the science practices? How will the Science Practices Free response from Anecdotal teacher completion of the Confidence Survey students about their observations and IB internal confidence in the student check-ins assessment impact science practices students’ confidence in understanding and using the science practices? How will the Science Identity Free response from Anecdotal teacher completion of the Survey students about their observations and IB internal perceived science student check-ins assessment impact identity students’ science identity? Students completed Science Practice Assessments (Appendix B). These were short assessments focused one each one of the science practices. This was repeated each day until they have completed a short formative assessment about each of the science practices. These make up the pre-test for mastery of the science practices. A rubric was used to assess the student mastery on a scale of 0-3. The assessments were distributed as Google forms, interactive slides in a presentation, written assignments, or as components of a larger assignment. At the end of the treatment period, students will be given one focused formative assessment per day focusing on a different science practice each day. 17 These were discussed in class and feedback was given to students individually and as a class. The assessments were tied into the current content being covered in class. The pre- assessments focused largely on biomes and the Amazon rainforest, whereas the post- assessments focused on air quality. Students were not informed that they were completing a pre- and post-assessment of the science practices so as not to sway their efforts. Students took two Likert scale tests: Science Practices Confidence Survey (Appendix C) and Science Identity Survey (Appendix D). The Science Practices Confidence Survey contained affirmative statements about each of the science practices. Students selected from Agree, Mostly Agree, Mostly Disagree, and Disagree for each statement. For data analysis, each response was given a numerical value: Agree = 4, Mostly Agree = 3, Mostly Disagree = 2, and Disagree =1. The Science Identity Survey contained fifteen statements about science identity that students responded to with YES!!!!, yes, no, or NO! to show their agreement with the statement. These were also converted into a numerical value: YES!!! = 4, yes = 3, no = 2, and NO! = 1. These surveys were given as an independent time assignment. It was indicated that completing the survey would have no bearing on their grade. The survey was adapted from a previous science identity study with questions in each of the science identity components: recognition, fascination, value, and competency (Vincent-Ruiz, 2007). At the end of the treatment, students were given the same confidence and science identity survey they took before the treatment. Students completed a Post-IA Survey (Appendix E) to identify things they learned during the IA process and to provide 18 feedback. The teacher made anecdotal observations through the treatment and assessment process. The data was processed in multiple ways. Students who did not complete the pre- and post- for that data collection instrument were removed in order to accurately complete paired t-tests on the data. In order for the data to be considered significant in this study, the p-value needed to be less than 0.1. Averages were calculated and compared as well as range. Microsoft Excel was used to process the data. Qualitative data was collected through teacher notes and students open response surveys. At the end of the pre-survey and post-survey on science identity, students were asked to comment on their science identity. Throughout the process, teacher observations were collected focusing on perceived student effort, engagement, and conditions in which the data was collected. By collecting quantitative and qualitative data, I will get a better picture of the results of this study. The sample size provides enough data points to make strong conclusions about the results. By providing free response questions on the student survey, the responses are truly student driven. 19 DATA ANALYSIS Each assessment was tabulated in a spreadsheet for analysis. In order to do a paired t-test, only students who had completed the pre-assessment and post-assessment of each assessment were included. This meant that the sample size (n) for the different assessments are varied. Also, each scaled survey was given a numerical value. Qualitative data was examined for trend in free response questions and their agreement with the quantitative findings. The teacher anecdotal observations were studied for issues with experimental design and the conditions during the data collection pieces. Quantitative Results Table 3. Averages of student responses to the Science Practices Assessments. Science Practice n Fall 2020 Spring % Mean 2020 Change p-value Mean Asking Questions 55 2.13 2.04 -4.23 0.322 Developing and Using 57 1.98 2.12 7.07 0.336 Models Planning and Carrying Out 54 1.91 1.91 0 1.00 Investigations Analyzing and Interpreting 70 2.61 2.29 -12.26 0.000917 Data Using Mathematics and 66 2.05 1.92 -6.34 0.288 Computational Thinking Constructing Explanations 67 2.34 2.61 11.54 0.0132 Engaging in Argument 64 2.48 2.47 -0.40 0.877 From Evidence Obtaining, Evaluating, and 62 2.77 2.67 -3.61 0.223 Communicating Information. 20 15 Asking Questions Developing and Using Models Planning and Carrying Out Investigations Analyzing and Interpreting Data Using Mathematics and Computational Thinking 10 Constructing Explanations Engaging in Argument From Evidence Obtaining, Evaluating, and Communicating Information. 5 0 -5 -10 -15 Science Practices Figure 2. Percent change between pre-assessment and post-assessment in the science practice assessment scores (n varies by practice). The change in analyzing and interpreting data was significantly lower after completion of the IA. The pre-assessment average was 2.61 with the post-assessment averaging 2.29 with a p-vale of 0.000917 (Table 3) where a p-value less than 0.1 will be considered significant for this study. Constructing explanations averages showed a significant improvement from 2.34 to 2.61 with a p-value of 0.0132. Other areas stayed very similar in the pre and post assessment averages. Percent Change Between Pre- and Post- Science Practices Assessments 21 Table 4. Averages of student responses to the Science Practices Confidence Survey (n=75). The numerical representation aligns as follows: 4 = YES!!!!, 3 = yes, 2 = no, and 1 = NO! Statement Fall Spring % 2020 2021 Change p-value Mean Mean I can ask a scientific question that could 3.49 3.68 5.44 0.127 lead to an experiment. I can construct a model of a scientific 3.45 3.52 2.03 .0587 concept showing parts and how they interact. I can design an experiment with an 3.53 3.53 0 1.00 independent, dependent, and controlled variables. I can look at data in a graph or table and 3.65 3.57 -2.19 0.276 identify trends and patterns. I can use math to make calculations with 3.45 3.49 1.16 0.7335 data, including average, mode, percent change, and others. I can write an explanation using data to 3.80 3.65 -3.95 0.0858 support my reasoning. (Claim-Evidence- Reasoning) I can argue my point of view using data 3.65 3.65 0 1.00 and evidence to support my point of view. I can read a passage about scientific 3.39 3.57 5.31 0.188 information and communicate what I have learned from it to others. 22 6 Asking Questions 4 Developing and Using Models 2 Planning and Carrying Out Investigations Analyzing and Interpreting Data 0 Using Mathematics and Computational Thinking -2 Constructing Explanations Engaging in Argument From -4 Evidence Obtaining, Evaluating, and Communicating Information. -6 Science Practices Figure 3. Percent change between the pre-survey and post-survey on science practice confidence (n=75). Average student confidence in constructing and using models was significantly increased from 3.45 to 3.52 (Table 4). Constructing explanations averages significantly dropped from 3.80 to 3.65. Other areas stayed very similar between the pre-survey and post- survey averages. Percent Change Between Pre- and Post- Confidence Survey 23 Table 5. Averages of student responses to the Science Identity Survey (n=59). The numerical representation aligns as follows: 4 =Agree, 3 = Mostly Agree, 2 = Mostly Disagree, and 1 = Disagree. Statement Fall 2020 Spring % Mean 2021 Change p-value Mean I am a science person. 2.63 2.61 -0.76 0.821 My family thinks of me as a "science 2.29 2.29 0 1.00 person". My friends think of me as a "science 2.12 2.20 3.77 0.132 person". My teachers/mentors think of me as a 2.20 2.27 3.18 0.472 "science person". After a really interesting science lesson is 2.58 2.76 6.98 0.0400 over, I look for more information about it. I need to know how objects work. 2.75 2.75 0 1.00 I want to read everything about science. 2.20 2.20 0 1.00 I want to know everything about science. 2.56 2.46 -3.90 0.293 I want to know how to do everything that 2.39 2.36 -1.26 0.727 scientists do. Knowing science is important. 3.47 3.61 4.03 0.0445 Knowing science helps me understand 3.49 3.54 1.43 0.568 how the world works. Thinking like a scientist will help me do 3.20 3.24 1.25 0.698 well. I think I am very good at coming up with 2.69 2.73 1.49 0.709 questions about science. I think I am very good at doing 2.68 2.63 -1.87 0.582 experiments. I think I am very good at figuring out how 2.49 2.68 7.63 0.0701 to fix a science activity that didn't work. 24 10 8 6 4 2 0 -2 -4 -6 Questions from the science identity survey I am a science person. My family thinks of me as a "science person". My friends think of me as a "science person". My teachers/mentors think of me as a "science person". After a really interesting science lesson is over, I look for more information about it. I need to know how objects work. I want to read everything about science. I want to know everything about science. I want to know how to do everything that scientists do. Knowing science is important. Knowing science helps me understand how the world works. Thinking like a scientist will help me do well. I think I am very good at coming up with questions about science. I think I am very good at doing experiments. I think I am very good at figuring out how to fix a science activity that didn't work. Figure 4. Percent change between the pre-survey and post-survey on science identity (n=59). Percent change between pre- and post - treatment 25 In the area of science identity, two areas were statistically significant in their improvement. More students agreed that after an interesting lesson that they would seek out more information on the matter. The average moved from 2.58 to 2.76 (Table 5). More students agreed that knowing science is important with the average moving from 3.47 to 3.61. Another notable area is an increase in students’ feeling like they can figure out how to fix a scientific experiment that has did not work. Many areas stayed very similar in the pre and post assessment averages. Figure 5. Science Practices Assessment Totals for Fall 2020 and Spring 2021, (n=19). 26 Figure 6. Science Practices Confidence Survey totals for Fall 2020 and Spring 2021 (n=75). Figure 7. Science Identity Survey Totals for Fall 2020 and Spring 2021, (n=59). 27 Using the totals for each of the measurement tools yielded little differences between the pre- and post- scores of each tool. The averages for the science practices assessment totals, science practices confidence survey, and science identity survey showed little to no change between the pre and the post administration of them. The science practices confidence survey totals showed an overall increase in total score for students in the middle quartiles (Figure 6). Qualitative Results Student feedback (n=60) by survey after the internal assessment yielded interesting results. Despite the data above showing analyzing data as an area where students did not make improvements in, one third of the students referenced learning more about analyzing and interpreting data in a free response question about what they learned during the process. This varied from learning more about how to graph and represent data to looking for trends in data. One student said they learned about “presenting scientific data in multiple ways.” Another trend were comments about how to work with data. One student said they learned “how to take secondary data and manipulate it on my own. How to make scientific conclusions and write about it in a way that makes sense.” Like the previous quote, many students mentioned the self-efficacy. A student said they learned “how to set up and experiment that wasn’t already set up or guided by a teacher.” Another student said” I learned how to design an experiment and do statistical testing on data I collected myself.” Fifty-seven percent of students agreed that the internal assessment taught them how to do different science practices. The statements 28 with the most agreement included “The IA is good preparation for college.” with 72% agreeing and “The IA pushes me academically.” with 70% agreeing. At the end of the science identity survey, students were asked to freely comment about their science identity. This question was optional. In the pre-survey, one student commented “I don’t really have a ‘science identity’, but I am happy learning more about science because I truly do want to know how the world itself works. It’s all so mind boggling.” Several students commented about liking science, but not being good at it. Students mentioned that they like certain areas of science and not others. One student commented “I do like science a lot, but I don’t consider myself a ‘science person’” In the post-survey, several students mentioned again that they like it, but didn’t feel that they were good at it. Teacher anecdotal observations looked at the strength of the study and the conditions in which data was collected. The pre-assessments of the science practices were done in between December and February. This was around winter break as well as the semester break. Commitment to school work tends to be stronger at the start of a new semester. The post-assessments on the science practices were completed in March. During this time, announcements were made about returning to in-person school, which was stressful for students and staff. This may have impacted attendance and effort on the post-assessments. Another factor that impacted student effort was the typical senior slide, where seniors show less commitment to their academics due to being very close to graduating. This was true for my students as tardiness and absences increased. More 29 students were coming to class late and leaving the distance learning meetings early and general lack of engagement. Despite the reputation of the IA, very few students attended office hours, sent emails, or stayed after class for assistance. This was not helped by the Derek Chauvin trial for the murder of George Floyd starting in March. Since the school boundaries include the place where George Floyd was killed, many students had experienced the trauma of the event, the protests that followed, the destructive riots in their own neighborhoods, and especially the awareness around systemic inequities. With this trauma, every event connected with it brought up the feelings associated with the initial event. This led to a very disrupted school year emotionally with effects on academics. 30 CLAIM, EVIDENCE, AND REASONING The completion of the IB internal assessment did lead to increased mastery in constructing explanations and decreases mastery of analyzing and interpreting data. While the expectation was that all science practices would improve, a large part of the IA is explaining the results of the investigation. There was a slight increase in developing and using models, which could be due to the content of the class not just the IA process. IB ESS focuses on systems models of soil, water, air, and energy and the storages and their connections. Due to linking the science practice assessments to content in the class, the assessments are not totally equal and it is possible that one assessment may have been slightly more or less difficult than the other. In the instance of analyzing and interpreting data, the pre-assessment had fairly simple data, whereas the post-assessment had air quality data that the student had collected from a real time air quality website. This could have been the reason for the significant decrease in analyzing and interpreting data. Another factor is that all but two of the students are seniors, who are known to struggle with commitment to their academics as their senior year progresses, especially after college acceptance. The teacher observations included huge variety in the amount of effort put into these science practices assessments. This was even with the same students and not just between students. There is a difference between perceived confidence in constructing explanations and actual ability as seen by the average scores on pre- and post-science practice assessments and the science practices confidence surveys. This could be due to the fact 31 that the IA rubric is quite rigorous and many students are disappointed with their score. In 2018, IB awarded the highest score for IA’s that scored 24 out of 30 (International Baccalaureate Organization, 2018). This is equivalent to a B- in a standard grading scale. This could cause students to question their abilities concerning the science practices. Also being exposed to challenging experiences can make them more aware of their weaknesses. Students struggled with how to put everything together in a conclusion and discussion which could have led to the decrease in confidence in constructing explanation. Through overcoming difficult things, we may feel empowered. For many students, working with their own data with unknown outcomes, they grew in their ability to work with data. That could explain the increased confidence in mathematics and computational thinking. However, students felt less confidence in analyzing and interpreting the data. The science identity results showed that overall students’ perception of their science identity strengthened a small amount. There were significant increases “After a really interesting science lesson is over, I look for more information about it.” This could be due to the self-directed nature of the IA and the need to look for information about scientific items. Also since the IA topic is self-determined, students already had some interest in those areas. Student responses to “Knowing science is important.” also showed significant gains. This could be due to the content of the class being about environmental issues. Students often comment that they feel the content in the class is more relevant leading to it increasing in importance. 32 According to student feedback in the Post-IA survey, students self-identified what they learned in the process of completing the IA. Students agreed that the IA is a rigorous assignment, and that it prepares them for the future of more rigorous assignments in college. One student commented “I always hate IA's when I am working on them, but after they are done, I am happy I did them and am proud of my work.” While I cannot confidently say that the process of completing the IA will increase mastery of the science practices, confidence in the science practices, or science identity, I can still communicate that there is value in the assignment linked to preparation, achievement, and some areas of improvement in science practices. 33 VALUE Value of the Study and Consideration for Future Research Focus on the science practices should be a priority for all science teachers whether they teach IB or not. These practices will lead to students being better prepared to be contributing and critically thinking members of our society. This study found that there were some impacts on mastery and confidence in the science practices and in student’s science identity. This was even done during an incredibly stressful year of pandemic, social unrest, and distance learning. In the future, studies of mastery of science practices should be incorporated in general science classes as well. It would also be of interest to complete this study during a regular in-person school year. Just completing the elements of this study did push students to expand their science experience. Hopefully that will influence their lifelong interaction with science. Impact of the Action Research on the Author Not only has this project pushed me to improve my mastery of the science practices, but it has shown the importance of intentionality in our instruction. In order to complete the surveys and assessments, detailed planning was required. I see value in spending time to really dig in and analyze student data, but unfortunately unless I am in the midst of an action research project, I am racing to pull out vague trends and outliers in the precious spare time I have. In the future, I will be intentional with the science practices and keeping track of student data. Another component that will become part of my teaching is administering the science identity survey at the beginning and end of the 34 year. I found the results fascinating and wondered what I could do to help students understand the concept of science identity and to help students increase their science identity. This study has increased my interest in focusing on the science practices as well as continuing to use the Science Identity Survey. It is a professional goal of mine to see more students view themselves as “science people”. Between mastery of the science practices and improving students’ science identity, we can equip students to be the critical citizens we need to move forward as a society on important scientific issues, such as climate change, health care, and sustainability. 35 REFERENCES CITED 36 Archer, L., DeWitt, J., Dillon, J., Willis, B., & Wong, B. (2010) “Doing” science versus “being” a scientist: examining 10/11-year-old schoolchildren’s constructions of science through the lens of identity. Science Education, 94(4), 617-639. Carlone, H. B., & Johnson, A. (2007). Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching, 44(8), 1187-1218. Conger, D., Kennedy, A., Long, M. C., & McGhee, R. (2021). The effect of advanced placement science on students’ skills, confidence, and stress. The Journal of Human Resources, 56(1), 93-124. Fisher, D. (2009). The use of instructional time in the typical high school classroom. The Educational Forum, 73(2), 168-176. Keeley, P. (2008). Science formative assessment: 75 practical strategies for linking assessment, instruction, and learning. Corwin Press. About the IB. International Baccalaureate. www.ibo.org/about-the-ib/ International Baccalaureate Organization. (2018, May).Grade boundaries for Diploma programme coordinators [PDF file]. https://www3.dpcdsb.org/STFXS/Documents/IB-2018-05_Grade_Boundaries.pdf International Baccalaureate Organization. (2015) Environmental systems and societies guide. Mcneill, K., Katsh-Singer, R., & Pelletier, P. (2015). Assessing science practices: Moving your class along a continuum. Science Scope, 39(4), 21. Minneapolis Public Schools. Washburn High School 2020-2021 School Profile [PDF file]. https://washburn.mpls.k12.mn.us/uploads/2020-2021_school_profile.pdf National Research Council (NRC). (2012). A framework for K–12 science education: Practices, crosscutting concepts and core ideas. National Academies Press. National Research Council. (2013). Next Generation Science Standards: For states, by states. The National Academies Press. Popham, W. J. (2008). Transformative assessment. Association for Supervision and Curriculum Development. Quick Facts Minneapolis City, Minnesota (n.d.). United States Census Bureau. https://www.census.gov/quickfacts/minneapoliscityminnesota 37 Schwarz, C., Passmore, C., & Reiser, B. (2017). Helping students make sense of the world using next generation science standards. National Science Teachers Association. Summers, R., & Abd‐El‐Khalick, F. (2018). Development and validation of an instrument to assess student attitudes toward science across grades 5 through 10. Journal of Research in Science Teaching, 55(2), 172-205. Transitioning from Scientific Inquiry to Three-Dimensional Teaching and Learning. (2018). https://www.nsta.org/about/positions/3d.aspx Vincent-Ruz, P., & Shunn, C. (2018). The nature of science identity and its role as the driver of student choice. International Journal of STEM Education, 5(48). 38 APPENDICES 39 APPENDIX A INTERNATIONAL REVIEW BOARD APPROVAL 40 41 APPENDIX B SCIENCE PRACTICES ASSESSMENTS 42 Pretest Asking Questions: Look at the two pictures above showing the confluence of the Mississippi and Minnesota River. Ask two scientific questions about the images. 43 Rubric: 1 – Student wrote a simple question that could not be tested. 2 – Student wrote a simple question that could be tested. 3 – Student wrote a complex question that could be tested. Developing and using models: Construct a model of a tropical rainforest. On a piece of paper, draw a model of a tropical rainforest. Include three storages and four processes (flows). Submit a photograph of your model. Rubric: 1 – Student drew a simple system diagram with two of the required items 2—Student correctly drew a system diagram with five of the required items 3 – Student correctly drew a system diagram with all the required items Planning and carrying out investigations: 44 You are given bean seeds, a window, a heat lamp, a grow light, a thermometer, a graduated cylinder, potting soil, compost, fertilizer, cotton, pots, plastic bags, water, and rainwater. Construct a research question you could test. What is your independent variable? What is your dependent variable? List three controlled variables. Rubric: 1—Students construct a research question that is not testable and correctly identify one variable. 2— Students construct a research question that is testable and correctly identify two variables. . 3— Students construct a research question that is testable and correctly identify all variables. Analyzing and interpreting data: Construct and label a graph of the information provided on a piece of paper. Submit a photo of it. It does not need to be done on graph paper and exact, but should accurately represent the data. 45 Rubric: 1—Students partially construct a graph with large omissions or mistakes 2—Students construct a graph, but there are some omissions 3 – Students correctly construct and label a graph Using mathematics and computational thinking: 46 What is the sample size (N) for the June data set? What is the mean of the April data set? What is the median of the June data set? What is the percent change from April to June? Type enough to show how you got your answer. Rubric: 1--- Students correctly answer one question 2—Students correctly answer two questions 3 – Students correctly answer all questions Constructing explanations: 47 Explain the information in the graph above. Do you see any patterns or trends? Rubric: 1 – Student has written an explanation with no reference to information in the graphic 2 – Student has written an explanation with little evidence and/or connection to the graphic 3—Student has written a well-crafted explanation with strong evidence and/or connection to the graphic. Engaging in argument from evidence: 48 49 50 Based on the data above, specifically where should tropical rainforest conservation efforts be focused? Provide at least two pieces of evidence to support your claim. Rubric: 1—Student states a location for conservation efforts and includes a superficial reason. 2—Student states a location for conservation efforts and include on solid reason with evidence. 3 – Student states a location for conservation efforts and includes at least two solid reasons with evidence Obtaining, evaluating, and communicating information: Read the first seven paragraphs of the article. https://www.sciencenews.org/article/invasive-jumping-worms-damage-soil-threaten- forests 51 Summarize the information in this article into three well-crafted sentences: Rubric: 1—Student wrote one well-crafted sentence summarizing the article 2 – Student wrote two well-crafted sentences summarizing the article 3 – Student wrote three well-crafted sentences summarizing the article. Post test Asking questions Rubric: 1 – Student wrote one simple question that could not be tested. 2 – Student wrote one simple question that could be tested. 3 – Student wrote two complex question that could be tested. Planning and carrying out investigations 52 Using plants, design an experiment that will test their tolerance to climate change. What is your research question? What is your independent variable? What is your dependent variable? Describe three controlled variables. Developing and using models Look in your book pages 274-277 (6.1) and other resources and draw a labeled model of the greenhouse effect. Include four types of greenhouse gases. Take a picture of your model and submit it here. Rubric: 1 – Student drew a simple system diagram with one of the required items 2—Student correctly drew a system diagram with two of the required items 3 – Student correctly drew a system diagram with all the required items Analyzing and interpreting data Gather Air Quality Index data on both cities five times over 2 consecutive days. For example, today at 2pm and 8pm and tomorrow at 9 am, 1 pm, and 6 pm. They must be at least four hours apart. Create a graph showing the air quality index for both cities on the same graph- remember to take into account time differences. 53 Rubric: 1—Students partially construct a graph with large omissions or mistakes 2—Students construct a graph, but there are some omissions 3 – Students correctly construct and label a graph Using mathematical and computational thinking PM 10 Data for the Upper Midwest according to the EPA https://www.epa.gov/air-trends/particulate-matter-pm10-trends Year Mean 2000 62.64 2001 59.36 2002 61.21 2003 64.43 2004 64.71 2005 65.5 2006 56.29 2007 60.98 2008 49.60 2009 45.5 2010 53.71 2011 50.57 2012 48.79 2013 42.57 2014 44.93 2015 48.57 2016 48.79 2017 46.29 54 2018 47.29 2019 44.64 Calculate the average PM10 measurement for the upper Midwest between 2000-2019. Calculate the percent change from 2000 to 2019 for PM10 measurements for the upper midwest. Rubric: 1--- Students have provided an answer. 2—Students correctly answer one questions 3 – Students correctly answer both questions Constructing explanations Evaluate the air quality of each city. Use three pieces of evidence from your data table. Imagine this is like a CER with the question “What is the air quality like in the city of ___?” Consider factors impacting the air quality in your reasoning. Rubric: 1 – Student constructs a simple explanation of the city’s air quality. 2 – Student constructs an explanation and includes one piece of evidence 3 – Student constructs a well-crafted explanation with three pieces of evidence Engaging in Argument from evidence Justify which city has better air quality. Again, like a CER with the question “Which city has better air quality?” Provide three pieces of evidence to support your claim. 55 Rubric: 1 – Student has an argument for which city has better air quality 2 – Student has an argument for which city has better air quality with at least one piece of supporting evidence 3 - Student has an argument for which city has better air quality with at least three pieces of supporting evidence Obtaining, evaluating, and communicating information https://phys.org/news/2021-03-oceans-emitting-ozone-depleting- cfcs.html?fbclid=IwAR3_mpqaydthxTDh-E8- 4f_BxNpNDaQdMHwL_jlLRVo1RIq3S8MA4iL7HBs Write a four-five sentence paragraph summarizing the article. Rubric: 1—Student wrote one well-crafted sentence summarizing the article 2 – Student wrote three well-crafted sentences summarizing the article 3 – Student wrote four well-crafted sentences summarizing the article. 56 APPENDIX C SCIENCE PRACTICES CONFIDENCE SURVEY 57 Science Practices Confidence Survey Participation in this research is voluntary and participation or non- participation will not affect a student’s grade or class standing in any way. Place an ‘x’ in the box that best represents your response to the statements. Mostly Agree Disagree Mostly Agree Disagree I can ask a scientific question that could lead to an experiment. I can construct a model of a scientific concept showing the parts and how they interact. I can design an experiment with an independent, dependent, and controlled variables. I can look at data in a table or graph and identify trends and patterns. I can use math to make calculations with data, including average, mode, percent change, and others. I can write an explanation using data to support my reasoning. (Claim-Evidence- Reasoning) I can argue my point of view using data and evidence to support my point of view. I can read a passage about scientific information and communicate what I have learned from it to others. 58 APPENDIX D SCIENCE IDENTITY SURVEY 59 Science Identity Survey: Participation in this research is voluntary and participation or non- participation will not affect a student’s grade or class standing in any way. Place an ‘x’ in the box that best represents how you feel. YES! yes No NO! I am a science person. My family thinks of me as a “science person” My friends think of me as a “science person” My teachers/instructors think of me as a “science person” After a really interesting science activity is over, I look for more information about it. I need to know how objects work. I want to read everything I can find about science. I want to know everything about science. I want to know how to do everything that scientists do. Knowing science is important. Knowing science helps me understand how the world works. Thinking like a scientist will help me do well. I think I am very good at coming up with questions about science. I think I am very good at doing experiments. I think I am very good at figuring out how to fix a science activity that didn’t work. *Adapted from “The nature of science identity and its role as a driver in student choices” by Paulette Vincent-Ruz and Christian D. Shunn. 60 APPENDIX E POST IA SURVEY 61