DETERMINING THE IMPACT ON THE PROFESSIONAL LEARNING OF GRADUATES OF A SCIENCE AND PEDAGOGICAL CONTENT KNOWLEDGE- BASED GRADUATE DEGREE PROGRAM by Alyson Mary Mike A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Education in Education MONTANA STATE UNIVERSITY Bozeman, Montana April, 2010 ©COPYRIGHT by Alyson Mary Mike 2010 All Rights Reserved ii APPROVAL of a dissertation submitted by Alyson Mary Mike This dissertation has been read by each member of the dissertation committee and has been found to be satisfactory regarding content, English usage, format, citation, bibliographic style, and consistency, and is ready for submission to the Division of Graduate Education. Dr. Elisabeth Swanson Approved for the Department of Education Dr. Joanne Erickson Approved for the Division of Graduate Education Dr. Carl A. Fox iii STATEMENT OF PERMISSION TO USE In presenting this dissertation in partial fulfillment of the requirements for a doctoral degree at Montana State University, I agree that the Library shall make it available to borrowers under rules of the Library. I further agree that copying of this dissertation is allowable only for scholarly purposes, consistent with “fair use” as prescribed in the U.S. Copyright Law. Requests for extensive copying or reproduction of this dissertation should be referred to ProQuest Information and Learning, 300 North Zeeb Road, Ann Arbor, Michigan 48106, to whom I have granted “the exclusive right to reproduce and distribute my dissertation in and from microform along with the non- exclusive right to reproduce and distribute my abstract in any format in whole or in part.” Alyson Mary Mike April 2010 iv ACKNOWLEDGMENTS I would like to acknowledge those people who supported me through the process of completing my dissertation. I would like to thank my committee – especially Elisabeth Swanson for her reviewing, editing, commenting and overall support from start to finish. Dr. Art Bangert provided insights and recommendations in preparing the statistical analysis. Dr. Peggy Taylor and Dr. John Graves for their continued encouragement and editorial comments. In addition, a huge thank you to Micki MacGregor for all the magic she does in making sure all the i’s are dotted and the t’s are crossed! I would like to thank Tim Joern for his help with coding and being my cheerleader. I would also like to thank Jill Miller for reading, editing, transcribing, and her continued support. She was a constant sounding board through the process. Finally, I would like to dedicate this dissertation to my mom, Gillian Mike (1931- 2005). She impressed upon me the importance of taking on a challenge and seeing it to completion. This research was supported by the Center for Learning and Teaching in the West (NSF Award #0119786) vTABLE OF CONTENTS 1. INTRODUCTION ....................................................................................................1 Background ...............................................................................................................1 Statement of the Problem..........................................................................................4 Purpose of the Study .................................................................................................6 Research Questions...................................................................................................6 Definitions of Terms .................................................................................................8 Significance of the Study..........................................................................................9 Outline of the Study................................................................................................13 2. LITERATURE REVIEW .......................................................................................15 Introduction.............................................................................................................15 Professional Learning .............................................................................................15 Categorizing Professional Learning .................................................................16 Summary...........................................................................................................25 Professional Growth through Professional Learning..............................................26 Professional Learning and MSSE .....................................................................31 Summary...........................................................................................................33 Global Graduate Program Recommendations ........................................................34 Graduate Program Recommendations ..............................................................34 Other Graduate Program Advocates .................................................................35 Graduate Program Curriculum Recommendations...........................................36 Summary...........................................................................................................43 Graduate Programs as Professional Development..................................................44 Essential Features of Professional Development....................................................46 Structural Components of Professional Development......................................48 Core Components of Professional Development..............................................49 Content Focus ...................................................................................................50 Active Learning ................................................................................................52 Coherence .........................................................................................................54 Summary...........................................................................................................57 Professional Development Standards .....................................................................58 The Context Standards......................................................................................60 The Process Standards ......................................................................................61 The Content Standards......................................................................................62 Summary...........................................................................................................64 Distance Education .................................................................................................66 Community of Inquiry Model.................................................................................67 Essential Components of the Online Learning Environment ...........................69 Summary...........................................................................................................72 vi TABLE OF CONTENTS – CONTINUED Connecting Professional Learning, Professional Development and the Community of Inquiry Model ....................................................................74 Summary...........................................................................................................80 Distance Education Supports Geographically Separated Learners.........................81 Summary...........................................................................................................83 Summary: The Need for Further Study ..................................................................83 3. RESEARCH METHODOLOGY............................................................................89 Introduction.............................................................................................................89 Program Context ...............................................................................................90 Participant Context............................................................................................93 Research Procedures ...............................................................................................95 Overall Design of the MSSE Professional Learning Survey..................................98 Pilot Survey.....................................................................................................102 Survey Implementation...................................................................................103 Data Analysis ..................................................................................................104 Overall Design of the Case Study.........................................................................105 Case Participants.............................................................................................107 Pilot Case Study Interviews ............................................................................108 Case Study Implementation ............................................................................109 Case Study Data Analysis...............................................................................110 Study Fidelity........................................................................................................112 Delimitations/Limitations .....................................................................................113 4. RESULTS OF THIS STUDY...............................................................................116 Introduction...........................................................................................................116 MSSE Student Survey...........................................................................................116 Section I Student Survey – Professional Experience and Education..............117 Precision of Responses and MPLS Instrument Reliability .............................120 Section II Student Survey – MSSE Students’ Science Content Understanding and Knowledge.......................................................................121 Section III Student Survey – Professional Learning.......................................126 Professional Learning through Using Research to Inform Best Practice........................................................................................131 Professional Learning through Collaboration.....................................133 Professional Learning through Reflective Practice.............................135 Professional Learning about Supporting Student Achievement .........137 Areas of Least Influence .....................................................................138 Content Knowledge and Associated Learning Strategies ...................141 vii TABLE OF CONTENTS – CONTINUED Section IV Student Survey – MSSE Students’ Enhancement and Change in Practice Skills and Knowledge ......................................................144 Open Ended Survey Responses Regarding Science Content Knowledge ............................................................................146 Open Ended Responses on Science Content Knowledge: Connections to the Pedagogy of Teaching Science ............................148 Open Ended Responses on Science Content Knowledge: Connections to Action Research.........................................................149 Open Ended Responses on Science Content Knowledge: Connections to Categories of Professional Learning..........................149 Open Ended Responses on Science Content Knowledge: Perspectives on Distance and Face-to-Face Learning ........................151 Summary.........................................................................................................152 Survey Responses Regarding Pedagogy for Teaching Science......................153 Open Ended Responses on Science Pedagogy: Connections to Content Knowledge ...................................................155 Open Ended Responses on Science Pedagogy: Connections to General Pedagogy and Pedagogical Content Knowledge...................155 Open Ended Responses on Science Pedagogy: Connections to Action Research.........................................................158 Open Ended Responses on Science Pedagogy: Connections to Professional Learning ................................................159 Open Ended Responses on Science Pedagogy: Perspectives on Distance and Face-to-Face Learning ..................................................160 Summary.............................................................................................161 Survey Responses Regarding Science Curriculum.........................................162 Open Ended Responses on Curriculum: An Overview......................164 Open Ended Responses on Curriculum: Connections to Action Research..............................................................................................167 Open Ended Responses on Curriculum: Connections to Professional Learning .........................................................................168 Open Ended Responses on Curriculum: Perpectives on Distance and Face-to-Face Learning ..................................................169 Summary.............................................................................................169 Survey Responses Regarding Diverse Learners .............................................170 Open Ended Responses on Supporting Diverse Learners: An Overview.......................................................................................172 Open Ended Responses on Supporting Diverse Learners: Connections to Action Research.........................................................175 Open Ended Responses on Supporting Diverse Learners: Connections to Professional Learning ................................................175 viii TABLE OF CONTENTS – CONTINUED Open Ended Responses on Supporting Diverse Learners: Perspectives on Distance and Face-to-Face Learning ........................177 Summary.............................................................................................178 Summary of Student Surveys..........................................................................179 MSSE Faculty Survey............................................................................................181 Section I Faculty Survey – Professional Experience and Education ..............182 Precision Responses and MPLS Instrument Reliability .................................183 Section II Faculty Survey – MSSE Students’ Science Content Understanding and Knowledge.......................................................................184 Section III Faculty Survey – MSSE Students’ Professional Learning ...........189 Section IV Faculty Survey – MSSE Students’ Science Content Understanding and Knowledge.......................................................................198 Summary of Faculty Survey .................................................................................202 MSSE Student Case Studies .................................................................................205 Charles ............................................................................................................208 Motivation to Pursue a Degree ...........................................................209 Inquiry – Pedagogical and Content Influences ...................................210 Social Capital – Social Connections ...................................................213 Critical & Responsive Learning – Reflective Practice .......................214 Valuing Learning ................................................................................215 Content Knowledge ............................................................................216 Online Learning ..................................................................................219 Sue...................................................................................................................220 Motivation to Pursue a Degree ...........................................................220 Inquiry – Pedagogical and Content Influences ...................................221 Social Capital – Social Connections ...................................................223 Critical & Responsive Learning – Reflective Practice .......................224 Valuing Learning ................................................................................225 Content Knowledge ............................................................................226 Online Learning ..................................................................................227 Action Research ..................................................................................228 Barb.................................................................................................................229 Motivation to Pursue a Degree ...........................................................229 Inquiry – Pedagogical and Content Influences ...................................230 Social Capital – Social Connections ...................................................234 Critical & Responsive Learning – Reflective Practice .......................235 Valuing Learning ................................................................................238 Content Knowledge ............................................................................239 Online Learning ..................................................................................242 Action Research ..................................................................................242 Julie .................................................................................................................244 Motivation to Pursue a Degree ...........................................................244 ix TABLE OF CONTENTS – CONTINUED Inquiry – Pedagogical and Content Influences ...................................245 Social Capital – Social Connections ...................................................249 Critical & Responsive Learning – Reflective Practice .......................250 Valuing Learning ................................................................................251 Content Knowledge ............................................................................253 Online Learning ..................................................................................259 Caleb ...............................................................................................................260 Motivation to Pursue a Degree ...........................................................261 Inquiry – Pedagogical and Content Influences ...................................261 Social Capital – Social Connections ...................................................263 Critical & Responsive Learning – Reflective Practice .......................264 Valuing Learning ................................................................................265 Content Knowledge ............................................................................266 Online Learning ..................................................................................270 Sean.................................................................................................................271 Motivation to Pursue a Degree ...........................................................271 Inquiry – Pedagogical and Content Influences ...................................272 Social Capital – Social Connections ...................................................273 Critical & Responsive Learning – Reflective Practice .......................274 Valuing Learning ................................................................................275 Content Knowledge ............................................................................276 Online Learning ..................................................................................277 Action Research ..................................................................................277 Summary of Case Studies ...............................................................................278 Summary of Overall Findings.........................................................................281 5. FINDINGS, CONCLUSIONS, AND IMPLICATIONS......................................286 Introduction...........................................................................................................286 Summary of the Study ..........................................................................................286 Findings.................................................................................................................289 The Student MSSE Professional Learning Survey .........................................290 The Faculty MSSE Professional Learning Survey .........................................294 Case Studies ....................................................................................................297 Conclusions...........................................................................................................299 Research Question 1 .......................................................................................299 Research Question 2 .......................................................................................304 Research Question 3 .......................................................................................311 Implications...........................................................................................................312 Recommendations for a Comprehensive, Research and Experience-Based Framework for Graduate Level Professional Development for Science Teachers ................................................................312 xTABLE OF CONTENTS – CONTINUED Recommendation to Explicitly Strengthen the Focus on Student Learning In Graduate Professional Development for Science Teachers........................313 Recommendations for MSSE Faculty and Program Staff .............................316 Considerations for Teachers Selecting a Master’s Program...........................321 Future Research ....................................................................................................321 Additional Research on Professional Learning...............................................322 Additional Research on Graduate Programs...................................................323 Additional Research on the MSSE Program...................................................323 Summary...............................................................................................................325 REFERENCES CITED...............................................................................................328 APPENDICES ............................................................................................................338 APPENDIX A: MSSE Student Informed Consent for Participation in Research Form...............................................339 APPENDIX B: MSSE Faculty Informed Consent for Participation in Research Form...............................................342 APPENDIX C: MSSE Student Professional Learning Survey .......................345 APPENDIX D: MSSE Faculty Professional Learning Survey.........................358 APPENDIX E: Comparison of Language and Phrasing Between the Learning How to Learn Survey and the MSSE Student Professional Learning Survey.................................................368 APPENDIX F: Comparison of Language and Phrasing Between the Learning How to Learn Survey and the MSSE Faculty Professional Learning Survey.................................................371 APPENDIX G: Comparisons of Language and Phrasing from the 2000 National Survey of Science and Mathematics Education, Question 12b, and the MSSE Student Professional Learning Survey and the MSSE Faculty Professional Learning Survey374 APPENDIX H: Descriptive Statistics for Questions 12b of the 2000 National Survey of Science and Mathematics Education .......376 APPENDIX I: Comparisons of Language and Phrasing from the eMSS (Electronic Mentoring for Student Success) Teacher Growth Survey –Mathematics and the MSSE Student Professional Learning Survey.................................................379 APPENDIX J: Alignment of the Research Questions and Methods of Quantitative Analysis with the Questions of the MSSE Professional Learning Survey.................................................382 APPENDIX K: MSSE Student Case Study Interview Questions ....................386 APPENDIX L: MSSE Student Professional Learning Portfolio .....................389 APPENDIX M: MSSE Student Professional Rubric ........................................397 APPENDIX N: MSSE Student Professional Learning Portfolio Rubric..........400 xi LIST OF TABLES Table Page 1. Effective Professional Development Components from the Research Literature ...........................................................................................11 2. Summary of Professional Learning Components: Research and Practitioner Literature .......................................................................................26 3. Research Literature and Policy Recommendations for Graduate Programs............................................................................................44 4. Effective Professional Developments Structural Components from the Research Literature .....................................................................................49 5. Effective Professional Development Core Components from the Research Literature ..........................................................................................56 6. Overview of the MSSE Course Offered ...........................................................92 7. MSSE Program of Study...................................................................................93 8. MSSE Students Currently Enrolled and Graduates, 2009 ................................94 9. MSSE Student Characteristics, 2009 ................................................................94 10. Overview of Research Exploring Professional Learning of the MSSE Graduates...............................................................................................96 11. Alignment of the Dimensions of Professional Learning and Case Study Interview Questions for the MSSE Students ........................................106 12. MSSE Respondents Graduation Dates, (N = 183)..........................................118 13. MSSE Student Perceptions of the Influences of Course Experiences on Knowledge and Practices in the Following Areas Related to Science Content ..............................................................................................122 14. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Science Content Most Influenced by Course Experiences, (N =369) ....................................................................................125 xii LIST OF TABLES - CONTINUED Table Page 15. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Science Content Least Influenced by Course Experiences, (N=369) .....................................................................................126 16. MSSE Student Perceptions of the Influence of Course Experiences on Professional Learning. Descriptive Statistics and Frequency of Responses, (N=369) ...................................................................................127 17. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning through Using Research to Inform Best Practice Most Influenced by Course Experiences, (N=369) ......133 18. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning through Collaboration Most Influenced by Course Experiences, (N=369).........................................135 19. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning through Reflection Most Influenced by Course Experiences, (N=369).........................................137 20. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning About Supporting Student Achievement Most Influenced by Course Experiences, (N=369) ..................138 21. MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning Least Influenced by Course Experiences, (N=369).........................................................................140 22. MSSE Student Perceptions of the Influence of Course Experience on Science Content Understanding and Knowledge, and on Learning Strategies for Furthering Content Understanding. Descriptive Statistics and Frequency of Responses, (N=369)..................................................................142 23. MSSE Student Perceptions of Areas of Science Content Understanding and Associated Learning Strategies Most Influenced by Course Experiences, (N=369)....................................................................144 24. Comparison of MSSE Students’ Perceptions of Influence on Science Content Knowledge by Course Type, ANOVA Comparisons of Means Across Course Types and Science Content Knowledge .................................145 xiii LIST OF TABLES - CONTINUED Table Page 25. Comparison of MSSE Students’ Perceptions of Influence on Pedagogy for Teaching Science by Course Type, ANOVA Comparisons of Means Across Course Types and Pedagogy for Teaching Science............................153 26. Comparison of MSSE Students’ Perceptions of Influence on Science Curriculum by Course Type, ANOVA Comparisons of Means Across Course Types and Science Curriculum...........................................................163 27. Comparison of MSSE Students’ Perceptions of Influence on Teaching Diverse Learners by Course Type, ANOVA Comparisons of Means Across Course Types and Diverse Learners ...................................171 28. MSSE Faculty Perceptions of the Influence of Course Experiences on MSSE Students’ Knowledge and Practices in the Following Areas Relating to Science Content. Descriptive Statistics and Frequency of Responses, (N=24) .....................................................................................185 29. MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Science Content Most Influenced by Course Experiences, (N=24) .......................................................................................188 30. MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Science Content Least Influenced by Course Experiences, (N=24) .......................................................................................189 31. MSSE Faculty Perceptions of the Influence of Course Experiences on the MSSE Students’ Professional Learning. Descriptive Statistics and Frequency of Responses, (N=24).............................................................190 32. MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Professional Learning Most Influenced by Course Experiences .....................................................................................................196 33. MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Professional Learning Least Influenced by Course Experiences .....................................................................................................198 34. MSSE Program Experiences Perceived by Faculty to Foster the MSSE Students’ Pedagogical Content Knowledge. Descriptive Statistics and Frequency of Responses, (N=24).............................................................199 xiv LIST OF TABLES - CONTINUED Table Page 35. MSSE Faculty Perceptions of Students Pedagogical Content Knowledge Most Influenced by Course Experiences .....................................201 36. MSSE Faculty Perceptions of Students Pedagogical Content Knowledge Least Influenced by Course Experiences ....................................202 xv LIST OF FIGURES Figure Page 1. Community of Inquiry Model.............................................................................4 2. Professional Development Components for a Change in Practice ..................56 3. Practical Inquiry Model ...................................................................................67 4. Community of Inquiry Model...........................................................................68 5. Relationships Between Interactions and Learning............................................70 6. Effective Components for Professional Learning, Graduate Programs, and Professional Development as Described by the Research Literature and Policy Recommendations...........................................................................88 7. Types of University Degrees of the MSSE Students and Graduates ..............119 8. Charles Portfolio Excerpt................................................................................218 9. Barb’s Diagnostic Learning Log.....................................................................232 10. Barb’s Content Form and Function Outline....................................................241 11. Julie’s Commentary on Student Data .............................................................248 12. Julie’s Project Handout ...................................................................................255 13. Student Work Sample on Observation............................................................256 14. Question of the Day Student Work Sample....................................................257 15. Investigation Student Work Sample ...............................................................258 16. Video Segments on Caleb’s Blog ...................................................................269 17. Diagram from Caleb’s Blog............................................................................269 18. Photograph of Geologic Formation from Caleb’s Blog..................................270 xvi LIST OF FIGURES – CONTINUED Figure Page 19. Essential Components for Professional Learning, Graduate Programs and Professional Development as Described by the Research Literature and Policy Recommendations ........................................288 20. Graduate Program Recommendations ............................................................301 xvii ABSTRACT This study examined the professional learning of participants in a science and pedagogical content knowledge-based graduate degree program, specifically the Master of Science in Science Education (MSSE) at Montana State University. The program’s blended learning model includes distance learning coursework and laboratory, field and seminar experiences. Three-quarters of the faculty are scientists. The study sought to identify program components that contribute to a graduate course of study that is coherent, has academic rigor, and contributes to educator’s professional growth and learning. The study examined the program from three perspectives: recommendations for teachers’ professional learning through professional development, components of a quality graduate program, and a framework for distance learning. No large-scale studies on comprehensive models of teacher professional learning leading to change in practice have been conducted in the United States. The literature on teachers’ professional learning is small. Beginning with a comprehensive review of the literature, this study sought to identify components of professional learning through professional development for teachers. The MSSE professional learning survey was designed for students and faculty, and 349 students and 24 faculty responded. The student survey explored how course experiences fostered professional learning. Open-ended responses on the student survey provided insight regarding specific program experiences influencing key categories of professional learning. A parallel faculty survey was designed to elicit faculty perspectives on the extent to which their courses fostered science content knowledge and other aspects of professional learning. Case study data and portfolios from MSSE students were used to provide deeper insights into the influential aspects of the program. The study provided evidence of significant professional learning among science teacher participants. This growth occurred in key categories defined in the literature including inquiry, building social capital, critical and responsive learning, valuing learning and building content knowledge. In general, findings from students and faculty were well aligned. Based on respondents’ recommendations, the author recommended a stronger emphasis on pedagogical content knowledge, analysis of student work, and strategies for differentiating instruction for diverse learners. Conclusions include a framework and roadmap elements to inform those working to improve graduate education or professional development for science teachers. 1CHAPTER 1 INTRODUCTION Background This study sought to examine the professional learning of the participants of a science and pedagogical content knowledge-based graduate degree program, specifically the Master of Science in Science Education (MSSE) at Montana State University. The characteristics of such a program were explored through the lens of research and policy recommendations for graduate education and professional development standards for teachers. The MSSE program is available asynchronously online and this study examined the impact this feature has on the program participants and graduates. The MSSE program was established in 1996 as an intercollegiate and interdisciplinary degree program sponsored by three of Montana State University’s (MSU) colleges: Agriculture, Education, Health and Human Development, and Letters and Science. The program is a distance education program coordinated through the Division of Graduate Education. Graves (2002) provided an overview of the goals of the program, which include increasing scientific literacy, improving pedagogical content knowledge, increasing content knowledge, and engaging teachers in action research. In this study, the MSSE program was examined from the perspective of professional learning and professional development for educators, graduate education recommendations, and the Community of Inquiry Model (Garrison, Anderson, Archer, 2000) for online learning. 2Professional learning is the application and use of different sources of knowledge, often derived through engagement in professional development. The opportunity to engage with other educators in meaningful dialogue, to reflect upon that activity, and to implement those ideas into practice is at the heart of professional learning (Pedder, James, MacBeath, 2005). Professional learning can occur through participation in a graduate degree program, a professional development opportunity, a graduate course, a professional learning community, or any activity that creates change in teacher practice. The Council of Graduate Schools and other graduate organizations identified key features of a quality program. These recommendations are global ideas that promote excellence through broad themes such as diversity, access, and accountability. The emphasis in graduate education has shifted from passive learning to the active engagement with the material and the subsequent use of that knowledge (Hinumi, 2005). This is accomplished by integrating the key features into the enacted curriculum of a graduate program. This study examined the characteristics of the MSSE program in relation to the recommended features of a graduate program. Professional learning outcomes may be weak or non-existent if the activity designed is not of high quality. The American Education Research Association [AERA] (2005) had examined what impact professional development, and the consequential professional learning, may have on student learning and what specifically makes professional development “high quality”. Professional learning and professional development have common aspects. Professional development is another lens through which to view the MSSE program. The term “Professional Development” covers a vast 3range of learning situations and activities. Underlying all of these learning activities and strategies is the need for professional development that is content focused, incorporates pedagogical content knowledge, allows for collaboration and reflection, and is embedded within teacher practice. Another perspective by which to examine the MSSE graduate program is through the characteristics of an online learning program. Many of these elements are captured in the Community of Inquiry Model developed by Garrison, Anderson, and Archer (2000). The MSSE program is an asynchronous online program. The Garrison et al. model was based on online communities of learning. The model builds on three components that all interact with each other: Teaching Presence, Social Presence, and Cognitive Presence. Teaching Presence encompasses everything from the design of the online environment to the facilitation skills of the instructor that lead toward learning outcomes. Social Presence is the ability of the instructor to engage participants in a manner, both socially and emotionally, which creates the perception of engagement in discussions with “real people.” Cognitive Presence is the act of constructing meaning through discussion and reflection. The interaction of these three components can be seen in Figure 1, which clearly outlines the interrelatedness and interconnectivity of the components of the model. 4Figure 1. Community of Inquiry Model Garrison et al. (2000, p. 88). The transfer of the graduate program professional development experiences to the individual’s practice in the classroom constitutes the professional learning of the educator, one of the goals of a program such as MSSE. The process through which graduate education professional development becomes internalized by the educator and ultimately incorporated in instructional practice defines professional learning (Fullan, 2006). Within the MSSE program, this professional learning is primarily facilitated through online, asynchronous courses. These courses form each participant’s program of study. Statement of the Problem The literature surrounding teacher professional development points to numerous recommendations of what is deemed as high quality professional development (Kedzior, 2004; Desimone et al., 2002; Kleiman, 2004; Kennedy, 1998; Loucks-Horsley, Stiles, & Hewson, 1996; Darling-Hammond and McLaughlin, 1995). The MSSE program, both intentionally and serendipitously, seems to have embedded these recommendations from the professional development literature through the structure and requirements of the 5program. This study sought to identify the program components that have been created within the MSSE program that contribute to a graduate course of study that is coherent, has academic rigor, and contributes to the educator’s professional growth and learning. Few studies have been conducted on the MSSE program. Graves’ 2002 study illustrated the empowerment or professional efficacy of the graduates as a result of the action research component of the MSSE program. Horizon Research (2001) looked at the MSSE program as part of a two-part evaluation study, but only as a small-scale component of a larger study. The Horizon study examined the cumulative impact of taking National Teacher Enhancement Network (NTEN) courses, a MSU program that provides online graduate level science courses for science teachers, such as connectedness to other teachers, teaching strategies, enthusiasm for teaching, and content knowledge. No other studies have examined the MSSE program. The science and pedagogical content knowledge-based graduate degree program has the potential to benefit and impact the professional growth and learning of the graduates, yet no evidence exists confirming this potential. Furthermore, no research studies have specifically explored how such a program has contributed to science teachers’ professional learning and growth. The Master of Science in Science Education (MSSE) program offered through Montana State University offers a large majority of its graduate courses through an asynchronous online format, which allows access to a wide array of participants (Obbink & Tutill, 1998). Harris-John and Ritter (2007) stated in their study, “In a highly competitive world laced with a variety of technological devices and software, it becomes 6imperative that this training is offered in a manner that is both convenient and relevant to the worker, and that includes offering training in an e-based format” (p. 7). No research has examined the MSSE program through this lens and the potential impact an online science and pedagogical content knowledge-based graduate degree program has on the graduates. Purpose of the Study The purpose of this study was to explore the professional learning of participants of a science and pedagogical content knowledge-based graduate degree program through a concurrent mixed-methods design. The study examined such a program from three perspectives: professional learning through professional development, recommendations for a quality graduate program, and accessibility for geographically distributed participants, as the program is offered online. A science content-based program for teachers, such as MSSE, has the potential to create strong science educational leaders through the collaboration with other teachers and scientists and the lab and field experiences offered in the courses. Research Questions This study attempted to answer the following research questions: Research Question 1: How do the characteristics of the MSSE program compare to 1) research and policy recommendations for graduate education and 2) to professional development standards for teachers? 7a. What are the formal program requirements for the MSSE program? b. What is the intended program experience? c. How do these formal requirements and intended experiences align with recommendations for graduate education programs and teacher professional learning in research and policy literature? Research Question 2: How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived by students, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from a program teachers’ perspective, contribute to participants’ professional learning? c. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? d. What evidence of teacher professional learning can be identified? Research Question 3: How does the distance mode of delivery of this science and pedagogical content knowledge-based graduate program impact the science teachers’ professional learning? In this mixed methods study, statistical, quantitative results were obtained from a survey administered to current and former program participants. Qualitative results from open-ended survey questions and from a case study of six MSSE students, selected through maximum variation sampling, came from portfolios of evidence of personal 8professional learning and interviews. A faculty survey was administered and the statistical and quantitative analysis added another layer of depth to this study. Definition of Terms Professional Development: opportunities to learn new or advanced skills, pedagogy, content, advanced exploration of resources, and can include technology to support teacher professional growth (Grant, 1996). Distance Education: an educational or learning event in which the instructor and students are separated by distance and geography and can occur synchronously or asynchronously. For the purpose of this study, it refers to computer-mediated asynchronous delivery (Commission on Colleges, 2006). Professional Growth: the transfer of new or advanced skills, pedagogy, content, and resources into instructional practice (Peine, 2007). Instructional Practice: decisions made by teachers regarding curriculum choices for their students (Saskatchewan Education, 1997). Pedagogical Content Knowledge: providing appropriate learning strategies and techniques based upon a deep understanding of content (Shulman, 1987). Disciplinary Content Knowledge: a deep understanding of “big ideas” as well as intricate details of a particular discipline (Shulman, 1987). Professional Learning: the transfer of new or advanced skills, pedagogy, content, and resources into one’s personal practice through reflection (Pedder, James, MacBeath, 2005). 9Action Research: examining one’s practice through systematic inquiry. It includes examination of how students learn and how well they learn with the intent of providing insight and reflection into one’s practice and moving toward increased student achievement (Mills, 2003). Significance of the Study Increased accountability in education has charged the educational community to redouble its efforts to support the professional learning of practicing teachers. Effective programs that lead to increased professional growth and learning are not always readily available to educators locally. Through the lenses of professional learning and professional development, graduate education recommendations, and the Community of Inquiry model, this study identified the characteristics of the MSSE program and the influence on teachers’ professional learning. The work in professional learning done by Pedder (2007) was extensive. In one study, data was collected from thousands of teachers in England. Pedder collaborated with other researchers in large-scale studies in the area of professional learning. Pedder identified four aspects of professional learning. Those aspects include: A. Inquiry: Using and responding to different sources of evidence and carrying out joint research and evaluations with colleagues. B. Building social capital: Learning, working, supporting, and talking with colleagues to discuss how students learn. C. Critical and responsive learning: Occurs through reflections, self-evaluation, 10 experimentation, and by responding to feedback. D. Valuing Learning: Deepening understanding and learning through talking about and valuing learning with others (p. 235). These aspects were used to examine the MSSE program for professional learning of the program participants and are defined further in Chapter Two. Graduate education in the U.S. responds to policy and recommendations from numerous graduate education organizations. The Council of Graduate Schools (CGS) is one prominent organization that outlined recommendations for effective graduate programs. The CGS, along with other graduate school advocates, outlined ten areas that an effective graduate program should address. They include student support, academic rigor, a coherent program of study, content focus, research, interdisciplinary content, a capstone project, communication skills, knowledge applications, and comprehensive exams. These areas serve as a lens through which to examine the characteristics of the MSSE. The literature identified key research-based features for effective professional development for teachers (Garet, Birman, Porter, Desimone, Herman, Yoon, 1999; Kedzior, 2004; Desimone et al., 2002; Kleiman, 2004; Kennedy, 1998; Loucks-Horsley, Stiles, & Hewson, 1996; Darling-Hammond and McLaughlin 1995). Table 1 provides an overview of those features. 11 Table 1 Effective Professional Development Components from the Research Literature. Recommendations Definitions Structural:  Extended and Continuous Professional Development Sustained, extensive contact hours, opportunities for practice and follow up.  Participants in Similar Professional Contexts Similarity between educators who participate in the program with teaching assignments, content, etc. Core:  Content Focused A focus on deep understanding of disciplinary content knowledge, including how to deconstruct the knowledge for students.  Active Learning Discussions about pedagogical content knowledge, instruction, planning for implementation, analysis of student work, and presenting, reading and writing about the content with others.  Coherence Connection between the professional development and teachers’ curriculum, standards, and student assessment measures. The National Staff Development Council built upon those key features of effective professional development stating, “When teachers have access to high-quality, results- driven, content specific staff development, their students’ academic achievement increases” (NSDC, 2001, p. 3). The MSSE program is a graduate degree program, but is also a professional development experience. Professional learning is the goal of professional development and is a mechanism to examine the MSSE program. The NSDC also encouraged teachers to seek out e-learning professional development, which allows for increased opportunities that are flexible, convenient and 12 cost effective. Garrison, Anderson, and Archer’s Community of Inquiry model was used to examine the online aspect of the MSSE program. Looking at the online portion of the program from the perspectives of social, teaching, and cognitive presence allowed another aspect of the MSSE program to be examined. The MSSE program offered through Montana State University is a graduate degree program that appears to include the key elements of professional learning, effective graduate education, professional development and learning, and online learning. No studies have been conducted on the MSSE program design to examine the effectiveness of the program and the program’s influence on the professional growth and learning of current and past graduates. The purpose of this study was to begin to explore those aspects of the MSSE program and add to the literature on effective science and pedagogical content knowledge-based graduate degree program. The MSSE program was examined from the point of view of recommendations from the Council of Graduate Studies as well as other higher education advocates and policy makers with respect to effective graduate programs. The MSSE program was examined in terms of what constitutes effective professional development and the resultant professional learning of the program participants. Lastly, the MSSE program was examined from the perspective of online learning and its influence of program graduate and participants. The results helped inform the MSSE program staff of the impact the program has on its graduates, but may also inform similar programs regarding effective policies, procedures, and practices. While the MSSE program is successful based on the large number of graduates, 13 evaluations, and testimonials, the basic premises of its success have not been investigated thoroughly. From a larger perspective, the study may offer insights into the design and development of other pedagogical, content focused science graduate programs. The study may benefit other graduate programs or professional developers in creating programs that influence professional learning. This study may inform how professional learning may be addressed through different types of courses, i.e. online, face to face, field, hybrid, etc. The study may offer suggestions on how to consider the mix of pedagogy and content that influences professional learning. Finally, blending the recommendations of graduate programs and professional development this study may inform administrators in creating a sustainable and ongoing learning experience. Outline of the Study Chapter Two of this study examined research related to what constitutes an effective science and pedagogical content knowledge-based graduate degree program through the lens of professional learning. Professional learning is an intended outcome of professional development and the literature on professional development was reviewed. Chapter Two also explored the various areas of teacher learning, in particular, the construct of changes in instructional practice and improved content knowledge. Chapter Two explored the recommendations of effective graduate programs based on policy and research. Online learning was examined from the lens of online professional development and Community of Inquiry model. 14 The third chapter addressed the methodology employed to investigate the influence of the MSSE program on teachers’ professional growth and learning. Results and interpretations from the descriptive and inferential statistical analyses of the collected data were presented in Chapter Four. Chapter Five discussed the findings and implications of this study of the MSSE program as it relates to content based graduate degree programs, professional development, professional learning, access to all, and suggestions for further research. It advised other science and pedagogical content knowledge-based graduate degree programs of effective policies, procedures, and practices. 15 CHAPTER 2 LITERATURE REVIEW Introduction The review of the literature examined relevant research that pertains to professional learning, effective graduate programs, professional development and the associated features as a mechanism for professional learning. The Community of Inquiry model as the framework for online learning was also reviewed. The literature review is divided into the following sections: professional learning, graduate program recommendations, essential features of professional development, and the impact on professional growth, standards of professional development, online learning including recommendations for online professional development, distance education, the Community of Inquiry Model, and needs of teachers who are separated geographically from institutes of higher education. Professional Learning Those individuals who continue to increase their knowledge base pertaining to their work in a profession such as teaching are engaged in professional learning. Professional learning can happen through university course work, professional development, or informal channels, each of which can take many forms. Professional learning, as defined by Pedder et al. (2005), based on over six years of research and thousands of surveys across the United Kingdom, “is an embedded feature of teachers’ 16 classroom practice and reflection, is extended through consulting different sources of knowledge, is expanded through collective activity, and deepened through talking about and valuing learning” (p.209). Professional learning stems from the transference of new knowledge to classroom practice. Bransford, Brown, and Cocking (1999) in their synthesis of teacher learning, added that research in this area is an emerging research topic with limited data. The majority of the research that does exist is in the form of rich case studies. These studies provide an initial understanding about teachers’ professional learning as they attempt to change their classroom practice. Categorizing Professional Learning Pedder (2007) systematically examined results from the “Learning How to Learn” survey that was administered to over twelve hundred teachers in thirty-two schools in England in an effort to understand the key components needed for professional learning to occur. This was a continuation of a study completed in 2005 in which over one thousand teachers in forty-three different schools completed the same survey. Pedder used a factor analysis to analyze the data from the surveys. From this analysis, four dimensions emerged as the professional learning process. These dimensions include: A. Inquiry: “Using and responding to different sources of evidence and carrying out joint research and evaluations with colleagues” (Pedder, 2007, p.235). In addition, inquiry consists of examining good practice used in other schools. Teachers may read current research or use the Internet as a source of best practices as a means of improving one’s practice. Consulting with students on a regular basis about how they learn is also part of the inquiry process. In essence, 17 teachers use research and pupil consultation as sources of information. This information is combined with joint research and team teaching to improve professional practice. B. Building social capital: “Learning, working, supporting, and talking with colleagues to discuss how students learn” (Pedder, 2007, p.235). This dimension allows for collaboration with other staff members with the intent of focusing on student learning. Teachers suggest ideas to each other and then take steps to try out those new ideas. There are open discussions about what they, as teachers, are learning and how they are learning. There are opportunities to discuss best practices in how students learn. The discussions are collaborative, reassuring and supportive. C. Critical and responsive learning: “Occurs through reflections, self-evaluation, experimentation, and by responding to feedback” (Pedder, 2007, p.236). Teachers discuss experiences and practices, and based on feedback adapt it for other contexts. Through reflective practice, teachers identify professional learning needs. Teachers continue to experiment with different practices as a method of improving their work. Reflective practice is only part of this critical and responsive learning – student feedback, self-evaluations, and observations by other colleagues – all provide insight into a teacher’s practice. This dimension is characterized by a teacher’s responsiveness to ideas, suggestions, and feedback from multiple sources. D. Valuing Learning: “Deepening understanding and learning through talking 18 about and valuing learning with others” (Pedder, 2007, p.236). This dimension celebrates learning of both students and teachers. Academic success is noteworthy and should be recognized – both the successes of students and teachers. Teachers are also committed to the concept that all students are capable of learning. Learning should be an enjoyable experience for all involved. Professional learning in the context of teacher education is an outcome of many formal and informal learning experiences, including graduate education, professional development, and teachers’ daily experiences. Pedder’s (2007) comprehensive analysis of surveys, repeated over time, indicated that there are four broad dimensions that are associated with professional learning. Each of these four broad areas has specific components that collectively define the dimensions at a more granular level. Pedder’s dimensions serve to make connections with the other research on professional learning as examined below. Day and Gu (2007) conducted a longitudinal, multi-site, mixed methods study to examine professional learning over teachers’ careers. The authors looked at professional learning from the perspective that “it will enrich teachers’ knowledge base, improve their teaching practices, enhance their self-efficacy, and commitment to quality of service” (p. 425). A key finding from their study showed that professional learning is strongest when “responsive and differentiated support” is provided to meet the teachers’ needs (p. 439). The premise of responsive and differentiated support strongly connects to Pedder’s (2007) third dimension, critical and responsive learning. Pedder’s third dimension includes the ability to see how practices that work in one context may be adapted to other 19 contexts. Teachers modify their practice in light of evidence from feedback, self- evaluations, and evaluations from colleagues. Pedder’s results connected to Day and Gu’s findings in that professional learning meets the needs of teachers by being responsive to the teachers’ situation and context. Bolam, McMahon, Stoll, Wallace, and Greenwold (2005) conducted a 34-month study of professional learning communities across 393 schools. The premise of the research was to detemine if professional learning communities could sustain and promote professional learning. Their survey incorporated the ideas that professional learning is believed to be more effective when it is based on self-development and work-based learning. Key structures that promote professional learning emerged from their review. These structures include experiential learning, reflective practice, process knowledge, problem-based professional learning, professional collaboration and learning support as teachers use the new knowledge. These structures identified by Bolam et al. are similar to the dimensions Pedder identified, specifically the first, second, and third dimension. Pedder looked at professional learning through a lens that allows for reflection, collaboration, and using new knowledge in a supportive environment. Bransford, Brown, and Cocking (1999) devoted a chapter in their book to teacher learning. Their book, How People Learn, was the result of the work for two distinct committees of the National Research Council: the Commission on Behavioral and Social Sciences and Education, and the Committee on Developments in the Science of Learning. The book was based on a two-year project in which a committee reviewed and evaluated developments in the science of learning. They compiled the report into a practioner 20 guide about learning. The authors identified the opportunities for teacher learning, including graduate programs. “For the most part, teachers take graduate courses in education rather than in the subject matter of their teaching because of the lack of disciplinary graduate courses that are offered after school hours or during the summer” (Bransford, Brown, and Cocking, 1999, p.180). They stated that the generic nature of many professional learning opportunities diminishes effectiveness of the experience. For professional learning to be effective, they made the case that it must be learner centered and include a focus on pedagogical content knowledge. The authors emphasized there is power in teachers rethinking their disciplinary content knowledge and pedogogical content knowledge simultaneously. These authors also extracted key components that must be in place for teacher learning to occcur. They include the following;  Teachers need expertise in both subject matter and teaching.  Teachers need to have an understanding of both what the students know and how students acquire knowledge of the specific content that is being taught.  Teachers need a pedagogical content knowledge base, an understanding of the needs of diverse learners and cultures, and strategies to meet the needs of those diverse students.  A teacher’s focus is to support students with their learning and the application of that learning to new situations.  Teachers need an understanding of child development, strategies to best meet those developmental needs, and strategies to tap into prior knowledge of the 21 student based on their developmental level.  Teachers need a personal professional development plan that meets their personal and professional needs. The authors identified broad components that move toward effective teacher learning. The components identified by Bransford et al. (1999) are similar to Pedder’s (2007) dimensions of professional learning. The authors described the need of both pedagogical and content knowledge and an understanding of the needs of diverse learners. Pedder’s dimension of inquiry is similar in the sense that teachers look to research and examination of good practice in order to make informed choices and classroom decisions. Pedder’s dimension of building social capital focuses on student learning both what students are learning and how they are learning. Bransford et al. stated teachers need to identify their own professional development needs. Pedder’s dimension of critical and responsive learning encouraged teachers to identify their professional learning needs based on reflective practice. The National Center for Research on Teacher Learning (NCRTL), founded at Michigan State University’s College of Education in 1985, conducted a series of longitudinal research studies to examine teacher learning through 1995. The Teacher Education and Learning to Teach study consisted of more than 700 teachers at 11 sites throughout the U.S. who completed questionnaires before, during, and after their teacher preparation program. A smaller group of 160 teachers was interviewed and the researchers observed their teaching. Over 100 other studies conducted over the 10-year period examined 4 areas: how can teachers learn to teach in more powerful ways, how 22 can teachers transform their learning into better teaching, how can teachers connect content knowledge to diverse learners, and the complexities involved in teaching. The emphasis on teacher learning was based on research indicating that understanding occurs only when learners actively think about or try out new ideas in light of their prior knowledge. NCRTL’s guiding principles on teacher learning stated that teachers must be introduced to new ideas and have their experiences questioned and beliefs challenged to move learning forward. Programs that aim to impact teacher learning must engage teachers in a manner that broadens and deepens their subject knowledge to meet a diverse student population. Teachers must also have opportunities to reflect about their learning and the subsequent application into their practice. The NCRTL reported three tasks are essential for professional learning and must occur concurrently to be effective. They are  Teachers need to rethink their assumptions about teaching and learning by exposure to better ideas and challenges to question their own experiences and beliefs.  Teachers need to improve their subject matter knowledge and engage in new learning tasks. Over time, a deeper content knowledge enhances the ability of teachers to present content knowledge in a manner that allows connections for diverse learners.  Teachers must learn to reason about their work in the process of doing it through reflection of their practice. 23 The three tasks are similar to Pedder’s dimensions. NCRTL’s task of rethinking assumptions correlates to Pedder’s dimensions of inquiry, using and responding to different sources of evidence. Examination of best practice and research is used to improve practice. Pedder did not specifically identify subject matter knowledge as a dimension, meeting the needs diverse learners was addressed in the dimension of valuing learning with the inherent belief all students can learn. The ability to reason about one’s work is reflection, which is a dimension of critical and responsive learning. The NCRTL’s tasks and Pedder’s dimension parallel each other in regard to what comprises professional learning. Elmore and Burney (1997) prepared a report for the National Commission on Teaching and America’s Future. They described a local school district’s focus on attending to teacher learning. The report was a thorough analysis of the Community School District 2 in New York City. The district successfully used staff development to change instruction system-wide. In addition, the authors presented a protocol for staff development that may be duplicated in other districts as a mechanism to influence professional learning. The premise of the report was a general concern among educational reformers about the role of local school districts in the improvement of teaching and learning in schools. From the work in this district, the authors reported that professional learning that impacts instructional change is a four-stage process. The stages include awareness, planning, implementation, and reflection. Awareness is providing teachers with materials, experts, resources, and research in order to illustrate new ideas. This ties in closely with Pedder’s dimension of inquiry. Planning is the 24 opportunity to collaborate and design curriculum – the collaboration similar to Pedder’s building social capital dimension from the perspective of teachers talking and discussing how students learn and how the ideas generated by teachers are experimented with in the classroom. Implementation and reflection are closely related. In implementing new strategies, teachers constructively critique each other and provide feedback. The feedback then allows opportunities for reflection. The dimension of critical and responsive learning identified by Pedder resembles this stage. The Department of Educational Training (2005) in Melbourne, Australia, developed a practitioner’s guide of seven principles of highly effective professional learning. Although a practitioner’s guide, the principles were identified by the Department of Educational Training through an analysis of the research literature. Effective professional learning focuses on developing the core attributes of an effective teacher. It enhances teachers’ understanding of the content they teach and equips them with a range of strategies that enable their students to learn that content. It is directed towards providing teachers with the skills to teach and assess for deep understanding and to develop students’ metacognitive skills. (Department of Education and Training, 2005, p. 6) The Department of Education and Training stated that highly effective professional learning: 1. Increases teachers’ pedagogical content knowledge. 2. Is school based and is grounded in the day-to-day work of the teachers. 3. Is based on best practice and research. 4. Is collaborative, reflective, and offers opportunity for feedback. 5. Includes evidence and data driven from multiple sources. 25 6. Is ongoing, sustainable, and embedded in the culture of the school. 7. Is linked to the district’s goals and is system wide. Several of these principles can be connected to the dimensions that Pedder described. Principle 3 refers to best practice and research and Principle 5 refers to evidence to make educational decisions, both are similar to Pedder’s dimension of inquiry. Principle 4 has similarities to Pedder’s dimensions of building social capital in the sense of collaboration that is expected in professional learning. In addition, there are similarities to the dimension of critical and responsive learning in that reflection and feedback make for effective professional learning. Summary An examination of the components of professional learning and the four dimensions Pedder identified through his multiple studies – inquiry, building social capital, critical and responsive learning and valuing learning – revealed common themes in the research literature. An area that was not explicit in Pedder’s dimensions is a content focus for professional learning. Bransford et al. (1999) stated that pedagogical content knowledge is critical for professional learning. Examining one’s disciplinary content knowledge should occur simultaneously with an examination of pedagogy. The NCRTL presented compelling findings that content focused learning is key for professional learning. Table 2 summarizes the key components of professional learning. Table 2 includes the research presented in this section with connections to the dimensions of professional learning identified by Pedder et al. (2007). Professional learning is effective when it includes Pedder’s four dimensions of inquiry, building social capital, 26 critical and responsive learning, and valuing learning. In addition, there must be a focus on content knowledge and understanding. Using the lens of professional learning to examine the MSSE program provided insight into how the experiences in the program lead to professional learning and growth of the MSSE students. Table 2 Summary of Professional Learning Components– Research and Practitioner Literature. I n q u ir y : U s in g a n d re s p o n d in g to d if fe re n t s o u rc e s o f e v id e n c e a n d c a rr y in g o u t jo in t re s e a rc h a n d e v a lu a ti o n s w it h c o ll e a g u e s B u il d in g S o c ia l C a p it a l: L e a rn in g , w o rk in g , s u p p o rt in g , a n d ta lk in g w it h c o ll e a g u e s to d is c u s s h o w s tu d e n ts le a rn . C r it ic a l a n d r e s p o n s iv e le a r n in g : O c c u rs th ro u g h re fl e c ti o n s , s e lf -e v a lu a ti o n , e x p e ri m e n ta ti o n , a n d b y re s p o n d in g to fe e d b a c k . V a lu in g L e a r n in g D e e p e n in g u n d e rs ta n d in g a n d le a rn in g th ro u g h ta lk in g a b o u t a n d v a lu in g le a rn in g w it h o th e rs C o n te n t fo c u s e d Pedder (2007)     Day and Gu (2007)  Bolam et al (2005)    Bransford, et al (1999)     NCRTL (2007)     Elmore & Burney (1997)    Dept of Education & Training (2005)     Professional Growth through Professional Learning Professional learning can occur if key components are embedded with a professional development experience. An outcome of professional learning is a change in practice. The professional growth of an educator can be defined as the transfer of new 27 or advanced skills, pedagogy, content, and resources into instructional practice (Peine, 2007). McAleer (2008) developed a framework for the domains of potential teacher growth and development. This framework was developed through a synthesis of the research literature on professional growth with an emphasis on mathematics teachers. McAleer examined growth through two broad categories – knowledge for teaching and engagement and practices of teachers. Five subcategories comprised the category of knowledge for teaching. They include:  general pedagogical knowledge.  content knowledge.  pedagogical content knowledge consisting of specialized content knowledge, knowledge of content and teaching, and knowledge of student thinking.  curricular knowledge.  knowledge of how to support diverse learners.  knowledge of educational policies, practices, and objectives. The engagement and practices category consisted of two subsets: professional engagement and leadership, and reflective practices. Professional growth consisted of a deep content knowledge, as well as how to teach the content and address and meet the needs of all learners. Professional growth was also defined by involvement in professional activities or positions of leadership beyond the regular classroom. McAleer stated that reflective practice was also a key factor in professional growth. This framework for professional growth can be compared to Pedder’s dimensions of professional learning. Pedder’s dimension of inquiry, using different sources of 28 evidence and research to improve practice, can be related to McAleer’s knowledge of student thinking and knowledge of educational policies, practices, and objectives. The knowledge of supporting diverse learners connects to Pedder’s dimension of valuing learning, the idea that all students are capable of learning. These ideas were similar to the knowledge of teaching components McAleer presented in her framework. Pedder’s building social capital was based on collaboration with others to discuss ideas, pedagogy, and student learning. This dimension bears some semblance to the professional engagement and leadership of McAleer’s framework. Reflective practice, also identified by McAleer, was included in Pedder’s critical and responsive learning dimension. Further insight into professional growth and the relationship to professional learning was explored in the graduate education and the professional development sections of this literature review. The intent behind any professional learning is to create change – a change in instructional practice, a change in beliefs or attitudes, and a change in understanding. Professional development is one of several mechanisms that can support professional growth through professional learning. Desimone, Porter, Garet, Yoon, and Birman (2002) in their large scale study of professional development, stated that professional development can be a “cornerstone of systemic reform efforts designed to increase teachers’ capacity to teach to high standards” (p. 81). They also emphasized that professional development that is content focused impacts instructional practice and can have a positive influence on student achievement. 29 Guskey and Sparks (1991) examined profesional development that leads to professional growth. They identified three areas of a professional development opportunity that can lead to professional growth. The first was instrumentality, which refered to how clear and specific the pedagogical practices are explained. Congruence refered to the connectedness of the new practices with teachers’ present practice and philosophy. The last area they identified was cost – the time and effort to implement the new practices compared to the potential benefits of implementation. Pedder’s dimension of inquiry has some connections to instrumentality. Inquiry is examing research and best practice followed by implementation or connecting new practices within a teacher’s classroom. Linking professional development to the context in which teachers teach can also influence the effectiveness of professional learning. Applying the new knowledge to a participant’s teaching situation can lead to better instruction and student learning (AERA, 2005). Borko (2004), in her synthesis of the research literature, added another dimension to this idea by layering challenging conversations around this new knowledge to help embed this new knowledge in teachers’ practice. Loucks-Horsley, Hewson, Love, and Stiles (1996), in their NISE brief examinning professional development, stated explicitly that linking the new knowledge to classroom practice, as well as to the teachers’ prior knowledge, strengthens teacher learning. One area of professional growth that often emerges from content rich professional development is increased leadership capacity (Loucks-Horsley, Stiles, and Hewson, 1995). Often the result of such effective professional development is that participants 30 become agents of change beyond their classrooms. McAleer (2008) reported an increase in leadership for the math mentors in her study. The recent research in professional learning was examined by Putnam and Borko (2000). The authors used the lens of “situated perspective” in developing recommendations for best practice in teacher professional learning. The idea of teachers engaging in a professional community of discourse is one method that has shown promise in affecting teacher learning. Diverse groups of teachers discussing practice, embedded with different viewpoints and expert views, create rich conversations that impact the learning of the participants. Putnam and Borko stated, “University participants can bring to these communities the critical and reflective strands and modes of discourse that are important norms within the academic community. In additon, they bring research based knowledge” (p. 7). The research examined by Putman and Borko ties in with three of Pedder’s dimensions – different sources of information and research (inquiry), the ability to discuss them with other professionals at a deeper level (building social capital), and reflecting on the discourse (critical and responsive learning). Fishman, Marx, Best, and Tal (2003) developed a model of teacher learning and a framework to evaluate teacher learning using student performance as the indicator of growth. The premise of their research was that if teachers were learning, then their students should be learning as well. Their study examined multiple points of evidence to determine the content, strategies, sites, and type of media used to create professional development in the Detroit Public Schools. The study focused on middle school science teachers implementing a new curriculum. This mixed method study used student pre and 31 post tests, data from teacher focus groups, observations, and feedback to develop a framework. The authors concluded and emphasized that using smaller snippets or samples of student learning are the best method to track teacher learning. The authors recommended looking at student learning of specific concepts rather than large learning units. This method provided better evidence of teachers’ professional learning and changes in instructional practice. The model that Fishman, Marx, Best, and Tal presented can be connected to Pedder’s dimensions. The framework was developed by the authors using a variety of data sources to inform practice (inquiry) and analyze teacher learning . There was also an emphasis of teachers receiving multiple forms of feedback (critical and responsive learning). Fishman, Marx, Best, and Tal, designed professional development to affect teacher learning and Pedder’s dimensions were visible in the architecture of the professional development. Professional Learning and MSSE Graves’ (2002) study showed that, according to participants, one benefit of the action research component of the MSSE program was improved teaching and learning. Graves’ mixed method study examined the impact of action research on two cohorts of MSSE participants. Utilizing focus groups and teacher responses to open ended survey items, Graves’ thorough analysis indicated action research was a type of professional learning that impacts teacher practice. The MSSE curriculum also included self- reflection as part of the process in the courses offered. Graves’ research indicated that opportunities for self-refection by the participants were a powerful part of empowerment. 32 Graves’ study also included an examination of the collegial relationships formed in each cohort. Qualitative data indicated the sharing and interaction had a significant impact on the sense of empowerment of the participants. Graves’ study examined the construct of empowerment among the MSSE participants, which connect with Pedder’s dimensions of professional learning. The MSSE program was designed with cohorts of teachers collaboratively engaging in deep discourse about content and offering opportunities for self-reflection. The MSSE courses reflect Pedder’s dimensions of professional learning. Participants engage in a collaborative environment to discuss student learning and the implementation of those ideas (building social capital). Throughout the MSSE curriculum, participants are asked to try to incorporate new teaching practices in their own classrooms and reflect upon the experience (critical and responsive learning). The Horizon Research’s (2001) evaluation of the National Teachers Enhancement Network (NTEN) courses, many of which are used toward the MSSE graduate degree, indicated an increased content knowledge and a change in teaching practice of the participants. Opportunities for collaboration and an increased knowledge of instructional strategies provide a strong commitment to effective professional development through an effective graduate program. Again, the Horizon report identified collaboration and discourse about content as factors contributing to professional learning. The dimensions of building social capital, as defined by Pedder, are similar to the factors identified by Horizon Research. Pedder did not address content specific discussions, but rather recommended the more general act of discussing how students learn and exchanging 33 ideas, both components in the MSSE courses. Summary Professional learning can evolve from many experiences, but as Pedder et al. outlined, the most effective learning comes from a program that addresses the four dimensions; inquiry, building social capital, critical and responsive learning, and valuing learning. Learning is effective in impacting teacher practice if it allows for reflection through collaboration and deep dialogue. Day and Gu stated that when learning experiences are intentionally designed to meet the needs recognized by teacher participants, this strengthens the professional learning experiences. Bolam et al. (2005) added a new aspect to Pedder’s dimensions by including problem based learning. Bransford, et al stated that graduate programs can contribute to professional learning. The authors also found that programs that are learner centered and include a focus on pedagogical content knowledge are the most effective for professional learning, but are not prevalent options offered in many graduate programs. Professional growth of educators has been written about extensively in the professional development literature, although it can occur through other experiences. McAleer identified areas of professional growth which result from professional learning. Based on a comprehensive review of the research and practitioner literature, she identified two broad categories: knowledge for teaching, and engagement and practices of teachers. The components that comprise the categories identified by McAleer can be mapped to the dimension of professional learning described by Pedder. Quality professional learning experiences can affect teachers’ beliefs, knowledge, and attitudes. 34 Fishman et al. presented a model of professional development that has similarities to the professional learning design Pedder recommended. A design with Pedder’s dimensions and an emphasis on content specific knowledge has the potential to influence the professional growth and learning of teachers. Global Graduate Program Recommendations The Council of Graduate Schools (CGS) mission statement is “to improve and advance graduate education in order to ensure the vitality of intellectual discovery” (Council of Graduate Schools, 2006). The CGS is comprised of a collective of member colleges and universities across the country that offer master’s and/or doctoral degrees. The CGS advocates and promotes research and innovation in higher education. Institutes of higher education look to the CGS for policy, recommendations, and research for all facets of graduate education. Through its policy, position statements, and clarifying criteria for effective graduate programs, the CGS established essential components of a graduate program. Graduate Program Recommendations The Council of Graduate Schools (2005) identified components of a quality graduate program. They include:  a supportive program culture.  an informed faculty who can provide a first-hand perspective of the field and work place.  a variety of planned learning experiences ranging from traditional and 35 immersion/intensive courses to practice-centered learning, effective mentoring, a culminating experience or tangible products, and skill-building activities. These components can be tied to Pedder’s dimensions – specifically the dimensions of inquiry, building social capital, and critical and responsive learning. The CGS listed a supportive program culture as a component. This can be connected to building social capital in the sense that there is collaboration between faculty and the teachers in the graduate program focusing on the learning of the teachers’ students. The CGS also listed the component of informed faculty and the expertise they bring. This can be mapped to the dimension of inquiry. The faculty becomes a conduit of providing research and ideas of best practice to the teachers, an essential piece of the inquiry dimension. The CGS also recommended a range of planned learning experiences. This paralleled the dimension of critical and responsive learning. This dimension focuses on experimenting with different teaching practices, reflecting on the experience, and continually honing the practice. These dimensions involve using research and new knowledge, applying it to practice, and examining the content with an expert in the field. Other Graduate Program Advocates While the CGS is a leader in graduate education advocacy, numerous other organizations influence graduate programs as well. These organizations include the Southern Regional Educational Board (SREB), the Association of American Universities (AAU), and the Council for Higher Education Accreditation (CHEA), the Institute for Higher Education Policy, the American Council on Education, and the American 36 Distance Education Consortium. The SREB, a respected member of the educational community founded in 1948, is a nonprofit, nonpartisan organization that works with leaders and policy-makers in 16 member states to improve pre-K through postsecondary education. The AAU focuses on issues of importance to research universities. The CHEA, the Institute for Higher Education Policy, the American Council on Education and American Distance Education Consortium all provide recommendations for what constitutes effective graduate education. These recommendations from this collective group are designed for the administrative level. Specific curricular recommendations are also outlined by some of the organizations. Graduate Program Curricular Recommendations Hirumi’s (2005) comprehensive study examined what defines quality e-learning programs using the parameters outlined by six different higher educational organizations. Hirumi defined e-learning as technology supported learning. Using all of the previously listed graduate advocate organizations, with the exception of the CGS, Hirumi also included the American Federation of Teachers and the Open and Distance Learning Quality Council and systematically examined what it means to define a graduate program as a quality program. Hirumi’s study extracted five basic structures:  Institution – organizational commitment and support  Program – course development, curriculum and instructions, evaluation and assessment  Course – outcomes and objectives, methods, learning environments and 37 experiences  Student support – student services, admissions, requirements, research opportunities  Faculty support – faculty benchmarks, academic control, preparation Hirumi’s recommendations were broad, but began to categorize the necessary components in the actual graduate program. The CGS listed three components of effective graduate programs: a supportive program culture, informed faculty, and a variety of planned learning experiences. The CGS recommendations can be connected to those structures Hirumi reported. The CGS listed a supportive program culture as a recommendation. This is similar to Hirumi’s structure of student and faculty support. Student support included the admission process, the advising of students in the program and ensuring graduation of those enrolled in the program. Faculty support is ensuring that the faculty has the needed resources and professional development to support them in their endeavors. The next recommendation by the CGS was informed faculty. Hirumi’s study looked at the needs of the faculty, especially those teaching distance education courses. Hirumi identified a quality program as one that ensured that the faculty received appropriate training, support, and resources to adequately teach the courses being offered. Without these supports in place for the faculty, the integrity of the program is reduced. A variety of planned learning experiences for the graduate students were also recommended. Hirumi referred to these in the structures of program and course. These two structures focused on important instructional variables, such as objectives, content, 38 assessment, feedback, and media use. The CGS recommendations emphasized a multitude of learning experiences that will challenge the graduate student. Hirumi reinforced that idea with his study. A change Hirumi noted from previous studies of effectiveness of educational programs was the evaluation of outputs rather than inputs. The emphasis in graduate education has shifted to the learner’s engagement with the material and the subsequent use of that knowledge. “World-class benchmarks not only define what should be done but also delineate how well it should be done” (Hirumi, 2005, p. 315). These outputs were addressed in the program and course structures. Hirumi also referred to accrediting organizations that place an emphasis on outcomes, such as the National Council for Accreditation of Teacher Education (NCATE). Hirumi reported this allows flexibility for institutes of higher education in the manner in which they strive to achieve those outcomes. Kasworm and Hemmingsen (2005) examined graduate programs pertaining to adult education and human resource development through case studies in Denmark and the United States. They examined the changes that have evolved in graduate education, particularly the master’s degree, in both countries. Their findings indicated the professional preparation of adult education and human resource development professionals should focus on the development of critically reflective practitioners. The two programs that comprised the U.S. case study required participants to complete thirty-nine hours of course work and a capstone project. In synthesizing data from the programs, six components emerged which included: 39  Theory, research, and practice in relation to the characteristics of adult learners.  Instructional programs and practices.  Theory, research, and practice that focused on meeting the needs of diverse adult learners.  A research course for the practitioner to engage in examination of the research and in designing a research project.  Selecting courses to meet specific career and student needs.  A research capstone project that integrates of coursework and professional practice. The authors of this study found programs with these components have had a significant impact on the professional practice of participants and also on personal growth. Kasworm and Hemming pointed out that connecting theory to practice is a key element in an effective graduate degree program. In a national study of master’s degree programs, Haworth and Conrad (1997) examined what constitutes a quality program. Their study was based on 781 participant interviews focusing on the interaction of students, faculty, and administrators to extract the attributes of high-quality programs that emphasize student learning and development. They identified five program attributes, which include:  Diverse and engaged participants, including all current students and alumni, in the program. The intent is to use their insights of the students’ learning experience to improve the process.  A community of learners who have a similar vision of the program. 40  Collaborative discourse.  Breadth and depth of course work developed with connected program requirements, including a culminating project.  Adequate resources. Their work not only defined the attributes, but also presented a model in which to evaluate and improve graduate programs. Conrad, Duren, and Haworth (1998) conducted a comprehensive study of master’s programs in the U.S. The authors stated that there is a lack of literature describing both the student experience and perspective. Their study was an attempt to begin to fill in that void. The study consisted of over 800 interviews of faculty, administrators, students, and alumni of forty-seven graduate programs representing eleven fields of study. The goal of the study was to “sketch a national portrait of master’s education and to advance a theory of program quality” (Conrad, Duren, & Haworth, 1998, p.66). They found that a master’s degree program was a powerful professional development experience, especially for those in an education program. Programs that offered doing-centered learning, or applying the classroom based knowledge into a professional setting and the opportunity to put theory and research into practice, ranked highest among respondents. Minkel and Richards (1987), in a synthesis of the research literature, and through consensus of an expert panel, identified ten components of a quality master’s degree program. The authors also noted that the emphasis on a master’s degree has two outcomes. The master’s degree is a stepping stone to a doctoral degree or the master’s 41 degree is seen as a “professional degree”. The emphasis with the professional degree approach is directed toward the application of knowledge rather than original research. The quality criteria Minkel and Richards identifed are:  Tutorial Experience: personalized instruction and guidance.  Level of Sophistication: courses should be comprised of advanced disciplinary content and intellectual rigor.  Core of Planned Coursework: a coherent program that assures mastery of specified knowledge and skills.  Tool/Technique/Methodology Requirement: students acquire tools specific to the discipline.  Research Component: demonstration of mastery through a research component.  Extra-disciplinary Experience: exposure outside the immediate degree or discipline.  Culminating Experience: a capstone project.  Communication Skills: the ability to communication in a manner appropriate for the degree and discipline.  Application of Knowledge: a mechanism to apply the coursework to the profession.  Comprehensive Examination: an exam for the student to demonstrate the breadth of knowledge through participation in the program. The authors also viewed the graduate program as part of a university commitment. The extensive recommendations paralleled the broad themes identified by Hunumi and the 42 recommendations made by the CGS. The Council of Graduate Studies (2005) in the policy statement, Master’s Education: A Guide for Faculty and Administrators, outlined specific program requirements and aspects including a minimum number of courses, a core curriculum to be mastered and a culminating project and a faculty advisor for each student. The CGS also identified other elements that may be part of a master’s degree program. Not all elements may be included in a program, but their policy gudielines offered suggestions. They include:  An internship to allow for professional practice while offering faculty supervision and feedback.  A capstone or culminating experience to integrate proir learning.  A comprehensive examination.  A residency requirement that allows for interaction and mentoring between faculty and students. This requirement is more fluid with more distance learning opportunties. Utilzing technology for video conferencing, web-based course, etc., allows for more flexibilty and requires faculty to reach out to stuents in new ways.  Time limits for degree completion.  Completion of a minor field.  Mastery of a foreign language or research tool . Each master’s program should set requirements specific for their program and use the policy suggestions as a guide. 43 Summary Graduate programs look to the CGS and other higher educational organizations for recommendations of policy and practice. The CGS offered three broad components that a graduate program recommends: supportive program culture, informed faculty, and a variety of learning experiences for the graduate students. Hirumi’s study built upon those foundational pieces, and began to organize them into subsets. Both the CGS and Hirumi’s research identifed key areas that universities need to address in developing, evaluating, and maintaining quality programs. The research literature on graduate degree programs is sparse, particularly in examining student experience and perspective as noted by Conrad, Duren, and Haworth (1998). A concerted effort by this researcher to examine the literature on graduate programs based on recommendations from faculty and librarians, and using the bibliographical references of the studies referenced, confirmed the literature base is small. There are several research studies and policy statements that make recommendations for developing a quality program. The policy recommendations presented an overview of components that an institute of higher education should consider in developing a graduate degree program. Those recommendations are in presented in Table 3. 44 Table 3 Research Literature and Policy Recommendations for Graduate Programs. Program Components S tu d e n t S u p p o rt A c a d e m ic R ig o r C o h e re n t P ro g ra m o f S tu d y C o n te n t F o c u s R e s e a rc h In te rd is c ip li n a ry C a p s to n e P ro je c t C o m m u n ic a ti o n S k il ls K n o w le d g e A p p li c a ti o n C o m p re h e n s iv e E x a m Hinumi (2005)     Kasworm and Hemmingsen (2005)        Haworth and Conrad (1997)       Conrad, Duren, and Haworth (1998)   Minkel & Richards (1987)           Council of Graduate Studies (2005)       Graduate Programs as Professional Development One type of provider of a credible professional development is an accredited university or college. Technological advances in distance education have allowed higher education to offer online professional development, including graduate degree programs. The MSSE program offered through Montana State University offered professional learning in a pedagogical content knowledge and content-based master’s degree program. The MSSE home page states, “Courses may be taken for professional development or to earn a graduate degree” (MSSE, 2009). Isenberg’s (2002) monograph made the case for providing quality professional development for educators through master’s degree programs. Isenberg provided an in depth synthesis of the literature of the National Board process and how institutes of higher education can use that process to improve their own programs. The monograph 45 was designed to serve as a guide for higher education programs that want to incorporate the National Board Certification process into the current program of study being offered. Isenberg examined master’s degree programs that use the National Board for Professional Teaching Standards (NBPTS) core values as a foundation for the course of study. The monograph outlined the aspects of a quality master’s program and illustrated how these components are needed for graduate programs to be a source of quality professional development. Educational reform efforts place a heavy emphasis on student learning, and strong professional development is a key element. Isenberg explained that the NBPTS supported that goal of professional development. These recommendations are addressed in the next section as part of the essential features of professional development. Selke’s (2001) descriptive report of the types of master’s degree programs also noted the significance of the National Board Certification (NBC) process. Selke found that many master’s programs designed for educators were including the NBC process within the master’s program design. Selke’s report examined the focus of different types of master’s degree programs that educators seek. Selke, in her review of the literature on master’s programs in the U.S., identified three types of master’s degrees: the master’s of arts in teaching, traditional master’s degree in education, and a practitioner’s master’s program. Selke emphasized that all of the types of master’s degree are professional development. “ ‘Practitioners’ master’s programmes do not conceptualize the master’s programme as a terminal degree that parallels a teaching career, but rather as one component on a continuum of professional development with the potential to transform a teaching career” (Selke, 2001, p.206). Selke’s definition of a practitioner master’s 46 program appears to align with the intent of the MSSE program as the MSSE program is a type professional development and with the intent of influencing teaching practice. While there are few studies that examine graduate programs as professional development, the studies that exist represent clearly that a master’s program for educators is a form of professional development. Some master’s programs are considering including the NBC process within their master’s program as the process provides “a programmatic context that facilitates the development of knowledge, skills, and dispositions” (Selke, 2001, p. 208). Each study offered recommendations pertaining to the aspects of the master’s program that should be considered in the design. These recommendations are discussed in the next section. Essential Features of Professional Development Professional development for teachers includes all opportunities, both formal and informal, that offer ways to enhance one’s skills and improve one’s teaching practice. The National Staff Development Council (2009) used terms such as “comprehensive,” “substantiated,” and “intensive” when describing professional development, all with the goal of increasing student achievement. The research literature on professional development is extensive and offers recommendations of what constitutes effective professional development. In addition, the research literature is broad in identifying design features that should be included for effective professional development. This literature review did not address all of those characteristics, only those that are relevant to the study of the MSSE program. 47 The research literature on professional development is vast, yet a large majority of the studies can be connected to examine common themes. Garet, Birman, Porter, Desimone, Herman, and Yoon (1999) went into sweeping detail in their book detailing the comprehensive national study they conducted examining professional development. The authors evaluated the Eisenhower Professional Development Program through three strands. The first was a national profile compiled from interviews and survey data. Over 450 interviews were conducted with the directors of Eisenhower-assisted professional development programs. Over 1000 surveys were collected and analyzed. The second strand consisted of a set of case studies in selected districts and states. The intent was to gather more detailed information about the types of professional development being offered under the auspices of the Eisenhower Professional Development Program. These sites were selected because of the diverse approaches being offered in the professional development experience. The case studies included in-depth site visits and interviews. The third strand was a longitudinal study of teacher change using those same selected districts. Again, thorough case studies in this strand included classroom observations in elementary, middle, and high school. The teachers who were observed were asked in- depth questions regarding topics taught, students’ performance goals, and their professional development participation. Through a series of studies, Garet et al. examined the relationships between a variety of features of professional development and change in teaching practice in a cross-sectional, national probability sample of teachers. The features were categorized as structural or core. Structural features are the overall design of the professional 48 development experience. Structural features include the duration of the activity and the collective participation of the participants. The types of activities that offer active learning define core features. Core features include the engaging teachers in discussions about teaching and coherence. Coherence is a way of meeting the teachers’ needs through a focus on content. Garet, et al. provided a comprehensive examination of core features and is used as a standard framework for much of the ongoing research in professional development. Numerous other studies augmented Garet’s et al. research with similar findings about professional development structure and activities. The study by Garet et al. was the framework of professional development components used to examine effective professional development in the study of the MSSE program. Structural Components of Professional Development Several studies identified structural components of professional development. Kedzior and Fifield (2004), in their review of the research literature of effective professional development, found that extended and continuous professional development is more effective than one time or episodic approaches. Kennedy (1998), through a comparative examination of professional development programs, concurred that professional development can be effective if it is distributed over time, but only if the content is worthwhile. Garet et al. (2001), in their large scale empirical comparison of effects of different characteristics of professional development on teachers' learning, found longer duration of the professional development allowed for more in-depth conversations on content, student learning, and pedagogy. 49 Garet et al. also examined the structural features of the collective participation of the participants. They defined this collective participation as active learning of teachers who were in similar professional contexts. These groups of participants in the study were all associated in a similar professional manner. These groups engaged in deep conversations about concepts, skills, and problems - core type features. It is the professional communication between teachers who share a professional context that can contribute to a change in teacher practice in the classroom (Maldonado, 2002). Loucks- Horsley, Stiles, and Hewson (1996) also found this association to be powerful. Professional communication between teachers with similar professional contexts can move these teachers to be teacher leaders and agents of change in their schools. Table 4 summarizes the effective structural components of professional development that are relevant to this study. Table 4 Effective Professional Development Structural Components from the Research Literature. Recommendations Definitions Structural:  Extended and Continuous Professional development Sustained, extensive contact hours, opportunities for practice and follow up.  Participants in Similar Professional Contexts Similarity between educators who participate in the program with teaching assignments, content, etc. Core Components of Professional Development Core features of professional development are also needed along with structural features. Core features are the activities associated with collective participation over an 50 extended time period. They are developed to meet the needs of the teachers in a particular setting. Discussions about pedagogy and discussions that focus on content are key core activities. The AERA (2005), in a research policy brief, elaborated on the core features of professional development, particularly the focus on subject matter. Referencing several studies that examined teacher practice and student achievement, professional development that connected how students learn subject matter, instructional practice specific to subject matter, and increasing teacher’s knowledge of subject matter were most effective. The brief concluded that content focused professional development was a key aspect in effective professional development. Professional development that focuses on how students learn, pedagogical content knowledge, instructional practice, and disciplinary content knowledge can lead to improved student achievement. In addition, professional development that connects to the teachers’ curriculum, standards, and accountability measures leads to better instruction and student learning. Garet et al. referred to this connection as coherence. Content-Focused Teachers begin to actively engage their students in developing ideas once they have a deeper understanding of the content (Garet et al., 2001). Kedzior and Fifield (2004) drew those same conclusions from their synthesis of the literature. Borko (2004) argued that when teachers have a powerful understanding of the content, consisting of subject matter knowledge for teaching, understanding of student thinking, and instructional practices, it enabled them to foster students’ conceptual understandings. 51 Garet et al. (1999) concluded content was a core feature that improved the effectiveness of professional development. Numerous other studies pointed to a focus on content as a critical component for effective professional development. One finding of Loucks-Horsley, Stiles, and Hewson’s (1996) synthesis of the research literature and policy recommendations surrounding professional development was that teachers with strong disciplinary content knowledge were better able to address the needs of diverse learners regarding the content. Knowing how students learn is critical, but recognizing how all students learn moves toward increased student achievement for all students. Kennedy (1998) performed an analysis of the research literature in search of professional development that included evidence of student learning and concluded that effective professional development is content focused. The research literature clearly indicated content is important, but identifying appropriate content for instructors to teach is part of that challenge of professional development. Ball (2000) centered much of her work on teachers of mathematics, but her findings are applicable to science teachers. Ball stated that content knowledge that matters for teaching must be identified. Once it is identified, it must be put into the context of the teacher’s practice in the classroom. Teachers must also be able to deconstruct content knowledge in a manner that enables it to be broken down conceptually for students. 52 Active Learning Another core component of professional development identified in the research literature is active learning. Active learning pertains to all parts of the teaching practice, including disciplinary content knowledge, potential misconceptions, pedagogical content knowledge, the associated instructional practice, and discussions around those areas. When teachers have the opportunity to construct and investigate new knowledge, this translates into better teaching practice. Lewis (2002) described this as an act of professionalism in which teachers seek out new information to hone their practice. Active learning is comprised of several features, all of which lead teachers to examining and discussing pedagogy, while allowing teachers the opportunity to practice what they have learned. Active learning is facilitated through engaged, meaningful discussions of the new information gained from the professional development activities and with intent of changing teacher practice. Garet et al. identified four components of active learning. Active learning can be observing other teachers, as well as being observed; planning for implementation of the new knowledge in the classroom; analysis of student work; and presenting, leading and writing about the content. Observation, whether observing others or being observed, can impact teacher practice. It was not a component that explored in this study, as the majority of the courses in the MSSE program are asynchronous and offered over geographic distance. While it may be speculated that some participants in the MSSE program do engage in this type of active learning, it is not an area that was studied. 53 Creating opportunities for learning the subject matter as well as opportunities to use it in the classroom is another aspect of active learning. Borko (2004) in her article examining the research on professional development and its impact on teacher learning, clarified successful programs were not purely about the subject matter, but instead were about how students learn that subject matter. Ball (2000) identified the challenges teachers face in integrating content and pedagogy into their classroom practice. Understanding content and utilizing appropriate instructional practices to convey content are essential skills of effective teaching. Teachers must be provided opportunities to link the new knowledge from the professional development to their classroom. Garet et al. outlined parameters that can lead to implementation. One strategy was that the teachers must have the opportunity to participate in the same lessons they expect their students to experience. By engaging in the lessons, it offers teachers an opportunity for feedback and stimulates discussion about the challenges that may be experienced in their own context. Analysis of student work is another facet of active learning. By delving deep into students’ written responses, teachers can gain insight into misconceptions, reasoning, and problem solving skills of their students. Reviewing student work allows teachers to differentiate instruction and better meet the needs of their students. Garet et al. found opportunities to examine student work were rare. Timble (2005) stated in her research summary of strategies that work to improve student learning, that reviewing student work was best practice and leads to improved student performance. Continual assessment of 54 student work is a practice of teachers who strive toward increased student achievement (Darling-Hammond, 1996). The last component of active learning is presenting, leading, and writing about the knowledge learned from the professional development. Synthesizing the information to share with someone else or teach caused the teacher to examine the information in depth (Garet et al., 1999). Using new knowledge to teach through a lecture, demonstration, or presentation requires an understanding of the information. Lieberman (1996) included that opportunities for teachers to develop or write curricula with others were part of active learning. In this collaborative event of writing, reviewing, enacting, and revising lessons teachers have the opportunity to use their professional learning in a meaningful way. Coherence Effective professional development needs to connect and link to the context of the teacher. The professional development must connect to the curriculum being used, the standards that are used, and the accountability measures that are required by the district and/or state. The AERA (2005) described two studies that examined the impact of connecting the professional development to the curriculum being used by the teachers in a research brief. In instances where the professional development focused on the curriculum being used, there was a much higher level of adoption of the practices learned through the professional development. Garet et al. added that aligning the professional development to the standards used by the teachers also resulted in implementation of the new knowledge and practices in the classroom. Lieberman (1996) concurred that if the 55 professional development was consistent with the schools curriculum and expectations for student performance a change in teaching practice wass more probable. Coherence also includes the connectedness to a larger professional development plan. The one-day workshop or weekend session is not designed to build on previous experiences or to begin a foundation of subsequent professional development experiences with the same focus. Coherence links to the structural feature of duration. A coherent professional development opportunity was one in which there are opportunities to link and build upon experiences, allowed for more advanced topics and skills, and went into greater depth of the content (Garet et al., 1999). Effective professional development results in professional learning. Graduate programs, which may be considered a form of professional development, have characteristics that overlap the key features of both professional development and professional learning. Figure 2, a graphical representation developed by the AERA (2005), illustrates the connections of essential components of professional development and the resultant professional learning – a change in instructional practice. 56 Figure 2. Professional Development Components for a Change in Practice (AERA, 2005). Table 5 summarizes the effective core components of professional development that are relevant to this study. Table 5 Effective Professional Development Core Components from the Research Literature. Recommendations Definitions Core:  Content Focused A focus on deep understanding of disciplinary content knowledge, including how to deconstruct the knowledge for students.  Active Learning Discussions about pedagogical content knowledge, instruction, planning for implementation, analysis of student work, and presenting, reading and writing about the content with others.  Coherence Connection between the professional development and teachers’ curriculum, standards, and student assessment measures. 57 Summary Professional development is an opportunity to learn new or advanced skills, pedagogy, and content. It is one mechanism in which teachers can increase their professional learning. The intent behind professional development is to create a change in teacher practice and student achievement. Garet et al. (1999), in their comprehensive study developed a framework for effective professional development that included both structural and core features. This framework was used to examine the MSSE program as a professional development experience for teachers. Structural features are the architecture of the professional development. Two structural features emerged as a lens through which to examine the MSSE program. They included extended and continuous profession development and participants in similar professional contexts. Professional development that allowd for extended conversation and opportunities for more in depth conversations about the content were more effective. When those discussions included participants in a similar teaching or professional context, the impact on instruction change was more likely. Core components were the activities that the teachers engage in during the professional development. Content focused professional development that allowed teachers to go in-depth into the subject matter and offered opportunities to unpack that knowledge for their students was a core component that influences instructional change. Active learning, or the opportunities to discuss, co-construct lessons, analyze student work, and the analysis one’s own practice was influential in creating a change in practice. 58 Finally, the professional development must have coherence and be linked to the teacher’s classroom. Professional Development Standards Numerous professional organizations have policy recommendations for professional development. All of the organizations are in alignment that ongoing professional development throughout one’s career is essential to ensure high quality instruction for students. The professional development standards relevant to this research study are those from the National Staff Development Council [NSDC] (2001), the National Research Council [NRC] (1996), the National Board for Professional Teaching Standards [NBPTS] (1987), the Southern Regional Education Board [SREB] Online Professional Development Standards, and the National Science Teachers’ Association [NSTA] position statement. The NSDC initially established standards for professional development in 1995 in an effort to provide guidance for schools and districts to develop effective programs. The standards were updated in 2001 and were guided by three questions. Those questions were; what are all students expected to know and be able to do?; what must teachers know to insure students meet those expectations?; and what is the focus of professional development to meet those goals? The standards developed by the NSDC were broken into three broad areas: Context Standards, Process Standards, Content Standards. In 2006, the SREB used the NSDC standards and adapted them for online professional development as a means to address the unique features inherent in online professional 59 development. As a call to action to increase the scientific literacy of students in the U.S., the National Research Council published the National Science Education Standards in 1996. The standards also included professional development standards for science teachers. “The Standards describe a vision of the scientifically literate person and present criteria for science education that will allow that vision to become reality” (NSES, 1996, p. 11). These standards were developed based on exemplary practice and research. A similar call to action created the National Boards for Professional Teaching Standards. The Carnegie Forum on Education and the Economy’s Task Force on Teaching as Profession released a final report, A Nation Prepared: Teachers for the 21 st Century (NBPTS, 2008). The recommendation in the report called for the creation of a board to clarify what all teachers should know and be able to do. Shortly thereafter, the NBPTS released their core propositions, which were the standards in which the National Board Certification program was founded. NSTA is the largest professional organization for science teachers in the world and supports and promotes professional development and has developed a position statement regarding professional development. NSTA drew from the research that suggested high quality science instruction can significantly impact student achievement. In May of 2006, NSTA published a position on professional development. Based on both the practitioner and research literature, NSTA outlined standards for professional development in science. 60 The Context Standards The context standards focused on improving learning for all students through resources, leadership, and learning communities. The NSDC emphasized that learning communities are a driving force behind professional development. The NBPTS Core Proposition 5, Teachers are Members of Learning Communities, emphasized a similar theme. The SREB also referred to learning communities as an essential part of the context standard. Collaborating with colleagues was not only a form of best practice in professional development, but also a dimension of professional learning. Continuous instructional improvement, guided by district personnel, was also part of the context standard. The intent behind this standard is that professional development is ongoing and meets the needs of the teachers. The professional development should also connect and link to the daily practice of the teachers. Both the NSTA position statement and the NRC reiterated this same theme. The SREB supported the implementation of online professional development as key component in a district’s overall plan. The contextual connection of the professional development has a direct association to the recommendations of effective professional development in the research. The NSDC context standards also connected to the recommendations in the research literature for effective professional development with regard to collaboration and extended duration of the professional development opportunity. The last component of the context standard referred to ensuring the necessary resources were available to support teacher learning and collaboration. NSTA has several references to resource support in the position statement. Sustainability was a 61 theme that resonated with all the standards. Sustainability included support from the district to simply provide quality professional development, funding, time, materials, and ongoing administrative. The NRC also advocated for resources, especially those specific to science teaching. The Process Standards The process standards of the NSDC recommended that educators’ address the learning of all students through data driven decisions, research based design, collaboration and evaluation of the professional development. The intent of the process standard was to provide educators with data and research to make informed decisions about what all students are expected to know and be able to do and what teachers must know and do in order to ensure student success. The process standard is reiterated in all of the other professional development standards. The NBPTS Core Proposition 3, Teachers are Responsible for Managing and Monitoring Student Learning, resonated with the process standard. In order to deliver effective instruction, teachers must have a thorough understanding of their students from assessments. Using multiple measures, the data can inform practice. NSTA’s position statement also addressed the process standard by integrating professional development with the assessment practices. NSTA also included opportunities for teachers to analyze and reflect on the assessments and use that information to drive instructional practice. The SREB emphasized using disaggregated student data for analysis. The NRC included analysis of data as a means of professional growth and learning. Using opportunities to examine ones’ teaching practice through analyzing 62 student work was one method by which feedback is collected on growth and learning. Collaborating with other teachers during this process allowed for extending the learning of both teachers and students. This connected with the NBPTS, Core Proposition 4 – Teachers Think Systematically about Their Practice and Learn from Experience. Critically examining one’s practice on a regular basis increased skills and the ability to better meet the needs of students. Another aspect of the process standard that emerged from the SREB is that the design of the professional development should address a variety of learning styles through the selection of appropriate technologies. The learning styles of participants can be addressed through a variety of online learning experiences that connect to teachers’ practice. A hybrid approach is also recommended when possible. The flexibility of “just in time” professional development support the needs of the participants. The Content Standards The last broad category of the NSDC standards was the content standards. The content standards sought to address learning of all students through equity, quality teaching and family involvement. The essence of the standard was understanding and appreciating all students, creating safe, orderly and supportive learning environments, and holding high expectations for their academic achievement lead to improved student learning. This closely connected to the dimension of valuing learning from the professional learning literature. The content standards also focused on deepening educators' content knowledge, providing them with research-based instructional strategies to assist students in meeting rigorous academic standards, and prepared them to use 63 various types of classroom assessments appropriately. The last part of the content standard included providing educators with knowledge and skills to involve families and other stakeholders appropriately. The aspects of the content standard closely resembled the standards from the NRC, NBPTS, and NSTA. Equity is a common theme in all standards. The Standard D of the NRC emphasized the history and culture of the school environment should be part of any professional development. Core Proposition 1 of the NBPTS focused on making learning accessible for all. An understanding of cultural and family differences is critical for effective teaching. The NSTA position broadly stated science education must serve the needs of all students. The SREB extended the idea of equity in access to the technology of the educators engaged in the professional development. Quality teaching improves the learning of all students by deepening teachers disciplinary content knowledge and pedagogical content knowledge (NSDC, 2001). The NRC strongly emphasized content in all of the standards. In Standard A, the emphasis of professional development was on essential science content and using inquiry methods to learn the content. Standard B continued the emphasis on strong content understanding and added an emphasis on pedagogical content knowledge to make the content accessible to all students. In Standards C and D, the NRC included access to new knowledge and research, collaborating with other teachers about the content, and the coherence of the content within their professional context. The intended outcome of professional development is “transformative learning experiences that confront deeply held beliefs, knowledge, and habits of practice” (NSTA, 64 2006, p.2). Professional development that is designed to challenge one’s thinking and deepen content knowledge should be the driving force in the design. NSTA also added that pedagogical issues and the understanding of how students learn is an essential part of professional development. The SREB added that the online instructor should also be highly qualified in the specific content knowledge of the professional development and online learning strategies. The NBPTS core Proposition 2 clearly articulated teachers should know their content and have mastery over the subjects they teach. A deep understanding of content, and the instructional strategies needed to engage students are the key elements of this standard. Teachers should be aware of misconceptions, potential gaps in content and skills of their students, and design lesson to address them. The last part of the content standards focused on including all of the stakeholders in a student’s education. Again, all of the standards included connections with families and communities. Both the NRC and NSTA expanded those connections to science organization, university faculty, and experts in the field. The intent is that all available resources are utilized in supporting the learning needs of all students. Summary In comparing the NSDC’s, NRC’s, NSTA’s, SREB’s, and NBPTS professional development standards to the recommendations of effective professional development from the research literature, it is evident there are common themes that emerge. The professional development research literature pointed to key components that define effective professional development as structural and core features. 65 Structural features were collaboration and duration. Collaboration was part of all the professional development standards. The context standard included learning communities, opportunities for teachers to engage in professional discussions about their professional learning. Collaboration with other teachers offers opportunities for feedback and better instructional practice. The content standard emphasized that the professional development should be ongoing and connected to the teaching context. This was similar to the recommendations of duration described the research literature. The process standard focused on making data driven decisions and using those decisions to determine what students know, what students need to know, and what teachers must do to have students reach those goals. This may be accomplished through analysis of student work. Engaging in collaborative discussions with others about student work, revising instructions, planning for implementation to parallel the core feature and active learning are all aspects of the process standard. The last standard was content. Teachers need a mastery of content and how to apply that content to their own instructional practice to meet the needs of a diverse student population. Content was a core feature described in the research literature that is a necessary component of effective professional development. The content standard also described the importance of equity and involving families and communities in meeting the educational goals of all students. This characteristic is very similar to the valuing learning dimensions of professional learning. 66 Distance Education Distance education offers a unique opportunity to further ones educational pursuits that fits within ones professional and personal needs. Bender (2003) cited four reasons that students and administrators believe online learning is effective: increases convenience, enhances flexibility, eliminates time and travel to class, and alleviates classroom space issues. Students who complete online classes have a great deal of satisfaction from the experience, which may be attributed to the convenience and flexibility to the learner (Extension, 2006). Another aspect of online learning that contributes to its appeal is that it breaks down cultural barriers (Bender, 2003). The asynchronous format and the distance that separates all participants allows an interaction that may not be replicated in a face to face class which in turn allows each member of the class an equal voice regardless of culture or race and focuses the attention on the discussion. The majority of the MSSE courses may be taken through online, asynchronous, computer mediated communication. From the perspective of online, asynchronous, computer mediated communication, Garrison, Anderson, and Archer (2000) developed a framework for examining meaningful educational experiences in a text-based environment. The Community of Inquiry model was used as a lens through which to examine the MSSE program, as it is widely used in other distance education research. 67 Community of Inquiry Model A group of individuals engaged in a task or common goal can be referred to as a community of practice. Learning is a process of social participation, a community of practice (Wenger, 1998). The Community of Inquiry, proposed by Garrison, Anderson, and Archer (2001), used the idea of a community of practice in the online environment. Inquiry was a main component of the model as it provides the questioning and exploration of a topic or experience. Through interacting and collaborating with others in an online environment, participants experience an inquiry process. This is represented in Figure 3. Figure 3. Practical Inquiry Model, Garrison et al. (2000). The Community of Inquiry model clearly outlined the three necessary elements that intertwine with each other in order to create an effective online environment: teaching presence, social presence, and cognitive presence (Garrison, Anderson, & Archer, 2001). The use of computer-mediated conferencing requires written communication to develop the model proposed by Garrison et al. Vaughan (2004) reiterated that written 68 communication was associated with critical thinking. It set up a reflective component as well as encouraging rigor in the discourse. Figure 4 illustrates the Community of Inquiry model. Figure 4. Community of Inquiry model Garrison et al. (2000). Rourke, Anderson, Garrison, and Archer (1999) broke down the elements of the Community of Inquiry model very succinctly. The first element in the model was cognitive presence, which was defined as the ability of individuals to construct meaning through the community of practice. Teaching presence included designing and managing learning sequences, providing subject matter expertise, and facilitating active learning. The third element was social presence, defined as the ability of learners to project themselves socially and emotionally in a community of inquiry. All three elements are needed to create the optimal online learning experience. Garrison, Anderson, and Archers’ (2000) seminal article elaborated on each of the elements in the model. It can be extrapolated from the literature that cognitive presence is essential, yet it is not sufficient to sustain a group of learners without a sense of 69 collaboration and community. While Garrison, et al. presented the research base that illustrated the effects of cognitive presence, they also found flaws with much of the methodology presented. Despite these shortcomings, there was empirical evidence that a social presence was needed to create cognitive presence. Social presence requires forethought and effort by all of the online participants. In computer-mediated conferencing, the social cues exhibited in a face-to-face setting are absent. Once again, Garrison et al. (2000) presented the body of research that showed that other social cues can have the same effect. The use of emoticons or other symbols to convey an affective perception may also be used. The most effective social cue is through the written communication that sets up a tone of respectful and challenging questioning. The final component in the Community of Inquiry model was teaching presence. Garrison, Anderson, and Archer (2001) suggested that teaching presence consists of three main categories: design and organization, facilitating discourse, and direct instruction. All three components are necessary to promote meaningful individual and social knowledge construction. Essential Components of the Online Learning Environment Within the online environment, there are multiple types of interaction that the learner has with the experience. Swan (2004) extracted four types of interaction through her review of the research literature examining interaction in the online environment. They included interaction with content, interaction with instructors, interaction with classmates, and interaction with course interface, i.e. the technology used for the 70 interaction. In addition, she meshed those interactions with the Community of Inquiry model to illustrate how those interaction types could be portrayed within the model. See Figure 5 for Swan’s adapted model. Figure 5. Relationships between Interactions and Learning, Swan (2004). Both Swan’s compilation and other research indicated that interaction between faculty and students and student-to-student was the necessary for a satisfactory experience (Picciano, 2002; Chickering & Gamson, 1987; Kumari, 2001; Fulford and Zhang, 1993). Shea, Pickett, and Pelz (2003) in their follow up study with students enrolled in courses through the SUNY Learning Network (SLN), concluded that teacher presence in the form of course organization and instructor facilitation equated to a high level of satisfaction in the course. While there is a great focus on faculty-student interaction by the very nature of teaching and learning, there must also be an importance placed on peer interaction. Swan 71 (2002) examined correlations between twenty-two design factors and students perceptions of satisfaction, and learning, and interaction. Data was collected from 73 courses from the State University of New York Learning Network. Swan not only stated that interactions with peers were essential for satisfaction, but the perceived learning was tied into the interaction. A positive experience with interaction equated to a high level of satisfaction and perceived learning. Jung, Choi, Lim, and Leem (2002) in their investigation of the types of interactions with undergraduates in Korea, found that the social presence was a strong indicator of course satisfaction as well as motivation. Interaction is paramount in creating a successful online experience. Swan’s (2004) adapted Community of Inquiry model illustrated that interaction with content, peers and faculty is a key factor in online learning. Social presence is only part of the model. Garrison (2003), in his paper outlining the components of cognitive presence, clarified that no part of the model occurs in isolation, but rather through integration and balance of all three. Deep and meaningful learning is possible by reflective and collaborative discourse, part of the cognitive presence. Through reflection and collaboration, learners create meaning from the experience. Engaging in discourse with others, the social presence facilitates the cognitive presence. Shea et al. (2003) confirmed Garrison’s statement, stating, “Cognitive presence is the extent to which students are able to construct and confirm meaning through sustained discourse in a community of inquiry, and it is achieved in concert with effective teaching presence and satisfactory social presence” (p. 65). No part of the model stands on its own; each part is an integral part of the other. 72 Teaching presence also integrates with cognitive and social presence to complete the model. The instructional design and organization of the course falls into the realm of teaching presence. The actual facilitation of the course and the associated direct instruction are also part of the teaching presence. Shea, et al. utilized a survey to measure teaching presence as it related to student satisfaction. The survey was sent to over six thousand students with a response rate of 31%. The authors clarified this is a low response rate and indicated the results were more suggestive than conclusive. The survey measured the interaction of students and faculty for both facilitation and direct instruction. Students reported high levels of satisfaction with the design and organization, and this indicated students expected a high level of interaction from the instructor. They concluded that the interaction between social presence and teaching presence were related – one cannot exist without the other for a satisfactory experience. Anderson, Rourke, Garrison, and Archer (2001, p. 5) made this connection again defining, “teaching presence as the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes.” Summary The Community of Inquiry presented an effective model for computer-mediated learning. Each of the three components of the model can be used to address the recommendations of effective professional learning, graduate programs, and professional development. 73 Cognitive presence is ability of individuals to construct meaning through a community of practice. Reflective and collaborative discourse is at the heart of cognitive presence. In examining the recommendations for professional learning, graduate programs and professional development emphasized a need for learners collaborate to make meaning of the new information presented. The research literature for professional learning and professional development also recommended reflection as part of the process of contextualizing learning in one’s practice. Teaching presence was defined as the instructional design, facilitation, and direct instruction to create a meaningful educational learning experience. The learning experience created by the teaching presence required both the cognitive and social presence to be part of the process. The literature on professional learning can be connected to teaching presence if the instructional design of the course includes examination of best practice and research leading toward a change in teaching practice. The facilitation and direct instruction components of teaching presence can be connected to the recommendations of professional learning through open discussions. The instructional design and facilitation can be implemented to support active learning, content, and coherence – all part of the professional development recommendations. Social presence was defined as the ability of learners to project themselves socially and emotionally in a community of inquiry. Both the professional development and professional learning research can be linked to social presence. Both bodies of research recommended collaborative work that included interaction, feedback, and reflection. It was this rich discussion about students and student learning with other 74 teachers, and personalizing those experiences, which connected social presence to the literature on professional leaning and professional development. It was the ability to connect with others in a distance education experience that allowed one to assimilate and personalize the experience. The Community of Inquiry model provided a foundation for the distance learning that occurred in a science and pedagogical content knowledge-based graduate degree program. Teaching presence can provide both the organization and instructional components of good content. The pedagogical principles used by the instructor can be modeled through teaching presence and social presence. The teaching and social presence created within the course can support the cognitive presence of the participants. Part of the cognitive presence included opportunities for participants to collaborate and discuss pedagogical strategies and content and develop their ideas and perspectives. The three components of the model supported the recommendations for effective professional learning, graduate programs, and professional development. Connecting Professional Learning, Professional Development, and the Community of Inquiry Model Designing an online course or program should not occur by happenstance. Rather, the research literature should drive the design and pedagogical strategies used in the learning experience. The experience that a student has in an online course should match the academic rigor and expectation in a face-to-face course. Developing an online course that supports the Community of Inquiry model, as well as recommendations from the professional learning and professional development literature for science and 75 pedagogical content knowledge-based graduate degree, requires careful consideration and planning. A policy recommendation for online course development stems from the Institute for Higher Education Policy. In 2002, the Institute for Higher Education Policy established a series of benchmarks for success for online courses. These benchmarks outlined guidelines for good practice. The policy statement identified twenty-four benchmarks considered essential for a quality online learning experience. These benchmarks can be connected to the Community of Inquiry Model, in particular teaching presence. Several benchmarks focused on attributes of course development, teaching and learning, course structure, faculty support, and evaluation and assessment. These benchmarks would be the equivalent of a checklist of items that might be included in the teaching presence of a course. Benchmarks for a quality online experience are then combined with best practices in teaching. In 1987, Chickering and Gamson published the “Seven Principles for Good Practice in Undergraduate Education.” These seven principles were a guiding document for faculty in promoting good teaching practices. Chickering and Ermann (1996) modified those original seven principals for distance education experiences. The authors responded with principles for online learning that addressed the unique aspects of computer-mediated learning. The first principle of good practice encouraged communication between students and faculty. Technology offered additional opportunities for communication with students through email and discussion, while increasing participation and contribution 76 from diverse students. Chickering and Ermann claimed that this connection was a motivating factor for students. This first principle can be related to social presence in the Community of Inquiry Model. Interaction between faculty and students was one of the types of social interaction that defines social presence. Increased collaboration was encouraged and supported in online learning and allows for more open communication between students was the second principle identified by Chickering and Ermann (1996). Social presence was part of this principle as student to student and student to faculty are types of interaction that can increase course satisfaction. In addition, this connection and collaboration with others supported the recommendations from the literature from professional learning and professional development on collaboration (Pedder, 2007; Garet, et al., 2001). Offering opportunities to discuss the content, get feedback, and reflect on the process also supported cognitive presence. Kleinman’s (2004) white paper illustrated examples of professional development that strongly focused on “cohorts”, a group of educators participating together in a professional development, and expounded on the use of the application of the online work being applied to the classroom. The application of the new knowledge was tried and then successes and challenges were shared with the community for an increased value of the professional learning The next principle identified by Chickering and Erman was active learning. Active learning techniques were supported not only through communication tools, but learning by doing – using technological tools in the field, simulations, and opportunities 77 for insight using these tools. Chickering and Ermann, in their third principle, emphasized that the opportunities for active learning using technology are vast. The definition of active learning by Chickering and Ermann define active learning as talking about learning, reflecting upon learning, relating it to personal experience, and then applying to daily lives. This definition resembled the definition of active learning from the professional development literature. In the context of professional development, active learning can be observing other teachers, being observed, planning for implementation of the new knowledge in the classroom, analyzing student work, and presenting, leading and writing about the content (Garet, et al. 2001). Relating the learning to one’s practice is similar to coherence as described in the professional development literature, connecting the teacher’s context. The concept of active learning can also be connected to the literature on professional learning. Professional learning consists of inquiry, examination of good practice to improve teaching practice. The application to one’s practice is evidence in active learning and professional learning. The dimension of social capital also connects to active learning through the process of talking about student learning and applying it in the classroom. Professional learning also includes critical and responsive learning – the feedback of active learning (Pedder, 2007). The process of active learning supports the Community of Inquiry model through the teacher presence of instructional sequence, to the social presence created in discussion, and cognitive presence of assimilating the new knowledge into one’s teaching practice. 78 Maintaining a written record of what has transpired and utilizing other technological tools such as video, editing capabilities of word processing software, etc, expands opportunities for feedback and was the fourth principle identified by Chickering and Ermann. The authors clarified that students needed scaffolding to assess their prior knowledge and needed feedback to assess their growth and understanding of new knowledge. Computer mediated instruction provides a written record of all that has transpired and offers multiple methods of providing feedback and opportunities for reflection. Once again, this emphasis on feedback and reflection relates to the literature on professional learning (Pedder, 2007: NCRTL, 1995; Elmore & Burney, 1997). This process of feedback and reflection supports cognitive presence. The anytime, anywhere capability of most distance learning allows coursework to fit into busy schedules. Chickering and Ermann referred to this as time on task. The flexible nature of online learning allowed for increased efficiency and time spent learning, rather than commuting. This defined the fifth principle. They also stated access to resources online allowed students who are geographically distant to have the same opportunities as those students on campus. Access to electronic resources was more time efficient. The authors indicated that a better use of time contributed to better interaction between participants in the course. Social presence was again supported by increased opportunities to discuss the material with colleagues and faculty. Focused time on task related to the recommendation from the professional development literature that extended and continuous professional development is effective in impacting instructional practice (Kedzior & Fifield, 2004). 79 High expectations were established through good practice in online learning as per Chickering and Ermann’s sixth principle. Utilizing technology allowed participants to access to additional information on the Internet and the experience can be personalized. Technology allowed for opportunities to have work peer-reviewed and work samples and the expectations can be easily posted for all to review. The collaboration of peer-review connects to the professional learning recommendations. Critical and responsive learning, a dimension of professional learning, included teachers sharing ideas and getting feedback, observations by other colleagues, reflecting on that feedback (Pedder, 2007). This recommendation for good practice also incorporated teaching presence by setting the high expectation, social presence trough peer reviews, and cognitive presence through reflection. Lastly, Chickering and Erman stated that technology offers multiple avenues for learning that met the needs of all students. Technology addresses a variety of learning styles and the instructor has numerous options available to support the learning styles. Through course design and instruction, teaching presence can impact good practice by meeting the needs of a diverse student population. The authors emphasized self- reflection and collaboration were strategies of good practice in distance education. By meeting the needs of diverse learners with these strategies, the recommendations of professional learning and the Community of Inquiry model were supported. Collaboration is a component of inquiry and building social capital and social presence. Reflection is the foundation of critical and responsive learning and cognitive presence. This principle also supported the recommendation from professional learning of valuing 80 learning. All students can learn and that learning should be recognized is the recommendation from the literature. It connects with the idea that technology supports diverse learners and all learners can be successful. Summary Chickering and Ermann’s updated “Seven Principles of Good Practice in Undergraduate Education” provided recommendations for good practice for distance learning. The authors emphasized that if technology is to be fully utilized to its potential, the recommendations for good practice should be consistent with the essence of the Seven Principles. The modified Seven Principles aligned with the Community of Inquiry Model and the literature on professional learning and professional development. The Community of Inquiry Model consisted of a balance and connectedness between teaching, social, and cognitive presence. Chickering and Ermann described in the updated Seven Principles that the instructional design and structure, setting high expectations, and active learning were all components established by the instructor. These components were all part of the teaching presence. Emphasized throughout the principles is the opportunity for interaction between the faculty and students and between the students. The collaboration and active learning is social presence. Within the aspects of collaboration, feedback and reflection are key components. Feedback and reflection are what contribute to cognitive presence. Professional learning consists of inquiry, the examination of research and new information and discussing it with colleagues. Building social capital was also a component of professional learning and emphasizes collaboration with colleagues on how 81 students learning, sharing ideas, and implanting those new ideas into practice. Critical and responsive feedback consists of feedback and reflective practice. Valuing learning is the idea that all students are capable of learning. These recommendations from the literature aligned with the updated Seven Principles. The Seven Principles emphasized collaboration, reflection, discussions about learning, and meeting the needs of diverse learners. Chickering and Ermann’s updated Seven Principles can also be connected to the literature on professional development. Effective professional development incorporates active learning. Active learning was one of the principles. In addition, the principles recommended connecting the learning to one’s daily life and experience. This is similar to the recommendation of coherence – connecting the professional development to the teacher’s instructional context. Distance Education Supports Geographically Separated Learners The pace of new information and knowledge is ever increasing in all fields, especially in education due to reform efforts. Educators must keep up with those changes through professional learning. A science and pedagogical content knowledge-based graduate degree program is one type of professional development that can support geographically distant learners, while meeting the participants’ need for content and new information. Distance education can offer the mechanism to accomplish professional learning goals. Harris and Ritter (2007) advocated that e-based learning meets the needs of educators with its “any time, anywhere” philosophy. 82 Perdue (2003) reviewed the research regarding web-based professional education. Her analysis explored the use of distance learning, as well as the motivation and deterrents of online learning. A multitude of professions utilize web-based learning as a means to keep current in their respective fields. Most professions require continuing education and distance learning opportunities make this task easier and more convenient. A study by Thomas-Goodfellow, Pedue, and Valentine (2001) examined motivation for online professional development for respiratory therapists. They used a self-completed, mailed survey to participants in a web-based continuing education course. There were 110 respondents to the survey. Their study identified eight motivators for distance learning including flexibility and convenience. Valentine (2002, p. 2) reiterated these themes stating distance education “can meet the promise to deliver classes to a geographically broad and diverse population.” The U.S. Department of Education (2003) examined distance education program across the U.S. in both 2-year and 4-year Title IV-eligible, degree-granting institutions. The report outlines key findings in regard to distance education. Access for geographically distant students was a priority for these institutions. Neal and Miller (2004) built upon these common themes. In their review of distance education, the authors identified similar factors that have affected the popularity and rise of distance education offerings. The increased accessibility is also a key factor in the popularity of distance education. Accessibility has multiple layers including attracting diverse student populations and those students with disabilities for which online courses are may offer easier access than attending a traditional course. One other benefit is that participants in 83 the courses have equal opportunities for their voice to be heard. Course participants have access to experts who may provide a learning opportunity without requiring travel. Summary Distance learning provides opportunities for professional development that may not have been available to teachers separated geographically from a campus. The increase in distance professional development is rapidly growing. Participants report it is both convenient and cost effective. It saves travel time and allows teachers to engage with other educators in a collaborative manner anytime, anywhere. Increased access is often are reasons cited as a benefit to online learning. Summary: The Need for Further Study Educators who continue to improve their pedagogical skills and content knowledge are engaging in professional learning. The research on professional learning identified five dimensions of professional learning that have the potential to impact student learning. Inquiry is the process of reading the literature, examining best practice while collaborating with others. Building social capital emphasizes the collaboration further. In this dimension teachers exchange ideas, share insight to their own learning, and discuss how students learn. Another dimension of professional learning is critical and responsive learning. This dimension centers on reflective practice and feedback from other teachers. The responsiveness of that reflective practice is what makes this dimension a powerful learning experience for the teacher. Valuing learning is another part of professional learning. Not only is learning celebrated and recognized, but the 84 underlying core value is that all students can learn. Pedder (2007), through extensive teacher surveys and case study data from teachers in England, characterized and defined these dimensions of professional learning. Professional learning that emphasizes disciplinary content knowledge is another aspect of professional learning. Understanding the concept at a deeper level allows teachers to differentiate instruction to better meet the needs of a diverse student population. In addition to the increased content knowledge, a focus on pedagogical content knowledge and the science specific strategies to implement in the classroom is necessary as well. The research literature on professional learning included a focus on content along with the dimensions Pedder identified (Day & Gu, 2007; Bolam, et al., 2005; Bransford, et al. 1999; Elmore & Burney, 1997). A graduate degree program can be a form of professional development. The Council of Graduate Studies, along with other graduate school organizations, has recommendations and policy statements that clarified what a quality graduate program should entail. The research literature on graduate program is sparse and the majority of recommendations and guidelines come from policy and position statements from the various graduate school organizations. The policy and research literature identified ten components that should be included for an effective graduate program. The recommendations for graduate programs had some similarities to both the professional learning and professional development literature. All had an emphasis on content. All had an emphasis on the application of the content knowledge, through discussions and collaboration. Both the graduate program and the dimension of inquiry 85 in professional learning included the examination of research. The literature on professional development identified coherence and alignment of the professional development to a teacher’s context and curriculum. The graduate literature also referred to a coherent program of study, a program that assured mastery of specified knowledge and skills. In examining coherence in the MSSE program, the mastery of knowledge and skills can be connected to the coherence recommenation of professional development. The coherence of both content and pedagogoical content knowlegde can be a lens in which to examine the MSSE program. Communication skills were also a recommendation from the literature. Graduate students should possess the ability to communicate in a manner appropriate for the degree and discipline. This can be connected to the literature on both professsional learning and professional development. Collabortion, discussion, and consultation with other teachers and students, all require communication skills. In addition, the graduate recommendations included student support. This consisted of everything from financial support to advising of the graduate students. The literature emphasized that a graduate program should ensure the academic rigor of the program. The Council for Graduate Studies offered suggestions on how that can be evaluated, including internal and external evaluations. The research also pointed to programs that had interdisciplinary components as an effective characteristic. Programs that cut across disciplines and/or field of studies offer students a more global perspective in their course of study. The last two recommendations were characteristics that allowed for a performance based assessment and demonstration of understanding by the teacher 86 enrolled in a graduate program. A capstone project and a comprehensive exam rounded out the graduate program recommendations for an effective graduate program (Hinumi, 2005; Kasword and Hemmingsen, 2005; Haworth & Conrad, 1997; Conrad, Duren, and Haworth, 1998; Minkel & Richards, 1987, Council of Graduate Studies, 2005). The third lens that was used to examine the MSSE program was professional development. Garet, Birman, Porter, Desimone, Herman, Yoon’s (1999) comprehensive study was the framework used for the professional development component. Professional development was characterized by two features, structural and core. The structural features were the architecture of the professional development experience. The research literature identified several structural components, but two were pertinent to this study. Professional development that is sustained over time, includes extensive contact hours, and offers opportunities for practice and follow up was one of structural professional development feature that was examined. The literature identified that extended and continuous professional development impacted teachers’ knowledge (Kedzior & Fifield, 2004; Kennedy, 1998; Garet, et al., 2001). The second structural recommendation is participants who share a similar teaching context. There was power in the learning of between educators who participated in the program with similar teaching assignments and content (Maldonado, 2002; Loucks-Horsley, Stiles, & Hewson, 1996; Garet, et al., 2001). In addition to structural components were core components. Core components were the activities that the teachers engaged in during the professional development. A focus on content was one of the core activities. As previously mentioned, a focus on 87 content was mirrored in the literature from professional learning and effective graduate programs. Active learning was another core feature that was used to examine the MSSE program. Discussions about pedagogical content knowledge, instruction, planning for implementation, analysis of student work, and presenting, reading and writing about the content with others comprised the essence of active learning. Once again, there were similarities to the professional learning literature. Inquiry, building social capital and critical and responsive learning, all professional learning dimensions, can be connected to active learning. Both recommended discussions, collaboration, trying new strategies and sharing the results with other teachers. A final core recommendation was coherence. Coherence refers to a connection between the professional development and teachers’ curriculum, standards, and students. As stated previously, a connection can be made between coherence of professional development and a coherent graduate program. There is little research examining the impact on the professional learning of graduates of a science and pedagogical content knowledge-based graduate degree program, such as MSSE. There is no established method of examining the effects of the professional learning or using artifacts to demonstrate the learning. Structures of effective components emerge from the literature on professional learning, graduate programs, and professional development. Those components are illustrated in Figure 6. 88 Figure 6. Effective Components for Professional Learning, Graduate Programs, and Professional Development as described by the research literature and policy recommendations. While many of the descriptive terms for professional learning, professional development, and graduate programs varied, the definitions and intent were similar as described previously. This research study identified the impact of professional learning of the MSSE graduates. The findings from this study added to the limited research on science and pedagogical content knowledge-based graduate degree programs. In addition, this study informed the MSSE leadership about the program and its influence on the professional learning of its graduates. 89 CHAPTER 3 RESEARCH METHODOLOGY Introduction This chapter describes the design of this study and the research procedures used. Measures utilized to improve the study quality and the potential delimitations and limitations of the study are presented. The Master of Science in Science Education (MSSE) program provided the context for this study. The program, along with its students, graduates, and faculty are described in the sections that follow. Other contextual and background information necessary for understanding the objectives of the study are provided. In addition, the selection process of study participants is outlined. This research involved a concurrent mixed-methods design utilizing both quantitative and qualitative techniques (Creswell, 2007). In a concurrent study, both quantitative and qualitative data are used to seek similarities within the results. Both data carried equal weight in the study. In this study, two surveys were used to provide an overview of how the MSSE program addressed the dimensions of professional learning. The case studies provided a more in-depth analysis of the data as to where and how the professional learning occurred. Case studies were utilized for in-depth study and comparison (Patton, 2002). Case study data was used in this study to provide an understanding of the professional learning that occurs as a result of participation in the MSSE program. The case studies were layered or nested, in which several individual cases are used to evaluate a program 90 (Patton, 2002). Case studies were only conducted with students in the MSSE program, not faculty. The research procedures section described the different data collection methods used in this study including 1) a survey of graduates and current participants in the MSSE program 2) a case study of six graduates and current participants in the MSSE program and 3) a survey of MSSE faculty. The case study data included the MSSE Professional Learning Survey (Appendix C) administered to all students and the open ended, standardized interviews. In addition, portfolios demonstrating evidence of professional learning from the graduates and current participants in the MSSE program were collected. The Institutional Review Board of Montana State University mandated that all requirements for the protection of human subjects be met in this study. The IRB approval for this study (Appendix A and Appendix B) included that all case study participants were provided with consent forms which were signed and securely filed. In addition, names and identifying information of all the participants, MSSE faculty and staff, and specific courses were changed to maintain confidentiality. Program Context The Master of Science in Science Education (MSSE) program is a science and pedagogical content knowledge-based graduate degree program available to participants from across the country, as well as internationally. The MSSE Program is unique in that it is an inter-college, interdisciplinary program housed in the Division of Graduate Education at Montana State University-Bozeman. In 1996, the Montana Board of 91 Regents of Higher Education approved the MSSE degree program. It is a joint venture among the Colleges of Agriculture, Education, Health and Human Development, Letters and Science, and the Division of Graduate Education (MSU, 2008). The Department of Intercollege Programs for Science Education administers the program. The technical support for the program’s online courses is managed from the Burns Telecommunication Center, Montana State University’s Extended University. MSU-Bozeman faculty members of the Departments of Biology; Ecology; Chemistry and Biochemistry; Earth Science; Education, Health and Human Development; Land Resources and Environmental Science; Mathematics; Microbiology; Plant Science and Plant Pathology; Physics; and other related areas teach most courses. Approximately 91% of the instructors have doctorate degrees in the subjects they teach. Science courses are offered through the appropriate science content departments (MSSE, 2008). The majority of the courses may be taken through online, asynchronous, computer mediated communication. The program of study for most students includes approximately 80% of the coursework being completed online. This offers students, who are primarily practicing science teachers, the flexibility and convenience to participate in a graduate level program while continuing their careers. Students are required to be on campus for a minimum of two weeks as they pursue their MSSE degree, one week for at least one lab or field based campus course. A second week on campus is required to present the culminating capstone project during the annual symposium in science education. Most students complete the degree within 2-3 years, with 46% completing the 92 program within 2 years. There is a six-year window of completion for master’s degree coursework that is required by the MSU Division of Graduate Education. As of Spring 2009, there were 81 courses offered in the MSSE program and by the National Teachers Enhancement Network (NTEN). NTEN is an MSU online professional development program for K-12 teachers of science and has offered graduate level science courses since 1993. Table 6 provides for an overview of types of courses offered. Table 6 Overview of MSSE Types of Courses Offered. Types of Courses Discipline MSSE NTEN Courses TOTAL COURSES Biology 9 1 10 Chemistry & Biochemistry 8 1 9 Earth Science 9 2 11 Education 11 11 Electrical Engineering 1 1 Health and Human Development 2 2 Geography 1 1 Geology 4 4 Land Resources and Environmental Sciences 2 2 4 Mathematics 1 1 1 Microbiology 5 2 7 Plant Sciences 3 3 Physics 6 9 15 Range Science 1 1 93 Thirty total credits are required for the MSSE degree which include 12 core education credits consisting of an assessment course and an action research sequence designed to lead to the capstone project, 12 credits from a multidisciplinary selection of science courses, and 6 elective credits from either science or education (Table 7). Table 7 MSSE Program of Study. MSSE Required Courses Credits Education (includes capstone) 12 Science Courses 12 Electives (science or education) 6 A three-person faculty committee is formed to advise each student during enrollment in the program and through the design, implementation, write-up, and presentation of the capstone project and paper. Participant Context Students are admitted on a rolling basis throughout the year through an application and admission process. Applicants must have a bachelor’s degree in science, science education, K-12 education, or other related area. Applicants must have two years of successful teaching experience, which can range from a traditional school setting to informal science settings such as a museum or a position in outdoor education. At the end of the 2008/2009 academic year there were 333 educators enrolled in the degree program. All were at different stages of their programs of study. At the end of the 2009 summer session, teachers from 50 states and 16 foreign countries had 94 participated in the program (Table 8). By the end of the summer 2009, 413 MSSE degrees had been awarded in the program’s history. Table 8 MSSE Students, Currently Enrolled and Graduates, Summer 2009. Geographic Distribution Grade Level Instate Teachers Out-of-state Teachers Total Teachers Elementary 27 10 37 Middle School 29 104 133 High School 71 253 324 Community College 7 27 34 Other Non-traditional & dual teaching assignment 34 120 154 Total 168 514 682 Many students begin their experience in the MSSE program by enrolling in NTEN or MSSE courses as non-degree/continuing education students. The initial experience with these courses lead many participants to apply for admission in the MSSE program. Table 9 provides an overview of the demographics of the MSSE students as of the summer of 2009. Table 9 MSSE Student Characteristics – Summer 2009 Gender Number of Participants Male 271 Female 411 Ethnicity White 576 African American 1 Asian 5 Hispanic 7 Hawaiian 1 American Indian 9 95 Research Procedures The purpose of this study was to explore the professional learning of current students and graduates of the MSSE program at Montana State University – Bozeman. Based on the research literature, it appeared that effective graduate degree programs embedded many components in the design that were consistent with the identified dimensions of professional learning. These included inquiry, building social capital, critical and responsive learning, valuing learning, and a focus on content (Pedder, 2007; Day & Gu, 2007; Bolam et al., 2005; Bransford et al., 1999; Elmore & Burney, 1997). In addition, based on the research literature, it appeared that effective graduate degree programs embedded in the design many components that were consistent with the strategies used in effective professional development for educators (NSDC, 2001; AERA Association, 2005; Ball, 2000; Garet et al., 2001; Desimone, et al., 2002; Kennedy, 1998; Borko, 2004; Kedzior and Fifield, 2004; Maldonado, 2002). These strategies included active learning, a content focus, coherence, extended and continuous participation, and participants in similar contexts. Lastly, the recommendations in the policy and research literature for an effective graduate program were a lens through which to examine the MSSE program (Hinumi, 2005; Kasword and Hemmingsen, 2005; Haworth & Conrad, 1997; Conrad, Duren, and Haworth, 1998; Minkel & Richards, 1987; Council of Graduate Studies, 2005). These dimensions of professional learning, key components of effective graduate programs and professional development were used to structure this study and influence the selection and creation of data collection methods and instruments, which included case study and survey data from both MSSE students and survey data 96 from the faculty. Sample size and selection for the survey, participant and faculty case study sample size and selection were addressed. Table 10 provides an overview of this study. Table 10 Overview of Research Exploring Professional Learning of the MSSE Graduates. Research Questions Data Collection and Analysis Participants and Timeline 1. How do the characteristics of the MSSE program compare to 1) research and policy recommendations for graduate education and 2) to professional development standards for teachers? a. What are the formal program requirements for the MSSE program? b. What is the intended program experience? c. How do these formal requirements and intended experiences align with recommendations for graduate education programs and teacher professional learning in research and policy literature? Literature Review and interviews of program staff Interviews of program staff Interviews of program staff Literature review interviews of program staff Spring & Summer 2009 Spring & Summer 2009 97 Table 10 Continued. Research Questions Data Collection and Analysis Participants and Timeline 2. How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from a teachers’ perspective, contribute to participants’ professional learning? c. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? d. What evidence of teacher professional learning can be identified? Case interviews & portfolios of MSSE students MSSE Professional Learning Survey – participant perceptions & portfolios of current and former MSSE students MSSE Professional Learning Survey – participant perceptions & portfolios of current and former MSSE students MSSE Professional Learning Survey – participant perceptions Portfolios of current and former MSSE students Selected MSSE students for case study – Summer/Fall 2009 All current and former MSSE students July- September, 2009 All current and former MSSE students & selected students for case study, Summer/Fall 2009 Selected MSSE students for case study - Summer/Fall 2009 3. How does the distance mode of delivery of this science and pedagogical content knowledge- based graduate program impact the science teachers’ professional learning? Case interviews of MSSE students Summer /Fall 2009 98 Overall Design of the MSSE Professional Learning Survey The quantitative data for the research study was collected using the MSSE Professional Learning Survey (MPLS) developed by this researcher based on Pedder’s four dimensions of professional learning (Appendix C). The research on professional learning also indicated that a focus on content knowledge was a measure of professional learning (Day & Gu, 2007; Bolam et al., 2005; Bransford et al. 1999; Elmore & Burney, 1997). Thus, questions related to content knowledge were included in this survey. The literature described professional learning as having the characteristics of inquiry, building social capital, critical and responsive learning, valuing learning, and a focus on content. Pedder et al. (2005) described the development of a survey called the Learning How to Learn Staff Questionnaire. The survey consists of three parts. Only part B of the Learning How to Learn Staff Questionnaire was applicable to this study. James et al. (2003) outlined the research base for the questions that were developed in the original survey. The Learning How to Learn survey was pilot tested and revised several times before the final questions were established. Over 2000 teachers had responded to this survey. The Learning How to Learn Staff Questionnaire used a correlation analysis. The Staff Questionnaire was administered twice with a 55% return rate from the schools in the study in 2002 and 2004. Exploratory factor analysis in the original study led to the identification of four factors accounting for 43.8% of the variance of respondents’ scores. The four factors identified for Part B of the Learning How to Learn survey on which the MSSE Professional Learning Survey was based included the following: 99  Factor 1: Inquiry: a. Using different sources of knowledge b: Inquiry: Reflection and change  Factor 2: Collective Learning  Factor 3: Mutual Support and reassurance  Factor 4: Valuing Learning These four factors were what Pedder et al. used to develop the dimensions of professional learning, the framework for this study. A reliability analysis showed a generally high level of reliability based on Cronbach’s alpha for the factors that were identified. Cronbach’s alpha for each of the four factors in Part B were .8998, .8847, .8238 and .7520, respectively (James, et al., 2003). Because the Learning How to Learn survey was developed in England, there was one minor change that this researcher made in terminology. The researcher changed the word pupils to students to reflect a more common term used in the United States. The phrasings of the statements were modified to make the statement more personal to the respondent and connect to an overarching question stem. For example, in the original survey, question five stated, “students are consulted about how they learn most effectively” (James, et al., 2003). This was modified to, “consulting and observing students about how they learn most effectively”, and connected to the overarching question. Appendix E outlines the changes made from the original Learning How to Learn Questionnaire and the MPLS for the MSSE students. The reliability of the MPLS is discussed in the next section. 100 An MPLS was also developed for the MSSE faculty (Appendix D). It was similar to the survey used for the MSSE students. Terminology and phrasing were modified by the researcher to reflect a faculty perspective rather than a graduate student perspective. For example, question five on the original survey stated, “Staff engage in team teaching as a way of improving practice” (James, et al., 2003). This was modified to – “Engaging in team teaching with someone at the MSSE teacher’s school as a way of improving practice”. Appendix F outlines the changes made for the faculty survey. Questions on the MPLS that related to content specific professional development came from the 2000 National Survey of Science and Mathematics Education survey administered by Horizon Research, Inc. Specifically, question 12b was used in the MPLS for this study as it addressed content focused professional development as outlined in the literature. The 2000 National Survey of Science and Mathematics Education was administered to 5,756 mathematics and science teachers across the United States. The questions used in this study maintained the original language with some minor changes. The original questions and the changes made to the questions for this study are outlined in Appendix G for both the student and faculty survey. The questions used in this study from the 2000 National Survey of Science and Mathematics Education focused on the impact of professional development on the respondents’ content knowledge. The results were expressed in percentages or means with the standards errors in parentheses and are shown in Appendix H. No reliability data was generated for the questions selected for use in the MPLS. Reliability data was only provided on selected composite variables, and Question 12 was not included in any 101 of those composites that included reliability data. (S. Smith, personal communication, February 2009). The questions on the MPLS that related to changes in practice, skills, and knowledge were modified from the eMSS (Electronic Mentoring for Student Success) Teacher Growth Survey –Mathematics created by McAleer (2008). Questions from Section IV of the MPLS that focused on content knowledge, pedagogical content knowledge, curricular knowledge, and supporting diverse learners were modified for this study. McAleer used these questions to elicit responses about how specific areas in the eMSS program impacted mentor teachers’ professional growth. The modified questions used in the MPLS elicited how specific program components have enhanced the MSSE students’ professional learning. The original questions developed by McAleer and the modifications made to the phrasing and language for the MPLS are outlined in Appendix I for the student survey. The questions used in this study that asked respondents about the influence of science content and understanding come from the 2006 Learning Science Online study. The questions used were only a small subset of questions from the original study. The original study was an aggregate study of 40 online science course offered in 2004-2005. Clark and Rowe used these questions as part of a study to determine an overall picture of online science courses and to create a profile of participants and instructors in those courses. Eight questions were used that were part of the Individual Learning Opportunities section. The reliability analysis for these questions was 0.76. Five questions were from the Social Learning Opportunities. Reliability analysis for the 102 Social Learning Opportunities was 0.80 (Asbell-Clark & Rowe, 2007). The questions on the MPLS were not modified, rather the original language was maintained. Pilot Survey Both the MSSE student and faculty surveys were piloted. All surveys were administered online using Survey Monkey. A request to participate in the survey was sent electronically to the participants from the MSSE Program Director, Dr. Peggy Taylor, and included a brief statement of the research study and contact information of the researcher. The notification of the survey from Dr. Taylor informed participants of the legitimacy of the survey and may have contributed to a higher response rate. Non- respondents received a follow up email within two weeks from the researcher. A sample of 16 current MSSE students was asked to complete the pilot student survey. This preselected group of teachers was currently enrolled in one or more of the MSSE courses. Pilot survey respondents were asked for comments regarding clarity and understandability of the questions. Modifications were made as needed for the final MPLS. These included minor wording changes for clarity. As the number of MSSE faculty is small (approximately 33), a pilot study would reduce the number of respondents later. As the faculty survey closely paralleled the students’ survey, the pilot student survey was used to provide one level of feedback. Expert feedback from four members of the education faculty reviewed and commented on the survey. Three members of the science faculty also reviewed and provided expert feedback. Modifications were made as needed for the final MPLS. These included 103 adding more content specific questions. The content specific questions were identified as section two on both the student and faculty survey. The MPLS for the students and faculty was designed with feedback from the MSSE program staff. The MSSE program director and faculty members familiar with both the goals of this study and the MSSE program contributed to the development of the survey. This expert feedback contributed to the content validity of the instrument. Survey Implementation The MPLS survey was revised based on feedback from the pilot study. The final survey was administered to all MSSE students, both currently enrolled and graduates of the program. The MSSE program staff maintains contact information of all of the MSSE students. Dr. Peggy Taylor, the MSSE Program Director, made the initial request to the MSSE students to participate in the survey. Similar to the pilot survey, the initial correspondence from Dr. Taylor added legitimacy to the survey. Communication from the MSSE office may have contributed to higher response rates. The survey was administered online using Survey Monkey to 610 MSSE students and graduates who had an active email address on file with the program office. Each MSSE student was provided a link to participate in the survey. All participants were assigned a number to maintain confidentiality and only the researcher had access to names and assigned numbers. A follow up email was sent to all non-respondents at two weeks and four weeks. As the survey was originally sent out during the summer when many teachers may not be checking email as frequently, the survey was resent to all non- respondents at the end of September to increase the number of respondents. A follow up 104 email was sent in mid-October. Three hundred sixty nine MSSE students responded to the survey, a 60% response rate. The faculty survey was also revised based on feedback from the pilot study. It was administered online to thirty-two MSSE faculty. The request to participate in the survey was sent out electronically from Dr. Peggy Taylor, the MSSE Director with the same rationale as the student survey. All participants were assigned a number to maintain confidentiality and only the researcher had access to names and assigned numbers. A follow up email was sent to all non-respondents at two weeks and four weeks. Twenty-four faculty members responded for a 75% response rate. Data Analysis Survey responses attempted to reveal the ways in which MSSE program promoted the professional learning of the participants, influenced understanding of content knowledge, and pedagogical content knowledge. All of the survey responses were analyzed descriptively revealing the ways in which these teachers’ knowledge, skills, and practice was enhanced or changed. The data was also analyzed for themes and trends. A one within repeated measures analysis was used to analyze the responses in Part IV, questions 1- 4 asking about students’ enhancement and change in practice, skills, and knowledge. Qualitative analysis was used to identify trends and themes in the open- ended responses. Appendix J provides a summary of how the research questions and methods of quantitative analysis align with the questions of the MSSE Professional Learning Survey. 105 Overall Design of the Case Study Using case studies, the professional learning of the MSSE students was explored through the perspective of the participants. Case studies offer multiple perspectives in examining data. Patton (2002) stated that well constructed case studies are both holistic and context sensitive. Patton described holistic as providing an overall picture of the focus of the research through the case study. Context sensitive referred to the case study providing in-depth information about each case. Data must come from multiple sources as a means to improve the validity of the study. Yin (2009) recommended multiple sources of evidence to develop “converging lines of inquiry” (p. 115), a process of triangulation. Using multiple sources of evidence in a case study is more likely to present more accurate findings. The questions for the case study were developed by the researcher based on the literature characterizing professional learning as having the characteristics of inquiry, building social capital, critical and responsive learning, valuing learning, and a focus on content. The questions were also developed to reflect attributes of professional development. The MSSE student interview questions are found in Appendix K. The MSSE student portfolio was developed by the researcher to capture evidence of professional learning through instructional materials and student work samples. Providing an opportunity to reflect on the submitted artifacts offered insight into the professional learning of the case study respondents. The portfolio was a way to connect current practice in the classroom with a MSSE course to demonstrate evidence of 106 professional learning. The portfolio provided another source of data to strengthen the case study. The components of the MSSE student portfolio are found in Appendix M. The interview questions and portfolio for the students were designed with feedback from the MSSE program staff. The MSSE program director and faculty members familiar with the both the goals of this study and the MSSE program contributed to the development of the survey. This expert feedback contributed to the content validity of the instrument. Table 11 illustrates the alignment of the dimensions of professional learning with the specific MSSE student interview question that addresses that dimension. Table 11 Alignment Of The Dimensions Of Professional Learning And Case Study Interview Questions For The MSSE Students. Dimensions of Professional Learning MSSE Student Interview questions A. Inquiry: Using and responding to different sources of evidence and carrying out joint research and evaluations with colleagues. Do you use sources of evidence (such as research, consultation with students and colleagues) differently than before to understand and modify your teaching practice? B. Building social capital: Learning, working, supporting, and talking with colleagues to discuss how students learn. Are there any ways in which your tendency to collaborate with colleagues [MSSE or others] to explore how students learn, and to experiment with new approaches, have been modified? (Probe for specific examples.) C. Critical and responsive learning: Occurs through reflections, self- evaluation, experimentation, and by responding to feedback. Are there ways that your reflection on practice has been influenced? (Probe for specific examples.) Are there ways that you respond to feedback (from other teachers, students, administrators, parents) differently now 107 Table 11 continued. D. Valuing Learning: Deepening understanding and learning through talking about and valuing learning with others Has the value you place on your own learning changed in any way? Explain. Have the ways you convey to your students that you value their learning changed? Explain E. Content Knowledge: A deep understanding of “big ideas” as well as intricate details of a particular discipline To what extent has your knowledge of science content been enhanced as a result of your participation in MSSE? Explain (Probe for specific examples such as changes in depth of understanding of “familiar” topics; grasp of connections across topics; updating information; knowledge of research methods...) To what extent has your knowledge of science pedagogy, and your tendency and ability to implement varied approaches, been have been influenced as a result of your participation in MSSE? Explain (Probe for specific examples of understanding of “advanced” science teaching practices, frequency of implementation, tendency to use more varied practices.) Case Participants Purposive sampling was used to select participants for the participants’ interviews and portfolio. Krathwol (2004) recommended using this sampling technique as it “will better inform the researcher regarding the current focus of the investigation” (p. 172). The case study participants, both current students and graduates of the MSSE program, were selected based on their willingness to create a portfolio demonstrating their professional learning from a specific MSSE course and participate in an interview. Six current students and graduates of the MSSE participants were selected in summer 2009 based on the following criteria: 1) students in good standing with the MSSE program and 2) students who have completed at least five courses, including one 108 or two science courses, and/or the first two core education courses toward the MSSE degree. The selected participants represented a cross section of the demographics of the MSSE program including geographically distant teachers and teachers who work in hard- to-staff schools. Students were selected through maximum-variation sampling. Maximum variation sampling, a type of purposeful sampling, involves selecting a wide range of variations, such as professional backgrounds, culture, teaching experience, etc. This allowed this researcher to maximize the diversity relevant to the research questions with such a small sample (Cohen & Crabtree, 2009). The researcher contacted the MSSE Program Director, Peggy Taylor, for a list of names of students who would fit the profile needed for the study. Several names were provided for each demographic profile. The researcher then randomly selected individuals and contacted them about participation in the case study. Pilot Case Study Interviews The MSSE student interview questions were piloted with six MSSE students. A request to participate in the interview was sent electronically to the pre-selected participants from the MSSE Program Director, Dr. Peggy Taylor. It included a brief statement of the research study and contact information of the researcher. The notification of the pilot interviews sent from Dr. Taylor informed participants of the legitimacy of the interviews. This preselected group of teachers were currently enrolled in the MSSE program or preparing to graduate. Modifications to the questions were made by the researcher in the first two interviews in order to provide clarity for the student of the questions being 109 asked. All of the interviews were recorded, transcribed and reviewed with a member of the MSSE faculty steering committee. Slight modifications were made in the phrasing of the MSSE Student Case Study Interview Questions for the final interview questions (Appendix K). The pilot student interviews provided one level of feedback. Expert feedback was received from four members of the education faculty who reviewed and commented on the interview questions. The MSSE program director and faculty members familiar with the both the goals of this study and the MSSE program contributed to the development of the interview questions. This expert feedback contributed to the content validity of the instrument. Case Study Implementation As modifications were made to the interview questions during the pilot, three of the six interviews were selected as part of the final case study. These original participants also met the demographic profile created through purposeful sampling to ensure a representative sample of MSSE students. These participants were then contacted about further participation by completing a portfolio. The researcher explained the MSSE Student Professional Learning Portfolio was used to provide evidence of professional learning (Appendix M). Artifacts for the portfolio may include:  Electronic posts from the MSSE course for participants currently enrolled in a course.  Examples of lessons, units, or projects created.  Student work samples. 110  Examples from the capstone project. Other artifacts suggested by the participants were considered as an indicator of professional learning such as observation reports, evaluations, documentation of growth from colleagues, etc. Participants were asked to provide a rationale for the selected artifacts. The portfolio included a professional learning self-reflection. Participants were provided with a description of the dimensions of professional learning and asked to describe how each dimension related to their experience in the MSSE program. The self-reflection was part of the MSSE Student Professional Learning Portfolio (Appendix M). Case Study Data Analysis Analysis of interviews and portfolios was ongoing. Patton (2002) recommended three steps in the analysis of the case studies. The steps included organizing the raw data, creating a case record, and writing a descriptive narrative. Categories were identified as they emerged from the data analysis. Those categories were then subdivided to better reflect the data’s themes. The raw data consisted of the transcript of each interview and the portfolio. The data was organized into categories and subcategories. Excerpts of the interviews and portfolios were identified for use in the case narrative. The case narrative was a descriptive picture of each case created after reviewing the data (Patton, 2002). Inter-rater reliability is the extent to which two or more individuals agree. Inter-rater reliability addressed the consistency of the implementation of a rating system; therefore, another researcher coded the raw data. The co-rater was a doctoral level science educator. 111 The co-rater read through each interview transcript and was provided a list of numbered excerpts from the interview. The co-rater then coded these statements based on the categories and subcategories that emerged from the data. The co-rater was also asked to look for other categories. If discrepancies existed, the co-rater and this researcher discussed them and the categories were edited accordingly. The co-rater was also provided a copy of the narrative and was asked to read and comment about the narrative to ensure that each case was accurately presented. A co-rater was also used to verify the portfolio data. The portfolio data was analyzed using a rubric developed by this researcher. The co-rater was provided information and training on the dimensions of professional learning. The training consisted of reading the literature review of professional learning and several conversations to discuss the aspects of each dimension. The co-rater was then presented with the raw portfolio data and asked to score the portfolio using the same rubric. If discrepancies existed, the co-rater and the researcher discussed them and the rubric was edited accordingly. The co-rater was also provided a copy of the narrative of the portfolios and was asked to read and comment about the narrative to ensure that each case was accurately presented. Triangulation occurred through the different data sets collected in this study. A comparison of themes and ideas that surfaced from the case studies provided one perspective. The case study data was compared to the recommendations from the professional learning literature. For both the case study interviews and the portfolio, a rubric was used to measure the professional learning dimensions. The Professional 112 Learning Reflection Rubric for Interviews and the MSSE Student Professional Learning Portfolio Rubric are found in Appendices N and O. Study Fidelity Using a mixed methods approach to collect data offered triangulation that addressed issues of internal validity. Creswell (2003) defined a mixed method study as one in which data collection is both numeric and textual. Using the survey from the MSSE students and faculty, and the case study interviews and portfolios of students, these diverse methods provided an understanding of the research questions. The survey was broad and allowed the researcher to generalize. The MSSE Student Case Study Interview Questions and MSSE Student Professional Learning Portfolio data allowed for the examination of the research questions on a more granular level. Denzin (1978) stated that triangulation can partially overcome the deficiencies inherent from a single researcher. Lincoln and Guba (1985) reiterated triangulation as a method to improve validity. Viewing data from difference sources and perspectives was a strategy to improve the quality of a study. Both qualitative and quantitative data collection methods have benefits and shortcomings. Quantitative methods, such as the survey utilized in this study, allowed for large numbers of respondents. Quantitative methods are often associated with researcher objectivity and can be replicated by others (Creswell, 2003). Shortcomings stem from the fact that the researcher selected the questions based upon the research literature and may not document other perspectives and perceptions. 113 Qualitative methods also have advantages and disadvantages. This study focused on participants, current and former, in the MSSE program and faculty. Capturing perceptions from these two groups allowed for rich descriptions that provided a unique insight to the MSSE program and allowed the researcher to document unanticipated perspectives and perceptions. Qualitative methods have the disadvantage of being more difficult to generalize. Creswell (2003) advocated the use of a mixed methods approach as long as there was clarity, through detailed documentation, in the data collection methods being utilized. In addition, the researcher merged qualitative and quantitative data to answer the research questions being posed. The survey questions provided a broad context for the study in which generalizations emerged. The more detailed data from the interviews have the potential to enhance and provide detail to the generalizations in the survey. Delimitations and Limitations The survey used in this study only involved current MSSE students and graduates of the program. There were 349 graduates of the program and 333 students currently in the program that were potential participants, yet the number of respondents to the survey limited the study. Non-respondents are always problematic since their lack of participation can affect the conclusions drawn from the analysis of the data. Since this study only collected data from the MSSE program, generalizability to other science and pedagogical content knowledge-based graduate degree program is limited, but possible through the recommendations provided in the literature and emergent themes in the data. 114 As a cross section of MSSE students was selected, the results can be generalized specifically to the MSSE program, but can also inform a larger audience of gradate programs and professional developers. Responses to survey questions were of concern since respondents can potentially answer questions not as they see themselves, but as they would like to see themselves. Without firsthand, thorough observations of the respondents, the survey data collected was subject to a variance in accuracy. However, the portfolio provided evidence to support the self-perception of professional learning. As there are no control groups, it was challenging to determine if MSSE was the sole contributor to the professional learning of the participants. Even with a control group, it would still present challenges. Other extraneous factors may have influenced the respondents besides participation in the MSSE program and provided rival explanations (Krathwol, 1998). Efforts were made through the case studies to understand other influences on the professional learning. The MSSE Student Case Study Interview Questions asked for specific examples, which provided information to understand the influences. Another challenge was that participants had self-selected participation in the MSSE program. Self-selection can be the result of a higher level of motivation. Teachers may have had other motivations for participation beyond increasing their professional learning. Another limitation of the Student MPLS was the number of courses a respondent had taken when answering the questions. The perception of influence may be limited to 115 only a single course. Other respondents may have taken several courses at the time of the survey. Participants were selected based the demographic profile by the researcher. The MSSE Program Director provided the names of potential participants. The final selection process was left to the researcher to remove influence by the MSSE program staff. The Program Director was asked to send an email to the selected participants after they had been selected to provide legitimacy to the request. 116 CHAPTER 4 RESULTS OF THIS STUDY Introduction This chapter presents the results of the analysis of data collected in this mixed methods study. In keeping with the same consistency and format of the discussion of the data collections procedures in Chapter 3, the following results are presented and discussed in five sections: 1. The MSSE Professional Learning Survey for Students 2. The MSSE Professional Learning Survey for Faculty 3. Summary of the MPLS 4. Case Studies of MSSE Students 5. Summary of Case Studies 6. Summary of Overall Findings A brief, explanatory introduction is given at the beginning of each section, followed by the presentation and discussion of results, and concludes with a brief summary of the results obtained in that specific area of the study. The last section provides an overall summary of all of the findings. MSSE Student Survey This section describes the results of the analysis of the MSSE Student Professional Learning Survey, which can be found in Appendix C. The survey was 117 designed to assess the ways in which the MSSE program influenced the professional learning of the participants and which program components contributed to this learning. Specifically, this survey was designed to answer the following research questions: Question 2: How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from a program teacher’s perspective, contribute to participants’ professional learning? c. What evidence of teacher professional learning can be identified? Question 3: How does the distance mode of delivery of this science and pedagogical content knowledge-based graduate program impact the science teacher’s professional learning? Appendix J provides a concordance of the research questions methods of analysis and the MPLS questions. There were 369 student surveys and 24 faculty surveys returned and included in this analysis. The response rate for the student survey was 60 percent and the response rate for the faculty survey was 75 percent. Section I Student Survey –Professional Experience and Education The purpose of this section of the survey was to gather background and contextual information on the participants. The largest percentage (39.4%) of the respondents was 118 from graduates of the MSSE program. The remaining respondents were still enrolled in the program with 17.7% in their first year, 22.3% in their second year and 16.8% in their third year. A small group (3.8%) of the respondents were in their fourth year. The graduate respondents ranged from all years of the program with the majority graduating in the last three years (Table 12). Not all respondents indicated the year that they graduated. Table 12. MSSE Respondents Graduation Dates, (N = 183) Year of Graduation Number of Respondents Percentage 1999 8 4.37% 2000 6 3.28% 2001 3 1.64% 2002 4 2.19% 2003 13 7.10% 2004 10 5.46% 2005 25 13.66% 2006 22 12.02% 2007 31 16.94% 2008 22 12.02% 2009 27 14.75% Unknown 12 6.55% Respondents were also asked about their years of teaching experience in science. One third (31.5%) of the respondents had between five and ten years of teaching experience with nearly one quarter (23.9%) who taught fifteen or more years. Only 6.3% of the respondents had taught between one and two years as a teacher of record in science, yet had also taught other subjects in other years. In addition, 4.1% had not been 119 a teacher of record in science. The MSSE program also attracts a number of informal science educators who may not be teaching in a school setting. In examining the educational background of the respondents, 45% of the MSSE students and graduates have attained an undergraduate degree in biology. The rest of the respondents had a variety of science degrees with chemistry and broad field science being reported as the next highest with 14% each. Fifty-four respondents entered the MSSE program with an elementary education degree. In the category of Other Education, twenty respondents reported having a master’s degree, with thirteen who had a master’s in biology, and ten in earth/space science. Eight of the respondents had earned a doctorate degree. Figure 7 shows the various degrees of the MSSE students and graduates who responded to the survey. Figure 7. Types of University Degrees of the MSSE Students and Graduates. 120 The vast majority of the respondents were currently teaching (91.6%). The grade levels and courses spanned the spectrum of science with high school biology, a range of high school courses in the physical sciences, and middle school life and earth science being subjects taught most frequently. Other respondents reported teaching environmental science, elementary science, math, college chemistry, and advanced placement courses. The respondents to the survey represented twenty countries with the majority of the respondents from the USA. All fifty states were represented with the largest number of respondents living in Montana, followed by California, and Colorado. The U.S. Territories were represented by Guam and the U.S. Virgin Islands. Precision of Responses and MPLS Instrument Reliability A 95% confidence interval was calculated based on the percent of respondents returning useable questionnaires (surveys) for analysis. Three hundred and sixty-nine of the 610 MSSE program graduates returned completed surveys yielding a 60% response rate. The 95% confidence interval for a 60% response and a population of 610 is +3. This result indicated there was a 95% confidence that the true percentage of the student MSSE population choosing a certain response would be within + 3 of the percent of the sample choosing the same response. For example, if 65% of the MSSE student sample “Agreed” with a certain statement on the MPLS, then it is expected that between 62 and 68 percent (65% + 3%) of the entire MSSE student population would also indicate that they “Agreed” with the statement. The MPLS also has a very high confidence level for responses. A sample size of 369 indicates a 99 percent confidence that the true 121 population percentage would fall within the 95% confidence interval. In other words, if 65% of the sample “Agreed” with an MPLS statement, there is a 99% confidence the true percentage of the population “Agreeing” with the statement would fall within the 62 – 68 percent confidence interval. The MPLS was found to be highly consistent for assessing the professional development constructs the questionnaire items were written to assess. The Cronbach’s Alpha for the each section of the MPLS was high indicating high internal consistency reliability. The overall internal consistency reliability represented by Cronbach’s Alpha for the Student MPLS was .95. The Cronbach’s Alphas for each section of the Student MPLS were as follows: Content Understanding and Knowledge = .836, Professional Learning = .946, Content Knowledge = .834. Section II Student Survey – MSSE Students’ Science Content Understanding and Knowledge The purpose of Section II of the survey was to gather information about the experiences of the MSSE program and the influence of the experience in fostering science knowledge and practice. Table 13 provides the means, standard deviations, and the number and frequency of responses for each survey question from the MSSE students for this set of questions. In Section II of the Student MSSE Professional Learning Survey, the questions were assessed using a five-point Likert scale with the value of 1 representing Not at All and a value of 5 representing To a Great Extent on questions one through thirteen. Specifically, the question asked, “To what degree did your experiences in the MSSE program foster your knowledge and practice in the following areas 122 pertaining to your role as a science learner?” A total of 369 teachers responded to the survey, although for some parts of the survey the number of responses dropped because sections were left blank or questions were skipped. Although the Likert responses were ordinal data, item and combined items mean may be compared because the data was not found to be significantly non-normal. In addition, the statistical results were supported by comments from a later section of the survey that solicited open-ended comments when appropriate. Table 13 MSSE Student Perceptions of the Influence of Course Experiences on Knowledge and Practices in the Following Areas Relating to Science Content. Descriptive Statistics and Frequency of Responses, (N = 369). Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 1. Working on problems based in contexts from my daily life. (N = 354) 3.57 1.003 3.1% (11) 10.2% (36) 30.5% (108) 38.1% (135) 18.1% (64) 2. Analyzing and drawing conclusions from data, observations, and other forms of scientific evidence. (N = 355) 3.86 .956 1.4% (5) 7.3% (26) 22.8% (81) 40.6% (144) 27.9% (99) 3. Providing evidence to support my scientific ideas. (N=355) 3.75 .950 2.3% (8) 5.6% (20) 28.7% (102) 40.8% (145) 22.5% (80) 4. Participating in on-site fieldwork (e.g., water testing, species counting in a natural setting, astronomical observations of the night sky). (N=355) 3.56 1.349 12.7% (45) 9.9% (35) 17.2% (61) 29.0% (103) 31.3% (111) 5. Working with pen and paper problem sets (e.g., problems from a textbook or worksheet). (N=354) 2.90 1.232 16.1% (57) 21.8% (77) 28.0% (99) 23.4% (83) 10.7% (38) 123 Table 13 - Continued 6. Interacting with physical materials or models (e.g., mixing solutions, building circuits, scale models). (N=354) 3.19 1.223 10.5% (37) 18.9% (67) 26.3% (93) 28.8% (102) 15.5% (55) 7. Using computer-based animations, games, or simulations (e.g., virtual dissection, SimEarth, flash interactives). (N=354) 2.89 1.333 21.8% (77) 15.5% (55) 27.1% (96) 22.0% (78) 13.6% (48) 8. Articulating scientific ideas in a discussion. (N=355) 4.13 .883 0.6% (2) 4.5% (16) 16.1% (57) 39.2% (139) 39.7% (141) 9. Reflecting upon earlier scientific ideas. (N=352) 3.89 .975 1.1% (4) 8.5% (30) 21.0% (74) 38.6% (136) 30.7% (108) 10. Reflecting upon the scientific ideas of other students. (N=353) 3.96 .968 1.1% (4) 8.5% (30) 17.3% (61) 40.2% (142) 32.9% (116) 11.Carrying out procedures of scientific investigations designed by instructors or course developers (e.g., lab exercises, kitchen experiments). (N=355) 3.56 1.093 4.2% (15) 13.0% (46) 25.1% (89) 36.3% (129) 21.4% (76) 12. Designing my own scientific investigation(s) (e.g., developed hypothesis or question and procedure. (N=352) 3.43 1.139 6.0% (21) 15.3% (54) 27.3% (96) 32.4% (114) 19.0% (67) 13. Carrying out procedures of scientific investigations I designed (e.g., collected data, made observations. (N=353) 3.48 1.154 5.9% (21) 13.6% (48) 27.8% (98) 30.9% (109) 21.8% (77) Note. The numbers in parenthesis represent the frequency of participant responses. Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, a mean rating of 3.75 or more was used to support the “more influenced” determination for an item. As there was a tendency for students to assign higher ratings 124 than faculty, a slightly higher mean was used for student results than for faculty results for determining the threshold for designating an area as “more influenced” by course experiences. The decision to handle the student and faculty thresholds differently was made after consultation with a university mathematician with expertise in applied mathematics and statistical analysis. Results reported in Table 13 indicated that almost 79% of MSSE students ranked the ability to articulate scientific ideas in a discussion as the area with more influence on their science knowledge and practice as fostered by the program experiences. Further analysis using a combined rating found that MSSE students perceived that an emphasis on reflection was an area influenced by the program experiences. Respondents reported the opportunities to reflect upon the scientific ideas of other students (73.1%) and to reflect upon earlier scientific ideas (69.3%) were areas fostering their knowledge and practice through program opportunities. One student commented, “The MSSE program has allowed me to reflect on my own teaching experiences as well as my content knowledge.” Other areas that were influenced by program experiences included; analyzed and drew conclusions from data, observations, and other forms of scientific evidence (68.5%); provided evidence to support my scientific ideas (63.3%); worked on problems based in contexts from my daily life (56.2%). Respondents reported the capstone project, which is part of the action research sequence, as area that was influenced in supporting the ability to examine data and to use that data to create a change in practice. While many of the capstone projects are pedagogically focused, the educational research processes students are exposed to during 125 the capstone, drawn from the social sciences, parallel the research the students have been exposed to in their MSSE science courses. One student described the research process stating, “The capstone project based on classroom research-developing surveys, analyzing data, reporting, and applying the results to the classroom contributed to my science understanding.” The mean rating for the areas that the MSSE students perceived as being most influenced by the course experiences are shown in Table 14. Table 14 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Science Content Most Influenced by Course Experiences, (N = 369). M Articulating scientific ideas in a discussion. 4.13 Reflecting upon earlier scientific ideas. 3.89 Reflecting upon the scientific ideas of other students. 3.96 Analyzing and drawing conclusions from data, observations, and other forms of scientific evidence. 3.86 Providing evidence to support my scientific ideas. 3.75 Areas perceived to be “less influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of 1 (Not at All) and 2. The mean rating for each item was also used in the determination. Generally, a mean rating of less than 3 was used to support the “less influenced” determination for an item. Two areas stood out as being less influenced by the program to the MSSE students. Opportunities to use computer-based animations, games, or simulations (e.g., virtual dissection, SimEarth, flash interactives), if included in a course, were areas of knowledge and practice not perceived to be fostered by program experiences by 37.3% of 126 the respondents. Nearly 38% of respondents stated that working with pen and paper problem sets (e.g., problems from a textbook or worksheet) were less influenced by program experiences. The mean rating for areas pertaining to students’ science content understanding that were perceived as being least influenced by experiences in the MSSE courses are shown in Table 15. Table 15 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Science Content Least Influenced by Course Experiences, (N = 369). M Using computer-based animations, games, or simulations (e.g., virtual dissection, SimEarth, flash interactives). 2.89 Working with pen and paper problem sets (e.g., problems from a textbook or worksheet). 2.90 Section III Student Survey – Professional Learning The purpose of Section III of the survey was to gather information about the MSSE students’ professional learning. Table 16 provides the means, standard deviations, and the number and frequency of responses for each survey question from the MSSE students for this set of questions. In Section III of the Student MSSE Professional Learning Survey, the questions were assessed using a five-point Likert scale with the value of 1 representing Not at All and a value of 5 representing To a Great Extent on questions one through twenty-eight. Specifically, the question asked, “To what degree did your experiences in the MSSE program foster your professional learning, including your knowledge and practices, in the following areas pertaining to your role as a science 127 teacher? ” A total of 369 teachers responded to the survey, although for some parts of the survey the number of responses dropped because sections were left blank or questions were skipped. Although the Likert responses were ordinal data, item and combined items means may be compared because the data was not found to be significantly non-normal. In addition, comments from the open-ended responses from the Student MPLS that supported the statistical analysis were included when appropriate. Table 16 MSSE Student Perceptions of the Influence of Course Experiences on Professional Learning. Descriptive Statistics and Frequency of Responses. (N = 369) Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 1. Using my learning to improve students' learning. 4.55 .639 0.0% (0) 1.7% (6) 6.0% (21) 33.3% (116) 58.9% (205) 2. Drawing on good practice from other teachers as a means to further my own professional practice. 4.48 .679 0.3% (1) 3.5% (12) 8.1% (28) 36.1% (125) 52.0% (180) 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving my practice. 4.08 .874 1.4% (5) 7.2% (25) 18.4% (64) 39.1% (136) 33.9% (118) 4. Using the Internet as one source of useful ideas for improving my practice. 4.24 .834 0.6% (2) 4.0% (14) 15.6% (54) 36.6% (127) 43.2% (150) 5. Consulting and observing students about how they learn most effectively 4.16 .903 2.3% (8) 8.6% (30) 15.2% (53) 36.5% (127) 37.4% (130) 6. Relating what works in my own practice to research findings and best practices. 4.21 .865 1.7% (6) 4.9% (17) 16.7% (58) 38.2% (133) 38.5% (134) 128 Table 16 - Continued 7. Using insights from my professional learning to feed into my school’s or organization's policy development 3.67 1.050 7.8% (27) 15.0% (52) 26.0% (90) 30.6% (106) 20.5% (71) 8. Understanding how instructional strategies that work in one context might be adapted to other contexts. 3.91 .896 0.6% (2) 11.5% (40) 20.7% (72) 38.6% (134) 25.1% (87) 9. Reflecting on my practice as a way of identifying my professional learning needs to better meet the needs of students. 4.31 .746 2.3% (8) 4.0% (14) 12.1% (42) 39.4% (137) 42.2% (147) 10. Experimenting with my practice as a conscious strategy for improving teaching and learning. 4.43 .719 0.3% (1) 3.4% (12) 10.1% (35) 33.3% (116) 50.0% (174) 11. Modifying my practice in light of performance and feedback from students 4.35 .768 0.3% (1) 4.6% (16) 9.8% (34) 34.2% (119) 46.3% (161) 12. Modifying my practice in light of published research evidence. 3.75 .835 3.7% (13) 7.8% (27) 28.5% (99) 37.2% (129) 18.2% (63) 13. Modifying my practice in light of evidence of self-evaluations of my teaching practice. 4.16 .780 1.7% (6) 3.4% (12) 13.5% (47) 39.7% (138) 37.4% (130) 14. Modifying my practice in light of evidence from evaluations of my teaching practice by administrators or other colleagues. 3.54 1.000 13.3% (46) 15.1% (52) 22.6% (78) 24.6% (85) 10.1% (35) 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving my practice. 3.07 1.255 18.1% (63) 15.8% (55) 19.0% (66) 21.3% (74) 10.1% (35) 16. Collaborating to plan my teaching. 3.30 1.175 11.5% (40) 20.4% (71) 21.6% (75) 23.3% (81) 12.4% (43) 17. Reading/discussing other teachers experiences in the classroom and giving each other feedback. 3.87 1.052 4.3% (15) 10.1% (35) 18.6% (64) 38.0% (131) 25.5% (88) 18. Engaging in team teaching with someone at my school as a way of improving practice. 2.87 1.381 23.0% (80) 13.2% (46) 18.7% (65) 11.8% (41) 11.5 % (40) 19. Turning to colleagues/peers for help if I have a problem with my teaching. 3.53 1.049 6.4% (22) 18.2% (63) 23.7% (82) 27.5% (95) 16.2% (56) 20. Posting suggestions, ideas or approaches for colleagues to try in class. 3.55 1.118 5.2% (18) 13.5% (47) 22.7% (79) 30.5% (106) 21.0 % (73) 129 Table 16 - Continued 21. Testing out new ideas in class. 4.34 .766 0.0% (0) 3.5% (12) 11.6% (40) 37.3% (129) 43.9% (152) 22. Discussing openly with colleagues what and how you the teacher are learning. 3.89 .976 2.9% (10) 9.5% (33) 21.6% (75) 37.4% (130) 25.3% (88) 23. Frequently using informal opportunities to discuss how students learn. 3.85 .989 2.3% (8) 11.8% (41) 22.7% (79) 34.8% (121) 25.3% (88) 24. Offering other colleagues reassurance and support. 3.91 .973 1.7% (6) 9.2% (32) 20.2% (70) 38.3% (133) 27.4% (95) 25. Believing that all students are capable of learning. 4.10 .946 3.2% (11) 5.5% (19) 15.5% (54) 36.5% (127) 37.1% (129) 26. Encouraging students in my school to enjoy learning. 4.29 .859 2.0% (7) 4.9% (17) 14.9% (52) 31.0% (108) 45.7% (159) 27. Regularly celebrating student success. 3.90 1.017 2.6% (9) 11.0% (38) 23.6% (82) 31.1% (108) 29.4% (102) 28. Discussing with colleagues how students might be helped to learn how to learn. 3.97 .883 2.3% (8) 6.0% (21) 21.65 (75) 40.2% (140) 27.0% (94) Note. The numbers in parenthesis represent the frequency of participant responses. The data was analyzed and organized based on the research questions of this study. Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, a mean rating of 3.75 or more was used to support the “more influenced” determination for an item. As explained earlier, since students tended to assign higher ratings than faculty, a slightly higher threshold was established for identifying areas perceived by students to be “more influenced” by course or program experiences. It was apparent from this data that the MSSE students perceived that experiences provided by the instructors influenced their professional learning. The data was analyzed 130 and organized based on the research questions of this study. Four groupings were formed based on questions and responses that focused on similar ideas or aspects of teaching and learning. Those four areas, designated here as “themes” for discussion purposes, included Using Research to Inform Best Practice, Collaboration, Reflective Practice, and Supporting Student Achievement. The theme of Using Research to Inform Best Practice was defined by the questions that focused on accessing research on teaching and learning and implementing those research based ideas, lessons and strategies within one’s classroom. This theme also included teachers experimenting in their own classes as a means to improve their practice. Survey questions and responses that related to reading research (practitioner reports and journal articles), using the Internet as one source of ideas, and utilizing the research findings about best teaching practices were placed in this group. The theme of Collaboration was defined by the survey responses to questions that focused on working with others for support, feedback, and new ideas. Offering colleagues’ support, reading/discussing together, team teaching, requesting advice from or posting ideas for colleagues; and using informal opportunities to talk about how students’ learn were the types of responses that were grouped into the theme of collaboration. The theme of Reflection encompassed questions and responses pertaining to self- evaluation as a means to improve teaching practice. Though the process of reflection, one’s own professional learning needs could be identified. Questions and responses 131 pertaining to using various kinds of evidence and feedback to influence change, and to improve teaching practice, were also grouped under this theme. The final theme was Supporting Student Achievement. This theme focused on believing in all students, and creating a positive learning experience for them. The questions and data that centered on encouraging students to enjoy learning and celebrating successes were also grouped under this theme. The other question that was included in this grouping was believing that all students were capable of learning. These four themes were used below to describe how the program experiences influenced the participants’ professional learning. Professional Learning through Using Research to Inform Best Practice. It was clearly evident from the results reported in Table 16 that using my learning to improve students' learning (92%) was reported to be an area of professional learning influenced by course experiences. One student commented, “I have gained tremendously in science content knowledge and how to apply that knowledge to my classroom…The combination of content and pedagogy helps me give my students what they need in class.” Other comments described particular courses that had direct application to the MSSE teachers’ classroom. “The (Field Course) gave the most useful hands-on information that I can use with my students.” A middle school teacher remarked, “ (Course) integrated my prior knowledge into useable and practical laboratory applications for my middle school students.” Further analysis indicated that the MSSE students perceived experimentation with their practice in order to improve learning (83.3%) was an area influenced by the 132 program. Comments from the students indicated that the MSSE program offered opportunities to experiment with their practice in order to find what worked for them. “(Courses) allowed me to examine my practices and learn new ways to help my students,” stated one student. Other comments focused on the overall experience that “helped me expand my teaching techniques and expose me to new practices.” Results reported in Table 16 also indicated that nearly 80% of the respondents ranked using the Internet as one source of useful ideas for improving my practice as an area significantly influenced by the experiences provided by the instructors. A twenty- year veteran teacher remarked, “Doing the Action Research preparation showed me some new ideas and got me using the internet more for research.” Another student said, “I have learned how to use the internet for resources on certain topics and how to improve a specific area of teaching.” Further analysis found that other areas in which the majority of the MSSE students felt course experiences influenced their professional learning was drawing on good practice from other teachers as a means to further my own professional practice (76.5%) and reading research (practitioner reports and journal articles) as one source of useful ideas for improving my practice (73%). Many comments identified the capstone project and the associated research as a learning experience. Others identified a specific course. As one student said, “The campus classes were great. I felt they built my knowledge base and exposed me to current research in various fields.” Others viewed the exposure to research more holistically. “Being forced to use research to support changes 133 in the classroom instead of making changes that I hoped would make a difference,” stated one student. There were others areas in the Using Research to Inform Best Practice aspect of professional learning that were influenced. Further analysis indicated that: understanding how instructional strategies that work in one context might be adapted to other contexts (63.7%) and testing out new ideas in class (81.2%) were all reported as areas that were influenced to some degree by the course experiences made available to students. Another area of professional learning was influenced was modifying my practice in light of published research evidence (55.4%). The mean ratings for the areas of professional learning connected to Using Research to Inform Best Practice that were most influenced by course experiences as perceived by the faculty are shown in Table 17. Table 17 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning Through Using Research to Inform Best Practice Most Influenced by Course Experiences, (N = 369). M 1. Using my learning to improve students' learning. 4.55 10. Experimenting with my practice as a conscious strategy for improving teaching and learning. 4.43 21. Testing out new ideas in class. 4.34 4. Using the Internet as one source of useful ideas for improving my practice 4.24 6. Relating what works in my own practice to research findings and best practices. 4.21 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving my practice. 4.08 8. Understanding how instructional strategies that work in one context might be adapted to other contexts. 3.91 12. Modifying their practice in light of published research evidence. 3.75 Professional Learning through Collaboration. The collaborative practice of learning from other colleagues (88.1%) was perceived as an area of the MSSE students’ 134 professional learning influenced by course experiences. A large number of responses spoke to the influence of “wonderful peer collaborations.” One student clearly articulated that the collaboration influenced his or her professional learning by stating, “Interactions with the professor were all very helpful and collaborations with my peers on WebCT has definitely helped me grow as a science teacher.” WebCT is an online course based management system. Results reported in Table 16 indicated that nearly 66% of the respondents reported program experiences influenced the area of offering other colleagues’ reassurance and support. One student commented that “ongoing communication and support from my colleagues and faculty supervising my Capstone project” fostered his or her professional learning. One individual described the MSSE program as “a forum for motivated educators to give constructive criticism and support for each others' practice.” Results further indicated the MSSE Program fostered professional learning in the area of frequently using informal opportunities to discuss how students learn (60.3%). Comments often connected theses types of experiences with the online discussion groups or conversations in the vans traveling to a field site. These were described as “student- led discussions of successful practices.” Discussing openly with colleagues what and how you the teacher are learning (62.7%) was another form of collaboration that appeared to be influenced by program experiences. “The online component of discussing with the other teachers in each class and ‘hearing’ how they teach has influenced me the most,” affirmed one student. 135 Reading and discussing other teachers’ experiences in the classroom and giving each other feedback (63.5%) and discussing with colleagues how students might be helped to learn how to learn (67.2%) were areas of collaborative professional learning reported in the results that were influenced by the program experiences. “We had in depth (online) dialogue on student learning and how to effectively teach and assess,” declared a student. The mean ratings for the areas of collaborative professional learning that were most influenced by program experiences are shown in Table 18. Table 18 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning Through Collaboration Most Influenced by Course Experiences (N = 369). M 2. Drawing on good practice from other teachers as a means to further my own professional practice. 4.48 24. Offering other colleagues reassurance and support. 3.91 23. Frequently using informal opportunities to discuss how students learn. 3.85 22. Discussing openly with colleagues what and how you the teacher are learning. 3.89 17. Reading/discussing other teachers experiences in the classroom and giving each other feedback. 3.87 28. Discussing with colleagues how students might be helped to learn how to learn. 3.97 Professional Learning through Reflective Practice. The MSSE program experiences fostered reflective practice in multiple ways. Reflecting on my practice as a way of identifying my professional learning needs to better meet the needs of students was an area of professional learning experiences identified by many students (81.6%) as 136 influenced by the MSSE program. One comment described the overall reflective process, as “It was an amazing personal journey!” Another student summed up the experience as “The (courses) have really made me cognizant of my own teaching practices and areas that I need to improve upon.” Another student was more eloquent and stated, “This class made it clear to me that, while I like to learn about and teach science to students, there is much more that I can learn about my own teaching and that there are simple and useful ways to evaluate teaching practices and student learning in your own classroom.” Modifying practice based on feedback and reflection (80.5%) was reported as an area of professional learning influenced by the program. One student stressed, “The most valuable aspect of the courses has been the courses that have forced me to reflect on my own teaching in order to help improve student learning.” Another student asserted, “The MSSE program encouraged me to reflect and think critically about pedagogy for the first time.” One other area found to be influenced by the program experiences was modifying my practice in light of performance and feedback from students (77.1%). “The on- campus, lab-based classes I have taken also required me to reflect on ways I could develop my pedagogy skills…” stated one student. “The readings and discussions that I had to do in my education courses made me reflect deeply about how I teach science,” asserted another student. One student stated that the self-study that was part of his/her capstone project “made me more reflective and aware of my teaching.” The mean ratings for the areas of professional learning through reflection that were most influenced by program experiences are shown in Table 19. 137 Table 19 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning Through Reflection Most Influenced by Course Experiences, (N = 369). M 11. Modifying my practice in light of performance and feedback from students. 4.35 9. Reflecting on my practice as a way of identifying my professional learning needs to better meet the needs of students. 4.31 13. Modifying my practice in light of evidence of self-evaluations of my teaching practice. 4.16 Professional Learning about Supporting Student Achievement. The results reported in Table 16 indicated that almost 77% of the MSSE students ranked encouraging students in my school to enjoy learning as the area of professional learning greatly influence by the experiences offered by the instructors. Many of the MSSE students described their experiences as “positive.” One student felt the experience of having one-on-one time with the instructor was influential. This same student wanted to embed that practice in his or her own class and stated the MSSE program supported him or her with “How to arrange class work so students get individual time to meet with the teacher.” Further analysis revealed that the MSSE students perceived consulting and observing students about how they learn most effectively (73.9%) was an area influenced by program opportunities. Respondents reported that believing that all students are capable of learning (73.6%) was a dimension of professional learning fostered by the MSSE program experience. Many comments identified that learning how to use formative and 138 summative assessments offered a lens through which to view all of the students. “Using assessment to drive instruction helps to differentiate learning experiences for our students,” stated one MSSE teacher. For some students the professional learning was about creating awareness, “I try to be more aware of how different materials and approaches can help more students.” For another student, “The development of the capstone project made me feel empowered to individualize each student's education without compromising the flow for the class.” Further analysis indicated that regularly celebrating student success (60.5%) was also perceived to be an aspect of professional learning influenced by the MSSE program. The mean ratings for the areas of professional learning associated with supporting student achievement that were most influenced by program experiences are shown in Table 20. Table 20 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning about Supporting Student Achievement Most Influenced by Course Experiences, (N = 369). M 26. Encouraging students to enjoy learning. 4.29 5. Consulting and observing students about how they learn most effectively. 4.16 25. Believing that all students are capable of learning. 4.10 27. Regularly celebrating student success. 3.90 Areas of Least Influence. Areas perceived to be “less influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale 139 ratings of 1 (Not at All) and 2. The mean rating for each item was also used in the determination. Within this subset of areas perceived as “less influenced” (see Table 20 below), the mean ratings were greater than 3 with one exception. In general, a relatively small percentage of respondents saw these areas as significantly influenced by program experiences. Yet in some instances the results were mixed, as about the same percentage of students found the area more influenced as those who found it less influenced. Nearly all of the aspects of professional learning that were less influenced by the program involved collaboration with colleagues. Further analysis of the questions showed they referred to collaborative activities that could be challenging for geographically distributed students to undertake. Two other areas of professional learning received ratings of 4 or 5 from slightly more than 50% of respondents, yet the mean rating was less than 3.75. The areas of posting suggestions, ideas or approaches for colleagues to try in class (51.5%) and using insights from my professional learning to feed into my school or organization's policy development (51.5%) were selected as influential by a higher percentage of respondents than the other aspects of professional learning whose mean ratings were less than 3.75. A relatively small proportion of MSSE students perceived that the program experiences had influenced several types of joint collaboration presented in the survey. Areas that were perceived as having been less influenced were engaging in team teaching with someone at my school as a way of improving practice (44.8%), and carrying out joint research/evaluation with one or more colleagues as a way of improving my practice (33.9%). Interesting to note, 31.4% of the MSSE students identified carrying out joint 140 research/evaluation with one or more colleagues as a way of improving my practice as being influenced by the program. Modifying my practice in light of evidence from evaluations of my classroom practice by administrators or other colleagues (28.4%) was an area perceived as having been less influenced, yet 34.7% rated it as an area influenced by the program experiences. Over 30% of the students rated collaborating to plan my teaching as an area less influenced by the course experiences, yet over a third of the respondents felt it was relatively greatly influenced. Turning to colleagues/peers for help if I have a problem with my teaching was an area perceived as slightly less influenced by the program experiences than other areas addressed in the survey. Although 44% found this area to be significantly influenced by program experiences, 24.5% perceived it as less influenced. The mean ratings for the areas of professional learning that were perceived as least influenced by program experiences are shown in Table 20. Table 20 MSSE Student Perceptions of the Types of Knowledge and Practice Relating to Professional Learning Least Influenced by Course Experiences, (N = 369). M 14. Modifying my practice in light of evidence from evaluations of my classroom practice by administrators or other colleagues. 3.54 19. Turning to colleagues/peers for help if I have a problem with my teaching. 3.53 16. Collaborating to plan my teaching. 3.30 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving my practice. 3.07 18. Engaging in team teaching with someone at my school as a way of improving practice. 2.87 141 Content Knowledge and Associated Learning Strategies. Another dimension of professional learning was content knowledge, which was defined as developing an understanding of the “big ideas” as well as the intricate details of a particular science discipline. This dimension also included providing appropriate learning strategies and techniques for understanding science concepts. Table 22 provides the means, standard deviations, and the number and frequency of responses for each survey question in this section. In Section III of the Student MSSE Professional Learning Survey, the questions were assessed using a five-point scale with the value of 1 representing Not at All and a value of 5 representing To a Great Extent on questions 29 through 35. Specifically, the question asked, “To what degree did your experiences in the MSSE program foster your professional learning, including knowledge and practices, in the following areas pertaining to your role as a science teacher?” The N was 369, although in some part of the survey the N dropped as some respondents left a section blank or skipped a question. The Likert responses were ordinal data, so the magnitude of the means allows for comparison between the assessed areas, as they were not non-normal. 142 Table 22 MSSE Student Perceptions of the Influence of Course Experiences on Science Content Understanding and Knowledge, and on Learning Strategies for Furthering Content Understanding. Descriptive Statistics and Frequency of Responses. (N = 369). Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 29. Deepening my own science content knowledge. (N = 347) 4.41 .829 0.9% (3) 3.5% (12) 6.3% (22) 32.3% (112) 57.1% (198) 30. Understanding student thinking in science. (N = 347) 4.02 .863 0.9% (3) 4.3% (15) 18.7% (65) 45.4% (158) 30.7% (107) 31. Using inquiry/investigation-oriented teaching strategies. ((N = 348) 4.08 .945 1.7% (6) 4.6% (16) 17.0% (59) 38.2% (133) 38.5% (134) 32.Using technology in science instruction. (N = 345) 3.74 1.053 3.8% (13) 8.7% (30) 24.1% (83) 37.7% (130) 25.8% (89) 33. Assessing student learning in science. (N = 346) 4.25 .900 1.4% (5) 4.0% (14) 10.4% (36) 36.4% (126) 47.7% (165) 34. Teaching science in a class that includes diverse student populations (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities). (N = 346) 3.19 1.210 10.7% (37) 18.2% (63) 30.3% (105) 24.6% (85) 16.2% (56) 35. Providing a challenging science curriculum for my students. (N = 346) 3.99 .924 2.0% (7) 4.0% (14) 19.7% (68) 42.5% (147) 31.8% (110) Note. The numbers in parenthesis represent the frequency of participant responses. Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, an item with a mean rating of 3.75 or more was placed in the “more influenced” category. A higher mean was used for students than was used with the faculty results as students tended to assign higher ratings than faculty. 143 Over 89.4% of the respondents perceived that the MSSE program greatly influenced the deepening of their science content knowledge through experiences provided in the courses. Nearly all of the students commented on a class, classes, instructors, or experiences that “contributed significantly to my content knowledge.” Respondents reported that the MSSE program experiences also influenced their ability to support their own students’ content learning, for example by fostering skills in assessing students in science (84.1%). Course experiences were reportedly powerful in changing classroom practice pertaining to formative and summative assessment. One student remarked, “We wrote and used CAT's and probes with our students. These tools made an immediate and positive impact on my teaching. I put them right to use and value how they inform me about my students' learning.” Over 75% of the respondents reported the MSSE program experiences influenced the area of understanding their own students’ thinking in science, using inquiry investigative teaching strategies, and providing a challenging curriculum for their students. Using, modeling, and experience with inquiry learning generated numerous responses from the students. As one student stated, “This was among the best courses that I ever took, online, part of the MSSE program or otherwise. It was well structured and truly showed how science inquiry should be taught and carried out”. The mean ratings for the areas of professional learning about science content understanding, and strategies for promoting such understanding, that were perceived by students as most influenced by program experiences are shown in Table 23. 144 Table 23 MSSE Student Perceptions of Areas of Science Content Understanding and Associated Learning Strategies Most Influenced by Course Experiences, (N = 369). M Assessing student learning in science. 4.25 Deepening my own science content knowledge. 4.41 Understanding student thinking in science. 4.02 Using inquiry/investigation-oriented teaching strategies. 4.08 The only area pertaining to science content knowledge, or strategies for promoting content learning, that did not receive a relatively high rating from over 50% of the respondents was teaching science to diverse student populations. Only 40.8% of the respondents felt this area was influenced “To A Great Extent” (rating of 5), or slightly less (rating of 4) through MSSE program experiences. Section IV Student Survey – MSSE Students’ Enhancement and Change in Practice Skills and Knowledge The purpose of Section IV of the survey was to gather information about the MSSE students’ enhancement and change in practice, skills, and knowledge as affected by course type. A one within subjects repeated measures of analysis of variance was used to compare teachers’ mean ratings to determine which types of MSSE courses most influenced their science content knowledge. Science content knowledge was compared across the types of courses offered in the MSSE program: on campus science lab courses, on campus science field course, online science courses, education required core courses, education elective courses, or the capstone project. The means and standard deviations for each type of course are presented in Table 24. 145 Table 24 Comparison of MSSE Students’ Perceptions of Influence on Science Content Knowledge by Course Type, ANOVA comparisons of Means Across Course Types and Science Content Knowledge. Response Scale MSSE Course Types 1 = Not at All 5 = To a Great Extent O n -C a m p u s s c ie n c e la b b a s e d c o u rs e s (n = 1 1 2 ) O n -C a m p u s s c ie n c e fi e ld b a s e d c o u rs e s (n = 1 1 2 ) O n li n e s c ie n c e c o u rs e s (n = 1 1 2 ) E d u c a ti o n R e q u ir e d C o re C o u rs e s (n = 1 1 2 ) E d u c a ti o n E le c ti v e c o u rs e s (n = 1 1 2 ) C a p s to n e P ro je c t (n = 1 1 2 ) R e s u lt s o f 2 -t a il e d P o s t H o c T e s ts (p > .0 5 ) Science Content Knowledge 3.92 (1.164) 4.25 (.973) 4.26 (.857) 3.71 (1.166) 3.69 (1.139) 4.11 (1.085) 2, 3 > 1,4,5 6 > 4,5 Note: Standard deviations are in parenthesis. Mauchly’s test of sphericity was found to be significant for this analysis, therefore adjusted F ratios based on the Greenhouse-Geisser correction were reported to control for Type I error rate. Results from this analysis indicated a significant effect for types of courses F (3.582, 397.56) = 8.757, p < .001. Results from pairwise comparisons revealed that science field courses were found to be a significantly greater influence on teachers’ science content knowledge than the science on campus lab courses, education required courses, and the education elective courses. The mean rating for science field courses did not differ significantly from the online science courses or the capstone project. Further comparisons indicated that the online science courses were found to be a significantly greater influence than the science 146 lab course, the education core course, and the education elective courses. The online science courses did not differ significantly from the science field courses or the capstone project. Additionally, the capstone project was rated as having a significantly greater influence on science content knowledge than the education core courses or the education elective courses. The capstone project did not different significantly from any of the science courses. Accompanying the Likert-style questions was an open-ended question designed to elicit more details and feedback about which types of courses influenced the respondents’ science content knowledge. Specifically, the question asked, “To what degree did the following MSSE program components foster your science content knowledge?” There were 341 responses to this item. The data was organized and coded based on the research questions of this study. The respondents cited a wide variety of program experiences, courses, components, and instructors that contributed to their science content knowledge. Quotes from the survey were included as they were written. Spelling and grammatical errors were left in the quotes that were typed by the participants. Open Ended Survey Responses Regarding Science Content Knowledge. It was apparent from the responses that the MSSE students perceived that the program influenced the science content knowledge. There were 328 responses that identified specific courses and experiences that influenced the science knowledge of the respondents. Respondents specifically named 46 different courses that were a significant influence and every course named was identified more than once. The most frequently named courses were in physics and geology. 147 There were several terms that respondents used frequently to describe the MSSE courses identified as influencing science knowledge. Three students used the term “interdisciplinary” to describe the selection of courses they took that influenced the science knowledge. Two respondents also used the term “integrated” and appeared to use it with a similar connotation to “interdisciplinary”. One student’s response explained, “…the program is an integrated science rather than focused on one area allowed me to take classes from disciplines I need more knowledge in.” The MSSE program was designed to be interdisciplinary across course offerings, and at times within individual courses, and this was an aspect of the program that participants valued. Five MSSE students used the terms “challenging” and “rigorous” to describe the science content. Several responses included specific skills learned as part of the MSSE program such as “essential assessment strategies, questioning techniques, and effective teaching methods” or “inquiry science” or “field observations, experiments, and hands-on learning”. Ten responses indicated the MSSE program “helped me fill in many gaps in my knowledge”. Four students stated that the MSSE program gave them “confidence” to teach science concepts. Overall, the respondents felt the courses provided solid content understanding of concepts that were relevant and “cutting edge.” There were three comments from respondents who that felt that the content did not meet their needs. One person wrote, “The science components could as easily been gleaned from a textbook. On-line science experiences have been no more than that.” Another student mentioned one class that was particularly influential, but summed up the rest of the experience when he or she said, “I feel like almost every other class was not at 148 a master's level, but at an introductory level.” The third person felt the students in a particular course had a better understanding of the science content than the instructor. Open Ended Responses on Science Content Knowledge: Connections to the Pedagogy of Teaching Science. Although the question stem addressed in this section pertained to science content knowledge, it appeared that the teachers experienced the program more holistically. Thus the responses often mentioned how one or more MSSE courses influenced the ability to teach science content. Sixty- four responses included a reference to methods for teaching science content. Thus equating or intermingling content and pedagogy perhaps mirrored the way teachers work. The responses that addressed pedagogical strategies for teaching science often mentioned a cluster of courses rather than a specific course that influenced the MSSE students’ science knowledge. One respondent stated, “…those classes because they not only allowed me to learn more about science, but also gave me strategies for teaching that information and made me think about what I know and how I learned.” Responses also included statements such as the courses “gave me insight into how my teaching can be improved.” Some teachers commented that the pedagogical strategies modeled by faculty during instruction provided insights into how the specific science concepts should be taught. While the majority of the courses mentioned were science courses, twenty-one responses identified the education courses, not including the capstone project, as influential in contributing to the MSSE students’ science content knowledge. Again, the connection between understanding the content and integrating the content within one’s 149 teaching seemed evident. One student described how the science courses improved his or her content knowledge and “the Educational courses have helped me understand how to improve my teaching.” The respondents perceived the strategies presented in the education courses provided an avenue to put the content knowledge into practice. Open Ended Responses on Science Content Knowledge: Connections to Action Research. The capstone project, which is the culmination of the action research component, had a significant influence on science content knowledge as perceived by the MSSE students. Nearly ten percent, 27 of 328 total responses identified the capstone as contributing to science content knowledge. One student stated, “The capstone project helps me put everything together.” Many of the respondents described the capstone experience as an event that pulled together all of the aspects of the MSSE program. The capstone was also a way for the MSSE students to connect the individual learning to the classroom. The application of content and pedagogical content knowledge was what made the experience powerful as perceived by the respondents. The action research component allowed, “reflection on my own teaching in order to help improve student learning.” The actual process of action research from “classroom research- developing surveys, analyzing data, reporting, and applying the results to the classroom,” connected the science to the pedagogy. Open Ended Responses On Science Content Knowledge: Connections to Categories of Professional Learning. Based upon the survey responses presented above, it is evident that one aspect of the MSSE program’s influence 150 on professional learning was contributing to science content knowledge. Over half of the 328 students, 162 in all, identified specific examples of this category of professional learning that occurred in the MSSE program. Many of these responses included the phrase “increased my content knowledge”, yet also referenced the pedagogical influences of the courses. Professional learning was evident through comments that indicated a change in classroom practice. Participants also described the connection between content and pedagogy as the influential professional learning experience. One student remarked, “I really enjoyed those classes [science] because they not only allowed me to learn more about science, but also gave me strategies for teaching that information and made me think about what I know and how I learned.” The metacognitive process described could be connected to critical and responsive learning, a dimension of professional learning. Another professional learning dimension that was noted in 28 of the comments focused on the collaboration with other MSSE students. One student responded, “The online discussions as part of every class creates a learning and sharing environment unmatched even by classroom experiences.” The collaboration with others was perceived to create a powerful professional learning experience by providing the opportunity to discuss content and how to teach that content. The opportunities to ask colleagues questions and discuss ideas allowed for participants to learn from each other. The ability to exchange ideas in this manner was building social capital, a dimension of professional learning. 151 Open Ended Responses on Science Content Knowledge: Perspectives on Distance and Face-to Face Learning. Analysis of the open-ended responses regarding science content knowledge indicated that on-campus science courses were cited slightly more often than online science courses when teachers were asked to give an example of a course or experience that had influenced this aspect of their learning. Nearly one third of the 328 respondents, or 127 in all, identified a field course or another category of on-campus course as influencing science content knowledge, while about one quarter, or 78 respondents, named an online science course. Slightly more than 20% of the respondents, or 64 in all, named both online and on-campus courses as contributing to the science content knowledge. The individuals who selected an on-campus course as influencing science learning often included a rationale with the statement. One student said, “…contact with scientists in the field--absolutely invaluable! Without this component, the MSSE program would lose a valuable component for teachers.” The responses that provided a rationale were passionate about the face-to-face aspect as described by this respondent, “The personal interaction with the extremely knowledgeable profs, the interaction with a wide variety of other students from many diverse backgrounds and interests and the hands-on component of getting outside the four walls of a classroom and getting immersed in the content is astounding!” Others found the personal connections and the time spent with other professionals a strength of the program. Those participants who identified specific online science courses, or online courses in general, as contributing to content knowledge were equally passionate about the experiences, often emphasizing the interaction with peers available in online courses. 152 One student commented, “The online collaborating with other teachers has been huge for me. I have learned a great deal from my peers and grown as a teacher from them.” Other comments mentioned personal growth as a result of online discussions. “Collaboration with other teachers has allowed me to identify my own deficiencies in my knowledge and teaching.” Interestingly, while teachers selecting on campus courses as influential often mentioned the importance of interaction with university scientists, those selecting online courses often highlighted the importance of interaction with peers. Summary. The opened-ended responses confirmed the statistical analysis in many respects. Although there was some variation in the types of courses respondents identified as having influenced the science learning, the field based, online and lab-based science courses were selected most often, and the education courses less often. Somewhat surprisingly, the capstone project, which is part of the action research sequence, also was perceived to influence the science content knowledge of the MSSE students. Some respondents expressed the connection between the inquiry processes used in the capstone, normally to study classroom phenomena, and inquiry processes introduced in the science courses. There was no clear preference between the online and on-campus courses with respect to influence on science learning. A variety of professional learning experiences were described in the open-ended responses. In keeping with the criteria for professional learning, these responses all alluded to a change in teaching practice. Several comments referred to the opportunities within the program for teachers to reflect on practice, a characteristic of critical and responsive learning. The process of reflection sometimes led to a concerted effort to 153 meet the needs of all students. This was an aspect of valuing learning, a dimension of professional learning. Lastly, respondents articulated that collaboration with colleagues or the instructors contributed to science content knowledge. Such collaboration is a type of professional learning referred to as building social capital. Survey Responses Regarding Pedagogy for Teaching Science A one within subjects repeated measures of analysis of variance was used to compare teacher’s mean ratings to determine which type of MSSE course most influenced knowledge of pedagogy for teaching science. Pedagogy for teaching science was compared across the types of courses offered in the MSSE program; on campus science lab courses, on campus science field course, online science courses, education required core courses, education elective courses, or the capstone project. The means and standard deviations for each type of course are presented in Table 25. Table 25 Comparison of MSSE Students’ Perceptions of Influence on Pedagogy for Teaching Science by Course Type, ANOVA Comparisons of Means Across Course Types and Pedagogy for Teaching Science. Response Scale MSSE Course Type 1 = Not at All 5 = To a Great Extent O n -C a m p u s s c ie n c e la b b a s e d c o u rs e s (n = 1 1 2 ) O n -C a m p u s s c ie n c e fi e ld b a s e d c o u rs e s (n = 1 1 2 ) O n li n e s c ie n c e c o u rs e s (n = 1 1 2 ) E d R e q u ir e d C o re C o u rs e s (n = 1 1 2 ) E d E le c ti v e c o u rs e s (n = 1 1 2 ) C a p s to n e P ro je c t (n = 1 1 2 ) R e s u lt s o f 2 -t a il e d P o s t H o c T e s ts (p > .0 5 ) Pedagogy for teaching Science 3.53 (1.156) 3.40 (1.143) 3.81 (.981) 4.08 (1.006) 3.81 (1.034) 4.39 (.771) 4, 6 >1, 2, 3,5 3, 5 > 1, 2 Note: Standard deviations are in parenthesis. 154 Mauchly’s test of sphericity was found to be significant for this analysis, therefore adjusted F ratios based on the Greenhouse-Geisser correction were reported to control for Type I error rate. Results from this analysis indicated a significant effect for types of courses F (3.131, 335.06) = 22.589. p < .001. Results from pairwise comparisons revealed that education core courses and the capstone project were perceived as having a significantly greater influence on pedagogy than all other types of courses offered. The mean ratings for the influence of education core courses did not differ significantly from those for the capstone project. Further analysis indicated that the online science courses and the elective education courses were rated as being significantly more influential that the on campus science lab or field courses. Results further found that the online science courses did not differ significantly from those of the elective education courses. Accompanying the Likert-style question was an open ended questioned designed to elicit more details and feedback about which types of courses influenced the respondents’ understanding of science pedagogy. Specifically, the question asked, “ To what degree did the following MSSE program components foster your knowledge of pedagogy for teaching science?” There were 341 responses to this item. The data was organized and coded based on the research questions of this study. The respondents cited a wide variety of program experiences, courses, components, and instructors that contributed to pedagogical content knowledge in science. Quotes from the survey were included as they were written. Spelling and grammatical errors were left in the quotes that were typed by the participants. 155 Open Ended Responses on Science Pedagogy: Connections to Content Knowledge. Once again it appeared that some MSSE teachers view the program holistically in that the responses did not delineate between content knowledge and pedagogy. Nine of the 341 responses indicated that the content influenced pedagogical understanding or the pedagogy of teaching science supported the understanding of the content. One student described this connection by stating, “The on- campus, lab-based classes I have taken also required me to reflect on ways I could develop my pedagogy skills...” In each of the nine responses, students identified a particular course or variety of courses that were influential on understanding of both science content and pedagogy. Either the program, or perhaps the culture of teaching, appeared to foster this blending of content and pedagogy in that teachers responding to the open-ended item on science pedagogy sometimes referred to content only, not pedagogy. For example, one student said, “The courses have given me greater knowledge of the content area as well as helping me to organize thoughts.” Through discussions with the instructors and other students, the MSSE students had opportunities to talk about and reflect on the experiences in a manner that was meaningful to the understanding of content and pedagogy. Open Ended Responses on Science Pedagogy: Connections to General Pedagogy and Pedagogical Content Knowledge. The MSSE program was perceived to have a strong influence on the pedagogy for teaching science on the students enrolled in the program. Two hundred eighty-eight of the 341 respondents mentioned that the program influenced pedagogy for teaching science. A student 156 summed up the experience by stating, “The transition from teaching science content to teaching for critical thinking and application were the greatest part of the experience.” Over two thirds of the responses, or 210 in all, identified a course or courses that contributed to the understanding of the pedagogy for teaching science. Further examination of the responses indicated that the MSSE students made general pedagogical references. Specific practices and implementation of instructional strategies were not clearly evident. Pedagogical content knowledge is defined as providing appropriate learning strategies and techniques based upon a deep understanding of content (Shulman, 1987). It was difficult to decipher exactly what was shared or learned in the discussions that related specifically to pedagogy or pedagogical content knowledge. Despite the generalities, pedagogical themes emerged from the data. One theme that emerged from the responses was the strong influence that assessment strategies had on the MSSE students’ practice. Over 15% of the responses to this open-ended item, 50 comments in all specifically included a reference to assessment. One student remarked, “Understanding how to assess individual learning in an informal way was an important part of this learning process.” The responses indicated that understanding and using formative and summative assessment strategies had a profound impact on the teachers’ classroom practice. Another common theme in the responses regarding influences on the teachers’ knowledge of science pedagogy, was that the opportunities to collaborate, discuss, and share ideas with other educators influenced the teachers’ pedagogy. “By sharing with other teachers I gained insight into practical ideas that could be implemented in my 157 classroom, and ideas that could be implemented school-wide to improve my practice,” stated one student. While specific strategies were not mentioned, the implication was made that pedagogy was influenced. The shared experience of the course(s) offered common ground in which to discuss pedagogical strategies specific for the content. Ten percent of the respondents, or 38 teachers in all, felt that the social connections and in- depth dialogue provided a powerful mechanism to build pedagogical content knowledge (PCK). A cluster of 15-20 responses added that the ideas and resources shared in the courses were influential on pedagogical understanding. These resources were often a part of a specific course named in the response, yet a small group of responses identified the larger collective of students and instructors sharing resources as a benefit of the course(s). The exchange of ideas and resources at times had direct impact on practice. For example, one student commented that as a result of resource sharing in the program, “My teaching has become more student-focused, rather than content-focused.” Reflective practice was also reported to have influenced the pedagogical understanding of the MSSE students. “I felt that much was learned by reflecting on my teaching strategies and by communicating with other science teachers,” said one MSSE student. The opportunities for reflection created introspection into one’s teaching practice and “fostered our critical thinking and problem solving skills.” Several responses alluded to the reflective process forcing the MSSE students to examine practice in a manner that had not been done prior to participation in MSSE. This reflection “helped me learn about my own teaching.” 158 A small handful of comments indicated there was no influence on science pedagogy in any of the courses. One student stated, “ I did not change the pedagogy of my teaching. Most of the theories and strategies I already used in my teaching.” One veteran educator remarked, “…that after teaching for over 20 years, I don't expect great advances in content or pedagogy knowledge.” Overall, the majority of courses that influenced the MSSE students’ understanding of pedagogy were the education courses. “I am disappointed when the class is entirely content based and there is little to no communication with other educators regarding their classroom practices,” said one student. Not embedding material in the content courses that explicitly addressed how to teach the target content appeared to be a missed opportunity. Open Ended Responses on Science Pedagogy: Connections to Action Research. A strong influence on the pedagogy of teaching science of the MSSE students was the capstone project. One quarter of all of the responses indicated the capstone and the action research process was the MSSE program component or experience that contributed the most to the knowledge of pedagogy for teaching science. As this was the culmination of the MSSE degree and the focus was on the MSSE students’ classroom, it was not surprising that the capstone rated so high. Many of the responses stated that the capstone “pulled it all together for me.” Using best practice and research, the capstone offered the opportunity to examine one’s teaching practice in-depth. “The capstone project allowed for reflection of my teaching practices. I feel like I can better assess the validity of various teaching strategies and 159 lessons in my science classes,” said one student. Another response indicated the capstone “assisted my development as a science educator.” Other responses honed in on the fact that capstone supported the MSSE students’ ability to focus on an area they felt needed improvement. For many respondents the action research was the initiation into a continued practice of evaluating one’s teaching. “I have continued performing action research in my classroom to give me insights into my teaching practice,” said one student. Other students mentioned that the literature review offered insights into teaching practices. Using research to modify or change practice were outcomes that the teachers said resulted from the exposure to the action research process. Open Ended Responses on Science Pedagogy: Connections to Professional Learning. Many of the professional learning experiences focused on generalities of the pedagogy of teaching science aforementioned. The exposure to using assessments was a commonly mentioned learning, as were opportunities to learn from others in discussions online and face-to-face. In addition, reflective practice was cited as a professional learning experience in 20 out of 341 responses, or about five percent of all responses. Several of the responses focused on a new outlook on examining one’s teaching practice. The professional learning experiences empowered the MSSE students to self- reflect, evaluate, and assess teaching practices. “I feel like I can better assess the validity of various teaching strategies and lessons in my science classes. I can make sure that my teaching is (in) agreement with best research practices and is effective for the learning of 160 my students,” remarked one student. These comments tied into critical and responsive learning, a dimension of professional learning. A few responses identified understanding inquiry as the professional learning that occurred in the program. One student commented, “My experience as a student in that course was eye-opening in terms of what good inquiry looks like.” Others stated that the modeling of inquiry opened up a “whole new teaching style I continue to use to this day.” Truly understanding inquiry was a result of participating in the MSSE program, according to these respondents, that can be connected to the professional learning of inquiry. A subset of the MSSE population came to the program with a limited pedagogical base. One student commented, “My undergrad degree was very content based with little information on HOW to teach it.” A small number of responses mentioned that exposure to problem based learning and the 5 E model were deficits in previous education experiences. One student referred to professional learning, that is learning resulting in changes in practice, by stating, “the outstanding education required core courses helped to guide and develop my teaching skills in a more directed (research based) and constructive (constructivist philosophy) manner.” Once again, these comments can be categorized as representing the professional learning dimension of inquiry. Open Ended Responses on Science Pedagogy: Perspectives on Distance and Face-to-Face Learning. Analysis of the 341 open ended responses in this category indicated that over one third identified as influencing respondents’ science teaching were online courses. The online courses mentioned were 161 primarily education courses. Nearly half of the remaining comments, or roughly one- third of all the responses, were not specific enough to clarify if the influence was associated with an online or on campus course. A key theme that emerged from the responses was that the online discussions with others in the courses were highly valued as influencing pedagogy. One student stated, “Discussing with the other teachers in each class and ‘hearing’ how they teach has influenced me the most.” Opportunities to share success and challenges with others were deemed to be a powerful way to learn. The online discussions not only allowed students to share and exchange ideas, but also offered an opportunity to reflect on the conversations. Summary. Once again the opened-ended responses confirmed the statistical analysis; the education core courses and the capstone project were a greater influence on the teachers’ understanding of science pedagogy than all other types of courses, yet interestingly the content courses were cited as contributing in this area too. The capstone project, which is part of the action research sequence, also was perceived to influence the science teaching knowledge of the MSSE students. The statistical analysis indicated the education core courses did not differ significantly from the capstone project in this respect and the open-ended responses confirmed this statistical analysis. The analysis of the open-ended responses identified four recurring themes in the data despite specific comments identifying pedagogical or pedagogical content knowledge. Many MSSE students reported an increased understanding of how to effectively assess student learning. Another theme that appeared often in responses was 162 the value MSSE participants placed on opportunities to collaborate with other teachers. This can be linked to building social capital, a dimension of professional learning identified by the literature. A third theme that emerged was that the resources and ideas that were shared among program participants, including use of the Internet, influenced participants’ science teaching practice. Finally, the informal and formal opportunities to reflect on teaching that occurred in the program also surfaced as an influence on participants’ classroom practice. Survey Responses Regarding Science Curriculum Another cluster of survey items asked teachers the degree to which various types of MSSE courses influenced the knowledge of science curriculum, including how to arrange topics over time. The types of courses offered in the MSSE program whose influences were compared included: on campus science lab courses, on campus science field course, online science courses, education required core courses, education elective courses, and the capstone project. A one within subjects repeated measures of analysis of variance was used to compare teachers’ mean ratings regarding how different types of MSSE courses affected science curricular knowledge. The means and standard deviations for each type of course are presented in Table 26. 163 Table 26 Comparison of MSSE Students’ Perceptions of Influence on Science Curriculum by Course Type, ANOVA Comparisons of Means Across Course Types and Science Curriculum Response Scale MSSE Course Type 1 = Not at All 5 = To a Great Extent O n -C a m p u s s c ie n c e la b b a s e d c o u rs e s (n = 1 1 2 ) O n -C a m p u s s c ie n c e fi e ld b a s e d c o u rs e s (n = 1 1 2 ) O n li n e s c ie n c e c o u rs e s (n = 1 1 2 ) E d u c a ti o n R e q u ir e d C o re C o u rs e s (n = 1 1 2 ) E d u c a ti o n E le c ti v e c o u rs e s (n = 1 1 2 ) C a p s to n e P ro je c t (n = 1 1 2 ) R e s u lt s o f 2 -t a il e d P o s t H o c T e s ts (p > .0 5 ) Science Curriculum 3.30 (1.237) 3.14 (1.233) 3.64 (1.171) 3.57 (1.169) 3.43 (1.161) 3.67 (1.193) 5 > 2 3,4,6 > 1,2,5 Note: Standard deviations are in parenthesis. Mauchly’s test of sphericity was found to be significant for this analysis, therefore adjusted F ratios based on the Greenhouse-Geisser correction were reported to control for Type I error rate. Results from this analysis indicated a significant effect for types of courses F (3.425, 342.522) = 8.156, p < .001. Results from pairwise comparisons revealed that on-campus science field courses were significantly less of an influence on science curricular knowledge than either the education elective courses or the capstone project. Comparison of mean ratings indicated that there were no significant differences between the education elective courses and the other types of courses. Further analysis revealed that the online science courses, the education required courses, and the capstone project had a significantly greater influence 164 on science curricular knowledge than the on-campus science lab courses, the on-campus science field courses, and the elective education courses. Accompanying the Likert-style question was an open ended questioned designed to elicit more details and feedback about which types of courses influenced the respondents’ knowledge of science curriculum, including how to use resources and arrange topics over time. Specifically, the question asked, “ To what degree did the following MSSE program components foster your knowledge of science curriculum, including how to use resources and arrange topics over time?” There were 338 responses to this item. The data was organized and coded based on the research questions of this study. The respondents cited a wide variety of program experiences, courses, components, and instructors that contributed to knowledge of science curriculum. Quotes from the survey were included as they were written. Spelling and grammatical errors were left in the quotes that were typed by the participants. Open Ended Responses on Curriculum: An Overview. It was clearly evident from the responses that influencing the MSSE students’ knowledge of science curriculum, including how to use resources and arrange topics over time was part of the program. The National Research Council defines curriculum as “the way the content is delivered. It includes the structure, organization, balance and presentation of the content in the classroom” (NRES, 1996, p. 22). And although 234 responses indicated the program influenced the participants’ knowledge of science curriculum, just a few of the possible aspects of curriculum were mentioned in the comments. Furthermore improving one’s knowledge of curriculum seemed to be more of a serendipitous event, rather than a 165 planned outcome of the program based on an analysis of the comments. Over one quarter of the responses, or 84 in all, explicitly stated there was no influence, marked their response as “N/A”, or used their response to go off topic. One theme apparent in the responses was the resources that were part of each of the courses were useful in that teachers could integrate these into their own science curriculum. The majority of the responses (234 responses) indicated that the MSSE program components or experience were influential, yet the comments were lacking substance and evidence into what the influence was. Some individuals remarked that particular courses or the program as a whole comprised “a great source of resources” or that courses “provided numerous resources and have helped me focus my curriculum for this short class.” Another student commented, “I have more resources online than I did before which is a huge impact to my instruction.” It seemed that the exposure to new resources was influential in general. Of those responses that indicated the MSSE program influenced the understanding of science curriculum, 74 responses indicated that the online discussions were the vehicle for this influence. “They (courses) have provided an avenue of thoughtful discussion about how to bring relevance to the science that we are teaching, to the students' experiences with the world around them. Other educators have shared great ideas and have been an incredible sounding board,” stated one student. Online discussions of course content were perhaps the major context in which program participants addressed science curriculum. Yet, based on the NSRC definition of curriculum it was an area that seemed to be only superficially addressed in the MSSE 166 program. An elective science course that focused on the history of science curriculum was one course that was explicitly identified in 17 of the open-ended responses pertaining to curriculum. “ I was not familiar with the national standards nor the studies that had been done about American students in science, “was the response by one student who identified the curriculum course. Another student felt it “helped in revising my own science curricula.” Many of the other responses that identified the curriculum course only mentioned the course without any information explaining it’s influence, which may have been substantial, on teachers’ curricular knowledge. While a majority of the respondents felt that science curriculum was part of the MSSE program, there was a large number who did not agree. One respondent stated, “I cannot name one component that specifically contributed the most to my knowledge of science curriculum...I feel the course work informed my knowledge of resources, but there was little instruction on how to arrange topics over time.” While most respondents mentioned resources as a “take away” from the courses, several commented they did not feel those resources, and how to integrate these within a curriculum, were strongly promoted. One respondent said, “I don't really think that any of the courses helped me to know how to design a curriculum or arrange topics over time. There were a few resources that we used in some science content classes and in the education courses, but that was about it.” One individual wished curriculum design were a component when he or she stated, “I wished some of my classes did address the curriculum more and show how to use the information we were learning in the class and apply it in the classroom.” 167 The other spectrum of comments came from those individuals who felt the MSSE program should not be about designing curriculum. One comment reflected this succinctly, when he or she stated, “Fortunately, there wasn't a lot of time wasted on this stuff.” Two comments indicated that curriculum would be a difficult topic for MSSE since it was determined at the state level. “Different states have different requirements for pacing and content so I don't feel that MSSE program really addressed that and I don't think that it should. Our school districts handle that, I (came) to MSSE for science knowledge and teaching practices.” In part because the MSSE program serves students from across the U.S. and a number of foreign countries, some students assumed that science curricular knowledge would not be a particular focus of the program. Open Ended Responses on Curriculum: Connections to Action Research. Further analysis of the responses included 30 students who identified the capstone project as the component that influenced the knowledge of science curriculum. As with the other responses previously mentioned, the rationale was vague or missing completely in most of the responses. One student clarified why the capstone was an influence and stated, “The work on the Capstone Project helped me most with recognizing the need for in-depth planning for each topic or unit including the use of pre- and post teaching evaluations.” As with many of the responses, it seemed to be challenging for the respondents to clearly articulate how a component of the program was an influence. One student who identified the capstone stated, “My capstone project directly involved arranging topics over time in a slightly different way and using household resources to perform experiments.” While the student 168 answered the question, there was limited substance to the response. Open Ended Responses on Curriculum: Connections to Professional Learning. When considering course or program influences on the knowledge of science curriculum, many respondents focused on the exposure to a wide variety of resources and opportunities for discussions with colleagues as pathways for professional learning. There were no clear themes that emerged from the other responses. There was mention of professional learning experiences, but the responses were individual and not connected to the other teachers’ ideas. One student commented, “Many of the teachers I have come across may have benefited more if they came into education as a second career,” indicating that curricular learning may have been greater for MSSE participants who were not already experienced educators. Another comment focused on an experience in a course by stating, “extensive conceptual mapping was used to find how's one view on connected ideas changed through the course.” This response seemed to indicate that the concept map supported the understanding of curriculum in this course, but did not provide any supporting evidence that there was long reaching influence. It was interesting to note that there were ten responses that did not respond to the prompt provided. Two responses offered criticisms - one of a particular MSSE course and one commented, “what about negative experiences and classes that were a waste of time???? this survey is very biased.” Off topic responses included comments such as, “Found out that online courses are much harder than classes attended in person. I hated the fake conversations. I would encourage others to get their degree in a more traditional 169 way.” Open Ended Responses on Curriculum: Perspectives on Distance and Face-to-Face Learning. Analysis of the responses regarding science curriculum revealed that many described a course or experience that influenced the teachers’ understanding of science content. While some specific courses were mentioned, often the courses were referred to collectively as “online classes” or “required courses. Of those responses that mentioned specific courses, 74 identified courses that were offered online, and most of those were education courses. Twenty courses were identified as on-campus field or lab courses. Once again, many of the responses only stated a course name without any supporting details. One student who felt the field courses influenced his or her curriculum knowledge commented, “It is the only way to be consumed with a topic and really understand the material well. I am a hands on learner so this works best for me.” It seemed apparent that students had a difficult time articulating a clear response to this question. Summary. The open-ended responses supported the statistical analysis. While the responses were broad and often vague, the education courses and the capstone courses were mentioned frequently. In addition, the statistical analysis indicated the online courses were more of an influence on the teachers’ knowledge of science curriculum. This was supported by the open-ended responses. In examining themes in the data, there seemed to be three components that influenced science curriculum. The exposure to resources in the courses was deemed to 170 be influential. In addition, many students perceived the online discussions contributed to more in-depth understanding. Those students who took the elective curriculum course stated specifically how the course influenced practice. Nearly 25% of the 338 respondents to this item indicated the MSSE program did not contribute to their knowledge of science curriculum. The range of responses included those students who felt that a focus on science curriculum had no place in the MSSE program to those who felt it was an area of deficiency. Survey Responses Regarding Diverse Learners A one within subjects repeated measures of analysis of variance was used to compare teachers’ mean ratings to determine which type of MSSE course influenced the knowledge of how to support diverse learners. Influence on knowledge of supporting diverse learners was compared across the types of courses offered in the MSSE program: on campus science lab courses, on campus science field course, online science courses, education required core courses, education elective courses, or the capstone project. The means and standard deviations for each type of course are presented in Table 27. 171 Table 27 Comparison of MSSE Students’ Perceptions of Influence on Teaching Diverse Learners by Course Type, ANOVA Comparisons of Means Across Course Types and Diverse Learners. Response Scale MSSE Course Types 1 = Not at All 5 = To a Great Extent O n -C a m p u s s c ie n c e la b b a s e d c o u rs e s (n = 1 1 2 ) O n -C a m p u s s c ie n c e fi e ld b a s e d c o u rs e s (n = 1 1 2 ) O n li n e s c ie n c e c o u rs e s (n = 1 1 2 ) E d u c a ti o n R e q u ir e d C o re C o u rs e s (n = 1 1 2 ) E d u c a ti o n E le c ti v e c o u rs e s (n = 1 1 2 ) C a p s to n e P ro je c t (n = 1 1 2 ) R e s u lt s o f 2 -t a il e d P o s t H o c T e s ts (p > .0 5 ) Diverse learners 2.5 (1.202) 2.41 (1.107) 2.66 (1.134) 3.24 (1.237) 3.02 (1.135) 3.42 (1.277) 1 > 2 3 > 1, 2 4 > 1, 2, 3, 5 5 > 1, 2, 3 6 > 1, 2, 3, 4, 5 Note: Standard deviations are in parenthesis. Mauchly’s test of sphericity was found to be significant for this analysis, therefore adjusted F ratios based on the Greenhouse-Geisser correction were reported to control for Type I error rate. Results from this analysis indicated a significant effect for types of courses F (3.009, 312.905) = 36.163, p < .001. Results from pairwise comparisons revealed that all types of courses were significantly less of an influence on understanding the needs of diverse students than the capstone project. Further comparisons indicated that the education courses and the education elective courses were a significantly greater influence than any of the science 172 courses. Yet the education core courses were a significantly greater influence than the education elective courses. Further analysis revealed the on-campus science courses were significantly less of an influence that the online science courses. Finally, the on- campus science lab courses were a significantly greater influence than the science field courses. Accompanying the Likert-style question was an open ended questioned designed to elicit more details and feedback about which types of courses influenced the respondents’ knowledge of how to support diverse learners. Specifically, the question asked, “To what degree did the following MSSE program components foster your knowledge of how to support diverse learners (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities)?” There were 338 responses to this item. The data was organized and coded based on the research questions of this study. The respondents cited a wide variety of program experiences, courses, components, and instructors that contributed to the knowledge of how to support diverse learners. Quotes from the survey were included as they were written. Spelling and grammatical errors were left in the quotes that were typed by the participants. Open Ended Responses on Supporting Diverse Learners: An Overview. Analysis of the open-ended responses indicated that this was an area of the MSSE that was by far the weakest of those examined in the survey. One third of the responses indicated unequivocally that this was not an area emphasized in the MSSE program. Further analysis indicated that those who felt it was addressed did not identify very many courses that included it as part of the design. Even 173 more revealing was that only three science courses were identified as contributed to participants’ knowledge of diverse learners. Similar to the results in the previous section, one theme that emerged from the data was the perceived significance that the online discussions had on this aspect of participants’ learning. One student remarked, “Sharing ideas with the other teachers about how they might address these students' needs has helped the most.” Even those courses that were reported to have an influence on participants’ knowledge of supporting diverse learners were often mentioned with reservation, for example, by saying that a course, “at least touched on the subject.” This indicated that this component was not an obvious part of the course. About one third of the comments, 100 out of 338, indicated that any reference to supporting diverse learners was not an overt part of the course. Most of the ideas to support diverse learners were generated in the discussions with other teachers. Many commented that this was a missing component that needed to be addressed by the program. One individual reflected, “This was the biggest gap in my educational experience at MSU-Bozeman.” One respondent noted, “I teach in a very diversified school with a large variety of languages, races, nationalities, etc. and truthfully I felt that I had more to offer than I learned from others.” Another respondent remarked that the knowledge was attributed to other experiences and stated, “ I also have recently received a degree in elementary education. Most of the knowledge I have about #4 comes from this degree; I've hardly learned anything new from the MSSE program.” Another respondent mentioned that, “Diverse learners were addressed for behavior fairly often, 174 but rarely for academic suggestions.” The comments clearly indicated that meeting the academic needs of a diverse student population was a concern and a void in the MSSE program. Four respondents commented they felt that this was not an area the MSSE program needed to address. One respondent stated: I cannot recall anything in the program that dealt specifically with this issue. That is not really a bad thing. What I did learn were sound science teaching practices. A good teacher will then adapt those things to the students they have in front of them. I don't put the labels indicated on this question on my students. Another respondent mirrored this comment and said, “Again, all the courses have done a great job of provided jumping points for all of us ...it really is up to us to know our teaching situations and accommodate the programs for our students. Best the courses can do is to provide lots of ideas, knowledge ... and we, as students, go with it.” The claim that addressing diverse students’ academic needs was not a needed part of the MSSE program represented only a small fraction of the comments received. Another viewpoint that emerged from the survey was that a small subset of students, 3 of 338, felt personal diverse learning needs were not being met. One individual stated, “I don't feel that any of the professors differentiated in their classes.” Another individual described it from a lack of access when he or she stated, “From personal experience...You cannot take any field courses if you are handicap. This could become a problem in the future.” Again, these sentiments represented only a few responses. 175 Open Ended Responses on Supporting Diverse Students: Connections to Action Research. Although the MSSE program was not perceived to be a strong influence on knowledge of how to support diverse learners, those who found it to be an influence often reported that the capstone project was the experience that was influential. Sixty-one respondents identified the capstone project as the pivotal experience that provided insight into supporting diverse learners. The responses that identified the capstone project as influential were very individualized and clarified how the capstone project incorporated addressing the needs of diverse learners. “ I had to make sure that the interventions of the research study were accessible to all student areas and then adjust my teaching to make sure that underrepresented students were given an equal opportunity for learning,” replied one student. Another student commented about including access to the material for diverse learners through the capstone project, “One of my goals was to include all achievement levels of students.” The MSSE students selected the focus of the capstone projects independently, and those who chose to study support for diverse learners approached this from many directions. Open Ended Responses on Supporting Diverse Students: Connections to Professional Learning. The category of professional learning that was evident in the responses was critical and responsive learning, and the responses were very individualized regarding this aspect of professional learning. The responses reflecting the perspective that meeting the needs of diverse learners was part of the MSSE program, generally mentioned the capstone project if they provided details. This group also tended to state that meeting the needs of diverse learners was addressed 176 within MSSE in online discussions with others. Those individuals who made it a focus of the coursework or capstone articulated evidence of professional learning. One student commented, “Because the courses allowed me to focus on my own students, who are very diverse, I got a lot out of them in that area.” Awareness of supporting diverse learners may not be a focus of some MSSE students. “I've found that most of the ed courses reflect the classes you're teaching... so if you don't teach a diverse population, you don't touch on diversity,” said one student. Most science classrooms include students from diverse culture, language and income groups, as well as students with different engagement or performance levels and learning styles. However, some respondents viewed their own students as homogeneous, and therefore there was no perceived need for MSSE to focus further on supporting diverse learners. Differentiation was mentioned in four responses as an aspect of the program that began to address diverse learners. “In several classes we have discussed differentiated instruction and reaching a diverse range of abilities, but we focus very little on supporting learners with disabilities, etc.,” commented one student. It seemed apparent from the responses that while differentiation may be a topic that was broached, it was not extensive and inclusive of all students. Two comments from teachers who work with Native American students brought another perspective on professional learning. Both responses indicated the program had increased the awareness and understanding of the abilities of the students in their classes. One person said, “The MSSE program has helped me to understand that all children are 177 capable of learning no matter what their circumstances. There has been a lot of growth in my classes these past two years.” One of the responses was from a Native American teacher who felt the MSSE program made him/her feel included. “As a Native American, I was thoroughly impressed by the openness of the instructors and their desire to help me feel more valued and included - they also taught me how to help my students feel the same way.” While the majority of the comments regarding student performance differences alluded to students who were low performing, diverse students might also be high performing. Gifted students are also considered diverse learners. One respondent addressed these learners by stating, “Encountering challenging problems for high- achieving students. How to arrange class work so students get individual time to meet with the teacher.” It may be inferred that this MSSE student felt the program enabled him or her to meet the needs of high achieving students, although it was not clearly articulated. As many teachers in the program teach AP level courses, it may be a missed opportunity for the program if the needs of all diverse learners are not addressed. Open Ended Comments on Supporting Diverse Students: Perspectives on Distance and Face-to-Face Learning. Further analysis of the comments indicated only a small number of responses identified specific courses beyond the capstone that were perceived to be an influence in teaching diverse students. Four individual courses were mentioned in science. The rest of the courses mentioned, 32 in all, were broad groupings such as “ed core” or “online”. Of the total courses that were mentioned, 36 of 338, only two were on-campus 178 and the remaining thirty-four were online courses. Nearly twenty percent of the responses, 61 of 338, indicated a perceived influence identified the component or experience was the capstone project that typically involved action research. As the discussions with other students was a program component frequently cited as influencing knowledge of teaching diverse learners, it made sense that the online courses were more frequently mentioned. Summary. Again, the open-ended responses supported the findings of the statistical analysis. The capstone was determined to be a significant influence on understanding the needs of diverse learners compared to all other types of courses or program experiences. In addition, the education core courses, both required and elective, were reported as a more significant influence than the science courses. It should be noted that only four individual science courses were reported to have any influence on participants’ knowledge of supporting diverse students. This again, seemed to be a missed opportunity for the program. Addressing the needs of diverse learners does not seem to be a strong part of the MSSE program based on the open-ended responses. Those that felt it was part of the program credited the discussions with other students to be the component that was influential. A number of respondents felt this was a deficit of the program. Other comments reflected a different perspective. These comments ranged from those students who felt it was not a needed part of the program to those students who felt the program gave them greater awareness of the students taught. Strategies for differentiation were reported in four responses and another comment described how the 179 participant felt that the instructors do not differentiate as they teach MSSE students. Summary of Student Surveys The MSSE Student Professional Learning Survey results indicated that the different experiences in the MSSE program have influenced the professional learning of the MSSE students. Each of the four sections of the Student MPLS focused on a particular aspect of professional learning as well as providing insights into other influential aspects of the program. In Section II of the survey, the focus of the questions was which areas did the program experiences foster scientific knowledge and practice. Not surprisingly, the MSSE students reported that the area that was cultivated through program experiences was the ability to articulate scientific ideas in a discussion. Reflection on scientific ideas was perceived to be an area of great influence by the program experiences. Practicing and engaging in science process skills was also an area influenced by the program opportunities offered by the instructors. These skills included analyzing data and drawing conclusions from data and observations, and providing evidence for ideas. Using the course content in the MSSE students’ context was also a practice that was encouraged through the program experiences. Section III of the Student MPLS focused on aspects of professional learning. The data was analyzed and organized based on the research questions of this study. The data from the descriptive statistics appeared to be clustered into four groupings based on questions and responses. The groupings were focused on similar aspects of teaching and learning. Those themes included Using Research to Inform Best Practice, Collaboration, 180 Reflective Practice, and Supporting Student Achievement. Aspects of using research to inform best practice that were found to more influenced included using the MSSE students’ learning experiences to influence the learning of their students’ learning. Experimenting with teaching practices as well as using information from the Internet and research were area perceived to be significantly influenced by program experiences. An aspect of professional learning that was perceived to be influenced by the program was collaboration with other MSSE students. Throughout the survey and open- ended responses, the MSSE students consistently described the multiple opportunities for collaboration as an influential part of the MSSE program experiences. In addition, including not only self-evaluation and self-analysis but also classroom modifications, was also an area that was highly influenced through course experiences. The feedback and collaboration with others in the program appeared to enhance and encourage reflective practice. Lastly, the survey results indicated that the MSSE program experiences fostered teachers’ knowledge and practice in the area of supporting student achievement, including encouraging students’ enjoyment of the learning experience, believing that all students are capable of learning, and regularly celebrating student success. Furthermore, assessment strategies introduced in the MSSE program experiences were reported to be a strong influence on building teachers’ understanding in the area of supporting and meeting the needs of all learners. The survey results illustrated that the MSSE program appears to deepen science content knowledge. MSSE students also felt empowered to teach challenging curricula. 181 One area that was rated as being less influenced by the program experiences was teaching science to a diverse student population. The MSSE program experiences did not seem to foster the area of teaching science to a diverse student population as was reiterated in the results in Section IV of the survey. Section IV of the survey focused on determining which type of MSSE courses most influenced science content knowledge, pedagogy of teaching science, curriculum development and meeting the needs of diverse learners. Not surprisingly the science courses fostered science content knowledge more than the education courses or the capstone course. The education courses were more influential than the science courses in the pedagogy for teaching science. Of the types of science courses, the on-campus science courses were significantly more of an influence on pedagogy than the online or field courses. Science curricular knowledge was influenced more by the science lab courses than the other types of science courses, yet seemed to be more of a serendipitous event than a planned outcome in the MSSE program. Learning how to support diverse learners was influenced more by the education courses than the science courses although this was not perceived to be a strong area of influence across the MSSE program. MSSE Faculty Survey This section describes the results of the analysis of the MSSE Faculty Professional Learning Survey, which can be found in Appendix D. The survey was designed to assess the ways in which the MSSE program influenced the science content 182 knowledge and professional learning of the participants from the faculty perspective. Specifically this survey was designed to answer the following research questions: How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? Section I Faculty Survey - Professional Experience and Education The purpose of Section I of the survey was to gather background and contextual information on the participants. Seventy five percent, or 24 out of 33, of the MSSE faculty completed the survey. The MSSE faculty is quite experienced, as over 54% have served in a faculty role within the MSSE program for more than five years. One third of the faculty had between one and three years of experience teaching for the MMSE program. Of the faculty respondents, 37.5% reported teaching on-campus science field based courses or online science courses. Slightly less than a third of the faculty respondents taught only on-campus science courses. The education faculty comprised 21% of those responding. Over one third of all the respondents advised the capstone project. More than half (58.3%) were adjunct faculty with the remainder being tenured faculty at MSU. All the faculty members in chemistry, earth science, and education have 183 attained doctoral degrees. Nearly 80% of the biology faculty holds a PhD. While the vast majority of faculty were based at MSU, other faculty members were located at University of Great Falls; Casper College; Louisiana School for Math, Science, and the Arts/Louisiana Virtual School; LigoCyte Pharmaceuticals, Inc.; and the University of Wisconsin – Oshkosh. The departments represented by the faculty respondents included Chemistry and Biochemistry; Education; Earth Science; Physics; Ecology; Mathematics: Land Resources; and Cell Biology and Neuroscience. Precision of Responses and MPLS Instrument Reliability A 95% confidence interval was calculated based on the percent of respondents returning useable questionnaires (surveys) for analysis. Twenty-four of the 32 MSSE faculty returned completed surveys yielding a 75% response rate. The 95% confidence interval for a 75% response rate and a population of 32 is +9. A 95% confidence interval indicates that a true percentage of the population would select the same response within a range of + 9 percent. For example, if 65% of the sample “Agreed” with a certain statement on the MPLS, it would be expected that between 56 and 74 percent (65% + 9%) of the population to also indicate that they “Agreed” with the statement. The large confidence interval indicates a much more variable confidence level than that found for the student survey. A sample size of 24 for a population of 32 total faculty indicates that the true population percentage would fall within the 95% confidence interval. In other words, if 65% of the faculty sample “Agreed” with an faculty survey statement, there is a 99% confidence the true percentage of the faculty population “Agreeing” with the statement would fall within the 56 – 74 percent confidence interval. The preciseness of 184 responses from the faculty survey is much less than that of the student survey due to the small population size of MSSE faculty. The MPLS was found to be highly consistent for assessing the professional development constructs that the questionnaire items were written to assess. The Cronbach’s Alpha for the each section of the MPLS was high indicating high internal consistency reliability. The overall internal consistency reliability represented by Cronbach’s Alpha for the Faculty MPLS was .91. The Cronbach’s Alphas for each section of the Faculty MPLS were as follows: Content Understanding and Knowledge = .697, Professional Learning = .790, and Content Knowledge = .875. Section II Faculty Survey- MSSE Students’ Science Content Understanding and Knowledge The purpose of Section II of the survey was to gather information about the MSSE Students’ science content understanding and knowledge from a faculty perspective. Table 28 provides the means, standard deviations, and the number and distribution of responses from the faculty for this set of questions. In Section II of the Faculty MPLS, the items were assessed by respondents using a five-point scale ranging from 1 = “Not at All” and 5 = “To a Great Extent.” Specifically, the question asked, “To what degree do the course experiences you provide foster the MSSE teachers' professional learning, including knowledge and practices, in the follow areas?” The N was 24 for this section of the survey, although in some parts of the survey the N dropped to 22 as respondents left a section blank or skipped a question. The Likert responses were ordinal data, so the magnitude of the means allowed for comparison between the 185 assessed areas. For analysis purposes, the data in Section II was combined so that the categories of 4 and 5 represented a “great influence,” while categories 1 and 2 were combined to represent “none or little influence.” Table 28 MSSE Faculty Perceptions of the Influence of Course Experiences on MSSE Students’ Knowledge and Practices in the Following Areas Relating to Science Content Descriptive Statistics and Frequency of Responses, (N = 24). Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 1. Working on problems based in contexts from their daily life 3.41 1.436 18% (4) 4.5% (1) 22.7% (5) 27.3% (6) 27.3% (6) 2. Analyzing and drawing conclusions from data, observations, and other forms of scientific evidence. 3.91 1.151 4.5% (1) 9.1% (2) 13.6% (3) 36.4% (8) 36.4% (8) 3. Providing evidence to support their scientific ideas. 3.82 1.053 4.5% (1) 4.5% (1) 22.7% (5) 40.9% (9) 27.3% (6) 4. Participating in on-site fieldwork (e.g., water testing, species counting in a natural setting, astronomical observations of the night sky). 2.73 1.778 45.5% (10) 4.5% (1) 9.1% (2) 13.6% (3) 27.3% (6) 5. Working with pen and paper problem sets(e.g., problems from a textbook or worksheet). 2.86 1.583 31.8% (7) 13.6% (3) 9.1% (2) 27.3% (6) 18% (4) 6. Interacting with physical materials or models (e.g., mixing solutions, building circuits, scale models. 2.73 1.579 36.4% (8) 9.1% (2) 18% (4) 18% (4) 18% (4) 7. Using computer-based animations, games, or simulations(e.g., virtual dissection, SimEarth, flash interactives). 2.32 1.427 45.5% (10) 9.1% (2) 22.7% (5) 13.6% (3) 9.1% (2) 8. Articulating their scientific ideas in a discussion. 4.36 .953 4.5% (1) 0% (0) 4.5% (1) 36.4% (8) 54.5% (12) 186 Table 28 - Continued 9. Reflecting upon their earlier scientific ideas. 3.64 1.049 4.5% (1) 9.1% (2) 22.7% (5) 45.5% (10) 18% (4) 10. Reflecting upon the scientific ideas of other students. 3.77 1.066 4.5% (1) 9.1% (2) 13.6% (3) 50% (11) 22.7% (5) 11.Carrying out procedures of scientific investigations designed by instructors or course developers (e.g., lab exercises, kitchen experiments). 3.05 1.647 31.8% (7) 4.5% (1) 18% (4) 18% (4) 27.3% (6) 12. Designing their own scientific investigation(s) (e.g., developed hypothesis or question and procedure. 2.86 1.490 31.8% (7) 4.5% (1) 22.7% (5) 27.3% (6) 13.6% (3) 13. Carrying out procedures of scientific investigations they designed (e.g., collected data, made observations. 2.82 1.468 22.7% (5) 27.3% (6) 13.6% (3) 18% (4) 18% (4) Note. The numbers in parenthesis represent the frequency of participant responses. Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, an item with a mean rating of 3 or more was placed in the “more influenced” category. A lower mean rating was used with the faculty results as the faculty tended to assign lower ratings than the students. This may be due to the faculty self-assessing course experiences they provided with a certain level of modesty. Or as one faculty member suggested, the difference may be due to the fact that many of the respondents are research scientists, i.e. members of a profession that thrives on criticism and skepticism. It was evident from the results reported in Table 38 that more than 90% of the faculty perceived that the course experiences they provided influenced the MSSE students’ ability to articulate scientific ideas to a relatively great extent, which we can assume signifies an increase in students’ science content knowledge and understanding. 187 Further analysis indicated over 70% of the faculty reported that experiences in their courses influenced students’ ability to analyze and draw conclusions from data, observations, and other forms of scientific evidence, and to reflect upon scientific ideas, to a relatively great extent. In their open-ended survey responses, a large number of students described the capstone project as a mechanism through which their exposure to scientific research was put into action, but they did not refer as often to other course experiences where this occurred. “It helped with true data analysis and drawing conclusions based upon my own research,” commented one student. Another student made the connection between the capstone project and scientific research stating, “I feel that the capstone project contributed the most, understanding on a large scale, how important research and its analysis is to all science.” These student comments support faculty perceptions that the MSSE program influenced students’ understanding of scientific research, but not necessarily that this occurred in a broad swathe of courses. Other areas of note included providing evidence to support scientific ideas, as an area in which 68% of the faculty felt the course experiences they provided influenced MSSE participants to a fairly great extent, and reflecting upon one’s own earlier scientific ideas or those of other students, areas in which 63% and 73% of the faculty, respectively, felt their course experiences had positively influenced the MSSE students. The mean ratings of areas that faculty perceived as being most influenced by the course experiences they provided for MSSE students are shown in Table 29. 188 Table 29 MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Science Content Most Influenced by Course Experiences. M Articulated their scientific ideas in a discussion. 4.36 Analyzed and drew conclusions from data, observations, and other forms of scientific evidence. 3.91 Provided evidence to support their scientific ideas. 3.82 Reflected upon the scientific ideas of other students. 3.77 Reflected upon their earlier scientific ideas. 3.44 Areas perceived to be “less influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of 1 (not at all) and 2. The mean rating for each item was also used in the determination. Generally, an item with a mean rating of less than 3 was categorized as “less influenced”. There were four areas of fostering scientific content knowledge and practices that were ranked by the faculty as less influenced by the course experiences they offered. Over half of the faculty reported that the course experiences they provided did not affect MSSE students’ understanding of on-site fieldwork (e.g., water testing, species counting in a natural setting, astronomical observations of the night sky) to a significant extent. In addition, providing opportunities to interact with physical materials or models (45.5%) or using computer-based animations, games, or simulations (54.6%) was not included as part of the course design to a marked extent. Finally, over fifty percent of the faculty responded that experiences provided in their course did not influence MSSE students’ ability to carry out procedures of scientific investigations the student designed (e.g., collecting data, making observations, etc.) to a particularly great extent. The mean 189 ratings of areas pertaining to students’ science content understanding that faculty perceived as being least influenced by experiences in their courses are shown in Table 30. Table 30 MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Science Content Least Influenced by Course Experience, (N = 24). M Used computer-based animations, games, or simulations (e.g., virtual dissection, SimEarth, flash interactives). 2.32 Interacted with physical materials or models (e.g., mixing solutions, building circuits, scale models. 2.73 Participated in on-site fieldwork (e.g., water testing, species counting in a natural setting, astronomical observations of the night sky). 2.73 Worked with pen and paper problem sets (e.g., problems from a textbook or worksheet). 2.86 It should be noted that both the science faculty and education faculty responded to the same set of questions. Some MSSE courses focus heavily on science subject matter, others on how to teach science, and still others use a blended approach. Section III Faculty Survey-MSSE Students' Professional Learning The purpose of this section of the survey was to gather information about the MSSE students’ professional learning from a faculty perspective. In Section III of the Faculty MPLS, the questions were assessed using a five-point scale with 1 = “Not at All” and 5 = “To a Great Extent.” Specifically, the questions asked, “To what degree do the course experiences you provide foster the MSSE teachers’ professional learning, including knowledge and practices, in the following areas?” Once again, some questions have an N of less than 24 as some respondents left a section blank or skipped a question. 190 The Likert responses were ordinal data, so the magnitude of the means allowed for comparison between the assessed areas. Table 31 provides the means, standard deviations, and frequency of responses from the faculty for the questions that related to professional learning. In addition, comments from the open-ended responses from the Student MPLS that aligned with or vary from the faculty positions revealed by the statistical analysis were included when appropriate. Table 31 MSSE Faculty Perceptions of the Influence of Course Experiences on MSSE Students’ Professional Learning. Descriptive Statistics and Frequency of Responses, (N = 24.) Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 1. Use of learning by the MSSE teachers to improve their students’ learning. 4.10 .852 0% (0) 0% (0) 30.0% (6) 30.0% (6) 40.0% (8) 2. Drawing on good practice from other MSSE teachers as a means to further the MSSE teachers’ professional practice. 3.60 1.046 5.0% (1) 0% (0) 50.0% (10) 20.0% (4) 40.0% (8) 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving the MSSE teachers’ practice. 3.40 1.095 5.0% (1) 10.0% (2) 45.0% (9) 20.0% (4) 25.0% (5) 4. Using the Internet as one source of useful ideas for improving the MSSE teachers’ practice. 3.68 1.157 5.3% (1) 10.5% (2) 21.1% (4) 36.8% (7) 26.3% (5) 5. Consulting and observing the MSSE teachers about how they learn most effectively. 2.95 1.191 10.0% (2) 30.0% (6) 25.0% (5) 25.0% (5) 10.5% (2) 6. Relating what works in the MSSE teachers’ practice to research findings and best practices. 3.25 1.070 0% (0) 30.0% (6) 30.0% (6) 25.0% (5) 15.0% (3) 191 Table 31 - Continued 7. Using insights from the MSSE teachers’ learning in the course to feed into their school’s policy development. 2.15 1.182 35.0% (7) 35.0% (7) 15.0% (3) 10.0% (2) 5.0% (1) 8.Understanding how instructional strategies that work in one context might be adapted to other contexts 3.45 .887 0.0% (0) 15.0% (3) 35.0% (7) 40.0% (8) 10.0% (2) 9. Using reflection as a way for the MSSE teachers to identify their professional learning needs. 3.25 1.410 15.0% (3) 15.0% (3) 25.0% (5) 20.0% (4) 25.0% (5) 10. Experimenting with the MSSE teachers' practice as a conscious strategy for improving classroom teaching and learning. 3.05 1.432 25.0% (5) 5.0% (1) 25.0% (5) 30.0% (6) 15.0% (3) 11. Modifying the MSSE teachers' practice in light of performance and feedback in the course. 2.95 1.395 20.0% (4) 20.0% (4) 20.0% (4) 25.0% (5) 15.0% (3) 12. Modifying the MSSE teachers' practice in light of published research evidence. 2.74 1.368 26.3% (5) 15.8% (3) 26.3% (5) 21.4% (4) 10.5% (2) 13. In light of evidence of self- evaluations, the MSSE teachers’ modify their practice. 2.35 1.496 45.0% (9) 10.0% (2) 25.0% (5) 5.0% (1) 15.0% (3) 14. In light of evidence from evaluations of classroom practice by administrators or other colleagues, the MSSE teachers’ modify their practice. 2.05 1.276 45.0% (9) 30.0% (6) 5.0% (1) 15.0% (3) 5.0% (1) 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving the MSSE teachers’ practice. 2.20 1.399 45.0% (9) 20.0% (4) 15.0% (3) 10.0% (2) 10.0% (2) 16. Collaborating to plan teaching. 2.55 1.504 35.0% (7) 20.0% (4) 15.0% (3) 15.0% (3) 15.0% (3) 17. Reading/discussing other teachers’ experiences in the classroom and giving each other feedback. 3.20 1.508 15.0% (3) 25.0% (5) 15.0% (3) 15.0% (3) 30.0% (6) 18. Engaging in team teaching with someone at the MSSE teacher’s school as a way of improving practice. 1.40 .598 65.0% (13) 30.0% (6) 5.0% (1) 0% (0) 0% (0) 19. Turning to colleagues/peers for help if the MSSE teachers have problems with their teaching. 2.35 1.387 35.0% (7) 30.0% (6) 10.0% (2) 15.0% (3) 10.0% (2) 20. Posting suggestions, ideas or approaches for colleagues to try in class. 2.90 1.683 35.0% (7) 10.0% (2) 10.0% (2) 20.0% (4) 25.0% (5) 21. Testing out new ideas in class. 3.00 1.522 20.0% (4) 25.0% (5) 15.0% (3) 15.0% (3) 25.0% (5) 192 Table 31 - Continued 22. Discussing openly with colleagues what and how they, as teachers, are learning. 3.40 1.273 10.0% (2) 10.0% (2) 35.0% (7) 20.0% (4) 25.0% (5) 23. Frequently using informal opportunities to discuss how students learn. 3.35 1.226 10.0% (2) 10.0% (2) 35.0% (7) 25.0% (5) 20.0% (4) 24. Offering other teachers reassurance and support. 3.30 1.031 5.0% (1) 15.0% (3) 35.0% (7) 35.0% (7) 10.0% (2) 25. Believing that all students are capable of learning. 3.65 1.089 5.0% (1) 5.0% (1) 35.0% (7) 30.0% (6) 25.0% (5) 26. Encouraging MSSE teachers, and by extension their students, to enjoy learning. 4.30 .733 0% (0) 0% (0) 15.0% (3) 40.0% (8) 45.0% (9) 27. Regularly celebrating student success. 3.15 1.226 15.0% (3) 10.0% (2) 30.0% (6) 35.0% (7) 10.0% (2) 28. Discussing with colleagues how students might be helped to learn how to learn. 3.10 .978 0% (0) 35.0% (7) 25.0% (5) 35.0% (7) 5.0% (1) Note. The numbers in parenthesis represent the frequency of participant responses. Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, an item with a mean rating of 3 or more was categorized as “more influenced.” A lower mean rating was used as the threshold for “more influenced” with the faculty results as the faculty tended to assign lower ratings than the students. This may be due to the faculty self-assessing course experiences they provided, or belonging to a professional culture that often takes a critical approach. The results indicated that the faculty perceived encouraging MSSE teachers, and by extension their students, to enjoy learning, as an area on which the course experiences they provided had more influence (85%). One student commented, “I learned to learn in a way that will serve me for the rest of my life,” which appeared to support the survey results. “It reminded me of my own curiosity and taught me to be freer with my inquiries 193 and have a deeper passion to explore the answers,” was a reflection by another student. In addition, the course experiences were believed by faculty to influence the area of using the learning by the MSSE teachers to improve their students’ learning (70%). Many examples of the transference of information to the MSSE teachers’ classrooms were reported by the teachers themselves, including one who remarked, “I still use diagrams from that class when teaching my freshmen about mountain building and river erosion.” The results reported in Table 31 showed that another area in which the majority of faculty felt experiences in their courses had influenced teachers’ professional learning was using the Internet as a source of ideas for improving practice (63.1%). A student commented that the MSSE program “got me using the Internet more for research.” Further analysis indicated that just over half of the faculty felt that experiences provided in their courses influenced the teachers’ professional learning in the area of believing that all students are capable of learning (55%). A student who said, “As a Native American, I was thoroughly impressed by the openness of the instructors and their desire to help me feel more valued and included,” supported this perception. The faculty perceived that drawing on good practice from other teachers as a means to further professional practice (60%) was an area that was influenced by opportunities made available in their courses. Just 50% of the faculty reported that the experiences in their courses fostered understanding how instructional strategies that work in one context might be adapted to other contexts. 194 As reported by faculty in Table 31 reading research (practitioner reports and journal articles) as one source of useful ideas for improving the MSSE teachers’ practice (45%) was an area influenced by course experiences. Further analysis indicated that discussing openly with colleagues what and how they, as teachers, are learning (45%), frequently using informal opportunities to discuss how students learn (45%), discussing with colleagues how students might be helped to learn how to learn (40%) and offering other teachers reassurance and support (45%) were all reported by faculty as areas that were influenced to some degree by the course experiences made available to students. This connected to a large number of students’ comments that described how opportunities to discuss with colleagues were “valuable in helping me become more aware of my teaching.” Another student made the statement, “collaborating with other teachers has been huge for me. I have learned a great deal from my peers and grown as a teacher from them and feedback from my teachers on assignments,” which appeared to support the faculty results. The results reported in Table 31 indicated that reflection was an aspect of teachers’ professional learning that was perceived by the faculty as influenced by the course experiences they provided. The analysis indicated that using reflection as a way for the MSSE teachers to identify their professional learning needs and reading/discussing other teachers' experiences in the classroom and giving each other feedback were both perceived by just 45% of the faculty as having been somewhat influenced by the course experiences provided. Several student comments described how the MSSE program influenced their reflective practice. “I had not taken a great deal of 195 time reflecting on my teaching as my school was private and required virtually no teaching oversight. The MSSE program encouraged me to reflect and think critically about pedagogy for the first time,” explained one student. “The readings and discussions that I had to do in my (Courses) made me reflect deeply about how I teach science,” remarked one student. Forty-five percent of the faculty felt that the course experiences they provided fostered the MSSE teachers’ ability to experiment with their practice as a conscious strategy for improving classroom teaching and learning. A student commented, “The action research project has encouraged me to try different ideas in teaching science and also reflect on its effectiveness,” appeared to verify the faculty perception. “ (Course) forced me to examine my practice and led to an improvement of my practice,” said another student. Another area that was perceived to have been influenced to some extent by the MSSE course experiences was regularly celebrating student success (45%). Relating what works in the MSSE teachers’ practice to research findings and best practices (40%) was also perceived to have somewhat more influence on the professional learning of the students as a result of their course experiences. The mean ratings of areas of professional learning that were influenced by course experiences as perceived by the faculty are shown in Table 32. 196 Table 32 MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Professional Learning Most Influenced by Course Experiences M 26. Encouraging MSSE teachers, and by extension their students, to enjoy learning. 4.30 1. Use of learning by the MSSE teachers to improve their students’ learning. 4.10 4. Using the Internet as one source of useful ideas for improving the MSSE teachers’ practice. 3.68 2. Drawing on good practice from other MSSE teachers as a means to further the MSSE teachers’ professional practice 3.60 25. Believing that all students are capable of learning. 3.65 8.Understanding how instructional strategies that work in one context might be adapted to other contexts 3.45 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving the MSSE teachers’ practice. 3.40 22. Discussing openly with colleagues what and how they, as teachers, are learning. 3.40 23. Frequently using informal opportunities to discuss how students learn. 3.35 24. Offering other teachers reassurance and support. 3.30 6. Relating what works in the MSSE teachers’ practice to research findings and best practices. 3.25 9. Using reflection as a way for the MSSE teachers to identify their professional learning needs. 3.25 17. Reading/discussing other teachers’ experiences in the classroom and giving each other feedback. 3.20 27. Regularly celebrating student success. 3.15 28. Discussing with colleagues how students might be helped to learn how to learn. 3.10 10. Experimenting with their practice as a conscious strategy for improving classroom teaching and learning. 3.05 Areas perceived to be “less influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of 1 (not at all) and 2. The mean rating for each item was also used in the determination. Generally, an item with a mean rating of less than 3 was categorized as “less influenced.” 197 The areas that were reported by the faculty to be less influenced by course experiences were engaging in team teaching with someone at the MSSE teacher's school as a way of improving practice (95%), in light of evidence from evaluations of classroom practice by administrators or other colleagues, the MSSE teachers’ modify their practice (75%), and using insights from the MSSE teachers’ learning in the course to feed into their school’s policy development (70%). Other areas that were reported by faculty to be less influenced by course experiences were carrying out joint research/evaluation with one or more colleagues as a way of improving the MSSE teachers’ practice (65%), collaborating to plan teaching (55%), and modifying the MSSE teachers' practice in light of published research evidence (42%). Turning to colleagues/peers for help if the MSSE teachers have problems with their teaching (65%) was perceived by faculty as an area of professional learning that was less influenced by course opportunities, yet the MSSE students perceived this to be more influenced by almost the same percentage. The faculty also perceived that posting suggestions, ideas, and approaches for colleagues to try in class (45%) was an area of professional learning that was less influenced by course experiences, but 50% of the students found this to be an area more influenced. Further analysis indicated that a final area influenced less by course experiences was modifying the MSSE teachers' practice in light of performance and feedback in the course (40%), yet 45% of the MSSE students perceived it was more influenced. The mean ratings of areas that were perceived as having less influence on professional learning through course experiences as reported by the faculty are shown in Table 33. 198 Table 33 MSSE Faculty Perceptions of the Types of Student Knowledge and Practice Relating to Professional Learning Least Influenced by Course Experiences. M 18. Engaging in team teaching with someone at the MSSE teacher’s school as a way of improving practice. 1.40 14. In light of evidence from evaluations of classroom practice by administrators or other colleagues, the MSSE teachers’ modify their practice. 2.05 7. Using insights from the MSSE teachers’ learning in the course to feed into their school’s policy development. 2.15 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving the MSSE teachers’ practice. 2.20 13. In light of evidence of self-evaluations, the MSSE teachers’ modify their practice. 2.35 19. Turning to colleagues/peers for help if the MSSE teachers have problems with their teaching. 2.35 16. Collaborating to plan teaching. 2.55 12. Modifying their practice in light of published research evidence 2.74 Section IV Faculty Survey-MSSE Students’ Science Content Understanding and Knowledge The purpose of Section IV of the survey was to gather information about the MSSE students’ science and pedagogical content understanding. Table 34 provides the means, standard deviations, and frequency of responses from the faculty for the questions relating to the MSSE students’ science content understanding and knowledge. In Section IV of the Faculty MPLS, the questions were assessed using a five-point scale with 1 = “Not at All” and 5 = “To a Great Extent.” Specifically, the question asked, “To what degree do the course experiences you provide foster the MSSE teachers’ professional learning, including knowledge and practices, in the following areas?” Some questions have an N of less than 24 as some respondents left a section blank or skipped a question. 199 For analysis purposes, the data in Section III was combined so that the categories of 4 and 5 represented a “great influence”, while categories 1 and 2 were combined to represent “none or little influence.” Table 34 MSSE Program Experiences Perceived by Faculty to Foster the MSSE Students’ Pedagogical Content Knowledge. Descriptive Statistics and Frequency of Responses, (N = 24). Frequency of responses Question M SD 1- Not at all 2 3 4 5 - To a Great extent 29. Understanding student thinking in science. 3.60 1.142 5% (1) 10% (2) 30.0% (6) 30.0% (6) 25.0% (5) 30. Using inquiry/investigation-oriented teaching strategies. 4.05 .945 0.0% (0) 5.0% (1) 25.0% (5) 30.0% (6) 40.0% (8) 31. Using technology in science instruction (e.g. simulations, probe ware, digital imagery, etc.). 3.25 1.446 25.0% (5) 0% (0) 15.0% (3) 45.0% (9) 15.0% (3) 32. Assessing student learning in science 3.20 1.399 15% (3) 15% (3) 30.0% (6) 15% (3) 15% (3) 33. Understanding science curriculum, including how to use resources and arrange topics over time. 3.00 1.414 25% (5) 5% (1) 30.0% (6) 25% (5) 15% (3) 34. Teaching science in a class that includes diverse student populations (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities). 2.20 1.361 45% (9) 15% (3) 25.0% (5) 5% (1) 10% (2) 35. Adapting teaching to meet state assessment requirements 2.05 1.234 45% (9) 25.0% (5) 15% (3) 10% (2) 5.0% (1) 36. Adapting teaching to meet state standards or curriculum framework requirements. 2.05 1.276 50.0% (10) 15% (3) 20% (4) 10% (2) 5% (1) 37. Understanding educational policies and practices. 1.89 1.197 52.6% (10) 21.1% (4) 15.8% (3) 5.3% (1) 5.3% (1) Note. The numbers in parenthesis represent the frequency of participant responses. 200 Areas perceived to be “more influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of four and five (To a Great Extent). The mean rating for each item was also used in the determination. Generally, an item with a mean rating of 3 or more was categorized as “more influenced.” A lower mean rating was used as the threshold for “more influenced” with the faculty results as the faculty tended to assign lower ratings than the students. This may be due to the faculty self-assessing course experiences they provided, or due to normal behavior within their professional culture. The results reported in Table 34 indicated that the area of inquiry/investigation- oriented teaching strategies (70%) was perceived to be more influenced by the course experiences provided by the faculty. A large number of student responses indicated that “inquiry based teaching”, “learning through inquiry”, and “inquiry science” were key aspects of the MSSE program. One student commented, “My experience as a student in that course was eye-opening in terms of what good inquiry looks like,” which seemed to support the faculty perception. Slightly over half of the faculty felt that the area of understanding student thinking in science was influenced through course experiences. A student supported the perception of understanding student thinking when he or she remarked, “…challenging assignments, case studies, and etc. that taught me how to research, how to critically think, and apply scientific research content to how my students and I learn the best.” One other area of note that was somewhat influenced by course opportunities provided by instructors was using technology in science instruction (60%). It is interesting to note that only 40% of the faculty felt the MSSE course experiences 201 fostered the area of assessing student learning, yet this was a common theme in the student surveys as an area that was more influenced. The mean ratings of areas of pedagogical content knowledge most influenced through course experiences are shown in Table 35. Table 35 MSSE Faculty Perceptions of Students’ Pedagogical Content Knowledge Most Influenced by Course Experiences, (N = 24). M 30. Using inquiry/investigation-oriented teaching strategies. 4.05 29. Understanding student thinking in science. 3.60 31. Using technology in science instruction (e.g. simulations, probe ware, digital imagery, etc.). 3.25 Areas perceived to be “less influenced” by the MSSE program experiences were determined by combining the percentages of the Likert scale ratings of 1 (not at all) and 2. The mean rating for each item was also used in the determination. Generally, an item with a mean rating of less than 3 was assigned the “less influenced” designation. Further analysis of the results indicated one area of science content understanding and knowledge less influenced by courses experienced was understanding educational policies and practices (73.8%). In addition, adapting teaching to meet state assessment requirements (70%) was an area of professional learning less influenced through course experiences. Other areas of professional learning less influenced by course experiences provided by the faculty were teaching science in a class that includes diverse student populations (60%) and adapting teaching to meet the state standards or curriculum framework (65%). The mean ratings of areas 202 of content understanding and knowledge relating to professional learning least influenced through course experiences are shown in Table 36. Table 36 MSSE Faculty Perceptions of Students’ Pedagogical Content Knowledge Least Influenced by Course Experiences, (N = 24). M 37. Understanding educational policies and practices. 1.89 36. Adapting teaching to meet state assessment requirements 2.05 34. Teaching science in a class that includes diverse student populations (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities). 2.20 36. Adapting teaching to meet state standards or curriculum framework requirements. 2.05 Summary of Faculty Surveys When the MSSE faculty were surveyed about the extent to which they perceived their courses fostered science content understanding and knowledge, the following areas were noted as having more influence: analyzing and drawing conclusions, providing evidence to support scientific ideas, working on problems based in real-life, articulating scientific ideas in a discussion, engaging in science process skills and reflection on scientific ideas of both self and others. One of the goals of the MSSE courses is to connect science and/or education pedagogical content to the everyday practice of the MSSE students (Montana State University, 2010). The concerted effort to provide a best- practices approach to the development and delivery of course content is evidenced in the results of the faculty survey. 203 In Section II of the survey, the questions focused on the extent to which program experiences faculty provided fostered scientific knowledge and practice. Not surprisingly, the MSSE faculty reported that the ability to articulate scientific ideas in a discussion was an area cultivated in the program through course experiences. In addition, course experiences were said to influence the area of participants’ reflection on scientific ideas and analyzing and drawing conclusions from data, observations, and other forms of scientific evidence. These items exemplified practicing and engaging in science process skills. The areas of professional learning that included opportunities for on-site fieldwork and interacting with models were found to be slightly less influenced by the course experiences. It should be noted that not all MSSE courses are field-based nor use models in the teaching of content. Less than one third of the faculty respondents teach on campus courses which may account why these areas where rated as less influenced. Section III of the Faculty MPLS focused on professional learning. Using the Internet a source of ideas to improve teaching practice was an area of professional learning influenced by the course experiences. Similar to the student survey, the faculty results indicated that course experiences influenced an area of professional learning that included providing opportunities to read research, both practitioner and journal articles as way to improve practice. One of the case study participant’s comments supported this statement when he or she said, “You get exposed to all this cutting edge information from your content classes and then at the same time you get the research or background and the guidance from science education researchers.” 204 Reflective practice is a hallmark of professional learning. Both the faculty and the student survey results indicated that reflection was an area of professional learning influenced by course experiences. The faculty reported an area that was influenced to some degree by the course experiences was encouraging the MSSE students to regularly celebrate student success. An area that MSSE faculty perceived as influenced through experiences offered in their courses was the use of learning of the MSSE teachers to improve their students’ learning. Findings from the student survey reported earlier in this study appear to support this faculty position. An example of this alignment is provided by the following MSSE student response to an open-ended survey item: “The courses have offered a great opportunity to work with other students and use their experience to improve my teaching.” The faculty survey results illustrated that the MSSE program experiences influenced the pedagogical content knowledge and the area of deepening science content knowledge. Another area influenced by course experiences was using inquiry investigation-orientated teaching strategies. Teaching science to a diverse student population was an area that was less influenced by course experiences provided by the faculty. These results were similar to the student results with one exception. Just 40% of the faculty perceived that assessing student learning, which student responses sometimes linked to providing differentiated instruction for diverse learners, was an area influenced by course experiences. Yet was an area influenced by course experiences, yet the students described this as a far greater influence on their thinking than on any other aspect of pedagogy based on the student survey and open-ended responses. One 205 student’s comment supported this by stating, “…the formative assessments and the research behind formative assessments really, really revolutionized my teaching and I think really strengthened it.” MSSE Student Case Studies This section describes the professional learning of six MSSE students through case narratives of both graduates and current students. Each participant was an experienced teacher and all but one participant was teaching during the 2008-2009 and 2009-2010 school years, the period during which this study took place. The data in the narratives was gained from personal interviews and a portfolio, when provided, and was used to answer the following research questions: 2. How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived by students, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from a program teacher’s perspective, contribute to participants’ professional learning? c. What evidence of teacher professional learning can be identified? 3. How does the distance mode of delivery of this science and pedagogical content knowledge-based graduate program impact the science teacher’s professional learning? 206 In addition, the data was analyzed and synthesized across the six student cases to provide more in-depth answers to the research questions. Each case narrative provides a description of the case participant’s perceptions of professional learning that was a result of participation in the MSSE program. Categories that emerged within and across the cases included motivation to pursue a degree, classroom connections, pedagogical influences, content knowledge, impact on student learning, online opportunities, action research, reflective practice, and social connections. These categories were used to organize the data. Each narrative describes the influence of the MSSE program on the participants from the dimensions of professional learning using the themes that emerged from the data. Quotes from the interviews and the portfolios were used to support the findings. All names of the participants were changed for confidentiality. In addition, any other identifying information for MSSE faculty, staff, and courses were changed for confidentiality. The experiences that led to professional learning were also highlighted to illustrate the effective components of the MSSE program and the benefits to participants. Through the analysis of the MSSE student case studies, it was evident that each teacher valued a combination of program features, and this combination varied somewhat from case to case. Therefore, no clear distinct categories emerged from the data. Rather, three terms were used to describe prevalent program features that influenced the case study participants. These features included the classroom connector, the assessor, and the pedagogical learner. The features were developed after discovering philosophies and perspectives from the case studies through participation in the program and their own 207 classroom practices. There was overlap between categories and the MSSE students demonstrated behaviors or perspectives that fit into more than one feature. These are not mutually exclusive features. These features were used when appropriate to add another level of perspective when describing the case study. The “classroom connector” was defined as those MSSE students who attempted to connect their experiences in the program to their own classrooms. These teachers looked at their courses from the perspective that they wanted to increase their pedagogical and content knowledge, but in a manner that could translate directly into their own classrooms. They viewed the MSSE course from the lens of how their own students would directly benefit by enhancing their content knowledge. Those MSSE students who found an assessment course required by the program to be a transformative experience were delineated as “assessor”. The ideas of summative and formative assessment became the driving force in which they examined their own practice and all of the other MSSE courses they completed. They valued the feedback that was given by the MSSE faculty. Feedback became a way to inform how they were grasping the concepts and understanding their growth as educators. Reflection became an integral part of understanding how to meet the needs of their students It then translated into their own classrooms as a technique through which to inform themselves of their own students’ understanding of concepts. The “pedagogical learners” were those MSSE students who were educators in a community college or informal education setting. The MSSE students were teachers, yet had little to no formal pedagogical coursework. These MSSE students had a strong 208 science content background, yet had never been exposed to common K12 terminology or pedagogical background as others in their MSSE cohort. The pedagogical issues and strategies that were discussed became the lens through which they viewed their MSSE experience. In providing a clear picture of each category, direct quotations from the interview and the portfolio are included to illustrate the case study. The quotations and excerpts from the portfolio are presented as originally spoken or written. No attempt to correct grammar, spelling, or phrasing was made. The only corrections made were those that were needed to clarify the statements spoken or written. The names of the participants, MSSE faculty or staff, and course names were sometimes changed for confidentiality reasons. Word substitutions for confidentiality or clarity are enclosed in parentheses. Charles Charles is an experienced teacher from the west coast. He has taught high school science for more than 10 years. Charles teaches Biology, Honors Biology, and Earth Science. In addition, he serves as the department chair. Charles is also busy outside of school as a baseball coach and father to four children. Charles is very passionate about his personal educational goals and sought out a master’s program that had a strong science component. He was concerned about the large number of “drive through universities” that offered master’s degrees without substance. Charles described this in detail when he stated, “They were in and out with meaningless degrees in my opinion. I wanted to do science - science is my love. A colleague of mine was in the same 209 predicament: wanting to do a science master’s. He looked into this and signed on. He said the classes are ‘real’ classes.” Motivation to Pursue a Degree. Charles was motivated to earn a master’s degree because of the increased compensation his district offered for attainment of the degree. He also needed a program that would fit into his busy lifestyle and cause the least impact on his family. “I planned going back and earning my master’s probably zoology from UC-Davis. That’s what I expected. Marriage, kids, job, it just wasn’t going to happen. To be honest, I started seeing my colleagues earning more money so I was motivated to get my master’s again because it was going to put me on a much higher pay scale,” stated Charles. A colleague referred him to the MSSE program. Charles began by taking one course. He wanted to get a sense of the rigor of the science content and if the program was manageable with his work and family schedule. Charles had a positive experience with the initial course and moved forward in applying to the program. Charles’ overall impression of the MSSE program was that it had been “outstanding”. The science content was what drew Charles to the program. It was the content component that influenced Charles a classroom connector. Each and every course Charles took, he wanted to ensure what he was learning was being transferred to his classroom. He wanted his students to go beyond the textbook and connect with content that was “cutting edge”. Describing his experience in the (course), the initial course that convinced him this was the right fit for him, he stated: 210 …I was learning things that were beneficial to my classroom. I Decided to go for it. It’s been good because I feel like I am in a science major but at the same time every class I has taken had given me something to take back to the classroom. He wanted to share the passion he has for science with his own students by bringing into his classroom strategies and content that was beyond what was the norm. As Charles said, “We were able to take a lot of those labs that we did in class and I was able to work that into my curriculum.” Inquiry –Pedagogical and Content Influences. For Charles, the increased depth of pedagogical content knowledge and change in perspective on particular topics in science was a result of exposure to different courses in the MSSE program. Charles stated, “Little things like that give me a way to connect with the kids. I am wondering is there a way I can do that with my class.” Charles described numerous courses and instructors that offered insights or perspectives that added a new twist or dimension to his teaching. As a classroom connector, Charles viewed his experience from the lens of how an idea or strategy might be incorporated into his class. “My style is my style. What I’ve gotten is I’ve picked up useful info. …. I don’t know if it has changed my style but it has made me a better teacher, it has given me a more broad range of knowledge.” The content that Charles was exposed to during his time in the MSSE program also contributed to the inquiry aspect of his professional learning. The cutting edge research, the scientists sharing their research, and the opportunities to mirror the same laboratory techniques discussed in class added to the richness of the experience for 211 Charles. It also contributed to Charles connecting these experiences back to his classroom. “I think because of the enhanced knowledge I am able to give them a broader view of things rather than just –using our astronomy units. We were sticking to the text books and we are pretty much talking about those things, but now we can go further it’s going right back to them (the students).” Charles mentioned several courses that influenced his practice and his own content knowledge. In describing a course, he often made a reference to how it challenged his thinking about how to share this with his students. In describing his experience with the (course) he mentioned, “I hadn’t really got into how we see the universe and how life relates. When I was taught it was - here’s this and this is this and this is this - I got a whole different perspective. It opened my eyes to a lot of things.” He felt that this was something that his own students would “get into.” The online courses also added a new perspective of engaging his students with the content. Charles found the online discussions a comfortable way of talking about concepts with his colleagues. Charles described it as, “The online was good because I was participating in discussion. But if I were in a class having to raise my hand I wouldn’t have done it. It was much more comfortable for me.” For Charles, the online discussions allowed him to open up in a manner he had never done before. It was that insight into his own learning, which then moved him to connect his experiences with his own students. Charles described himself as someone reserved in small group settings. Charles surmised that if he felt this way about speaking up in a face-to face course, perhaps some 212 of his students had a similar apprehension. Charles took this experience of online discussions and conjectured his own students may benefit from an online experience: Even though I was into technology, outside of a website, I didn’t consider bringing it into a classroom. I didn’t realize how to do it but it was so simple after having done these classes. After doing these classes it was like “Why can’t we have discussions online? - Why can’t they log in after hours like we do?” This new perspective caused Charles to design and develop a weekly blog for his own students to discuss the content. “I wanted to increase the success of my classroom through a biology blog by increasing student participation.” This became the premise of Charles’ capstone research. In describing the use of an online blog to engage all of his students in a discussion, Charles reported: But what ended up happening with my class is that they ended up becoming a team of learners. Unknowingly. By the end they would realize and they would say things in their interview like - well I knew if I was wrong someone would come in and tell me what was right or I knew you were on there all the time, so I knew you were going to log in and say . . .” Usually I would send them private emails and then they could go back to the group. It did work pretty well. So I am going to incorporate it in my classes now on. Charles described this as one of the aspects of the MSSE program that most influenced his professional learning. Charles also wanted to include technology in his courses more frequently, but in a manner that influenced his students’ learning. Charles wrote about his goal to “increase student interest and to provide content” in his portfolio: While in the MSSE program I completed the (course) which required students to develop PowerPoint presentations in order to deliver subject specific content. Prior to this class, I hadn’t given thought to the use of PowerPoint presentations as a delivery tool for information in the classroom. The entire semester, I took great care to create 213 each presentation with the intention of using it during the appropriate units in the upcoming school year. Also, I downloaded several presentations written by other students (with their permission, of course) in order to use, as well. I like the thought of using PowerPoint presentations to provide information to the students. The use of this technology has allowed me to provide written information, images, and video during class lessons. Charles included an example of a PowerPoint he developed for his class in his portfolio. It was rich in physics and astronomy content, as well as containing vivid imagery. A university faculty member whose research focus is in physics/astronomy education reviewed the content of the PowerPoint. The faculty member reported that the content was not only appropriate for a high school course, but it was in much more depth than is generally covered in a similar science classes at this level. Social Capital –Social Connections. Connecting with other teachers in the MSSE program played a role in Charles’ professional learning. His strongest connections were made in the online courses. The online discussions offered a comfortable and nonthreatening way to engage in the course discussions with his colleagues. As Charles stated: I was the kind of student that comes even now to these presentations and I want to sit in the back, I don’t want to be noticed. I am going to do every thing that’s asked of me and I understand what’s going on but I don’t want to speak…that’s just the kind of student I was. In the online discussions, I was more likely—well, I had to get involved. It wasn’t as intimidating. The online discussion groups offered Charles the opportunity to engage and collaborate with others in a manner he had never done before in a face-to-face course. Charles articulated this when he said, “The discussion thing for me was the first time I had gotten 214 that involved in discussion. I have no problem collaborating with my colleagues or someone in a discussion group like that.” In his portfolio, Charles again alluded to the fact he was often one who sat back and did what was expected, but was challenged to engage with others including the instructor: When I was an undergrad I felt as if I was inferior to professors while in the classroom. I was there to complete assignments and earn enough knowledge to pass the course. During my studies at MSU in the MSSE program, I felt as if I was a contributing member of a team of learners within in class population. My opinions and contributions were valued as a professional. The MSSE program changed that perspective for him. Critical and Responsive Learning – Reflective Practice. Charles described himself as a reflective practitioner. Through his participation in the MSSE program, he described his reflection to be more focused. Charles felt his reflection was specific on a particular strategy or concept he was teaching. Charles also commented that he now reflected with colleagues with whom he worked. Charles described his professional growth in his portfolio. “The feedback received was invaluable as I grew towards my MSSE degree goals. It was the interaction between professionals that was probably most valuable during my course of study. Each member of the course, from students to professors were constantly providing feedback to help each other in the learning process.” For Charles, the reflective process also included a resulting action. He was open to what other MSSE students and faculty offered as suggestions and ideas, yet aware of the need to contextualize it. As Charles described: 215 …..someone who had something you liked and you would take a little bit from that person. And another teacher that you liked, you took this. I think I got a lot of that from the different classes. There wasn’t really a class that I thought wasn’t going to do it for me teaching wise. There were always some small components. There were certain things where you thought that will work for you but not for me it’s not my style. Charles felt strongly about incorporating new pedagogical strategies in his class. It required him to ponder it and play it out in his mind before taking action. His reflection made him think through the implementation of new strategies or concepts. Charles described a reflective moment from a (course). The instructor used a variety of PowerPoint’s in the presentation of the content. Charles felt they would have value in his own classroom: I got to thinking, “Wow, we’re sure spending a lot of time on these power points.” It dawned on me that he wants me to use these in my classroom. I thought about it and actually talked to a colleague of mine. I said, “Hey what do you think? You’ve done these in these classes too.” He said he thinks they’d be great but who has the time. I said I’ve got all these from this class. So I started using them in my Earth science class. Charles needed the reflection time to make sure the implementation of a new strategy would fit within his teaching context. Charles was not the type of teacher who immediately integrated a new strategy into his repertoire. He needed time to process it and figure out the best ways to use it with his own students. Valuing Learning. Charles’ motivation to enroll in the MSSE program was based on the fact the program is a science and pedagogical content knowledge-based graduate degree program. As a classroom teacher, Charles felt compelled to make sure that the content he was teaching was not only relevant to his students, but also current and up to 216 date. Science is an ever-changing body of knowledge and Charles wanted to impress that upon his own students. Charles described the caliber of the program in his portfolio when he stated, “The MSSE program contained professionals who valued their role as science instructors and professionals.” Initially, the motivation driving Charles’ master’s work was financial. His master’s degree would increase his salary. As he got more involved in the program, that motivation shifted, and he began to value his own learning from a different perspective. Charles stated, “Before I started doing this I wasn’t thinking about that, I was thinking about the money. But now I am glad that I went through the whole thing and it leaves you wanting to keep going.” Content Knowledge. One of the key factors that drew Charles to the MSSE program was the fact it was a science-based program. As much as he wanted to pursue a master’s degree, he had personal requirements for the program he selected. He had reviewed “drive-by universities.” These were online programs that he felt were inferior and only required a series of tasks to jump through without any substance. Charles wanted a program where he could pursue his passion of science. Charles initial foray into the MSSE program was a single course. He used this as a gauge of the type of program he was seeking. He immediately found value to the course as, “I was learning things that were beneficial to my classroom.” Charles had a strong commitment to his students and wanted to make sure his students were being exposed to relevant and current information. He wanted to go beyond the text, and the 217 MSSE program offered that to him. As Charles stated, “The thing about science is you have to keep up. If you don’t keep up you are way behind. More than any other subject.” Several classes made an impact on Charles and resulted in a direct connection to his classroom. Charles stated, “The experience made me more knowledgeable about science but in a way that I could use in my classroom.” In the (course A), he felt “the class enhanced my chemical knowledge of the greenhouse emissions and how they affect the warming process and the climate change process.” The (course A) clarified his understanding of the process in a way that he felt was simple and elegant – and something that his own students would grasp and understand. A second course caused Charles to reflect about its implications and influence on student learning. He stated, “Like with the (course B) thing, I was wondering - is there a way I can do that with my class? This whole experience gave me a broad range of things.” This was a new way of looking at things for Charles. As a former researcher he felt his focus was very “tunneled”. The background experiences influenced his teaching until he began the MSSE program. Connecting his own content learning to the learning of his students was a driving force for Charles. There was a third course that influenced his learning as well as that of his students. Charles had a research science background and described how he had done some water quality testing previously. Before participating in this third course, he never envisioned it as something he could do with his own students: I knew water quality from a research point of view and how to test water, but I hadn’t done it in a moving body of water. How to tell the pollution index and so on. The experience made me more knowledgeable about science but in a way that I could use in my classroom. 218 Even in an area of science Charles felt he was knowledgeable about before joining MSSE, a course experience took his learning to a deeper level. In addition, it also revealed ways that the subject matter and a related field investigation could be used within his own classroom to deepen the level of understanding with his own students. Charles summed up his professional learning in MSSE about content with a brief phrase, “It really strengthened my content knowledge.” As a classroom connector, it was important for Charles to connect his own learning to his students’ learning. By increasing his content knowledge, he was empowered to be a stronger classroom teacher. This excerpt from his portfolio describes the connection (Figure 8). Figure 8. Charles Portfolio Excerpt Charles provided student work samples from one of the courses he teaches. The activity he used for his portfolio was a direct result of a course he had taken in the MSSE program. He wrote that he developed “lab assignments related to the current course topic. The marriage of the presentation and lab activity provided a dynamic tool for 219 delivery of instruction. This is merely one example of how my scope of teaching has broadened as a result of skills learned within the MSSE program.” The work sample submitted related directly to his PowerPoint example. A university faculty member whose research focuses on physics/astronomy education reviewed the content of the work sample. While, there was a traditional approach to instruction, the astronomy content being examined in the lab activity was deemed to be very solid. The sample provided a clear sequence of concepts and learning progression. In addition, the questions being asked required students to articulate their understanding with examples. Online Learning. Charles represented a typical teacher, extremely busy with school and extracurricular activities, and raising a family of young children. Time away from his family and position was not an option for him as he sought a master’s program. The online aspect of the MSSE program offered a solution to his busy schedule and family commitments: But being able to go on when I had time—when everyone has gone to bed, during my prep period, on weekends. That was a big plus. I didn’t feel like the class was diluted because of those things. It was every bit as rigorous as a lot of classes I’ve taken or any class I’ve taken. Charles wanted a master’s program that was science based. But it was also important for him to find a program that could accommodate his lifestyle. Charles described the online opportunity as the only way he could earn an advanced degree. Charles stated, “I don’t know how I would have been able to do if it wasn’t for that. I don’t know how I would have went back to earn my master’s at a 220 respectful university.” The MSSE program offered Charles both the flexibility he desired and the opportunity to engage in an academically rigorous science program. Sue Sue is a veteran teacher who has taught for eighteen years. Sue teaches in south central U.S. in a town with a population of about 10,000. It is classified as a 3A school, a mid-size school for the state, with a student population of 418. Sue teaches Chemistry, Honors Chemistry, Advanced Placement (AP) Biology, and Anatomy and Physiology. Some years, if there is an interest and enrollment, she also teaches Advanced Placement Chemistry. Now that Sue has earned her master’s degree, a local community college will allow dual credit for her students for the AP biology course. Motivation to Pursue a Degree. Sue was motivated to get a master’s degree in science. Her rural location did not offer any opportunities for advanced degrees. Her personal research led her to the MSSE program. She stated: There’s nothing [master's degree] in science. There’s general education, there’s counseling degrees, and you can do those in the summers and at night and weekends. But in science, there’s nothing. This was the first one. I think there’s a couple more now-- Maryland and Mississippi now have one that has cropped up in the last couple of years. But this place has got it together and they’ve got so much experience at doing it. MSSE met her needs for the type of degree she was seeking and the online aspect offered her the ability to pursue a degree without taking a sabbatical leave. Sue speaks highly of the MSSE program and the experience. The on-campus classes were described as 221 “awesome.” The online courses exceeded her initial expectations. This was Sue’s first experience with online courses. Inquiry – Pedagogical and Content Influences. As a classroom connector, Sue’s classroom connections focused primarily on content, but she commented on some pedagogical strategies that she found powerful too. For example, Sue commented on her experience in MSSE discussions as follows: “There’s a different level of communication there than what there was before. Instead of - I’m the teacher in front of the room- it’s a give and take. An exchange with them like the exchange with your classmates.” Sue felt the MSSE program enabled her to facilitate her students learning rather than simply imparting information. Another pedagogical influence stemmed from the feedback she received in her courses. Sue began to ask her students to submit lab reports electronically. Using the reviewing feature in Microsoft Word, she commented on their work. Sue stated: I even started doing that [reviewing in Word] with my AP Biology kids. I started having them e-mail me their lab reports and I started typing on there. They thought that was just the coolest thing. I started actually doing that the way (Instructor) would do. That was the way he would do that: type back in a different color and send it back. And I took that and did that with those kids. They thought it was great. For Sue, it was not just that the students thought it was great technique, but she felt the feedback she was providing was really assessing their work and made her more accountable to the process. Sue remarked on several instances in which the research she did influenced her classroom instruction. Digging into the research allowed Sue to experiment with 222 different concepts that she had never used before with her students. Sue described the research aspect: The classroom research helps a lot. Because you are going to choose something that goes with your job and what you are doing to enhance that. It changes the way you address the students, the way you address the problems, the learning styles and all that. It just begins to come together. You look at everything a little bit differently when you are looking at it from the researcher’s point of view instead of the teacher: or you get both viewpoints, the researcher and the teacher point of view. Sue felt the research supported her making the classroom connections. “The work that they assign you is really applicable to whatever you’re doing. You can take some of your projects, take them right into the classroom and use them with your kids.” As a classroom connector, Sue translated her learning into her classroom practice. In one course, Sue was tasked with creating a Winogradsky column. Until this course, she had never heard of one. Her research, building one, and then asking her students to do the same expanded her realm of learning. Sue described the experience, “You might have done it before as a student, but I had never done it or even heard of it before. And so we researched all that and then we built them and then we reported on them. I just went right to the classroom and said - Look up this Winogradsky column and read about it, and go out and build you one. And they did, and they are still in my room.” Transforming her learning to her students’ learning was of great importance. Sue also had a limited lab science background from her undergraduate experience. One MSSE instructor created a special problems course for her to allow her the opportunity to expand her understanding of lab procedures and techniques. Sue stated, “I 223 never had a really good lab experience for certain things. He planned that and then I spent a week in the lab with one of his graduate students going through all the procedures and everything.” The time in the lab and the additional research allowed her confidence to begin to make changes in her own classroom through connecting her lab experiences to experiences for her students. As a classroom connector, Sue naturally described the MSSE experience as exposing her to a vast amount of information that has a practical application. Sue described this as, “They know we are teachers. And it seems that they have that in mind all the time so the things you learn you can put them right into your classroom.” Social Capital – Social Connections. Sue described the connections she made with fellow MSSE students as a valuable part of her professional learning. For Sue, the connections she made were another layer of the power of the MSSE program. She felt they were an “invaluable part of the program.” She summed it up with the comment; “I’ve kept contact with the people from ’07. People from last year were at my presentation today they just happened to be here. It’s a community of a different sort. It’s amazing how close you can get with people in a situation like that. The connection.” The social connections that Sue made allowed her opportunities to share ideas. The exchange with other MSSE students all across the country – and the advisors – allowed her to view teaching from a more global perspective. Sue stated, “It’s a trading of ideas and discussion. And you feel like you can always go back to your advisors forever now. You’ve got these contacts you can go back to and get information from.” 224 The exchange of ideas also supported her as a classroom connector by expanding her repertoire of ideas to use with her students. Sue teaches AP Biology and feels the social connections she made with her MSSE colleagues supported her in her challenge to teach this course. Opportunities to ask for ideas and “how you do this” from the larger group strengthened her teaching. One experience she described involved a person in China teaching International Baccalaureate biology and exchanging ideas with Sue and others on content and pedagogy. For Sue, this type of social connection strengthened the experience of the MSSE program. Critical and Responsive Learning – Reflective Practice. Sue found the MSSE program caused her to become a more reflective individual. As a classroom connector, Sue immediately described her reflection as being a student and how that it made her empathetic toward her own students. She stated, “You go back to being the student again and you realize what you are putting them through. How do they feel when they are sitting there? Because the first few years you remember, but then you forget what that felt like.” For Sue, relating her own experience to that of her students was part of the professional learning process and being a classroom connector. Sue described her reflection process in her courses as well, and the influence in her own practice. Sue said, “You sit there and think about it and you type it and then you say “um, no” and then you change it and you really put something into it instead of just popping off like that. It changes the way you interact with your students too.” She goes on to describe how the feedback that she got from other MSSE students and instructors 225 created a positive interaction. It was that positive interaction, based on reflection, which she wanted to share with her own students. Sue stated, “You want them to feel confident and feel successful. The way they make you feel when you get your feedback. Even though you may be being corrected, it’s in such a way that’s encouraging. So you want, in turn -oh, hey! I want my student to feel good like that about their work.” Again, Sue made the connections with her learning to that of her students. Valuing Learning. Sue walked away from the MSSE program with the sense of being valued and appreciated by the instructors. This perception was developed from the positive interactions she experienced with the MSSE faculty. Sue stated, “It’s changed me a lot to be a student again.” Sue took this to heart and wanted to continue to create a positive environment in her own classroom. Sue enjoyed her time on campus and much of that stemmed from the interactions with the MSSE faculty, MSU campus, and fellow MSSE students. Sue reflected on the experience: … it’s anywhere you go here, I don’t know what it is about this place, but anywhere you go you are going to get a smile and a cheerful response and a positive . . .. This is my third trip to campus and it’s anything that they can do to make it easier for you—whatever they can do, I’ve never seen another place like it. It was that same environment of positive learning that Sue wanted to duplicate in her home school. Sue also shared with her students her pursuit of an advanced degree. Many of her students may earn dual credit for some of their high school courses through online 226 courses. The shared experience of taking an online class was something Sue felt enabled her to relate more to her students. Content Knowledge. Sue relished the science content in the MSSE program. Sue described her undergraduate experience as limited, particularly in the area of laboratory science. The MSSE program nurtured her limited experience with a special course to begin to fill in the gap in her learning. Sue felt she had heard about many of the concepts that were discussed in her courses, but she did not feel she had a depth to her understanding. She described this insight saying; I think all the way back to the first summer of ’07 when I started with the thermal biology and learning about the MRAs and where it came from and all of that. I have heard of it since way back since in my undergrad days which is a long time ago. Can’t remember exactly when it was discovered. But I had heard about it at some point but I never knew exactly what it was so that just opened the whole field. The content information from the courses and the additional research background influenced Sue’s professional learning. Sue delighted in the new information and “cutting edge stuff” that the MSSE program had to offer. Each course was a new opportunity to embed the information from the course into her own classes, thus enriching the experiences of her students. An experience Sue fondly recollected was making biofilms. I have a fish aquarium in my room and I was, “Oh my goodness, I can grow a biofilm.” We had slides dangling in it and everything. We had to stop and learn about. . .this was cutting edge stuff guys and you need to know about that. I’ll even use it again this year. My anatomy and physiology classes had to write a report on some type of biofilm problem in the health field industry because they are all health career oriented. Got that right from (the instructor). 227 Sue’s enthusiasm for the content courses flowed over into her other descriptions of other experiences. Sue had recently taken a (botany course). Her voice got higher as she described the influence of the course: This past week I took the most fantastic. . .the (botany course) with (instructor). Oh my goodness, that was just unbelievable. ... and people ask what good is that going to do: (Course) when you come back here? They don’t understand, the way she approached it, we can go anywhere now in any part of the world and take the key from that country and go right out into that field. It’s not that easy, but you know what I mean. It applies to everything. . . I can go right home and do it. . . Sue always included a clause of how she took it back to her teaching context. Sue described another course that was influential as follows: “(The instructor) had experiments going up on the shuttle at the time that I was in his class. He was going back and forth to NASA. It’s just so inspiring and it’s right there. You feel like you’re on the cutting edge.” For Sue, these types of experiences not only pushed her professional learning, but renewed her interest in particular content areas. Online Learning. Sue teaches in a rural part of her state. She has access to a small community college, but no other institutions of higher education. Sue was not financially able to take a sabbatical to pursue a master’s degree. Sue had explored other options: I actually asked of my one-time favorite professors from my undergrad, because I went back to college and graduated later at age 40. So I asked him “Now, I want to get my master’s in biology, I’ve got this teaching job, I’ve got this lab, I’ve got all this stuff. Can’t we work something out so that I could do that?” He thought that was hilarious. They aren’t laughing now because this has all 228 come up around them. They thought distance education was just ridiculous. The other issue for Sue was she did not want to get a general education master’s degree. Sue stated, “There’s counseling degrees, and you can do those in the summers and at night and weekends.” But Sue wanted a science degree. Through some research she found the MSSE program. The appeal of the MSSE program was in part that it could be completed via distance. The ability to take rigorous science courses while still maintaining her position as a high school science teacher was a strength of the program. Sue said, “ There’s no way, being a traditional teacher in a rural community that you can go and be a traditional graduate student.” Sue summed up the online opportunity with, “If not for the distance part, I would not have been able to do it at all.” The online access offered Sue an opportunity she otherwise would not have had. Action Research. The MSSE program was Sue’s first exposure to action research. It offered her a different approach to looking at data. As she described her past experiences with analyzing student work, she often felt the intended outcome of the analysis was not evident. With the experience of the capstone project, she began to view data analysis with a different lens. Sue shared a story of how her school was identified by the state due to low test scores. A team from the regional educational service center provided professional development for their staff. As she described the process: People came to give a workshop because our science state test scores were low. Of course they brought out all the pages and 229 pages of data from the test. Normally I would be, ugh, glazed over. I would be, “I am not looking at any of those numbers, and I don’t care about that. I am just going to teach my stuff and my kids are going to learn it.” This time I was like, “Oh, data analysis. Well, ok, let me have some of those pages. I can figure out what’s going on here.” It gives the teacher/researcher outlook on everything. It’s really powerful. Her experience with her capstone project gave her insights into the data that allowed the professional development to be more informative to her. Overall, Sue found the action research “… fits in with your job and makes your job better and makes you more aware. It makes you a better teacher. It doesn’t take away from the job that you have to do that comes first.” Even with the action research, Sue connected it to her own students’ learning. Barb Barb is a young enthusiastic educator. She teaches in a north suburb of a large urban area. It is a large four- year public high school. The student population is around 2000 students. The population is predominately Caucasian, with 20% of the students from other ethnic groups. There is an English language learner program at the school, including a transitional program in biology that Barb teaches. In addition to the transitional program, Barb also teaches AP Biology, Honors Biology, and Biology Survey. Biology Survey is a lower level biology for struggling learners. Barb is very active in extracurricular activities. She coaches volleyball for boys and girls. Volleyball is both a fall and spring sport, so it leaves little free time. In addition, Barb has children of her own. 230 Motivation to Pursue a Degree. Barb initially looked at a master’s degree as a way of increasing her salary. Her husband is a stay at home dad, so they are a one- income household. The increase in salary was a strong motivator. In addition to the financial incentives, it was important for Barb to earn a science degree. Barb stated, “I wanted really badly to get a master’s or graduate degree in this content area, in science. This is actually the closest thing to it.” Barb was a strong champion of the MSSE program. She described it as a “phenomenal program”. The only suggestion she offered to improve the program was to “market it more than it already has been.” Barb felt the MSSE program epitomized a high quality master’s degree program. Inquiry – Pedagogical and Content Influences. The most significant influence on Barb was the assessment course. That course offered a new perspective to assessment that she had not considered up to this point. It also gave her a lens to view all of the courses she took in MSSE, as she observed how each instructor assessed her work and how could she use that approach to assessment in her own practice. It was this focus that identified Barb as an assessor. Barb described her use of formative assessment as, “I did very little of it before I started in the program. And now, it’s kind of transformed the way I do things.” Up to this point Barb had only taken online courses. But it was the rich discussions in the online courses that influenced her pedagogical strategies: And in some instances I tried like “Oh that sounds like such a cool example of how you got your kids through your content”. And I 231 tried it and it was a complete flop so I am like—ok. What happened there?? There are other things you can take from different teachers because it’s a whole community of different teachers working together. You are sharing/they are sharing their research and allowing you to grow more and use different learning strategies in your classroom that I don’t know that you would actually tap into as well as if you are sitting in a class of 50-60 students. Barb found this exchange of ideas powerful. The access to ideas from the online community met her professional learning needs. Barb discovered that the action research component of the MSSE program provided a way to make connections between the science content and assessment in her own classroom. While she felt that her students were learning, she didn’t have clear measures for assessing their growth. Barb surmised how the action research influenced her assessment practices: But when you can take action research and have it applicable to where your students currently are at, what you need to do to help them. They’re giving you feedback for your teaching and you’re giving them feedback for their learning. And they structure it in a way where you can actually look at data, statistically, from your classes as they are currently going on. I think that’s a huge benefit. Barb focused on data from her students as a means of assessing understanding. The inquiry aspect of professional learning was powerful for Barb, in particular the literature review for the action research she conducted for her capstone project. She found the process “one of the most important and beneficial of the whole capstone project.” In her portfolio, she wrote about the literature review, “It gives insights to what works from teachers across the globe and establishes a foundation for best practice. It is something that I would not have done if I had not been part of the MSSE program.” The literature review focused on assessment. 232 Barb’s portfolio illustrated the influence of assessment on her practice. She developed Diagnostic Learning Logs for her students as part of her action research. In her portfolio, she wrote about how she used them in her classes: The goals of the pretest with diagnostic learning log (were) to allow students to engage in metacognitive behaviors in the classroom, review their previous learning regarding background content in Biology and to allow the instructor to assess students’ prior knowledge. In reviewing their own work in this manner, students can develop analytic skills, develop problem solving skills, adapt their study habits based on analysis of individual work and cultivate a sense of responsibility for their own learning. Barb included a copy of the Diagnostic Learning Log in her portfolio as an example of how she utilized her professional learning from the MSSE program to influence her students’ learning. A copy of the log is included below in Figure 9. Diagnostic Learning Log 1. Briefly describe the test. What was it about? 2. Give one or two content examples of areas where you were felt most successful on the exam. Try to explain what things you did that made them successful. 3. Look at the questions you got wrong, how many were strictly knowledge based (i.e. vocabulary) and how many were application based (analyzing lab data)? 4. For each answer that was incorrect, explain what you did wrong in each case. Explain the content of the problem. Figure 9. Barb’s Diagnostic Learning Log 233 Barb used the logs as a way for students to earn points back on the exams, but her students’ reported other value in using the logs. Barb shared the reflections of her own students on the use of learning logs. “I really had to go back and understand the content I missed, and it helped my grade.” Another added, “I like being able to look up and explain things so that I am able to know the information.” A third student stated, “Diagnostic learning logs let me see exactly what I don’t know and how to correct it.” With regards to the diagnostic learning logs only two of the students mentioned earning points back and one student mentioned that, “it helps you go over your mistakes so you can get them right on the AP test.” The learning log was a powerful tool in assessing her students’ understanding. Barb shared some of her concerns about student understanding. The MSSE program gave her a new perspective on assessment that she felt allowed her to push her students academically. She stated: There are students, especially the last couple of years when I have been doing this [assessment], they are not used to it. And they are just like “can you just tell me what we need me to know?” But their depth of understanding is so shallow then. When you have them look at things or do activities from a different perspective, I really do think it increases their depth of learning. And so a year from know they’ll understand the process and they’ll understand the depth of learning rather than “I have no clue what we learned last year.” Understanding what her students know and don’t know by looking at research, ideas from colleagues, and best practice has strengthened her teaching. Barb explained the impact from another lens: Maybe this is my own perception: I think a lot of kids leave high school not remembering a lot even though they may be 4.0 students. I think there is so much content that we try to shove in them by the time they leave high school that if it’s not done in some kind of depth. The activities and how it’s structured and the content classes and the action research allows us to be able to really look at things from a 234 different perspective so we can really enhance the depth of understanding for our students. Assessors continually gather data to better understand student thinking. They may also introduce students to techniques to monitor their own learning. As an assessor, Barb focused on getting into the minds of her students. Her understanding of how to use assessments and building them into her practice was influenced by the MSSE program. In her portfolio, Barb wrote about this change: All students were able to identify what areas they knew were successful in and what areas they needed to review, even at times stating that they hadn’t heard of certain terms. Students were also able to indicate what questions they missed that were strictly knowledge based versus those that were application based. Students also indicated when they had a “misread the question” type of error. All of these factors give the students indicators of how to study and how to take tests in order to achieve at a high level. Being able to assess her students’ understanding accurately has empowered Barb and she also familiarizes students with techniques to monitor their own learning. Social Capital – Social Connections. Barb found the connections with others in her MSSE cohort to be a strong influence on her professional learning. She emphasized the shared experiences online and in the field courses added to her understanding of the content. The exchange with other MSSE students all across the country – and the world – allowed her to view teaching from a more global perspective. Barb stated, “And so to be able to look at people that are teaching in different countries, let alone different parts of the US, and see how they are handling things. And going back and forth and going, ‘Oh, I didn’t think about that—I’ll try this there.’ I think that’s just cool”. 235 The richness of different people and their perspectives resonated with Barb. Gathering assessment ideas from other teachers and determining how they might be used in their practice characterize assessors. As an assessor, Barb mentioned how she often discussed with other MSSE students their assessment ideas and strategies. Barb described this process in her portfolio: Being able to hear what other teachers were doing/struggling with throughout the country (and in other countries at times) was beneficial. Sometimes it was directly related to what I was trying and even when it wasn’t, it was often connected indirectly…Being able to share my work and get feedback from a different and “outsider’s” perspective, in my opinion, is an asset to the MSSE program. Developing her repertoire of assessment strategies was key to ensuring her students were learning. Barb stated, “I think, too, that the connections that you make with people are very important and valuable.” Barb felt that connecting with others and incorporating new ideas into her practice, both characteristics of her MSSE experience, were strengths of the program. Critical and Responsive Learning - Reflective Practice. Barb found the reflective practice that was part of the MSSE program reinforced her learning. It was through feedback from the professors and her reflection on that feedback that Barb shifted own assessment practices. Barb described this process when she stated, “I like the online discussions and the immediate feedback that your professors give you. When you read you have to go ‘hmm’ and maybe think awhile before you write something that makes sense and sounds good…just that small amount of additional thinking time almost forces 236 you to be more reflective.” Barb elaborated on the feedback received from MSSE instructors and how this affected her: The feedback the professors give you—like you (Sue) had mentioned earlier—is very constructive and in a very positive light and it really causes you, or caused me, to think about what I was doing or to even think about things from a different perspective than I had thought about. I think teachers have a control issue something or territory issue with their work in their classroom. It allows you to let go of some of that and really be open to constructive criticism because it is in a very non-threatening way. It was through this revelation that Barb began rethinking her assessment strategies. Assessors use the reflective process to ensure the assessments used are a clear measure of student understanding. She felt that feedback and opportunities to reflect like those she was given in her coursework would perhaps influence her own students’ learning. It started small, simply using the track changes feature to comment on her students’ formal lab reports. Barb felt the quality of her feedback to her students was becoming of higher quality, as she was able to focus on certain pieces of the lab in more depth. Barb felt this small step was the beginning of improving the ways she assessed her students. Barb described the process and the influence on her students: I’ll be typing, because I just type my thoughts in a different color or on the side. As I am reading their paper and then later I am like, “Oh, ok, you clarified this here.” I think this kind of stuff, the students see the process I am going through, and I think—and I haven’t done any data collection on this—but I think that may help them also see this is what I am looking for. When they see how I am going through grading the paper or why they are not getting certain things. I think I have found a lot less argument or digging for points because they say, “Oh, I see what you did.” It’s clear and for some reason because it’s “tight,” they get it more. I think even if I had hand written the same thing, it’s not as valid as if it’s typed. Maybe that’s. . .with their technology. . . 237 Barb’s words reflect the position that the onus of assessment should not fall only on the teacher, nor should the teacher be the only recipient of insights yielded by analysis of student work. Barb’s statement illustrates her perception that when teachers enable students to play a more active role in assessment, for example, by allowing them to glimpse teacher thinking while grading a lab report, it may influence their learning. Barb’s experience in the MSSE courses, particularly the online courses, and the feedback and reflective experiences was instrumental in her moving her practice forward in terms of assessment. Barb repeatedly linked her MSSE experiences to how it translated in her own classroom. From her portfolio, she described her shift in thinking about assessment: Though I used student surveys, I didn’t look at student academic work as data that could contribute to improving my practice as an educator. Furthermore, I did not really understand the difference between formative versus summative assessments even though I was a certified teacher, already had one graduate degree and was considered highly qualified by state and federal standards. Her learning in the MSSE program influenced her practice especially in the area of assessment. While she has implemented more formative assessments, including the learning logs, the reflection of what was happening in her classes caused her to move her practice forward once again. She wrote about this change in practice in her portfolio. It was based on her reflection on what was working and not working in her classroom: One of the changes I have made this year to my teaching practice was how I use formative assessments. As stated, last year when I was piloting the use of formative assessments in the classroom, one of the obstacles was to get students to complete them for the sake of learning, not for gaining points. I didn’t accomplish that last year. This year, I decided to set up my classroom with formative assessments as a non- 238 negotiable classroom activity. Students are required to complete all formative assessments as pass/fail. If they meet the requirements, they pass, if not they don’t and must repeat. Setting up the classroom like this from the beginning is hopefully setting a different learning environment than “learning to count points.” This idea is just one of the suggestions that came from not only online collaboration, but having the opportunity to discuss learning strategies with other presenters and advisors. Valuing Learning. Barb described in detail how she came to value the MSSE program due to the quality she perceived in the content and the action research component. As she got further into the program, those components reinforced the idea she had selected the right program to meet her needs. While Barb sought a master’s program for financial reasons, she also had an innate sense of wanting a high quality program that met her needs. Barb’s enthusiasm for her learning carried over to the classroom. She extended her own learning of new ideas and content into her own classes. She described one experience: So the class saw my excitement about learning something new when I am twice their age and have my own family. I think they pick up on that too. At some point they are like “Ok, Mrs. B enough with the biofilms we get that you’re excited about it and think it’s cool.” But I think that that really helps bring your rapport with your students to a different level that I don’t think would have been there. To be honest, my students for the last two years wouldn’t know what a biofilm was if I hadn’t taken the class. But there’s so many places where that is an application to help them understand what we are talking about in class. Barb shared her own learning with her students as a means of enhancing her own teaching. Her students were able to push their learning further especially with the 239 addition of Barb’s assessment strategies. Not only were Barb’s students going beyond the textbook, the feedback the students received from Barb pushed their thinking as well. Another layer to Barb’s valuing learning was addressing the needs of what she referred to as “marginalized students.” In her portfolio she wrote about how students were historically selected for the AP courses. This process did not “provide equal access to higher education options for all students who were interested in excelling in a science field.” The other complexity to the problem was that the learning needs of those students enrolled in the AP courses were not being met. She described how, “The strategies presented in the education classes and the ability to have autonomy to choose what works best for individual situations, was beneficial for both teachers and students.” She shared this knowledge with her colleagues in her department. Barb wrote, “For a variety of reasons, over the last 6 years, our school’s and department’s practices have changed and now a wider range of students have the opportunity to enroll in AP Sciences.” She attributed some of the shift in how the classes are taught and students are selected for the AP courses to her MSSE experience. Content Knowledge. During her interview, Barb frequently mentioned MSSE courses that had influenced her learning. She emphasized how she sometimes knew a concept at a superficial level, but the MSSE courses allowed her to go into depth and further her understanding. Barb also mentioned that science is an expanding area of knowledge and that it is a challenge to stay current. Staying on top of current research and having access to scientists conducting that research was “one of the biggest benefits” of the MSSE program. 240 She further notes that many of the courses connected directly to current research. Barb said, “You can really see—not that we’re not scientists as teachers---but you can really see what they [research scientists] are doing. There was just something in the news about a coral reef they are restoring and in the news there’s video of him [the MSSE instructor] doing it and we’re learning about it at the same time.” Being able to view and participate in ongoing research made Barb feel empowered with her new knowledge. Barb described content in several MSSE courses that had a direct influence in her classroom. The content she was learning was being translated to her own students. She remarked that often she was able to make connections to extending her students’ learning. She described one example that involved having students look at biomes. Rather than superficially look at the characteristics of a biome, her students could do “an activity that looks at how you map things on the ocean floor and how they relate to the organisms.” In addition, she proposed students could jigsaw the activity instead of every student researching every biome. She felt that engaging the students with the activity and then having them share their information with each would be a more effective strategy. Barb also made connections to pedagogical content knowledge. As an assessor, Barb was focused on increasing her students’ content knowledge, but had an interest in improving her delivery of the content. Assessing student understanding allowed her opportunities to reflect on her practice and the influence on her students. Barb stated: I think one of the things is getting the kids to be more actively, physically involved with the content. I am always looking for ways to get involved—and that’s part of what my research was actually on was-- Getting away from the lecture/do a lab/take a test format. I think that there’s so many places that that is what is done. All of the courses really gave me alternative ways so I don’t have to be standing up there. 241 In Barb’s portfolio, she included an example of a “content form and function outline.” This tool was a formative assessment strategy for subject area content. The outline is shown below in Figure 10. Barb stated how using this form for assessment provided her more insights into her students’ content understanding of the topic being discussed. Content, Form and Function Outline Content – what is it? Definition Form - how? Elaborate Function – why is it biologically important? Hydrogen bonds (example) Weak intermolecular , transient bond, contributing to electrostatic attraction Attraction between the partial negative bond of oxygen and the partial positive charge of hydrogen Surface tension resulting from collective hydrogen bonds – animals that walk on water Polarity Cohesion Surface Tension Adhesion High Specific Heat High Heat of Vaporization Versatile Solvent Unusual Phase Characteristics Dissociation/Ionization Figure 10. Barb’s Content Form and Function Outline 242 Barb accredited the MSSE program with providing her access to the content, as well as pedagogical strategies, particularly assessment, to influence her students. Online Learning. As a coach, mother, and the sole income provider for her family, Barb is cognizant of her time and commitments. The online aspect of the MSSE program, offered an opportunity to earn a master’s degree. Barb commented, “I am online a lot after my kids are in bed and there are other teachers popping up with things at the same time. I don’t think that if it was offered any other way that I would have been to do it at all.” The online aspect of the MSSE program made a master’s program accessible to Barb. While the online opportunity offered convenience, Barb felt the courses also offered the academic rigor she expected. The courses were intense, but the interaction with other participants also challenged her thinking. “I would say there is more time put into the online courses than people expect”, noted Barb. The different perspectives and opportunities to share ideas allowed for professional growth. Barb strongly felt that the online discussions allowed for more exchange of ideas than would be found in a traditional face-to-face course. Barb felt the MSSE program delivered a high quality program that offered additional opportunities for growth as a result of the online format. Action Research. Action research was not new to Barb. Her school had started an action research lab seven years ago. Her assistant principal’s familiarity with action research allowed her to get the MSSE program approved for her professional development plan. 243 Barb also felt that the action research project was something you could “really tailor to fit your interests and needs.” Barb indicated this was one of the strengths of the MSSE program. Barb stated, “The action research allows us to be able to really look at things from a different perspective so we can really enhance the depth of understanding for our students.” The MSSE program’s action research requirement afforded her the opportunity to conduct research that was applicable to “where your students currently are at and what you need to do to help them.” The action research project Barb completed offered her the opportunity to “look more at school climate and marginalized students.” As an assessor, Barb was perhaps more aware than many colleagues of the importance of gathering and sharing accurate information on the learning needs of all of her students. In the portfolio assembled for this study, Barb provided additional details about the influence of the action research project: The Action Research classes probably have had the biggest impact on my change in developing and implementing classroom activities. The focus on getting feedback from students and giving students feedback, combined with being provided with tangible examples of how to do this, has really influenced the way I teach. She developed two tools from her capstone project, the learning logs and the form and function outline. These tools have continued to be used and modified in her class. She wrote in her portfolio, “The students’ feedback most clearly indicated what was working and what wasn’t and most directly affected both immediate practice and long term teaching goals.” 244 Julie Julie is a science educator who had recently moved to a new state into a new position as an instructional coach. This was her first year out of the classroom. Julie had taught for thirteen years before transitioning into this new position. She credits the MSSE program in part for attaining this position. Julie said, “because of the program, it made me very qualified for this position.” Julie’s portfolio provided more details about her teaching context prior to becoming an instructional coach. She wrote that many of her students were English language learners. The community where she had taught was a resort area and more than one million tourists visited annually. Due to the nature of the community, the hospitality industry was the major employer. This meant jobs were abundant yet there was “little, if any, experience or specific skill sets including the ability to speak English.” Julie already had earned a master’s degree. In addition, she had experience with action research prior to enrolling in the MSSE program. Julie anticipated that she would be graduating in less than a year from the time of the interview. She still had a thesis to complete and finalize her research. She was well immersed into the MSSE program as she spent a summer on campus and was able to earn 14 credits. The vast majority of her coursework had been completed on campus. As she is doing a science research capstone, she had spent a large part of her time in the field. Motivation to Pursue a Degree. Julie’s interest in the MSSE program stemmed from the fact she “would prefer to do a hard science option” in a master’s program. Over a year ago, she had a conversation with the director of the MSSE program and shared her 245 interest in doing science research. It turned out that a scientist at a wildlife refuge in the region was looking for a graduate student to assist with a research project. It was shortly thereafter that Julie enrolled in the MSSE program and began her studies. Inquiry – Pedagogical and Content Influences. Julie described her transformational experience in the MSSE program as the exposure to learning how to really teach inquiry. The research she was exposed to on how students learn and how that could translate to classroom practice influenced her entire curriculum. Julie found the idea of assessing students to show understanding made the greatest impact on her work. While Julie was focused on teaching inquiry science, as an assessor she wanted to ensure her students understood the process skills of inquiry. She stated, “the formative assessments and the research behind formative assessments really, really revolutionized my teaching and I think really strengthened it.” Julie found the idea of process skills to be her focus. Assessing her students using process skills empowered her to advocate for additional resources and support. She stated, “I was then able to go to my principal and say, ‘Hey, I need these resources because research says.’ So, that was very, very powerful.” With an understanding of the power of assessment and the research to support her, advocacy for her approach to teaching was a result of being an assessor. Julie continued to describe how the idea of assessing students influenced her teaching: I’m an integrator so I cannot teach from a book for every different subject of the day, I don’t know how to do that. And so for me, integrating is everything. I wasn’t getting the results that I wanted year after year. So taking the process skills or teaching that, you’re having that light bulb go on and say, I have to teach them how to 246 think and I have to teach them how to think in each different skill area and then I have to have them reflecting on their own learning cognition in their writing and then maybe I’ll get the results. It was the personal growth in her teaching practice that made Julie feel more confident. Repeatedly Julie referred to the fact that the deep understanding of science inquiry was key in influencing her professional learning. She said, “Changing my practice of investigation. I was never a cookbook experiment teacher, but with the science inquiry process, that took my Foss kit teaching to an entirely new level.” As an assessor, Julie had a better understanding of her students’ understanding, which strengthen her inquiry teaching. In addition to the pedagogical changes, Julie felt she was able to assess her students’ content understanding more thoroughly from her experience in the MSSE program. One area that Julie discussed was the idea of misconceptions. She stated, “I’m covering a misconception. That is something I had taught before, but this took it to a whole new level and understanding that student misconceptions are there and what it takes, that has been my most powerful learning experience, in addition to my research project.” By developing her own content knowledge, Julie was able to support her students’ learning more effectively. The formative assessments she acquired in the MSSE program provided information about her students’ learning. This information led her to revise and improve her curriculum. Julie also explained how for the first time in thirteen years of teaching she had finally repeated curriculum, as she felt confident her students were learning. Confidence in her teaching was a result of being an assessor. She explained that she had 247 experienced frustration with what she had been teaching and wasn’t getting the results she had hoped. Using process skills to teach the content and “being willing to scrap all my plans,” she began seeing results. By adding formative assessments into her teaching, she was able to collect data on her students’ learning that informed her practice. She now knew if her students truly understood the process skills she was teaching. The exposure to using research for her classroom practice and for her research was a strong professional learning experience: And I was reading all of these articles, doing my review, reading scientific articles for the first time ….So “student” was working on her doctorate and she and I had talked at great lengths about the MSSE program and one of the things that she commented on that I completely concurred with, how many science teachers have actually read and analyzed and compared scientific articles? I’m going to have to say not many. When I would get a teacher’s schedule and she’s got 30 minutes of science a week, I’m going to guess that she hasn’t spent a whole lot of time reading scientific articles. Reading research became a way Julie built her confidence. Using research also supported her ideas and innovations. Julie provided examples of assessments she had used with her students in her portfolio. In one excerpt, Julie illustrated the data she had collected from her students and how that information would influence her practice. Her assessment and the data she was using are shown in Figure 11: Even though the test scores were all passing the students ranked themselves low and inconsistently on their mastery of each science inquiry process skill. As I looked at this data this was by far the most meaningful piece to me as an educator. It was interesting to realize that my students had such incorrect perception as to their abilities. This will be a fun area to encourage them and watch them grow. 248 Figure 11. Julie’s Commentary on Student Data Julie explained how this data allowed her insight into her students’ perceptions of their learning and understanding. Despite the fact that all those tested scored well on a quiz, her students’ perceptions of their learning were vastly different. This information informed her practice as she began to plan her next unit. She described her reflection on the process: I feel confident in my instructional methods of the scientific inquiry processing skills after administering this CAT. I will continue my instructional techniques .I will continue to have the student practice their writing skills daily. I have begun to see an improvement as they answer the QOD. The students still need guidance to wrote as much as they know regardless if they think they can’ explain it in English. Sometimes, it just take me speaking with the students individually and briefly to remind them that they really do know the answer. Once again, as an assessor, Julie evaluates her instructional methods as well as her students’ learning with the assessments she utilized. 249 Social Capital – Social Connections. Julie found the collaboration with her MSSE colleagues to be a valuable part of her learning experience. Julie felt isolated in her teaching practice and “it was very exciting for me to get out of my box and hear what other folks were doing.” Julie described two aspects of the social connections that influenced her practice - networking and confidence. Julie had taken a significant number of field courses. She described the time in the van with other MSSE students as a tremendous learning experience. Making connections with others who taught similar grades or disciplines offered opportunities to build teaching strategies. Conversations like, “Oh, you’re teaching 6 th grade. Hey, I’ve got this idea. Have you ever tried?” –were common occurrences that Julie described as one of the best parts of the courses she has taken. Julie looked to networking opportunities to gather new ideas, particularly assessment strategies. In addition, those conversations led to opportunities to network: So, I think the networking has been a very powerful tool that’s resulted from the types of collaboration that set up those face to face, as well as on line. And the face-to-face field classes, you’re in the van and the vans are fantastic because everybody sits and talks and exchanges ideas and I loved that! The connections made in the van were people she contacted throughout the year to share and exchange ideas. Julie also felt that the social connections empowered her with confidence in her abilities and knowledge: Having discussions online and for me, so many of the face to face discussions last summer made me feel like, “Oh, maybe I do know what I’m talking about …” and again, without that confidence, I wouldn’t be sitting in this office right now as the instructional 250 coach for the entire district. The confidence was evident in the way Julie spoke about all aspects of the MSSE program, including her own experience doing a field science research study, rather than a classroom study, for her capstone. In describing her confidence, she stated, “I’ll go back to the confidence, being able to say, hey, this wasn’t working. I learned about it. Here’s the research behind it. Here’s what I tried, and here’s my results.” This confidence stemmed not only from her science research, but her research into assessment. As an assessor, Julie grew in confidence in her abilities as a teacher as her ability to discover what her students knew improved. Critical and Responsive Learning - Reflective Practice. As an assessor, Julie keyed in on reflective practice as the part of the MSSE program that influenced her teaching practice. Reflection allowed her to see how to best meet the needs of her students by teaching inquiry. As her knowledge and skills improved, the ability to measure the impact on her own students was becoming more evident. Julie explained in the following statement from her interview: What is it that I want them to be able to do, understand and demonstrate? What do I want to see at the end of this? And how am I going to get them there? That was a very powerful, transformative experience; in addition to (Instructor’s) class where I had to really look at the different kinds of assessments and what is it that I’m looking for? How can I differentiate it and how can I -- for me, I always want to translate it into how can I make this work for English language learners. 251 As a reflective practitioner, Julie felt she was able to make better educational choices for her students as a result of her experiences in MSSE course on assessment. Assessors use reflection to make informed decisions about learning opportunities for their students. The online discussions offered Julie additional opportunities to become a better reflective educator. Julie stated, “Articulating your answer in writing is very different that spouting off what you’re thinking about at the moment. It’s at the tip of your tongue. So that reflective piece that was built in by just answering the discussion questions ended up being very powerful for me.” Julie found through reflection she wanted to keep the new ideas and thinking fresh. She developed binders to use and refer to later as a way of reflecting. “I actually have a binder. I’ve learned to keep evidence, keep a binder and so I have whole inquiry binder. I have my (course) unit binder, which really combines a lot of elements I have. I have student journals that I’ve kept.” Through this organizational method, Julie continues to reflect on her learning and how it integrates with her work. Valuing Learning. Julie’s enthusiasm for her research was evident by the detail she provided about her study. Julie pursued the MSSE program because it offered her the opportunity to do actual science research for her capstone project. While she felt there was value in action research, she had her sites set on immersing herself in a true scientific study. Julie described how the process influenced her and how she hoped to convey that to her students: So I think it’s enhancing my content area of knowledge as well as my own thinking skills. And I talked to a very good friend of mine who has a master’s degree in ecology and I said, what is it about going through these articles and disseminating the main point? I said, for some reason I’m feeling like there’s something going 252 on in my brain and I’m trying to identify it so that I can then be aware -- metacognition, but I also want to be able to encourage that in my students. Julie’s immersion into her research study stretched her thinking, skills, and content knowledge. Julie stated, “It was a very interesting process for me…I wouldn’t be in this way of thinking right now and I really feel like it has stretched my brain in amazing directions for critical things and analysis.” Julie summed her experience when she said, “ The whole investigation procedure has been monumental to me as a learner.” As an assessor, Julie examined her own learning and assessed her progress and understanding. Julie was eager to implement this same enthusiasm for learning with her students. Through the process of assessing her students, Julie began to get them to think about their own learning as explained in the statement from Julie’s interview below: They completed a process skills test, I guess; and at the end of it, for each skill, they had to give a value to themselves as learners. I feel like I’m giving myself a 5, I feel like I’m an expert at this. I can teach something about making predictions. Or a zero, I still have no idea what a prediction is. Or a 3, I feel pretty comfortable understanding what predictions are. And so what I did is I took those values as individuals and I took them as posters and I drafted them in Excel and I made a big poster and as their confidence increased over time, I made new posters to show them that they are taking responsibility for their own learning and that they’re ownership of that is very important for themselves as they build more confidence and as they gain more knowledge. Julie felt that using this method has caused her students to be more aware of their learning. Julie reported increased student engagement as a result of the student self- 253 assessment. A characteristic of an assessor is engaging students in the process of assessing their own learning. This was similar to Barb’s use of learning logs. Content Knowledge. Julie chose the MSSE program as it offered her the opportunity to do actual scientific research for her capstone project. This was a huge leap for Julie to immerse herself in the program as a science researcher. Julie stated, “I really have a very weak academic background in science and math. And so if you look at my history alone, you’d be scratching your head and wonder why I’m in the position that I am in today.” It was her interest in science and determination that led her to the MSSE program and her current position. Julie was very animated as she discussed her field research. She described her research project as starting “from the ground up and really being let loose by (two instructors) to make mistakes and then they’d reel me back in and going through that whole investigation procedure…. It’s just, oh, this is doing science rather than learning science.” For Julie, her experience in the program was about being in the role of a scientist and ultimately how she translated that experience for use with her students. As she described her research, Julie was specific about the skills she was learning. “And I go through all of them [notes] for my written review and really pull out some very specific things and then be able to make hypotheses to test for my investigation and also be able to make predictions.” These experiences translated back to her classroom as a need to teach her students science process skills so they could really do inquiry science. 254 While Julie felt the science research was a key element in her professional learning, there were other classes she described that influenced her content understanding: I didn’t know what recombinant RNA was. I didn’t know how to test for a certain gene and something so that you can replace it with another gene to make a better thing. You know, that was huge for me. The (course) reaffirmed something and it taught me so many other things…that was a great class because he had us go to the IPCC reports. Never would have done that. Gave me so much more factual information about global warming and climate change. So, I would have to say that all of the content classes enhanced my knowledge. Julie felt courses she took in the MSSE program greatly enhanced her content knowledge. As she had limited background knowledge in most science disciplines, often the content was new to her. Adding to the scientific research and content courses she chose to take, she also felt strongly that the instructors that taught in the program contributed to her understanding. She stated, “The quality of educators that they hire, the personal investment and background experience of (the program staff), they’ve got the infrastructure to support what goes out to the students.” In addition, the design of the courses allowed her to experience inquiry, but it was the attentiveness of the instructors that made the program what it was for her. “Some of the classes were more inquiry based and that was a very powerful experience. But the quality I think was there, and their responsiveness”, according to Julie’s statements during the interview. Julie’s portfolio provided examples of how her content knowledge was conveyed to her own students. In the first example, Julie provided examples of a final project for the quarter. The project was used to represent a culmination of the lessons presented in 255 her class. Students were offered opportunities to select a topic, research it more deeply, and then create a final project to present. As an assessor, Julie wanted to be able to assess her students understanding, but in a way that met their learning needs. To address this, she offered choice in the final project. The overview sheet that was used with her students is shown in the example in Figure 12. It was evident from this sample that Julie offered a variety of options for the final product to meet clear objectives that were provided to the students. Figure 12. Julie’s Project Handout 256 The MSSE program influenced Julie in the area of inquiry science, in particular using process skills to teach content. Julie included several examples of student work that illustrated how she used process skills to convey conceptual understanding. Julie made comments on their work, included in the examples, as well as a reminder that the samples were from students who were English language learners. The samples of student work were also indicators of students understanding. Blending her focus on teaching inquiry with assessment, Julie taught the content at the same time she was assessing their understanding. This seamless connection between content and pedagogy is a characteristic of an assessor. In the first sample shown in Figure 13, Julie showed an example of observing. The students generated their own questions as well as what types of observations would be relevant. Figure 13. Student Work Sample on Observation 257 The next example in the portfolio was of a Question of the Day (QOD). This was a formative assessment strategy used to gauge understanding at the beginning of a lesson or the next day after a lesson. As Julie had a large population of English language learners, she encouraged pictures and diagrams as a way to help students clarify their ideas. This example is below in Figure 14. The comments Julie made indicated that she is assessing understanding of the student by interpreting the diagram. Figure 14. Question of the Day Student Work Sample 258 A final example of student work from the portfolio, illustrates a student beginning to set up an investigation. Students were encouraged to design their own inquiry-based experiment. This example is seen in Figure 15 below. 259 Figure 15. Investigation Student Work Sample. Julie attributes her students’ abilities and skills to design an inquiry-based investigation from her learning in the MSSE program. In addition, she was able to adapt the content to what was meaningful for her own students. Online Learning. While the majority of Julie’s coursework was field or lab based , she experienced some online courses. Online was not her preferred method of learning. She stated in her interview, “I was totally resistant. I like talking to people. I like interacting with them so I thought, ‘on line learning?’ This is so weird.” Her preconceived notions of what an online course would be like were quickly dispelled. She felt it would be isolating and impersonal. Instead in was a powerful learning experience. Julie quickly found the community building that was part of her online courses not only a pleasant surprise, but also another venue to network and collaborate. Networking and collaborating are common traits of assessors, as they seek out new ideas 260 from others. In one course, she was able to share some of her insights from her own research. Julie felt that this type of collaboration and shared experiences would not have occurred face to face: I wouldn’t have done that outside of the class and I wouldn’t have done that if I were in a class because I don’t think the same type of discussion would have precipitated because we’re so busy as teachers and we’re so excited to talk to each other, that the details of what I was allowed to do in writing, I don’t think would have been articulated in person to person. It was the writing and reflection that occurred in the online courses that also influenced Julie’s professional learning. Julie recognized the reflective practice that was inherent in the online learning. The delay in responding due to writing caused her to be more careful and thoughtful of her responses: Articulating your answer in writing is very different than spouting off what you’re thinking about at the moment. It’s at the tip of your tongue. So that reflective piece that was built in by just answering the discussion questions ended up being very powerful for me. The written responses allowed her to go back and reread and review what she wrote and what others had responded. Julie stated, “And it was really interesting to see in writing what I had learned and I wouldn’t have done that outside of the class.” Once again, as an assessor, the reflection process allowed her the opportunity to assess her own learning. Caleb Caleb presented a different perspective for the MSSE program. A recent MSSE graduate, Caleb teaches at a community college. Caleb had previously earned a master’s degree in a specific science area. He teaches science courses at a community college 261 located in the eastern United States. He describes the school at which he teaches as affluent. Faculty members are encouraged to seek additional education and are provided with financial assistance for ongoing professional development. A friend of his was in the program and Caleb came to Montana with him one summer to see what the program entailed. He felt that what the program offered, especially the field courses, was what he was looking for to advance his career. Motivation to Pursue a Degree. Caleb’s motivation to continue his higher education was based on career advancement. At the community college where he teaches, the faculty has a tiered system of advancement. Additional coursework beyond a master’s degree up through the attainment of a Ph.D. offers both financial incentives and varying levels of professorships. As Caleb described, “Essentially, I accrued enough credit to make the jump to assistant professor and all along, I’ve been -- I said, well, if I’m going to go ahead and be earning credits, I might as well get another degree as well. So that was sort of like motivation for getting into the program.” A combination of career advancement and earning an additional degree was the impetus behind Caleb’s decision to enroll in the MSSE program. Inquiry – Pedagogical and Content Influences. As Caleb described the influences of the MSSE program on his practice, he clearly articulated that the pedagogical content knowledge and pedagogical practices were very revealing for him. As an instructor at a community college, the expectation was he knew his content, not how to teach it: Just to give this some context. I work in a building with a bunch of Ph.D.’s and people that have MS degrees and various other areas 262 like biochemistry, physics, biology, whatever. But very few, if any of them, have ever taken that education class. Yet, we’re all in the business of educating kids. It’s kind of this weird, screwed up situation where the requirements to get in the door is to prove that you can do science by having earned a master’s or Ph.D. Yet the job in here is not to do science. The job here is to educate. And I think that’s kind of weird. So for me in comparison with some of my peers in the program, and for them I think it was good. And everything that they’ve had, educational classes before and so some of this stuff they were not hearing for the first time. But for me, it was all new. Learning the pedagogy was empowering to Caleb, although his learning curve with terminology was steep. Caleb described one experience, “The first week of the first class, one of the discussion topics was how has your life changed since NCLB. I was like what is that? I had to ask. No Child Left Behind, you heard of it? And I was like, oh that. Right. Okay. That’s sort of outside my understanding.” Caleb didn’t describe himself as an innovative educator, as he stated, “And I give lectures and we do labs and I was told that those are of high quality but in terms of being an innovator, someone coming in as an educational practice, I’m not sure that’s me.” Caleb felt strongly that there were parts of the MSSE program he could adapt to his own teaching context. “And taking these on-line classes have given me ideas about ways to use blackboard better and also I guess what doesn’t really work and how are things can be richer than they are. I use a lot of imagery and stuff like that now,” said Caleb. Caleb viewed the teaching strategies used by the MSSE instructors as a way to influence his practice. He stated, “I’ve got a lot of interest in implementing technology into things so like I’ve got this blog and there’s regional field geology class, all the students there, it’s their final project has to be something that’s web sharable.” 263 Caleb was very confident in his understanding how he saw his own professional learning. While he found the education courses resonated with him the most, he viewed each MSSE instructor, including the scientists, as providing a new lens through which to view instructional strategies. Caleb stated, “If I ever find myself in a lousy presentation or a boring class, then I try and learn a little bit about like what’s going wrong with the ways that the presenter is presenting things. So I am kind of chewing on that even if the actual content of whatever they are saying are boring me to tears.” Social Capital – Social Connections. Caleb stated he did not have a strong need to connect or collaborate with his MSSE cohort. He stated, “Collaborating with MSSE colleagues. There are very few people in the MSSE program working in my same shoes.” He strongly believed that, as he was in a different teaching role than many of the K-12 teachers, he did not need to engage in discussions about classroom implementation or connecting content to curriculum. Caleb viewed his social connections from a different perspective. Caleb stated, “I think I served a good role in some of these classes where the professors were talking at a relatively high level. People weren’t quite sure what was going on so I could serve as a translator.” Caleb was confident in his content, which made him appear somewhat arrogant. He recognized that and stated, “I haven’t really hit it off especially well with my collaborative groups.” While interpersonal connections were not a strong aspect of the MSSE program for Caleb, he did connection with others through a blog. Caleb stated, “I read a geology blog and so sharing some of the stuff I learned through the program either that geology 264 blog with an audience that -- initially was intended to be like interested people here but once you put it out on the Internet, the whole world especially got reasonable readership around the planet.” This type of social connection suited Caleb, as he was the disseminator of information. In reading through Caleb’s blog, there were references to the social connections and learning that he experienced in MSSE: In the evenings, we discussed scientific papers about field technique, the Hell Creek Formation, taphonomy, and the extinction of the dinosaurs. All our meals were cooked for us by (Instructor), and so it was really ideal: Go out and learn all day, come back to camp to a hot meal, a cold beer, and a discussion of big picture ideas. My fellow teachers and I also played a lot of horseshoes and frisbee. To top it all off, when we got back to Bozeman yesterday, a group of us rented Jurassic Park and watched it over pizza and ale. While Caleb did not see the social connection as a strong part of his learning, his blog provides evidence to the contrary. Critical and Responsive Learning - Reflective Practice. Caleb described himself as a reflective practitioner, but the MSSE program influenced his reflective practice further. Caleb stated, “But I definitely got to be more reflective as a result of engaging with these programs. I really value that. I mean that’s something major that I’m taking away from participation in the program.” One area that caused Caleb to reflect on his practice was when he assessed his students. By utilizing assessment strategies with his students, he began to get a better grasp on their understanding. The results from those assessments caused him to reflect on his practice: 265 … I do a little what they call CAT - class assessment technique at the end of the class. …And it was the day I taught about minerals. And I got back that. A lot of people didn’t get what their own mineral was, which was a mineral break in a preferred direction. But I evidently didn’t explain that. I just basically said, okay, minerals, some of them break in very characteristic ways. But I didn’t get into the details of why, even though I laid the foundation for that earlier. So that really -- looking over that data seemed how ominous that was among their responses. I was like, wow, I totally didn’t cover this. So I went back in the next class, along with that information, and had a couple fresh analogies for them. Including like the splitting of a log. It’s easier to split one way than to split across the grains. And the same with minerals. …And then after that, they all claimed to have gotten it. That would be I guess one example where there was a misconception, or a lack of conception, and by reflecting on the feedback that they gave me, I was able to remedy it. Caleb explained that assessing and then analyzing the assessment, offered opportunities for him to reflect on his pedagogical practices more than he had in the past. Caleb said, “I really do appreciate being forced through the program to engage in much more reflective practice and data gathering and I really liked the aspect of basically asking students for feedback.” Valuing Learning. Caleb described himself as someone who has a “pretty high value to learning in general.” The structure that was part of the MSSE program was also something that resonated with Caleb’s learning style. The community college that Caleb is associated with encourages and supports continuing education of the faculty. Caleb capitalized on that opportunity, not only for career and salary advancement, but also for the sake of continuing his own professional learning. Caleb described this by saying, “It makes life interesting to figure things out. Gaining new appreciation for the way the world works.” 266 Caleb was more thoughtful about how he demonstrated his valuing of his own students’ learning. He grappled with what that would look like beyond personal praise or “when they figure something out, get excited.” Caleb built on this idea, noting that sharing his own excitement for learning could influence his students’ valuing of learning. Caleb took a variety of courses that allowed him to experience the geology of the Rocky Mountains. He wanted to extend his own students’ learning with his experiences in Montana. The end result was a field trip to Montana with his students to study geology. His excitement about the geologic formations generated an interest in his own students. Caleb stated, “So that was really important because it allowed me this year after I finished the MSSE degree to actually have a group of students from my community college to fly out to Montana and we did a two-week tour of the state, regional field geology class.” Content Knowledge. Caleb felt that he entered the MSSE program with a strong content knowledge understanding. Yet, he found value in the experience: I really appreciate the program. I found it different from my geology master’s in several ways. I didn’t find it as challenging in terms of the content that is such a good thing because I was losing sleep when I was in geology graduate school. Stressing out a lot. This was more relaxed but I wouldn’t say it was so relaxed that it lacked academic rigor. I definitely felt like I was expected to perform well in a way that I could be proud of for the program. Even with his confidence in his subject area background, Caleb found ample challenge in his science coursework in MSSE. For example, in the following passage he described the intricacies of interpreting the geochronology of a field site in eastern Montana that included some dinosaur remains: 267 The boundary formerly known as "K/T" is therefore between the two, and it records the changing of the eras: from Mesozoic to Cenozoic. I say "formerly known as K/T," since Tertiary is an archaic term that has been replaced (sort of) with Paleogene. The Paleocene is the first epoch in the Paleogene period. The sedimentologically-defined boundary between the two formations is the lowermost "significant" coal layer. We found this coal, the so-called "Z Coal," and you'd think that would be the K/Pg boundary, but it ain't that simple. True, there are dinosaurs below and no dinosaurs above, and it's also true that the Z Coal has been shown (rather shoddily, by the description we got) to have an iridium anomaly at its base. But there aren't any dinosaur fossils at all for 3 meters below the Z Coal, so the dinosaurs could have gone extinct well before the Z Coal was deposited (and before the iridium- rich clay layer was deposited). And of course, there's nothing in the deposition of a layer of coal that indicates it should be contemporaneous with a mass extinction -- it's just coal. Furthermore, the coal is lake coal, and the lake wasn't necessarily regionally extensive. It's a funny way of defining a critical geochronologic boundary: by the lowermost layer of lake coal in an area -- a criterion which could vary temporally from one place to another. Tricky business! He described numerous other examples of specific content experiences that influenced his thinking. Many of Caleb’s examples of describing the influences on his content knowledge were tangible items he could use in his own teaching. He specifically mentioned rock samples he collected and photographs that he took during different field courses. In addition, many of the courses he took added to his content understanding: And I really have gained not only a new appreciation for courses in the system that I didn’t really understand as well. The earth sciences. I have expertise in part of the earth sciences but not everything. And so it’s like really good for me to go and think about the screen depositional environment of the Hell Creek formation and I did that in a (course). 268 Although Caleb entered the MSSE program with confidence in his content understanding, he learned that he too held misconceptions, as illustrated in the following example: But even better was when I got something wrong and it was like one day, we were out looking at these stream deposits and the stream deposits I’m familiar with basically tilts in a certain direction. They tilt in a downstream direction. And so I was looking at the wall of the little canyon. I was like, (Instructor), these things are pointing the other way. He was like yeah. Well, that can actually happen if the stream flows high enough. And I was like what? It can be the opposite of what I thought was true. For Caleb, an experience that challenged his thinking was not only memorable, but gave him an appreciation for the MSSE program. Caleb’s blog served as his portfolio. The blog is a detailed description of his work and interests in geology, and the MSSE program often served as a backdrop: I was there on one of my four MSSE classes this summer, and I learned a lot. As many of you know, I'm trained as a structural geologist, not a sedimentologist. Though I use a lot of sedimentology (and fossils) in my (course), there is much I have left to learn. Some of those gaps got filled in this week during "Dino Camp," though. Caleb stated that one of the benefits of the MSSE program was learning about the Rocky Mountain geology. This added to his repertoire of geologic regions in the U.S. This new knowledge of the regional geology was a significant part of his blog during one of his trips to Montana. Photographs, diagrams, video, and scientific descriptions comprise the content on his blog. Examples of these and the detail to the content presented can be seen in Figures 16, 17, and 18. 269 Figure 16. Video Segments on Caleb’s Blog. Figure 17. Diagram from Caleb’s Blog. 270 Figure 18. Photograph of Geologic Formation from Caleb’s Blog. Online Learning. The majority of Caleb’s program of study was field courses. He took some online courses and found the experience to be mostly positive. The one exception was an online science course. His preference was taking field courses and he did not feel that the one science online course he took was well done. Although he did not like the online course, he still found value in the pedagogical strategies that were utilized. He found strategies that influenced his practice. Caleb stated, “The online classes have given me ideas about ways to use blackboard better and also I guess what doesn’t really work and how things can be richer than they are.” In addition to finding value in the pedagogical strategies that were utilized in the online courses, Caleb found the education courses to be very revealing. Caleb was not familiar with the pedagogy in teaching science and that added value to the experience. Caleb often put strategies he was learning in his online education courses to practice in 271 his own classroom immediately. The application from the course to his class was an aspect of the program he found valuable. He stated, “I think it’s one of the best things about this program is that we’re all there doing our job and then taking these online classes that are about doing our job at the same time.” Sean Sean’s teaching experience in K12 would be categorized as informal. His only formal classroom experience was during his student teaching. Sean described his teaching experience as “all my K-12 has been more non-classroom based or it’s been kind of short. It’s been classroom based but it’s been for short chunks where I’m going into the classroom and then otherwise my experience has been more extensively with university students.” Sean’s interests in science education are geared toward higher education as he is now enrolled in a doctoral program. Motivation to Pursue a Degree. Sean was looking for a master’s program that could bridge his interest in science education and science content, particularly environmental education. Another underlying goal was to continue toward a doctoral degree. Sean stated, “And so I was looking for a program that continued to combine both educational course work as well as science content course work – ecology and natural science course work and that was one of the things that drew me to the MSSE program.” Sean found that he was challenged academically, but his perspective allowed him to view the program differently than the K12 teachers in his cohort. Sean said, “I had to remind myself that it was designed as more of a professional -- it’s more of a professional 272 degree. And not really as much geared towards folks who are planning to move on afterwards.” Inquiry – Pedagogical and Content Influences. Although Sean had student taught in a high school classroom, most of his teaching experience stemmed from his graduate experience in another institutional setting. Sean described this experience as a “really tight-knit living and learning community where our learning -- these conversations moved beyond the classroom and beyond our teaching and occurred around the dinner table or during a run or out skiing.” In this teaching role, Sean taught students in grades K12 who were enrolled in the institution’s educational programs. Sean had the opportunity to learn about education practice and then try it with the students he was working with at the institution. As he described it, “we’d do course work and then be teaching and we’d do course work and teaching.” Sean found the advertised mixture of formal and informal learning experiences, for example, informal conversations with faculty and peers during online and field courses, matched the type of graduate program he was seeking. Sean transferred to Montana State University with a year of graduate school credit from a previous institution. Sean found that once he was immersed into the MSSE program it shifted his thinking from exploring “pure science” to also developing an understanding of how to teach science well” Once I was in the program, it was a wonderful kind of continuation of what I’d been doing, but then I found myself, through my own professional work, that I was actually interested in -- I was shifting a little bit away from purely teaching science to wanting to 273 continue teaching, but at the university level and exploring more interdisciplinary issues around -- that you can clump into environmental studies. So, that was kind of going on and I think it was one of the impetuses for then continuing with my studies afterwards. This “experience” caused Sean to think more about how to teach science. Sean had limited experience with assessment strategies prior to the MSSE program. As he said, “that was definitely newer material to me.” He found utilizing those, even in the informal teaching settings, informed his practice. Sean described how it influenced his teaching as, “Being able to use that kind of ‘in the process’ as compared to kind of waiting until the end to get more immediate feedback.” He did feel that he had not had a “chance to integrate and implement as much as I’d like to; but it’s been kind of tucked away in the back – it’s something that I want to try to work on integrating more.” Social Capital – Social Connections. Sean described his learning as very social, but as he wrote in his portfolio, “Unfortunately, this culture is not yet found throughout the field of education. When it exists, I am certainly a part of it.” He described the learning situation at the place where he began his graduate studies, and had opportunities to teach, prior to entering the MSSE program. The collaboration between the faculty and the students, and the social learning that occurred there were very powerful. Sean described it, “In that we had faculty members who were constantly coming out and observing us or having peers observe us as well. Giving us feedback and having the time to then debrief that as well, and talk through some of the observations and then reflect on those.” 274 Sean felt the MSSE program offered a similar collaborative situation. The way the courses were designed allowed opportunities for exchanging ideas with other MSSE students. Sean said, “I think I always had kind of a tendency to want to work with other folks to learn what was working for them, and what were some of the things they were bringing to their classes or their courses or their teaching, and where they were finding their successes and their failures.” Sean talked about the understanding of what good teaching looks like he gained through discussions with peers in MSSE, and that he will take away as he continues his education. Critical and Responsive Learning - Reflective Practice. Sean was a newcomer to reflective practice, but had, “opened up to the idea of being a reflective practitioner in my practice as an educator,” as he pursued his graduate work. Transferring into the MSSE program, “was definitely a nice segue because it helped to continue to cultivate that [reflective practice].” The MSSE program expanded Sean’s understanding of reflection and how it could influence his growth. Sean said, “that really opened my eyes to the role of feedback and the power of feedback and being open to how those can help with personal growth.” Sean had a unique perspective on reflection from his previous graduate work and teaching experience. He noted the differences in reflective practices between K12 teachers and university faculty. He felt the educational culture might influence professional growth differently. 275 Sean described that sometimes he felt in the online discussion that, “It doesn’t allow you to be as critical in regards of responses and feedback and learning; which I think the potential exists, but I think it’s inhibited.” When asked to elaborate, he stated: I think sometimes, too, the field of education or teachers are sometimes … it’s a group of folks that -- it’s different than like the academic field. Especially in the higher education in terms of people willing to challenge each other or kind of strive to get the best explanation or the best argument; or also being okay to disagree on things, which I believe has it’s downsides as well, but you don’t quite find that among teachers as much. Sean believed these insights would support his professional goals of working with teachers. His perspective was that reflection should challenge thinking in a way that causes professional growth. Valuing Learning. The path that Sean had taken for his own pursuit of higher education speaks loudly for his sense of the value of learning. Sean’s interest in science, led him toward thinking about how to teach science. It was that mind shift that caused him to think more about science education. Sean’s passion is environmental studies and the exposure to pedagogical content knowledge added a new dimension to this interest. Sean now is studying interdisciplinary issues around environmental science at the university level with the goal of teaching at the university level. In his portfolio, Sean offered more insight into his perceptions of how he valued learning. He felt the learning community that is created within a cohort of MSSE students, especially those who take the education courses together, not only enhances collaboration but also the valuing of learning. Sean wrote, “I absolutely loved watching the learning that took place for many of the teachers that had been in the classroom for so 276 many years. So from that sense, the cohort model does a good job at creating/ developing a learning community that fosters and values learning.” Content Knowledge. Sean felt he entered the MSSE program with a solid science content background. Prior to being in the MSSE program, Sean felt his higher education pursuits would be more “pure science.” The exposure to science pedagogy broadened his lens to include both science and science teaching. Sean also wrote that action research experience influenced his content understanding especially with his own professional goals of earning a doctoral degree. He wrote, “ I would also add that the Action Research course work developed my content knowledge in research methodologies which has proved to provide a better starting point now that I am back in graduate school in a program which values pluralistic methodologies.” Sean took both online and field science courses, which he described as “absolutely fantastic.” He felt that all of the courses he took, “I felt were really outstanding and really helped me to continue to increase my depth of understanding and expand in both of those areas.” But more than the content, Sean noticed how the courses were designed and delivered: I think that was the biggest thing, was just kind of from watching how they taught. As a participant, sometimes kind of stepping out and looking from the outside in to being like, what are they doing and how are they doing it; and why is this being effective right now for me? And kind of taking some notes. Sean’s teaching experience in formal settings was limited and he now viewed his courses with a new lens. And also because I came in with some foundational knowledge 277 and familiarity and then by taking those, it just kind of helped to make some connections and also give me kind of tools just by watching them on how they taught and I was kind of thinking about how I might pick up new ideas for activities or new ways to frame things or design courses and also ways to convey information as well. For Sean, the pedagogical approaches to teaching science content were the foundation of his professional learning in the MSSE program. Sean wrote in his portfolio, “I learned a lot just from watching how the faculty presented, framed and taught the courses.” Online Learning. The online component of the MSSE program was “one of the things that drew me to the MSSE program along with the ability to participate in the program on a somewhat low residency or remote basis,” according to Sean. While being on campus was an option, Sean continued his role as an informal science educator at the same time he was completing his master’s degree. Sean stated, “It allowed me the flexibility to continue to work in the field and to engage myself professionally. And also allowed me the opportunity to apply what was being learned within my own professional setting.” Action Research. Enrollment in the MSSE program allowed Sean to begin to pursue the connection between science and science teaching. Sean described it as, “a way to bridge the gap between theory and practice with undergraduate learning.” Sean was able to put into practice what he had only conjectured. His action research project blended multidisciplinary ideas of environmental education. Sean explained the process and his professional learning. I think the opportunity to one, kind of really dive into an idea or concept and to explore that inward depth. And then the ability to 278 and the requirement to really do something that impacted and was involved in work we were doing ourselves. There was an immediate connection there. And lastly, the idea of being very methodical with the work to really try to quantify or qualify what some of the outcomes were and how to improve the idea or concept. Sean wrote in his portfolio that he felt that through his action research project he “made leaps and bounds in effectiveness due to the process of mindful and methodical data collection and analysis.” In addition to his own learning from the capstone experience, he also found value in the capstone projects of others in his cohort. Sean said, “I was thoroughly impressed and it was amazing the opportunity to go and watch other people’s presentations to come together for that final symposium and to see the way that people have grown and what they have learned was absolutely outstanding and astonishing.” Summary of Case Studies Each of the case study participants brought a unique context and perspective through which to view the MSSE program. The dimensions of professional learning were evident in each case study, but the importance of particular dimensions varied according to participant. In addition, certain features or outcomes of the program were especially influential for particular case study participants. These features are evident in the labels selected to describe specific case participants including classroom connector, assessor, and the pedagogical learner. The classroom connectors focused on ways to bring the content learned in their coursework back to their own students. The assessors viewed their learning experiences from a perspective of how they could gain a better 279 understanding of their own students’ learning. The pedagogical learners were those MSSE students who entered the program with a strong confidence in their science content understanding, but more limited knowledge of teaching science, and then became intrigued by pedagogy. One of the first themes to emerge was reason to pursue a master’s degree. In all but one case, financial incentives from the teacher’s school or college provided the initial reason to earn a master’s degree. Ranked closely in importance to financial gain, was the desire to complete a master’s degree that was science based. There were several themes that emerged from the data, many of which connected with the dimensions of professional learning. The professional learning dimension of inquiry included being exposed to different sources of evidence and research and using that information to change practice. In all of the cases, the teachers reported that the MSSE program contributed to their use of research methods and evidence to increase their understanding of pedagogical strategies and content. Examples were provided in the case data that illustrated the influence of these experiences on teaching practice. Social connections were also a strong influence for all of the cases. Opportunities to collaborate contributed to the professional learning of each of the case study participants. Networking, sharing and exchanging ideas, and engaging in discussions with others about their students and student learning were an integral part of the MSSE program. Reflective practice is a way for teachers to grow and learn. While each of the case study participants felt they were reflective practitioners, the MSSE program pushed 280 their reflective thinking to new levels. Reflection was an inherent part of each of the courses and the opportunity to reflect on their own learning often translated into a change in practice in their own classrooms. Each of the case study participants chose to complete a master’s degree. While initially this may have been financially driven, the enrolling and participating in the program was a demonstration of valuing learning. Several of the case study participants shared their enthusiasm for learning with their own students, which in turn created a sense of valuing learning for their students. Most cases felt that as they began to assess their students more often, and provided feedback in greater depth, this also developed a sense of valuing learning among their students. The MSSE program attracted each of the case study participants because of the rich science courses that were available. Even those cases that came to the program with the perception of having a solid content foundation found the MSSE courses to be academically rigorous and influential in increasing their science content knowledge. Each case study participant cited specific courses that resonated with them and gave examples of adapting content from those courses for their own students. Another aspect of the MSSE program that was significant for the case study participants was the online opportunities. As many of the courses are offered online, they allowed access to people who are unable to attend a local college or university due to busy family and professional schedules, or geographic challenges. In addition, the online component allowed teachers to take courses while retaining their position. The flexibility afforded by asynchronous communication was an added feature. Many of the case study 281 participants would not have been able to pursue a master’s degree if it were not for the online component. The capstone is an action research project. It offered participants a chance to explore a topic in depth that is relevant to their teaching context. All of the case participants mentioned the influence of the action research on their teaching practice. It was a defining experience for many of the case study participants, which changed their perspectives or practice. The case study participants introduced in this section provided insights into the MSSE program. During interviews and through their portfolio materials, the case study participants provided evidence that all of the dimensions of professional learning were present in their MSSE experiences, although not always sought or valued to the same degree, and that their growth as educators was strongly influenced by the MSSE program. Summary of Overall Findings The results presented from both the Student and Faculty MPLS indicated that all of the dimensions of professional learning as recommended by the literature were evident in the MSSE program, although some aspects of each dimension were not evident. The case study data provided additional evidence of professional learning to support the survey results. In addition, the open ended responses from the student MPLS added another layer of evidence that supported professional learning was clearly part of the MSSE experience. 282 There were four broad areas that were analyzed across all data sets in an attempt to answer the research questions in this study. The first was to identify what course experiences in the MSSE program fostered the MSSE students’ content understanding and knowledge. Results from both the student and faculty survey indicated that course experiences influenced the MSSE students’ ability to articulate scientific ideas in a discussion to a great extent. Further analysis found that both the MSSE students and faculty perceived that course opportunities influenced the area of reflecting upon the scientific ideas of other students and reflection upon earlier scientific ideas. The case study data also supported the influence on content knowledge and practice as case study participants described in detail specific instances in which their content knowledge was impacted by the MSSE program. The next broad area examined across all data sets focused on to what degree MSSE program experiences fostered professional learning. The areas of professional learning that emphasized using research to inform best practice were reported as areas influenced by course experiences by both the faculty and students. The use of learning by the MSSE teachers to improve their students’ learning was rated as an area of influence in both surveys. Using the Internet and reading research was an area that was greatly influenced by course experiences as reported by both faculty and students. Several of the case study narratives described how reading research, both on science and educational topics, influenced professional learning. Both students and faculty perceived that course experiences influenced the area of collaboration to a great extent. Both the faculty and students perceived that course 283 experiences influenced the area of drawing on good practice from other MSSE teachers as a means to further the MSSE teachers’ professional practice. Offering colleagues’ support, discussions with colleagues, and informal opportunities to share experiences were areas perceived by both the faculty and students to influenced MSSE course experiences provided. Each of the case study participants commented on the collaboration that took place in the MSSE program and how those collaborative experiences enhanced the program. Reflective practice was an area that the course experiences influenced. Both faculty and students reported that course experiences influenced the practice of reflection as a way that professional learning needs were identified. In addition, reflective practice was seen as a tool to change and improve teaching practice that was influenced by the course experiences. Each of the case study participants described the influence of reflection on their teaching practice. The majority of the case study participants felt the MSSE program caused their reflection to be more focused. Supporting student achievement appeared to be influenced by course experiences. Results from the faculty and students surveys indicated that creating an environment that causes students to enjoy learning, believing that all students can learn, and celebrating their learning were areas fostered by the MSSE program experiences. The case study participants also supported the results from the survey. The case study data indicated that the feedback and positive tone established by the instructors was influential in creating constructive learning experiences. 284 The next broad section investigated the influence of program experiences on the areas of science content understanding and knowledge. Again the faculty and students responses were similarly aligned. Course experiences influenced the areas of using inquiry/investigation-oriented teaching strategies substantially, as reported by both faculty and students. However, the responses were mixed regarding the influence of course experiences on engaging students in science processes. For example, the proportions of faculty perceiving that their courses influenced students’ ability to conduct and design investigations were low. Both groups also perceived that course experiences fostered the area of understanding student thinking in science. Assessing students’ learning in science was an area also perceived as influenced by course experiences, although the MSSE students perceived its influence as much higher than the faculty. Once again, specific examples of how the MSSE program fostered science content understanding and knowledge were reiterated in the case studies. The last area of the survey that was analyzed was used to determine which course types offered in the MSSE program influenced the understanding of science content, pedagogy for teaching science, science curriculum, and teaching diverse learners. In each of the four open-ended responses, specific course types were identified as being significantly more influential, but there were two reoccurring themes that were evident in all of the responses. These themes included collaboration and reflection. Collaboration was a theme that appeared repeatedly in the open-ended responses. Opportunities to collaborate with other teachers were valued by the MSSE students. Collaboration can be linked to building social capital, a dimension of professional 285 learning identified by the literature. The collaboration with others was perceived to create a powerful professional learning experience by having the opportunity to discuss content and how to teach that content. The opportunities to collaborate, discuss, and share ideas with other educators influenced the teachers’ pedagogy, including pedagogical content knowledge. In addition, many students perceived the online discussions contributed to more in-depth understanding of content and meeting the needs of diverse learners. Informal and formal opportunities for reflection on teaching that occurred in the program were also highly valued by the MSSE students. Reflection, a characteristic of critical and responsive learning, was referred to in comments throughout all of the open- ended responses. Reflective practice was also reported to have influenced the pedagogical understanding of the MSSE students. In addition, reflection that occurred through completion of the capstone project was also a strong influence. This study sought to identify program experiences that led to professional learning. Both the student and faculty survey data and case study narratives indicated that the design of the MSSE program has created a program founded in best practices of professional development. The data presented in this chapter illustrated examples of professional learning that occurred through participation in the MSSE program. In addition, the case study narratives provided specific examples of program features that influenced the case study teachers’ professional learning. 286 CHAPTER 5 FINDINGS, CONCLUSIONS, AND IMPLICATIONS Introduction This chapter reports findings, conclusions, and implications of this study. The first section is a summary of the study that includes a review of the problem and the research questions. A brief overview of the focus of the literature review is included. The data collection methods are also reviewed in this section. The next section summarizes the findings of this study and includes a review of the statistical analysis and qualitative analysis. The conclusion section presents evidence to inform each of the research questions. The implications section addresses how this study can inform the work of other MSSE and graduate programs for science teachers, administrators, professional developers, and educators selecting a graduate program. The final section offers suggestions for future research. Summary of the Study The purpose of this study was to explore the professional learning of participants of a science and pedagogical content knowledge-based graduate degree program, in particular the MSSE program, through a concurrent mixed-methods design. The study examined such a program from three perspectives: professional learning through professional development, recommendations for a quality graduate program, and accessibility for geographically distributed participants. The literature review provided 287 recommendations for effective professional development, graduate programs, and a framework for online learning. The literature on professional learning was used to develop the surveys and interview questions. Data was collected from surveys administered to both MSSE students and faculty. In addition, case study data was collected in the form of interviews and portfolios from six MSSE students. This study sought to answer the following research questions. Research Question 1: How do the characteristics of the MSSE program compare to 1) research and policy recommendations for graduate education and 2) to professional development standards for teachers? a. What are the formal program requirements for the MSSE program? b. What is the intended program experience? c. How do these formal requirements and intended experiences align with recommendations for graduate education programs and teacher professional learning in research and policy literature? Research Question 2: How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? b. What program experiences, from the program teachers’ perspective, contribute to participants’ professional learning? 288 c. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? d. What evidence of teacher professional learning can be identified? Research Question 3: How does the distance mode of delivery of this science and pedagogical content knowledge-based graduate program impact the science teachers’ professional learning? The literature outlined recommendations for professional learning, professional development, and recommendations for quality graduate programs. Figure 19 provides a summary of the recommendations. Figure 19. Essential Components for Professional Learning, Graduate Programs and Professional Development as Described by the Research Literature and Policy Recommendations. Included in the population for this study were the MSSE students, current and graduates, and MSSE faculty. The MSSE students were asked to complete the Student 289 MSSE Professional Learning Survey. The survey had a 60% response rate. The intent of the survey was to elicit perceptions about the professional learning experiences in the MSSE program. From the survey respondents, six case study participants were selected through maximum variation sampling. The case study participants were interviewed and completed a portfolio to that was used to provide additional insights and evidence of professional learning. The faculty completed the Faculty MSSE Professional Learning Survey. The intent was to collect data from the faculty perspective on the MSSE students’ professional learning. The survey was administered to all faculty with a 75% response rate. The data from the surveys and the case studies along with recommendations from the literature were used to analyze, synthesize, and evaluate the results. Findings This section presents a review of all of the findings from the statistical analysis of survey data as well as the qualitative analysis of the open-ended survey questions and the case study data. In keeping with the same order and format of the discussion of the data collections procedures in Chapter 4, the following review of results are presented and discussed in three sections. 1. The Student MSSE Professional Learning Survey 2. The Faculty MSSE Professional Learning Survey 3. Case Study Findings 290 A brief, explanatory introduction is given at the beginning of each section, followed by a review of the results, and concludes with a brief interpretation of the results obtained in that specific area of the study. The Student MSSE Professional Learning Survey The MSSE Professional Learning Survey (MPLS) was administered to 611 MSSE students, both current students and graduates. There were four sections to the survey. The first section collected demographics, professional experiences, and education information about the MSSE students. Section II of the survey focused on students’ perceptions of the degree to which the program fostered their science content understanding and knowledge. Section III concentrated on students’ perceptions of course opportunities that were provided that fostered the professional learning of the MSSE students. Section IV examined the MSSE students’ knowledge gains and change in practice, skills, and knowledge with respect to the types of courses offered in the program. An analysis of Section II indicated that, according to students’ perceptions, the MSSE program, through course experiences, strongly influenced the ability to articulate scientific ideas. Opportunities to reflect upon earlier scientific ideas, reflecting upon the scientific ideas of other students, and opportunities to analyze and draw conclusions from data, observations, and other forms of scientific evidence were areas reported to be influenced through course experiences that enhanced students’ science knowledge. “(Course) helped with true data analysis and drawing conclusions based upon my own research,” remarked one student. Students’ responses in this section of the survey, 291 however, indicated that two learning opportunities were much less influential in the area of learning science content perhaps because they appeared to occur infrequently in the MSSE courses. These included the practice of working through problem sets with pen and paper and using computer based animations, games, or simulations. It should be noted that similar results were reflected in the faculty survey. Section III of the Student MPLS focused on professional learning. Four themes emerged from the data that were used to describe the findings. These themes identified by the researcher included Research and Best Practice, Collaboration, Reflective Practice, and Supporting Student Achievement. Both the students and faculty reported that course experiences influenced the broad area of research and best practice within professional learning. Research and best practice included reading research (practitioner reports and journal articles), using the Internet as one source of ideas, and utilizing the research findings about best practice as a means to influence teaching practice. These areas of professional learning were all influenced by the course experiences. As one student stated, “I can make sure that my teaching is agreement with best research practices and is effective for the learning of my students.” Both the faculty and student survey results indicated that using the Internet, reading research, and the use of learning by the MSSE teachers to improve their students’ learning were perceived to be areas that were fostered through program experiences. Collaboration was also a theme that emerged from the data. Collaboration included offering colleagues support, discussions with colleagues, and informal opportunities to share experiences and were areas of professional learning that were 292 influenced by course experiences. The “wonderful peer collaborations and extremely helpful instructional strategies” offered different perspectives for the MSSE students. In addition, opportunities to engage in reflective practice were rated as an area influenced by course experiences. Reflection was described as a way to identify professional learning needs and to influence change and improve teaching practice and was an area that appeared to be influenced by the program experiences. Lastly, supporting student achievement through encouragement, creating positive learning experiences, and believing all students can learn was rated as an area that was influenced through the MSSE program. Science content knowledge and understanding was another dimension of professional learning. Results indicated that most participants perceived the MSSE program experience greatly influencing the area of science content knowledge. Another area in which respondents reported a strong influence was in assessing students in science. One student reported that understanding of, “Assessments as a way of monitoring and helping students identify what they have learned,” was influenced by program experiences. According to student survey responses, the MSSE program also influenced students in the area of using inquiry, investigative teaching strategies, understanding their students’ thinking in science, and providing a challenging curriculum for their students. Results from the faculty surveys were more mixed regarding the extent to which the course experiences they provided fostered students’ ability to design or conduct science investigations. The only major area in which the program was not 293 reported as influential was fostering the ability to teach science to diverse student populations. Section IV of the survey was designed to gather information about the MSSE students’ learning gains and change in practice, skills, and knowledge as related to the type of program courses that were taken. Open-ended responses accompanied each of the questions. A one within subjects repeated measures of analysis of variance was used to compare teachers’ mean ratings to determine which type of MSSE courses most influenced their knowledge of science content, pedagogy for teaching science, science curriculum, and the ability to support diverse learners. The MSSE program courses can be broken down into six basic types that were used for comparison: on-campus science lab based courses, on-campus science field based courses, online science courses, education required courses, education elective courses, and the capstone project. In the area of science content knowledge, the statistical findings suggest on- campus lab based science classes were less of an influence on science content knowledge than were on campus field based science courses or online science courses. It may be that the on-campus lab classes were less of an influence as there are significantly fewer such courses offered and taken. The education core courses and the capstone project, which is a product of the core coursework, were rated by teachers as having a significantly greater influence on pedagogy for teaching science than the science field courses. Teachers rated the online science courses as a significantly greater influence on science curriculum knowledge than on-campus based field courses. One student stated, “Online courses where we research a topic & discuss via online chatting, blogging, 294 etc…was most helpful.” The education required core courses, education elective courses, and the capstone project were found to be a significantly greater influence on understanding the needs of diverse students than either the on campus or online science courses. Chapter 4 presents a detailed analysis of all of the findings of the Student MPLS. The Faculty MSSE Professional Learning Survey The MSSE Professional Learning Survey (MPLS) was administered to 32 MSSE faculty with 24 responding. There were four sections to the survey. The questions paralleled the student MPLS. The first section collected demographics, professional experiences, and the educational background of the faculty. Section II of the survey sought faculty perspectives on the MSSE students’ growth in science content understanding and knowledge. Section III concentrated on the professional learning of the MSSE students from a faculty perspective. Section IV sought to determine learning and growth regarding MSSE students’ science pedagogical content understanding and knowledge as viewed by the faculty. The results of Section II indicated that, as perceived by the faculty, the MSSE program influenced the students’ abilities to articulate scientific ideas. Opportunities to reflect on scientific ideas, and on participants’ previously held scientific ideas and analyzing and drawing conclusions from data, observations, and other forms of scientific evidence were also areas perceived by the faculty as being significantly influenced by the course experiences. Faculty responses in this section of the survey indicated that there were two areas that were much less influenced through program experiences. These 295 included using computer-based animations and interacting with physical models. The findings were similar to the results in Section II of the student survey. Given the structure of the survey questions, it is not clear whether experiences like these were rare in MSSE, or whether they occurred but were perceived as having little influence on participants’ knowledge and practice. Section III focused on the dimensions of professional learning. The faculty and students reported that course experiences influenced teachers’ knowledge and practice relating to accessing and using research, and to transferring lessons from the research into one’s science classroom. Both groups reported that course experienced fostered reading practitioner reports and journal articles on science and educational topics and discussions on how this information may be utilized as best practice“(Courses) have really helped guide my questioning in my research and develop my knowledge of pedagogy,” stated one student. These findings mirror the results in the Student MPLS. The area with little to no perceived influence by the faculty was the area of peers working together on joint research, either on a science or educational topic, to inform practice. The design of the MSSE program does not seem to accommodate this practice currently. Collaboration by offering colleague reassurances, reading and responding to feedback from colleagues and discussing learning were all areas reported by the faculty to be influenced as a result of the MSSE program and course design. These results were similar to the Student MPLS results. The faculty rated joint research and team teaching as areas less influenced by course experiences they provided. Once again, this result was similar to the Student MPLS results. 296 Reflection on practice and feedback are the foundation of critical and responsive learning. The area of reflection was a perceived to be strongly influenced by the MSSE program experience. Results from the faculty, reiterated by the students, indicated that the program opportunities influenced the area of reflection. One student described the reflection process, “….the process of trying new ideas, checking students for understanding and then reflection. This reflection is often forgotten but has been proved to be a key component to changing what and how we teach.” Based on the open-ended responses and case study data, it seemed apparent the instructors provide multiple opportunities for reflection for the MSSE students. Another area not perceived to be influenced substantially by course experiences was using insights from the MSSE teachers’ learning in a course to inform or influence the school’s policy development. We can speculate that the action research sequence, which may be tailored to district needs and priorities, could influence the district policy if the teacher was positioned and motivated to do so. In addition, there is a suite of high priority issues that would be relevant in any district in the country. Those issues could be included in the MSSE courses as a means of more strongly influencing policy locally. Lastly, the MSSE program experience was perceived as influential on teachers’ knowledge and practice about creating positive learning environments. The perception of valuing the learning of all students was an area that was influenced by the course experiences as perceived by the faculty. The MSSE students’ results verified this perception of the faculty. The survey, open-ended responses, and case study data confirmed that program experiences influenced the area of creating a positive learning 297 environment. The MSSE students reported in both case study data and the open ended responses that the faculty was responsive to their needs and provided positive constructive feedback that influenced their professional learning and growth. The next section of the survey examined the influence of course experiences on the scientific and pedagogical content understandings of the MSSE students as perceived by the faculty. The faculty rated gave mixed ratings to the influence of course experiences on using inquiry/investigation-oriented teaching strategies. They felt that teachers’ knowledge and practice regarding the use of technology, including digital imagery and probeware, were strongly influenced through course experiences. Through participation in the MSSE program and the associated experiences, the faculty perceived these opportunities influenced the deepening of content knowledge, assessment strategies, and understanding of student thinking of the MSSE students. One student’s comment support this finding when he or she stated, “I have gained tremendously in science content knowledge and how to apply that knowledge to my classroom.” In addition, addressing the needs and challenges of teaching diverse learners was an area in which the program experiences had little or no influence on professional learning as perceived by faculty. These findings were similar to the student survey results. Case Studies The case study data presented a lens into the MSSE program from the student perspective. The MSSE program was influential on the professional learning and growth of each individual in specific and unique ways. In addition, certain program features influenced how each case study participant viewed the program. It appeared from the 298 responses and the portfolios that the dimensions of professional learning were clearly evident. The interview questions were based on the Pedder’s dimensions of professional learning. With respect to the dimension of inquiry, in all of the cases the teachers reported that the MSSE program contributed to their understanding of pedagogical strategies and content through exposure to different sources of evidence, to research on science and on education, and by providing opportunities to apply the research and evidence to improve practice. The social connections and opportunities to collaborate were also reported as a strong influence on professional learning. Pedder described collaboration as the dimension of building social capital. Reflective practice was also a significant part of the program experience and reported as being influential on professional learning by the case study participants. Described by Pedder as critical and responsive learning, this dimension of professional learning was reported by the case study participants to have created a change in practice in their classrooms. The last dimension, referred to as valuing learning by Pedder, was also strongly influenced in the MSSE program. Each of the case study participants sought a master’s degree program that had a focus on science. The course offerings of the MSSE program met the need for science content courses yet offered a perspective for teachers. One case study participant described the two science courses he had taken stating, “I felt (courses) were really outstanding and really helped me to continue to increase my depth of understanding and expand in both of those areas.” 299 The case study participants also described the online component of the MSSE program as critical to their ability to pursue a master’s degree. Each of the case study participants was a busy professional juggling families and extracurricular duties. The asynchronous courses allowed each person to participate at times that were convenient for him or her while maintaining personal and professional obligations and commitments. Lastly, each case study participant described in detail their capstone project and how it influenced their teaching practice. The opportunity to select a project that meets the learning needs of each student allowed for each case study participant to become immersed in a topic that was relevant to their context. The case study data confirmed the survey results in providing rich detail and insights into the MSSE program. Conclusions The following section addresses each of the research questions posed in this study. The literature base that surrounds each question is presented when appropriate. The questions are also addressed in terms of which sources of data contributed to answering these questions. Research Question 1 The first research question that was intended to be answered by this study was: How do the characteristics of the MSSE program compare to 1) research and policy recommendations for graduate education and 2) to professional development standards for teachers? a. What are the formal program requirements for the MSSE 300 program? b. What is the intended program experience? c. How do these formal requirements and intended experiences align with recommendations for graduate education programs and teacher professional learning in research and policy literature? The Council of Graduate Schools (CGS) influences the recommendations for graduate programs. Both the CGS and a research study by Hirumi (2005) identified key areas that institutions of higher education need to address in developing, evaluating, and maintaining quality graduate programs. The CGS (2005) offered three broad components that a graduate program should include: supportive program culture, informed faculty, and a variety of learning experiences for the graduate students. Hirumi’s (2005) study built upon those foundational pieces and organized them into subsets. Conrad, Duren, & Haworth (1998) found that a master’s degree program for teachers was a powerful professional development experience, especially programs that offered learning that applied the classroom based knowledge into a professional setting and the opportunity to put theory and research into practice. Minkel and Richards (1987), in a synthesis of the research literature and through consensus of an expert panel, identifed ten components of a quality master’s degree program. They found when the emphasis was a professional degree the application of knowledge rather than original research was a key measure of success. The Council of Graduate Studies (2005) in the policy statement, Master’s Education: A Guide for Faculty and Administrators, outlined specific program requirements and aspects including a minimum number of courses, a 301 core curriculum to be mastered and a culminating project and a faculty advisor for each student. While all of these components are sugested in the literature, the literature does not present compelling arguments for all of them to be included, nor do the results of this study. In particular, the recommendation to include a comprehensive examination may be more traditon or legacy than based upon research evidence. The recommendations from the literature for a high quality master’s degree program, as compiled by this researcher, are illustrated in Figure 20. Figure 20. Graduate Program Recommendations 302 The professional development literature is quite extensive and offers a vast array of recommendations of what constitutes effective professional development. Garet, Birman, Porter, Desimone, Herman, and Yoon’s (1999) comprehensive national study was one of the pivotal studies that examined the components of effective professional development and was used as the framework for this study. The literature identified two broad components for effective professional development; structural and core components. Structural components include:  Extended and Continuous Professional development: Sustained, extensive contact hours, opportunities for practice and follow up.  Participants in Similar Professional Contexts: Similarity between educators who participate in the program with teaching assignments, content, etc. Core components include:  Content Focused: A focus on deep understanding of disciplinary content knowledge including how to deconstruct the knowledge for students.  Active Learning: Discussions about pedagogical content knowledge, instruction, planning for implementation, analysis of student work, and presenting, reading and writing about the content with others.  Coherence: Connection between the professional development and teachers’ curriculum, standards, and student assessment measures. Another area of the literature with recommendations for professional development came from the professional development standards. The professional development 303 standards relevant to this research study were those from the National Staff Development Council [NSDC] (2001), the National Research Council [NRC] (1996), the National Board for Professional Teaching Standards [NBPTS] (1987), the Southern Regional Education Board [SREB] Online Professional Development Standards, and the National Science Teachers’ Association [NSTA] position statement. These standards overlap with the recommendations for effective professional development. The formal program requirements for the MSSE program and the intended program experience are clearly articulated on the MSSE website (2009). The program identifies the interdisciplinary nature of the program and the opportunity to complete the majority of the courses via distance learning. The website clearly outlines the program requirements and features. With only a few exceptions, the formal requirements and intended experiences of the MSSE program seem to be aligned with recommendations for graduate education programs and teacher professional learning from the research and policy literature. In reviewing the recommendations for an effective graduate program, this research determined that the MSSE program addressed all of the recommendations. One recommendation that was not defined consistently in the literature was the component of research. Minkel and Richards (1987) who examined the differences between a master’s degree that was a stepping stone for a doctoral program and a professional master’s degrees found differences in the focus of the research. The emphasis with the professional degree approach is directed toward the application of knowledge rather than original research that may be the focus of study for a career. The action research that is 304 associated with a professional degree is applied in an immediate context in which the reseacher is also a participant or even a leader. As the MSSE program is intended as a professional degree, that level of research is appropriate for the professional degree. The MSSE program also adhered to the recommendations in the literature for effective professional development. The design of the program and the requirements appear to ensure that professional development opportunities meet the recommendations. Although some aspects of the core components of professional development are addressed in the MSSE program, these aspects could be strengthened. More specifically, there needs to be a stronger emphasis on pedagogical content knowledge a focus on the analysis of student work, and a stronger connection to the MSSE students’ curriculum that they are teaching. Recommendations to enhance this aspect of the MSSE program are found in the implications section. The professional development standards also appear to be addressed within the MSSE program. These standards focus on collaboration, using data to inform instruction, and mastery of content. Research Question 2 The next question this study intended to answer was: Research Question 2: How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers increase their professional learning because of their participation in such a program? 305 b. What program experiences, from a program teachers’ perspective, contribute to participants’ professional learning? c. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? d. What evidence of teacher professional learning can be identified? Five dimensions of professional learning, which were used for the analysis in this study, were identified in the research literature. Pedder’s comprehensive research indicated four dimensions. Pedder (2007, p. 235) identified the dimensions as inquiry, building social capital, critical and responsive learning, and valuing learning. Other research studies that supported the dimensions of professional learning identified by Pedder included Day and Gu (2007); Bolam, McMahon, Stoll, Wallace, and Greenwold, (2005); Bransford, Brown, and Cocking (1999); National Center for Research on Teacher Learning (1995); Elmore and Burney (1997); Department of Educational Training (2005). These other studies added one additional component that Pedder did not identify. This fifth dimension was content knowledge and it was determined that it must be included for effective professional learning to occur especially for studying a program like MSSE which has a strong emphasis on science content. Two surveys were administered in this study. The surveys were designed to collect data on each of the dimensions of professional learning that may be evident in the MSSE program. The first was the MPLS for students. It was used to ask the students about their perceptions of professional learning in the MSSE program. Four open-ended questions allowed respondents to provide more detailed evidence of professional 306 learning. In addition, six case studies provided more in-depth perspectives of professional learning and evidence as demonstrated in a submitted portfolio. A second parallel survey was administered to the MSSE faculty. This survey sought to understand, from a faculty perspective, the MSSE students’ professional learning. The results of this survey and the student survey are found in Chapter 4. As the results of the faculty and student survey were very similar they are used in tandem to answer Research Question 2. The first dimension of professional learning identified in the literature and used in this study was inquiry. Pedder (2005) defined inquiry as using and responding to different sources of evidence through reading research and using the Internet as well as carrying out joint research and evaluations with colleagues. According to survey responses, the MSSE program strongly promoted professional learning with some aspects of inquiry, in particular applying research on best practices. The open-ended responses identified both specific courses and experiences in which research and best practices were influential on professional learning. The case study data supports this finding with examples from the portfolios that added strong additional evidence of this dimension. In terms of research and best practice, the surveys, open ended responses, and case study data clearly indicated that these aspects of professional learning were promoted in the MSSE program. The only aspect of the dimension of inquiry identified by Pedder that was not perceived as being influential was joint research or team teaching. Due to the program design, collaboration with other MSSE students to conduct research is not an expected outcome. As participants are from all over the US, as well as some 307 international students, this would be a difficult, yet not impossible, aspect of professional learning to address. The original research on professional learning, on which the survey was based, focused on groups of educators who taught in the same school and district. This was very different from the population in the MSSE program. Some may conclude that this is not an aspect of inquiry that is appropriate for this program, yet schools that are geographically separated face many of the same issues and capstone studies involving multi site studies of a shared issue could certainly be conducted via the Internet. Cross- site educational studies are complicated, yet rewarding, and present a unique opportunity for programs like MSSE. The next dimension of professional learning identified in the literature and used for analysis in this study was building social capital or learning, working, supporting, and talking with colleagues regarding teaching and student learning. This study confirmed that this aspect of professional learning was strongly encouraged and promoted in the MSSE program. This dimension was categorized as collaboration in this study. Both students and faculty reported that collaboration to develop and share ideas was a program experience used widely in the MSSE program coursework. Online discussions were often cited as the program components that supported collaboration. Sharing experiences and providing feedback to others was an integral part of the discussions, both formally and informally. One case study participant described how the online discussions allowed him for the first time in his higher education experience to have a voice in classroom discussions. Throughout the open-ended survey responses it was made clear that opportunities to collaborate with others in the program, other than in summer field 308 experiences or online discussion, were not provided frequently. However, this omission was not reported to have a particularly negative effect on the professional learning that took place. The only components of building social capital identified in the literature that were not identified as being influential in the MSSE program were those that focused on joint research and evaluation or team teaching. Again, the demographics of the MSSE program, and the design of the program may not promote that aspect of building social capital. As stated previously, the literature that provided the framework for this study examined professional learning within the context of a district or school rather than geographically distributed participants. So in one sense it is not surprising that this aspect of professional learning is not included in the MSSE design. Yet in another sense this presents a promising opportunity waiting to be taken by distance programs like MSSE. Critical and responsive learning occurs through reflections, self-evaluation, experimentation, and by responding to feedback. This was the third dimension used in this study to examine professional learning. It is referred to as reflective practice in this study. Both faculty and students reported this was an area influenced by course experiences within the MSSE program. Teachers discussed experiences and practices and offered each other suggestions and ideas to implement in their practice. The feedback that the MSSE students received from instructors and other students often created a change in practice as reported in the data. One of the case study participants 309 described how the feedback she received from an instructor caused her to reflect on how she provided feedback to her own students. Reflective practice was a theme in the open-ended responses, and was cited as an influential part of the MSSE program. All case study participants provided specific examples of how the reflective process impacted their teaching practice. Through reflective practice, teachers were able to identify their own professional learning needs. Opportunities to engage in reflective practice provided insight into the teachers’ practice and were reported by both faculty and students to be an inherent part of the MSSE program. Academic success is noteworthy and the successes of students and teachers should be recognized. This was one aspect of valuing learning, deepening understanding and learning through talking about and valuing learning with others, a fourth dimension of professional learning identified in the literature and explored in this study. This aspect was sometimes referred to in this study as supporting student achievement. Within this dimension, teachers are committed to the concept that all students are capable of learning, they find ways to recognize, even celebrate, student learning, and they themselves model life long learning. The attitude that learning should be an enjoyable experience for all involved is evident in their classrooms. As reported by the student and faculty respondents, valuing learning was overwhelmingly promoted and embedded in the MSSE program. A case study participant described how one instructor went out of the way to provide an experience of working in a science research lab. Her previous lab experience was weak and she wanted an opportunity to improve her skill set. The instructor not only 310 made the arrangements for her, but made her feel valued by his actions. Once again, both the open-ended responses and case study data provided specific examples of how the MSSE program created a positive learning experience for the students. This was a practice that was then emulated in the MSSE students’ teaching practice. Both the research literature on professional development and the research on professional learning identify content knowledge as a dimension of professional learning. This was a fifth dimension used for analysis in this study. The MSSE program allows students to “develop interdisciplinary combinations of science courses” (MSSE, 2009) for their graduate programs of study. The analysis of faculty and student surveys indicated that content is addressed in the MSSE program from multiple angles. These include the content of the course, the instructors, and the collaboration with other MSSE students. The case study data and open-ended responses provided specific examples of courses and experiences that influenced the content knowledge of the MSSE students. Evidence of teacher professional learning in all of the dimensions was identified in both the open-ended responses to the student survey and the case studies. Chapter 4 of this study provides excerpts from the open-ended responses that specifically illustrate the dimensions of professional learning that helped in answering the research questions in this study. The portfolios submitted by the case study participants offered a wide range of evidence to support these dimensions of professional learning. These included PowerPoint presentations, a blog, student projects, and assessment tools. Some portfolios included student work samples that illustrated the transfer of learning to the student level. 311 Analysis of the survey and case study data clearly indicated that experiences in the MSSE program contribute to the professional learning of its participants. Research Question 3 The final question this study intended to answer was: How does the distance mode of delivery of this science and pedagogical content knowledge-based graduate program impact the science teachers’ professional learning? Distance learning provided opportunities for professional development that may not have been available to teachers separated geographically from a campus. In addition, distance learning offered flexibility and convenience for educators who have busy lives and multiple levels of responsibilities. The MSSE program offers participants the opportunity to complete a master’s degree with a minimal amount of time on campus. In examining the demographics of the MSSE students, participants represented all fifty states, as well as 20 foreign countries. This was a testament to the access of the program to a geographically distributed audience. The program can be customized to meet the needs of each individual almost regardless of where they reside. The strongest evidence to support the influence of the distance mode of delivery came from the case studies. Several of the case study participants remarked that the online component of the program was what allowed them to pursue a master’s degree and personal professional learning goals. Family obligations, extracurricular activities, or limited or no access to a university or other higher education facility nearby were cited as the reasons that the MSSE program was beneficial to their continuing education. As one participant stated, “if it weren’t for the online course, I couldn’t get a master’s degree.” 312 In addition, the online discussions offered the opportunity to reflect on and think about issues before responding to discussions. This experience is unlike opportunities in the face-to-face classroom. Implications The following section addresses the implications of this study. The implication section includes recommendations for those designing research-based professional development for science educators. Within these recommendations are issues that are worthy of consideration. From a larger perspective, the study may offer insights into the design and development of future pedagogical, content focused graduate programs. The study added to the literature on effective science and pedagogical content knowledge- based graduate degree programs. The results may inform similar programs regarding effective policies, procedures, and practices. In addition, the study may benefit professional developers in creating programs to influence effective professional learning for graduate level professional development for science teachers. The study also explored the aspects of the MSSE program to inform the MSSE program staff of the program’s influence on professional learning. Recommendation for a Comprehensive, Research and Experience-Based Framework for Graduate Level Professional Development for Science Teachers Past research and literature on professional development, professional learning and graduate programs clearly outlined recommendations for the design and implementation of effective programs. These guidelines should serve as the starting 313 point for the initial design and architecture of a graduate program. As seen in this study, these same recommendations can be used as a method to examine program effectiveness. While there are overlaps in the recommendations from the Council of Graduate Studies and other higher education policy makers, there are common aspects across the research and policy literature that might be used to influence professional learning. Professional developers and administrators of graduate programs should look to these policy recommendations and embed those features in their programs if their intention is to create a change in practice through professional learning. While the research on graduate programs from the student perspective is small, there are few recommendations in those existing studies that specifically address graduate programs from a students’ perspective. Those recommendations co-exist with the recommendations in the graduate program recommendations. They can be used as another layer of features to include in the design of professional development. The MSSE program whether intentionally or serendipitously has clearly embedded many of these recommended features and serves as a model program. Recommendation to Explicitly Strengthen the Focus on Student Learning in Graduate Level Professional Development for Science Teachers The education climate today demands high levels of accountability from all stakeholders. In addition, it is expected that professional development outcomes include some measures of impact on student achievement. The open-ended responses included references to examining student achievement data and how to use that data. One student remarked, “The (course), which was an elective, also helped me to see and try different 314 techniques to use for student achievement.” Another student commented, “My capstone project data forces me to look at support for diverse learners.” Guskey (2000) advocated that student learning outcomes should be the starting point of any professional development. Thoughtful planning of a professional learning experience should include connecting the professional development experience to student achievement. Within the MSSE program, this could be accomplished by adding K20 student outcomes as a part of all course assignments. The outcomes could be framed as what the MSSE participants’ students should be able to do after completing a learning experience or sequence designed by the MSSE student. In addition, examination of K20 student achievement data could be a required element in all course assignments, or at least within a major assignment within each course. If the goal of professional learning is to create a change in practice that influences student achievement, student achievement should be embedded in all components of the MSSE program that require any type of lesson planning or research on students’ learning. Another pedagogical strategy recommended by the literature is to examine student achievement through the process of analysis of student work. This recommendation was included as a part of active learning. Garet et al. (2007) advocate that the analysis of student work offers teachers the opportunity to not only identify learning gaps, but to begin to think about differentiation to meet the needs of their students. It was not clearly apparent in the data that analyzing student work was an integral part of the MSSE program. Assignments that require the analysis of student work should be an integral part of 315 any professional development at the graduate level. There are recommended processes for analyzing student work in the professional development literature and this is an integral part of the National Board Certification process. Establishing a program-wide protocol for the analysis of student work as part of a lesson design assignment would further address meeting the needs of a diverse student population. A core component of professional development is active learning that includes not only the analysis of student work, but also planning and implementing different learning strategies carefully designed to support students’ science learning. Creating a process of analysis of student work and including the other aspects of active learning would be a way of strengthening the MSSE program. An offshoot of the analysis of student work is creating differentiated learning experiences for students with varying academic preparation, cultural background, special needs, and ways of learning. Although the open-ended responses indicated that differentiation was a topic that was broached, it was apparent it was not addressed explicitly. Differentiation of learning experiences should be an added component to assignments that require the MSSE students to develop a lesson plan or module. The differentiation should be extensive and inclusive of all students. There were a number of open-ended student survey responses in this study reporting that identified differentiation for low achieving students, including students from diverse culture, language and income groups, or special education students was not addressed by the MSSE program. This omission probably occurs in many professional development and graduate programs for science teachers, and certainly needs to be addressed within MSSE or wherever a similar 316 omission occurs. Yet only one comment focused on high achieving students. As the MSSE program attracts a large number of teachers who teach AP course and higher-level courses, it would be an enhancement to the program to explicitly discuss differentiation for high achieving students. Recommendations for MSSE Faculty and Program Staff The participants view the MSSE program positively. The surveys, open-ended responses, and case study provide both qualitative and quantitative evidence that the MSSE program influenced professional learning. There are still recommendations that were offered by the students and supported by this researcher to improve and enhance the program even more. There are also recommendations from the literature that could be implemented to enhance the MSSE program. Focusing on content is a dimension of professional learning recommended in the professional development literature. One aspect of science content knowledge is the knowledge of science curriculum, including how to use resources and arrange topics over time. Curriculum as defined by the National Research Council is not an explicit part of the MSSE program, except in one elective course on this topic. The professional development literature recommends coherence, for example, by making connections between the professional development and the teachers’ curriculum, standards, and assessment measures. Given the fact that the program caters to a national and international audience, traditional curriculum development may not be an appropriate shift in the goals of the program. Rather, it may be more appropriate to link the content of the courses to the 317 curriculum and standards being used in the school district of the MSSE students. Being more explicit in expecting students to make the connection between their learning and the curriculum they are teaching offers an opportunity to influence the professional learning of the MSSE students. In addition, it may also benefit the MSSE students for the instructors to address the sequencing of the content in the courses. Explicitly stating why concepts are being sequenced and taught in a specific order, that is addressing learning progressions that faculty experiences or the research literature have shown to be effective, may offer a deeper understanding of the content. Sequencing content may be a component of science curriculum that perhaps should be more transparent in the design of the science courses. Connected to content and curricular knowledge is pedagogical content knowledge (PCK). While an overwhelming number of open-ended responses expressed that the MSSE program influenced the pedagogy of teaching science, no responses clearly described PCK. Addressing PCK in the science courses would add a more focused learning experience that would be more transparent for the MSSE students to use in their own classrooms. The professional development literature identified core components including a content focus, active learning, and coherence. Each of these components clarifies that deepening the content knowledge must include a deconstructing of that knowledge so it can be taught effectively to students. Discussions about PCK and planning instruction are essential is creating a change in practice. Connecting the content and the PCK to the MSSE students’ curriculum would strengthen the program. In both the faculty and student surveys, an area that was not addressed strongly in 318 the MSSE program was learning how to meet the needs of a diverse student population. Diverse learners range from low achieving to academically gifted students. Such learners may also include students with varying cultural backgrounds, students identified with special needs or requiring modifications or adjustments to assignments, and varying learning styles. While there are some references that this aspect is included in the education courses, it is an area that needs to be included as part of all courses. Only four science courses were identified in the open-ended responses as offering recommendations to meet the needs of diverse learners. Addressing diverse learning needs of students cannot be an exclusive practice in the education courses. In an era of increased accountability, teachers are expected to differentiate instruction to meet the needs of all students who may be diverse learners. While differentiation may begin to address the learning needs of most students, students identified with special needs require a more intensive approach. As more and more special education students are included in the general education classroom, there needs to be an increased awareness on adapting curriculum and making modifications. The data analyzed in this study found that the MSSE program influenced the professional learning of the MSSE students. From the open-ended responses and case study data, it was apparent that pedagogical strategies were an integral part of the program. It also appears that the formal program curriculum has a far greater influence on teachers’ thinking about assessment than on any other aspect of pedagogy. The assessment course is a required course that may have contributed to the significant number of comments about the influence of assessment strategies. It is recommended 319 that the MSSE program develop as powerful and coherent an approach to other aspects of teaching and learning. For example, this same emphasis could be placed on inquiry science teaching that is so central in the national standards. Technology integration would be another area that may be emphasized. The courses offered in the MSSE program are designed to meet the needs of a wide array of science educators. Many of the participants in the program may have may entered with a broadfield degree or may be teaching out of their content area. There is a broad range of science discipline specific courses offered, yet a number of MSSE students would recommend that more chemistry courses be made available. Currently there are nine chemistry courses offered. In reviewing the demographics of the MSSE enrollment, chemistry and broadfield science teachers make up a large proportion of the MSSE students. As chemistry is a challenging subject, the need for more chemistry courses, especially for those who teach it, may be warranted. It may be that there are enough chemistry courses offered, but an emphasis on pedagogical content knowledge in those that are offered would address the perceived need for more chemistry. It may be that the request stems from a need to better understand how to teach chemistry effectively. One of the core components of professional development is a focus on content and understanding how to deconstruct the knowledge for students. The experience level of educators enrolling in the program is vast as well. This creates chasms of understanding in both content and pedagogy. For the experienced educators who have engaged in a variety of professional development, the pedagogical strategies and/or the content information may not be new. Differentiating the 320 assignments and discussions and allowing students to serve as discussion leaders at times may offer those MSSE students who have higher levels of expertise to challenge their learning. This would also support the dimension of building social capital and the active learning recommended in the professional development literature. Many MSSE students may enter the program with diverse learning needs and abilities. It may be reasonable to assume a certain percentage of MSSE students have been identified as having a learning disability or other health impairments that impact their ability to participate in certain course experiences or their academic performance. Access to learning opportunities should be available to all who meet the graduate entrance requirements. The academic rigor does not need to be diminished, but rather differentiation for the MSSE student should be a part of any course to meet the adult learning needs of the participants. The MSSE program strives to differentiate at the program level, that same approach may be a consideration at the course level. Offering options for assignments or using different types of textual material to present content may support different learning styles of the MSSE students. Also, students with physical disabilities may be excluded from some learning opportunities like field courses. Appropriate accommodations should be offered to all students who have the mental aptitude for a course. The addition of science courses designed specifically for elementary teachers – for example, by focusing on national science education standards at their grade levels – was perceived as a need, as more elementary teachers have begun enrolling in the program. There seems to be a desire for more science courses with an elementary focus 321 particularly with a focus on highly qualified teachers and increased accountability Considerations for Teachers Selecting a Master’s Program Teachers who are exploring master’s degree programs should review the recommendations for a science and pedagogical content knowledge-based graduate degree program. These recommendations would inform the decision when selecting a master’s program. Science educators, especially those teaching multiple subjects, need to have a high level of understanding of content across many disciplines. Teachers should select a program that embeds the dimension of professional learning that includes a strong content component. Most teachers who seek a master’s degree seek a professional degree rather than a stepping-stone to a doctoral program. The program selected should include the recommendations for graduate education that support a professional master’s degree. These recommendations included specific program requirements and aspects including a minimum number of courses, a core curriculum to be mastered, a culminating project and a faculty advisor for each student. Future Research The following sections address areas of future research. In the review of the literature, it was found that the research in the areas of teachers’ professional learning leading to changes in practice, as well as research on the design of graduate programs for teachers, was very limited. As the MSSE program has not been the subject of many research studies, recommendations for future research on the MSSE program are also included. 322 Additional Research on Professional Learning The term professional learning is often used in the context of education in a more colloquial manner. The National Staff Development Council, the largest non-profit staff development association, has recently changed their purpose to, “Every educator engages in effective professional learning every day so every student achieves (NSDC, 2010)”. While a more detailed examination of the purpose included several of the dimensions of professional learning from the literature, there were still some discrepancies. The largest study on teachers’ professional learning and its dimensions came from the work of Pedder et al. (2005) in England. No large-scale studies on professional learning have been conducted in the United States. The focus of this study examined one aspect of professional learning in a graduate program, but there is room for additional studies on professional learning within school districts and other professional development programs. The additional research on professional learning was primarily from case studies. Most of the case studies offered limited understanding about teachers’ professional learning. The ultimate outcome from professional learning is a change classroom practice. The research literature reviewed for this study speculated on the influence of professional development experiences on teachers’ practice, but offered no corroborating evidence, particularly at the student level. Futher studies are needed to contribute to understanding how professional learning through professional development can influence a change in practice. Continued research through case studies examining student artifacts 323 would begin to create a better understanding of how professional learning of the teacher transfers to the learning of the student. Additional Research on Graduate Programs There are a number of policy advocates and organizations that make recommendations for graduate programs such the Council of Graduate Schools. These organizations serve to inform high education administrators on policy for compliance and quality. Yet the research on how these recommendations influence graduate programs are limited. The recommendations intent to set high standards, yet there is no measure of impact or effectiveness built into those recommendations. There is a lack of research on the student experience and perspective in graduate programs. The research that exists primarily consists of overviews of graduate programs in general. A few case studies are included in the literature, but additional research from the student perspective would add to the literature on effective graduate programs. Additional Research on the MSSE Program To date, there have only been two other studies on the MSSE program. Graves’ (2002) study examined the action research component of the MSSE program. The NTEN study by Horizon (2001) only looked at a subset of the MSSE courses. Based on the findings of this study, the MSSE program stands as an exemplar in professional learning and could serve as the basis for additional studies on professional learning. The case study data presented in this research began to tap into the transfer of their teachers’ learning to the students of the MSSE teachers. Additional research 324 examining the influence of the program on the MSSE students’ classrooms would inform the program staff of the effectiveness of courses and content. Student work samples, analysis of student work, student achievement data, and pre and post data of the MSSE teachers’ students would all be useful in determining the nature and extent of the transfer of learning. In addition, the interview questions focused on the dimensions of professional learning, which do not explicitly address pedagogical content knowledge. Additional questions about the pedagogical content knowledge addressed in the courses would have added depth to understanding the influence of PCK on the MSSE students’ practice. After reviewing the data collected in this study, modifications to both the student and faculty MPLS should be considered. Within the survey questions, the open-ended responses could be restructured to elicit more substantive answers from respondents. Open-ended questions should be included in the faculty MPLS which may offer additional insights into the faculty perspective. Combining the content questions in Section II and the pedagogical and content questions in Section III could make the survey more concise. The questions in these two sections are similar and combining the two would provide additional clarity to the intent of the questions. There were questions on the MPLS that focused on joint research with a peer and team teaching. Although there are creative ways in which those may be addressed, the questions should be rephrased to elicit more responses on collaboration, reflection, or best practice. These types of questions may provide stronger insights into the dimensions of professional learning. 325 The timing of the survey may have influenced the response rate. The final student MPLS was not administered until July, a time when many teachers are on vacation or infrequently checking email. As many email addresses were sent to schools, the emails may have been inadvertently deleted or forgotten over the summer. Similar timing issues may have impacted the response rate of the faculty survey. The faculty perspective on the MSSE students’ professional learning was included in this study. Research on the professional learning of the MSSE faculty as a result of their program experiences is another potential area of informative research. As many of the instructors are practicing scientists who are working closely with K12 educators for the first time since their own school days, further study of the influence the teachers have on faculty professional learning about science teaching and learning would be intriguing, and round out our understanding of the learning and growth of faculty and students in the MSSE program. In this study, the case study data presented artifacts demonstrating professional learning from the perspective of the MSSE students. Case study data provided by the faculty that would include artifacts of the intended professional learning of students and evidence of such may offer additional insights into the professional learning of the MSSE students. Summary This study explored the professional learning of participants of a science and pedagogical content knowledge-based graduate degree program through a concurrent mixed-methods design. The study examined the program from three perspectives: 326 recommendations for teachers’ professional learning through professional development, recommendations for a quality graduate program, and accessibility, as the program is offered online. A science content-based program for teachers, such as MSSE, has the potential to create strong science education leaders through the collaboration with other teachers and scientists and the lab and field experiences offered in the courses. Using student and faculty surveys and case study data from the MSSE students, the results indicated that the MSSE program influences professional learning, including changes in the participants’ own classroom practice. Specific descriptive statistics to support this statement are found in Chapter 4. Case study data that includes portfolio evidence of professional learning is also found in Chapter 4. There were five dimensions of professional learning identified in the literature: inquiry, building social capital, critical and responsive feedback, valuing learning, and content. All five dimensions must be present in a professional development experience if the expected outcome is a change in teaching practice. The MSSE program included all five dimensions in the design and delivery of the program. Professional organizations for graduate programs, and advocates for the value of a graduate education, recommend policy to institutions of higher education. A review of the policy recommendations indicated there are ten components of an effective graduate program. The MSSE program included all of them in the design of the program, except the comprehensive examination. The comprehensive examination may be a remnant of tradition rather than supported through any research. The MSSE program offers many of its courses through distance learning 327 asynchronous, computer-mediated communication. The flexibility of distance learning allows access for educators who may not be able to pursue a master’s degree any other way. The MSSE students represent all fifty states and twenty foreign countries. These demographics speak loudly to the opportunity created by the MSSE program to geographically distributed participants. Designing a science and pedagogical content knowledge-based graduate degree program that adheres to the recommendations in the literature for professional learning, graduate programs, and access through distance education requires vision and commitment from all levels of an institution of higher education. Based on the results of this study, the MSSE program emerges as a program that has done this successfully to influence the professional learning of its graduates. 328 REFERENCES CITED 329 American Council on Education. (2008). Strategic Plan. Retrieved February 3, 2008, from American Council on Education: http://www.acenet.edu American Distance Education Consortium. (2002). ADEC Guiding Principles for Distance Learning. Retrieved February 3, 2008, from American Distance Education Consortium: http://www.adec.edu American Educational Research Association. (2005). Teaching teachers: Professional development to improve student achievement. Research Points , 3 (1). Anderson, T., Rourke, L., Garrison, D., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5 (2). Asbell-Clarke, J. & Rowe, E. (2007). Learning Science Online: A descriptive study of online science courses for teachers. Journal of Asynchronous Learning Networks 11(3) Association of American Universities. (1998). Association of American universities committee on graduate education: Report and recommendations. Ball, D. (2000). Bridging practices -intertwining content and pedogogy in teaching and learning to teach. Journal of Teacher Education , 51 (3), 241-247. Bender, T. (2003). Discussion based online teaching to enhance student learning: Theory, practice, and assessment. Sterling: Stylus Publishing LLC. Bezzina, M. (1994). Empowering the principal through professional development. Paper presented at the annual conference of the Australian Teacher Education Association. Bice, L. (2005). Construction of knowledge about teaching practice and educating students from diverse cultures in an online induction program. Unpublished Doctoral Dissertaion . Montana State University. Bolam, R., McMahon, A., Stoll, L., Thomas, S., Wallace, M., Greenwood, A., (2005). Creating and Sustaining Effective Professional Learning Communities. Universities of Bristol, Bath and London Institute of Education. University of Bristol. Borasi, R., & Fonzi, J. (2002). Foundations (vol. 3): Professional development that supports school mathematics reform. National Science Foundation. Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher , 33 (8), 3-15. 330 Bransford, J., Brown, A., & Cocking, R. E. (1999). How People Learn: Brain, Mind, Experience, and School. The National Academies Press. Chickering, A. & Ehrmann, S. (1996). Implementing the seven principles: Technology as lever. AAHE Bulletin , pp. 3-6. Chickering, A., & Gamson, Z. F. (1987). Seven Princliples of good practice in undergraduate education. AAHE Bulletin , 39 (7), 3-7. Cohen D, Crabtree B. "Qualitative Research Guidelines Project." February 2009. http://www.qualres.org/index.html Commission on Colleges. (2007). Distance Education -Policy Statement. The Commission on Colleges Southern Association of Colleges and Schools. Conrad, C., Duren, K., & Haworth, J. (1998). Students' perspectives on their master's degree experiences: Disturbing the conventional wisdom. New Directions for Higher Education , 101, 65-76. Costello, A., & Osborne, J. (2005). Best practice in exploratory facotr analysis: Four recommnedation fro getting the most form your analysis. Practical Assessment, Research, & Evaluation, 10 (6), 1-9. Council of Graduate Schools. (2007). Banff Principles on Graduate Education. Washington, D. C.: Council of Graduate Schools. Council of Graduate Schools. (2007). Graduate Education The Backbone of American Competitiveness and Innovation. Council of Graduate Schools. Council of Graduate Schools. (2006). Public Policy. Retrieved February 3, 2008, from Council of Graduate Schools: http://www.cgsnet.org/Default.aspx?tabid=142 Council of Graduate Schools. (2005). Master's Education - A Guide for Faculty and Administrators. (D. Denecke, & P. Tate, Eds.) Washington, D.C.: Council of Graduate Schools. Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods Approaches (3 rd ed.). Thousand Oaks, CA: Sage. Creswell, J. W. (2007). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage. Darling-Hammond, L., & McLaughlin, M. W. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan , 76 (8), 597-604. 331 Darling-Hammond, L. (1996). The right to learn and the advancement of teaching: Research, policy, and practice for democratic education. Educational Researcher, 25(6), 5-17. Day, C., & Gu, Q. (2007). Variations in the conditions for teachers’ professional learning and development: sustaining commitment and effectiveness over a career. Oxford Review of Education , 33 (4), 423-443. Desimone, L., Porter, A., Garet, M., Yoon, K., & Birman, B. (2002). Effects of professional development on teachers' instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy , 24 (2), 81-112. Denzin, N. (1978). The Research Act A theorectical Introduction to sociological Methods (2nd Edition ed.). New York, NY: McGraw-Hill . Dotterer, R., Klasen, J., & Smith, S. (2001). Volume 1 Participant Satisfaction with NTEN and Overview of Impacts. Horizon Research. Eaton, J. (2006). An Overview of U.S. Accreditation. Council for Higher Education Accreditation (2006) . Elmore, R., & Burney, D. (1997). Investing in teacher learning: Staff development and instructional improvement in community school district #2, New York City. National Commission on Teaching & America’s Future. Extension, University of Wisconsin. (2006). Enhancing the educational impact of distance learning. Retrieved November 5, 2006, 2006, from http://www.uwex.edu/disted/cybela.cfm Fishman, B., Marx, R., Best, S., & Tal, R. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education , 19, 643-658. Fulford, C., & Zhang, S. (1993). Perceptions of interactions, the critical predicators in distance education. The Americal Journal of Distance Education , 7 (3), 8-21. Fullan, M. (2006). Leading professional learning: think 'system' and not 'individual school' if the goal is to fundamentally change the culture of schools. School Administration , 10 (5). Garet, M., Birman, B., Porter, A., Desimone, L., Herman, R., & Yoon, K. (1999). Designing effective professional development: Lessons from the eisenhower program. U.S. Department of Education. 332 Garrison, D. R. (2003). Cognitive presence for effective asynchronous online learning: The role of reflective inquiry, self-direction and metacognition. In J. Bourne & J. C. Moore (Eds.), Elements of quality online education: Practice and direction. Volume 4 in the Sloan C Series, Needham, MA: The Sloan Consortium. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: computer conferencing in higher education. The Internet and Higher Education , 2 (2-3), 87-105. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence and computer conferencing in distance education. American Journal of Distance Education , 15 (1), 7-23. Grant, C. (1996, May 1). Professional development in a technological age: New definitions, old challenges, new resources. Retrieved February 2008, from LSCNET: http://lsc-net.terc.edu/do.cfm/paper/8089/show/use_set-tech Graves, J. (2002). Determining teachers' perceptions of self empowerment through professional development facilitated by asynchronous distance delivery instruction in classroom-based action research. Unpublished Doctoral Dissertation . Montana State University. Guskey, T. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin Press, Inc. Guskey, T., & Sparks, D. (1991). Complexities in evaluating the effects of staff development programs. Paper presented at American Educational Research Association. Chicago. Harris-John, M., & Ritter, S. (2007). e_based Professional developement (e-PD) for effective teaching and leadership. Connections Module , 1. Haworth, J., & Conrad, C. (1997). Emblems of quality in higher education. Developing and sustaining high-quality programs. Needham Heights, MA: Allyn & Bacon. Higher Education Program And Policy Council of The American Federation of Teachers. (2000). distance education guidelines for good practice. American Federation of Teachers. Hirumi, A. (2005). In search of quality an analysis of e-learning guidelines and specifications. The Quarterly Review of Distance Education , 6 (4), 309-330. Institute for Higher Education Policy. (2007). About IHEP. Retrieved February 3, 2008, from Institute for Higher Education Policy: http://www.ihep.org/ 333 Isenberg, J. (2003). Using national board standards to redesign master's degrees for teachers: A guide for institutions of higher education. National Board For Professional Teaching Standards. James, M., Pedder, D., Swaffield, S., Conner, C., Frost, D., MacBeath, J. (2003). A servant of two masters: Designing research to advance knowledge and practice. A paper presented at the annual meeting of the American Educational Research Association. Chicago. Jung, I., Choi, S., Lim, C., & Leem, J. (2002). Effects of different types of interaction on learning achievement, satisfaction and participation in web-based instruction. Innovations in Education and Teaching International , 39 (2), 153-162. Kasworm, C., & Hemmingsen, L. (2007). Preparing professionals for lifelong learning: Comparative examination of master’s education programs. Higher Education , 451-468. Kedzior, M., & Fifield, S. (2004). Teacher professional development. Education Policy Brief , 15, pp. 1-6. Kennedy, M. (1998). Form and substance in inservice teacher education. NISE Brief , pp. 1-20. Killion, J. (2000). Log on to learn to reap benefits of online staff development, ask the right questions. Journal of Staff Development , 21 (3). Kleiman, G., (2004). Meeting the need for high quality teachers: e-Learning Solutions. Education Development Center, Inc. Washington, D.C: U.S. Department of Education. Krathwohl, D. (1998). Methods of Educational and Social Science Research An integrated approach (2nd Edition ed.). Long Grove, IL: Waveland Press. Kumari, D. (2001). Journal of Aysnchronous Learning Networks. Retrieved September 2, 2007, from http://www.aln.org/alnweb/journal/Vol5_issue2/Kumari/Kumari.htm Lewis. A. (2002). School reform and professional development. Phi Delta Kappan, 83 (7), pp. 488-489. Lieberman, A. (1996). Creating intentional learning communities. Education Leadership, 54(3), 51-55. Lincoln, Y., & Guba, E. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage. 334 Loucks-Horsley, S., Love, N., Stiles, K., Mundry, S., & Hewson, P. (2003). Designing professional development for teachers of science and mathematics. Thousand Oaks, CA: Corwin Press, Inc. Loucks-Horsley, S., Stiles, K., & Hewson, P. (1995). Principles of effective professional development for mathematics and science education: A synthesis of standards. NISE Brief , 1 (1). MacCallum, R., Widaman, K., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods. 4(1), 84-99. McAleer, D. (2008). Professional growth through mentoring: A study of experienced mathematics teachers participating in a content-based online mentoring and induction program. Unpublished Doctoral Dissertation. Montana State University. McLeskey, J., Tyler, N., & Flippin, S. (2004). The supply and demand for special education teachers:A review of research regarding the chronic shortage of special education teachers. The Journal of Special Education , 38 (1), 5-21. Mills, G. E. (2003). Action research: A guide for the teacher researcher. Upper Saddle River, NJ: Merrill/Prentice Hall. Minkel, C., & Richards, M. (1987). components of quality in master's degree programs. the master's degree: Jack of all trades . (J. Green, Ed.) Asheville, NC: State Higher Education Executive Officers Association. Montana State University. (2009, January 26). Division of Graduate Education. Retrieved November 1, 2008, from http://www.montana.edu/gradstudies/ Montana State University. (2008). Master of Science in Science Education. Retrieved September 12, 2008, from http://www.montana.edu/msse/ National Center for Research on Teacher Learning. (1991). Teacher education and learning to teach (TELT) study. Michigan State University. NSDC. (2010). About NSDC. Retrieved Oct 2009, from National Staff Development Council: http://www.nsdc.org/about/index.cfm National Research Council. (1996). National Science Education Standards. Washington, D. C. : National Academy Press. National Staff Development Council. (2001). NSDC's standards for staff development. national staff development council. 335 National Staff Development Council. (2001). e-Learnign for educators: Implementing the standards for staff development. Oxford: National Staff Development Council. Neal, L., & Miller, D. (2004). The basics of e-learning: An excerpt from handbook of human factors in web design. In R. W. Proctor, & K. L. Vu (Eds.), Handbook of Human Factors in Web Design. Lawrence Erlbaum Associates. Obink, K., & Tuthill, G. (1998). Taking the distance out of distance learning. Montana State University. Bozeman: Montana State University. Office of Vocational and Adult Education. (2007). U.S. Department of Education. Retrieved June 1, 2008, from Technology and Distance Learning: Pedder, D. (2007). Profiling teachers' professional learning practices and values: differences between and within schools. The Curriculum Journal , 18 (3), 231-252. Pedder, D., James, M., & MacBeath, J. (2005). How teachers value and practise professional learning. Research Papers in Education , 20 (2), 209-243. Peine, J. (2007). The educator's professional growth Plan: A process for developing staff and improving instruction. Thousand Oaks, CA: Corwin Press. Perdue, K. (2003). Web-based continuing professional education: Uses, motivations, and deterrents to participation. In M. Moore, & W. Anderson (Eds.), Handbook of Distance Education (pp. 615-630). Mahwah, NJ: Lawrence Erlbaum Associates. Picciano, A. (2002). Beyond student perceptions: Issues of interaction, presence, and perforance in an online course. Journal of Aysnchronous Learning Networks , 6 (1). Putnam, R., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning. Educational Researcher , 29 (1), 4-15. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (1999). Assessing social presence in aynchronous text-based computer conferencing. Journal of Distance Education , 14 (2), 50-71. Salmon, G. (2000). e-moderating: The Key to Teaching and Learning Online (2nd ed.). Oxon, England: RoutledgeFalmer. Saskatchewan Education. (1997). Native studies 30: Canadian studies curriculum guide. Regina, SK: Saskatchewan Education. 336 Selke, M. (2001). The Professional development of teachers in the United States of America: The practitioners’ master’s degree. European Journal of Teacher Education, , 24 (2), 205-214. Shea, P., Frederickson, E., Pickett, A., & Peltz, W. E. (2002). A preliminary investigation of "teaching presence" in the SUNY Learning Network. Sloan Asynchronous Learning Network Workshop . Shea, P., Pickett, A., & Peltz, W. (2003). A follow-up investigation of "teaching presence" in the SUNY Learning Network. Journal of Asynchronous Learning , 7 (2), 61- 80. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review , 57 (1), 1-22. Southern Regional Education Board. (2006). Standards for online professional development guidelines for planning and evaluating online professional development courses and programs. Southern Regional Education Board. Swan, K. (2002). Building communities in online courses; the importance of Interaction. Education, Communication and Information , 2 (1), 23-49. Swan, K. (2004). Relationships between interactions and learning in online environments. Retrieved May 25, 2007, from Sloan C Series: http://www.sloan- c.org/publications/books/interactions.pdf Taylor, P. (2007). The effect of facilitator training on the development and practice of participants in an online induction program for teachers of science and mathematics. Unpublished Doctoral DIssertation . Montana State University. The Institute for Higher Education Policy. (2000). Quality on the line benchmarks for succss in internet-based distance education. Washington, D.C.: National Education Association. Thomas-Goodfellow, L., Perdue, K. J., & Valentine, T. (2001). Respiratory therapist and the world wide web: Going online to satisfy continuing professional education requirements. Respiratory Care Education Annual , 10 (3-13). U.S. Department of Education. (2003). Distance education at degree granting postsecondary institutions: 2000-2001. National Center for Education Statistics. Washington, D.C.: NCES. Valentine, D. (2002). Distance learning: promises, problems, and possibilities. Online Journal of Distance Learning Administration , 5 (3). 337 Van Hanegan, J., Pruet, S. A., & Bamberger, H. J. (2004). Mathematics reform in a minority community: Student outcomes. Journal of Education for Students Placed at Risk, 9 (2), 189-211. Vaughan, N. Building faculty learning communities: New directions for teaching and learning. (M. Cox, & L. Richlin, Eds.) San Francisco, CA: Jossey-Bass. Weiss, I., Pasley, J., Smith, S., Banilower, E., & Heck, D. (2003). A study of K–12 mathematics and science education in the United States. Chapel Hill: Horizon Research, Inc. Worthington, R., & Whittaker, T. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806-838. Yin, R. (2009). Case Sudy Research Design and methods (4th Edition ed.). Thousand Oaks, CA: Sage. Zheng, L., & Smaldino, S. (2003). Key instructional design elements for distance education. Quarterly Review of Distance Education , 4 (2), 153-166. 338 APPENDICES 339 APPENDIX A MSSE STUDENT INFORMED CONSENT FOR PARTICIPATION IN RESEARCH FORM 340 SUBJECT CONSENT FORM FOR PARTICIPATION IN HUMAN RESEARCH AT MONTANA STATE UNIVERSITY Project Title: Determining the Impact on the Professional Learning of Graduates of a Master’s Degree Program for Science Teachers You are being asked to participate in a study exploring K12 teachers’ and university faculty members’ professional learning as a result of participating in the Master of Science in Science Education (MSSE) program. In order to document professional learning, data from MSSE students, both current and graduates, will be collected through surveys, interviews, and portfolios. If you agree to participate, you will be asked to complete an online survey examining your professional learning as a MSSE student. The survey will take approximately 15-20 minutes to complete and submit. You may also be asked to complete one or two opened ended, standardized recorded phone interviews. Phone interviews, each approximately 30 minutes in length, will be scheduled at a time convenient for you. You may also be asked to submit a portfolio. The portfolio may include instructional materials like lesson plans, lab design, PowerPoint presentations, handouts, materials generated in class, etc. It will also includes a brief cover page with an overview of the lesson and the intended outcomes, two or three student work samples from each lesson that are part of the instruction such as in-class assignments, assessments, journal entries, projects, homework, etc., a reflection page for each student work sample, and an overall reflection about your portfolio entry and your teaching. It is estimated that portfolios will take 4-6 hours to complete and submit. If you are selected to complete the interview and portfolio and agree to participate, you will receive a stipend of up to $300 to compensate for your time. This consent form serves to cover all of the data collected in this study, which will take place in the spring and summer of 2009. Your participation is voluntary and at any time you may refuse to participate. In all study efforts, your confidentiality will be maintained. No names or specific identifiers will be reported in this study. Results will be shared with my doctoral committee members at Montana State University and with the leadership of MSSE. The results may also appear in campus, regional or national research presentations, and publications. All information you share will remain confidential, minimizing any social risks associated with your participation. The risks for participating in this study are minimal. I foresee no physical risks associated with your participation. A possible benefit would be that your experiences and insights might affect future MSSE management decisions and directions. 341 If you have additional questions about this research, please contact Elisabeth Swanson or Alyson Mike at the Montana State University Science/Math Resource Center (406-994- 6768) or Peggy Taylor, Director, MSSE (406-994-1741). If you have specific questions about the rights of human subjects regarding this study, please contact the Chairman of the Institutional Review Board at Montana State University, Mark Quinn, (406) 994- 5721. ************************************************************************ AUTHORIZATION: I have read the above and understand the nature of my involvement in the research study on the Master of Science in Science Education (MSSE) program. I, ______________________________________, agree to participate in this evaluation. I understand that I may later refuse to participate, and that I may withdraw from the study at any time. I have received a copy of this consent form and the accompanying explanatory page for my own records. ________________________________________________ ________________ MSSE Student [current or graduate] signature Date 342 APPENDIX B MSSE FACULTY INFORMED CONSENT FOR PARTICIPATION IN RESEARCH FORM 343 SUBJECT CONSENT FORM FOR PARTICIPATION IN HUMAN RESEARCH AT MONTANA STATE UNIVERSITY Project Title: Determining the Impact on the Professional Learning of Graduates of a Master’s Degree Program for Science Teachers You are being asked to participate in a study exploring K12 teachers’ and university faculty members’ professional learning as a result of participating in the Master of Science in Science Education (MSSE) program. In order to document professional learning, data from MSSE faculty will be collected through surveys, interviews, and portfolios. If you agree to participate, you will be asked to complete an online survey examining your professional learning as a MSSE faculty member. The survey will take approximately 15-20 minutes to complete and submit. You may also be asked to complete one or two opened ended, standardized recorded phone interviews. Phone interviews, each approximately 30 minutes in length, will be scheduled at a time convenient for you. You may also be asked to submit a portfolio. The portfolio may include the syllabus, assignments, other course materials, and other artifacts that illustrate how faculty members support different aspects of teacher growth in their course(s). You will also be asked to submit an overall reflection about your portfolio entry and your course(s). It is estimated that portfolios will take 1-3 hours to complete and submit. This consent form serves to cover all of the data collected in this study, which will take place in the spring and summer of 2009. Your participation is voluntary and at any time you may refuse to participate. In all study efforts, your confidentiality will be maintained. No names or specific identifiers will be reported in this study. Results will be shared with my doctoral committee members at Montana State University and with the leadership of MSSE. The results may also appear in campus, regional or national research presentations, and publications. All information you share will remain confidential, minimizing any social risks associated with your participation. The risks for participating in this study are minimal. I foresee no physical risks associated with your participation. A possible benefit would be that your experiences and insights might affect future MSSE management decisions and directions. If you have additional questions about this research, please contact Elisabeth Swanson or Alyson Mike at the Montana State University Science/Math Resource Center (406-994- 6768) or Peggy Taylor, Director, MSSE (406-994-1741). If you have specific questions about the rights of human subjects regarding this study, please contact the Chairman of the Institutional Review Board at Montana State University, Mark Quinn, (406) 994- 5721. ************************************************************************ 344 AUTHORIZATION: I have read the above and understand the nature of my involvement in the research study on the Master of Science in Science Education (MSSE) program. I, ______________________________________, agree to participate in this evaluation. I understand that I may later refuse to participate, and that I may withdraw from the study at any time. I have received a copy of this consent form and the accompanying explanatory page for my own records. ________________________________________________ ________________ MSSE faculty signature Date 345 APPENDIX C MSSE STUDENT PROFESSIONAL LEARNING SURVEY 346 347 348 349 350 351 352 353 354 355 356 357 358 APPENDIX D MSSE FACULTY PROFESSIONAL LEARNING SURVEY 359 360 361 362 363 364 365 366 367 368 APPENDIX E COMPARISON OF LANGUAGE AND PHRASING BETWEEN THE LEARNING HOW TO LEARN SURVEY AND THE MSSE STUDENT PROFESSIONAL LEARNING SURVEY 369 Pedder et al (2005) - Learning How to Learn Staff Questionnaire, Part B Modified Questions for the MSSE Student Professional Learning Survey Overarching Question: How important are these practices and beliefs for creating opportunities for students to learn? Overarching Question: To what degree did your experiences in the MSSE program foster your professional learning, including your knowledge and practices, in the following areas: 1. Staff as well as students learn in this school. 1. Using my learning to improve my students' learning. 2. Staff draws on good practice from other schools as a means to further their own professional development. 2. Drawing on good practice from other teachers as a means to further my own professional practice. 3. Staff read research reports as one source of useful ideas for improving their practice. 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving my practice. 4. Staff use the web as one source of useful ideas for improving their practice. 4. Using the Internet as one source of useful ideas for improving my practice. 5. Students are consulted about how they learn most effectively. 5. Consulting and observing students about how they learn most effectively. 6. Staff relate what works in their own practice to research findings. 6. Relating what works in my own practice to research findings and best practices. 7. Staff are able to see how practices that work in one context might be adapted to other contexts. 7. Using insights from my professional learning to feed into my school’s policy development. 8. Staff use insights from their professional learning to feed into school policy development. 8. Understanding how instructional strategies that work in one context might be adapted to other contexts. 9. Staff reflect on their practice as a way of identifying professional learning needs. 9. Reflecting on my practice as a way of identifying my professional learning needs to better meet the needs of my students. 10. Staff experiment with their practice as a conscious strategy for improving classroom teaching and learning. 10. Experimenting with my practice as a conscious strategy for improving classroom teaching and learning. 11. Staff modify their practice in the light of feedback from their students. 11. Modifying my practice in light of performance and feedback from my students. 12. Staff modify their practice in the light of published research evidence. 12. Modifying my practice in light of published research evidence. 13. Staff modify their practice in the light of evidence from self-evaluations of their classroom practice. 13. Modifying my practice in light of evidence of self-evaluations of my classroom practice. 370 14. Staff modify their practice in the light of evidence from evaluations of their classroom practice by managers or other colleagues. 14. Modifying my practice in light of evidence from evaluations of my classroom practice by administrators or other colleagues. 15. Staff carry out joint research/evaluation with one or more colleagues as a way of improving their practice. 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving my practice. 16. Staff regularly collaborate to plan their teaching. 16. Collaborating to plan my teaching. 17. Staff regularly observe each other in the classroom and give each other feedback. 17. Reading/discussing other teachers experiences in the classroom and giving each other feedback. 18. Staff engage in team teaching as a way of improving practice. 18. Engaging in team teaching with someone at my school as a way of improving practice. 19. If staff have a problem with their teaching they usually turn to colleagues for help. 19. Turning to colleagues/peers for help if I have a problem with my teaching. 20. Teachers suggest ideas or approaches for colleagues to try in class. 20. Posting suggestions, ideas or approaches for colleagues to try in class. 21. Teachers make collective agreements to test out new ideas. 21. Testing out new ideas in class. 22. Teachers discuss openly with colleagues what and how they are learning. 22. Discussing openly with colleagues what and how you the teacher are learning. 23. Staff frequently use informal opportunities to discuss how children learn. 23. Frequently using informal opportunities to discuss how students learn. 24. Staff offer one another reassurance and support. 24. Offering other colleagues reassurance and support. 25. Staff believe that all students are capable of learning. 25. Believing that all students are capable of learning. 26. Students in this school enjoy learning. 26. Encouraging students in my school to enjoy learning. 27. Pupil success is regularly celebrated. 27. Regularly celebrating student success. 28. Staff discuss with colleagues how students might be helped to learn how to learn. 28. Discussing with colleagues how students might be helped to learn how to learn. 371 APPENDIX F COMPARISON OF LANGUAGE AND PHRASING BETWEEN THE LEARNING HOW TO LEARN SURVEY AND THE MSSE FACULTY PROFESSIONAL LEARNING SURVEY 372 Pedder et al (2005) - Learning How to Learn Staff Questionnaire, Part B Modified Questions for the MSSE Faculty Professional Learning Pilot Survey Overarching Question: How important are these practices and beliefs for creating opportunities for students to learn? Overarching Question: As a MSSE faculty member, how important are these professional learning practices and beliefs fostered for the MSSE students in your course(s)? 1. Staff as well as students learn in this school. 1. Use of learning by the MSSE teachers to improve their students’ learning. 2. Staff draw on good practice from other schools as a means to further their own professional development. 2. Drawing on good practice from other MSSE teachers as a means to further the MSSE teachers’ professional practice. 3. Staff read research reports as one source of useful ideas for improving their practice. 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving the MSSE teachers’ practice. 4. Staff use the web as one source of useful ideas for improving their practice. 4. Using the Internet as one source of useful ideas for improving the MSSE teachers’ practice. 5. Students are consulted about how they learn most effectively. 5. Consulting and observing the MSSE teachers about how they learn most effectively. 6. Staff relate what works in their own practice to research findings. 6. Relating what works in the MSSE teachers’ practice to research findings and best practices. 7. Staff are able to see how practices that work in one context might be adapted to other contexts. 7. Using insights from the MSSE teachers’ learning in the course to feed into their school’s policy development. 8. Staff use insights from their professional learning to feed into school policy development. 8. Understanding how instructional strategies that work in one context might be adapted to other contexts. 9. Staff reflect on their practice as a way of identifying professional learning needs. 9. Using reflection as a way for the MSSE teachers to identify their professional learning needs. 10. Staff experiment with their practice as a conscious strategy for improving classroom teaching and learning. 10. Experimenting with their practice as a conscious strategy for improving classroom teaching and learning. 11. Staff modify their practice in the light of feedback from their students. 11. Modifying their practice in light of performance and feedback in the course. 12. Staff modify their practice in the light of published research evidence. 12. Modifying their practice in light of published research evidence. 373 13. Staff modify their practice in the light of evidence from self-evaluations of their classroom practice. 13. In light of evidence of self-evaluations, the MSSE teachers’ modify their practice. 14. Staff modify their practice in the light of evidence from evaluations of their classroom practice by managers or other colleagues. 14. In light of evidence from evaluations of classroom practice by administrators or other colleagues, the MSSE teachers’ modify their practice. 15. Staff carry out joint research/evaluation with one or more colleagues as a way of improving their practice. 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving the MSSE teachers’ practice. 16. Staff regularly collaborate to plan their teaching. 16. Collaborating to plan teaching. 17. Staff regularly observe each other in the classroom and give each other feedback. 17. Reading/discussing other teachers’ experiences in the classroom and giving each other feedback. 18. Staff engage in team teaching as a way of improving practice. 18. Engaging in team teaching with someone at the MSSE teacher’s school as a way of improving practice. 19. If staff have a problem with their teaching they usually turn to colleagues for help. 19. Turning to colleagues/peers for help if the MSSE teachers have problems with their teaching. 20. Teachers suggest ideas or approaches for colleagues to try in class. 20. Posting suggestions, ideas or approaches for colleagues to try in class. 21. Teachers make collective agreements to test out new ideas. 21. Testing out new ideas in class. 22. Teachers discuss openly with colleagues what and how they are learning. 22. Discussing openly with colleagues what and how they as teachers are learning. 23. Staff frequently use informal opportunities to discuss how children learn. 23. Frequently using informal opportunities to discuss how students learn. 24. Staff offer one another reassurance and support. 24. Offering other teachers reassurance and support. 25. Staff believe that all students are capable of learning. 25. Believing that all students are capable of learning. 26. Students in this school enjoy learning. 26. Encouraging MSSE teachers, and by extension their students, to enjoy learning. 27. Pupil success is regularly celebrated. 27. Regularly celebrating student success. 28. Staff discuss with colleagues how students might be helped to learn how to learn. 28. Discussing with colleagues how students might be helped to learn how to learn. 374 APPENDIX G COMPARISONS OF LANGUAGE AND PHRASING FROM THE 2000 NATIONAL SURVEY OF SCIENCE AND MATHEMATICS EDUCATION, QUESTION 12B, AND THE MSSE STUDENT PROFESSIONAL LEARNING SURVEY AND THE MSSE FACULTY PROFESSIONAL LEARNING SURVEY. 375 2000 National Survey of Science and Mathematics Education survey administered by Horizon Research, Inc. Modified Questions for the MSSE Student Professional Learning Pilot Survey Overarching Question 12b. Considering all the professional development you have participated in during the last 3 years, how much was each of the following emphasized? Overarching Question To what degree did your experiences in the MSSE program foster your professional learning, including knowledge and practices, in the following areas: Deepening my own science content knowledge. 29. Deepening my own science content knowledge. Understanding student thinking in science. 30. Understanding student thinking in science. Learning how to use science specific teaching strategies. 31. Using inquiry/investigation-oriented teaching strategies. Learning how to assess student learning in science. 33. Assessing student learning in science Learning how to teach science in a class that meets the needs of diverse learners. 34. Teaching science in a class that includes diverse student populations (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities). 376 APPENDIX H DESCRIPTIVE STATISTICS FOR QUESTIONS 12B OF THE 2000 NATIONAL SURVEY OF SCIENCE AND MATHEMATICS EDUCATION 377 12b.1 Grade K-4 Science Teachers’ Opinions of Professional Development Emphasis Percent of Teachers Not at all 1 2 3 4 To a great extent 5 Deepening my own science content knowledge 28 (2.6) 24 (2.1) 30 (2.4) 13 (1.6) 7 (1.4) Understanding student thinking in science 27 (2.5) 19 (2.0) 32 (2.3) 15 (1.8) 7 (1.5) Learning how to use inquiry/investigation oriented teaching strategies 23 (2.2) 21 (2.1) 29 (2.2) 18 (1.8) 10 (1.8) Learning how to use technology in science instruction 39 (2.9) 22 (2.3) 23 (2.0) 9 (1.4) 7 (1.1) Learning how to assess student learning in science 30 (2.5) 23 (2.2) 30 (2.4) 13 (1.9) 4 (1.1) Learning how to teach science in a class that includes students with special needs 47 (2.5) 25 (2.2) 19 (2.2) 6 (1.3) 3 (0.8) 12b.2 Grade 5-8 Science Teachers’ Opinions of Professional Development Emphasis Percent of Teachers Not at all 1 2 3 4 To a great extent 5 Deepening my own science content knowledge 21 (3.0) 23 (3.3) 26 (3.4) 19 (3.6) 11 (2.2) Understanding student thinking in science 20 (3.1) 27 (3.1) 26 (3.4) 23 (3.3) 5 (1.3) Learning how to use inquiry/investigation oriented teaching strategies 15 (2.8) 20 (3.4) 29 (3.6) 24 (3.3) 12 (2.4) Learning how to use technology in science instruction 22 (3.3) 25 (4.0) 23 (3.4) 21 (3.1) 9 (1.7) Learning how to assess student learning in science 18 (3.0) 27 (3.7) 30 (3.2) 22 (3.3) 4 (0.9) Learning how to teach science in a class that includes students with special needs 39 (3.9) 28 (3.3) 20 (3.0) 10 (2.8) 3 (0.9) 378 12b.3 Grade 9-12 Science Teachers’ Opinions of Professional Development Emphasis Percent of Teachers Not at all 1 2 3 4 To a great extent 5 Deepening my own science content knowledge 24 (1.7) 22 (1.4) 27 (2.3) 17 (1.9) 10 (1.2) Understanding student thinking in science 19 (1.8) 26 (1.6) 34 (2.1) 15 (1.4) 6 (1.1) Learning how to use inquiry/investigation oriented teaching strategies 14 (1.5) 22 (1.8) 29 (2.0) 23 (2.3) 12 (1.4) Learning how to use technology in science instruction 11 (1.3) 19 (1.6) 23 (1.5) 30 (2.3) 17 (1.6) Learning how to assess student learning in science 19 (1.8) 27 (1.9) 30 (2.1) 18 (1.9) 6 (1.0) Learning how to teach science in a class that includes students with special needs 40 (2.1) 28 (2.4) 19 (1.5) 9 (1.4) 4 (1.7) 379 APPENDIX I COMPARISONS OF LANGUAGE AND PHRASING FROM THE EMSS (ELECTRONIC MENTORING FOR STUDENT SUCCESS) TEACHER GROWTH SURVEY –MATHEMATICS AND THE MSSE STUDENT PROFESSIONAL LEARNING SURVEY. 380 eMSS (Electronic Mentoring for Student Success) Teacher Growth Survey – Mathematics Modified Questions for the MSSE Student Professional Learning Pilot Survey To what extent do you feel that your mathematics content knowledge has been enhanced as a result of participation in the following discussion areas? To what degree did the following MSSE program components foster your science content knowledge? Which discussion areas or other aspect of the eMSS do you feel has contributed most to enhancing your mathematics content knowledge? Please specific in what ways the discussion area or aspect of eMSS has contributed to your professional learning? Please describe the MSSE program component or experience that has contributed the most to your science content knowledge. Please be as specific in your description as possible. Pedagogical content knowledge is defined as content knowledge for teaching that embodies understanding how to effectively represent and teach the subject knowledge to students as well as understanding what makes the learning of topics easy or challenging for students (Shulman, 1986) To what extent do you feel that you pedagogical content knowledge needed for teaching mathematics has been enhanced as a result of participating in the following discussion areas? To what degree did the following MSSE program components foster your knowledge of pedagogy for teaching science? Which discussion area or other aspect of eMSS do you feel has contributed most to enhancing your pedagogical content knowledge for teaching mathematics? Please specific in what ways this discussion area or aspect of eMSS has contributed to your growth. Please describe the MSSE program component or experience that has contributed the most to your knowledge of pedagogy for teaching science. Please be as specific in your description as possible. Curricular knowledge is defined as the knowledge of how to use resources and arrange topics over time to organize the study of a discipline (Shulman, 1986) To what extent do you feel that your curricular knowledge of mathematics has been enhanced as a result of participation in the following discussion areas? To what degree did the following MSSE program components foster your knowledge of science curriculum, including how to use resources and arrange topics over time? 381 Which discussion area or other aspect of eMSS do you feel has contributed most to enhancing your curricular knowledge of mathematics? Please specific in what ways this discussion area or aspect of eMSS has contributed to your growth. Please describe the MSSE program component or experience that has contributed the most to your knowledge of science curriculum, including how to use resources and arrange topics over time. Please be as specific in your description as possible. To what extent do you feel that your knowledge of how to support diverse learners has been enhanced as a result of participation in the following discussion areas? To what degree did the following MSSE program components foster your knowledge of how to support diverse learners (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities)? Which discussion area or other aspect of eMSS do you feel has contributed most to enhancing your knowledge of how to support diverse learners? Please specify in what ways this discussion area or aspect of eMSS has contributed to your growth. Please describe the MSSE program component or experience that has contributed the most to your knowledge of how to support diverse learners (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities)? Please be as specific in your description as possible. 382 APPENDIX J ALIGNMENT OF THE RESEARCH QUESTIONS AND METHODS OF QUANTITATIVE ANALYSIS WITH THE QUESTIONS OF THE MSSE PROFESSIONAL LEARNING SURVEY. 383 Research Questions 2. How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers grow because of their participation in such a program? b. What program experiences, from a program faculty perspective, contribute to the science teachers’ professional learning? c. What evidence of teacher professional learning can be identified? Analysis: Descriptive statistics 1. Use of learning by the teachers to improve their students’ learning. 2. Drawing on good practice from other teachers as a means to further the MSSE teachers’ professional practice. 3. Reading research (practitioner reports and journal articles) as one source of useful ideas for improving the MSSE teachers’ practice. 4. Using the Internet as one source of useful ideas for improving the MSSE teachers’ practice. 5. Consulting and observing the MSSE teachers about how they learn most effectively. 6. Relating what works in the MSSE teachers’ practice to research findings and best practices. 7. Using insights from the MSSE teachers’ learning in the course to feed into their school’s policy development. 8. Understanding how instructional strategies that work in one context might be adapted to other contexts. 9. Using reflection as a way for the MSSE teachers to identify their professional learning needs. 10. Experimenting with their practice as a conscious strategy for improving classroom teaching and learning. 11. Modifying their practice in light of performance and feedback in the course. 12. Modifying their practice in light of published research evidence. 13. In light of evidence of self-evaluations, the MSSE teachers’ modify their practice. 14. Modifying my practice in light of evidence from evaluations of my classroom practice by administrators or other colleagues. 15. Carrying out joint research/evaluation with one or more colleagues as a way of improving my practice. 16. Collaborating to plan my teaching. 384 Research Questions 2. How are experiences in this science and pedagogical content knowledge-based graduate degree program perceived, and how have program experiences influenced participants’ professional learning? a. In what ways do science teachers grow because of their participation in such a program? Analysis: Descriptive statistics 17. Reading/discussing other teachers experiences in the classroom and giving each other feedback. 18. Engaging in team teaching with someone at my school as a way of improving practice. 19. Turning to colleagues/peers for help if I have a problem with my teaching. 20. Posting suggestions, ideas or approaches for colleagues to try in class. 21. Testing out new ideas in class. 22. Discussing openly with colleagues what and how you the teacher are learning. 23. Frequently using informal opportunities to discuss how students learn. 24. Offering other colleagues reassurance and support. 25. Believing that all students are capable of learning. 26. Encouraging students in my school to enjoy learning. 27. Regularly celebrating student success. 28. Discussing with colleagues how students might be helped to learn how to learn. 29. Deepening my own science content knowledge 30. Understanding student thinking in science. 31. Using inquiry/investigation-oriented teaching strategies. 32. Using technology in science instruction 33. Assessing student learning in science 34. Teaching science in a class that includes diverse student populations (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities) 35. Providing a challenging science curriculum for my students. 385 b. What program experiences, from a program teachers’ perspective, contribute to participants’ professional learning? Analysis: Descriptive statistics, one within repeated measures analysis, and qualitative analysis 1. To what degree did the following MSSE program components foster your science content knowledge? 1a. Please describe the MSSE program component or experience that has contributed the most to your science content knowledge. Please be as specific in your description as possible. 2. To what degree did the following MSSE program components foster your knowledge of pedagogy for teaching science? 2a. Please describe the MSSE program component or experience that has contributed the most to your knowledge of pedagogy for teaching science. Please be as specific in your description as possible. 3. To what degree did the following MSSE program components foster your knowledge of science curriculum, including how to use resources and arrange topics over time? 3a. Please describe the MSSE program component or experience that has contributed the most to your knowledge of science curriculum, including how to use resources and arrange topics over time. Please be as specific in your description as possible. 4a. Please describe the MSSE program component or experience that has contributed the most to your knowledge of how to support diverse learners (e.g. students with disabilities, from underrepresented populations, economically disadvantaged, range of abilities)? Please be as specific in your description as possible. 386 APPENDIX K MSSE STUDENT CASE STUDY INTERVIEW QUESTIONS 387 MSSE Student Case Study Interview Questions I am conducting a study examining the professional learning of students, current and graduates, of the MSSE program. I would like to speak with you about your experiences with the MSSE program and your professional learning. A. Introduction For currently enrolled students:  How long have you been in the MSSE program?  When is your anticipated graduation date?  How would describe your overall experiences with the program to date? (Probe for types of courses completed (Face-to-face vs Online), quality or relevance of experience.) For graduates of the MSSE program:  When did you graduate from the MSSE program  How would describe your overall experiences with the program? MSSE faculty and staff are very interested in the ways the program affects participants’ learning and teaching practice, and the following questions address those areas. (We recognize that any participant may be affected in only a few of these ways.) B. Inquiry  Do you use sources of evidence (such as research, consultation with students and colleagues) differently than before to understand and modify your teaching practice? C. Building Social Capital  Are there any ways in which your tendency to collaborate with colleagues [MSSE or others] to explore how students learn, and to experiment with new approaches, have been modified? (Probe for specific examples.) D. Critical and Responsive Learning  Are there ways that your reflection on practice has been influenced? (Probe for specific examples.)  Are there ways that you respond to feedback (from other teachers, students, administrators, parents) differently now? E. Value Learning  Has the value you place on your own learning changed in any way? Explain.  Have the ways you convey to your students that you value their learning changed? Explain. 388 F. Content and Pedagogical Knowledge  To what extent has your knowledge of science content been enhanced as a result of your participation in MSSE? Explain (Probe for specific examples such as changes in depth of understanding of “familiar” topics; grasp of connections across topics; updating information; knowledge of research methods...)  To what extent has your knowledge of science pedagogy, and your tendency and ability to implement varied approaches, been have been influenced as a result of your participation in MSSE? Explain (Probe for specific examples of understanding of “advanced” science teaching practices, frequency of implementation, tendency to use more varied practices.) G. Evidence of professional learning.  If you were asked to describe your most powerful learning experience(s) in MSSE to a group of applicants, what would you tell them? (Probe for specific examples.)  If I visited the place where you teach and said, “Show me some proof – a new product, a strategy, even a changed student – of the impact of MSSE on your practice, what would you show me?” H. Distance mode of delivery  How has the distance mode of delivery of the MSSE program affected your professional learning? (Probe for examples regarding access to learning opportunities, personal learning style, richness of experience, tools to use with own students…) I. Other  Do you have any other comments or thoughts about the MSSE program that might be useful for me to know?  What other graduate or professional development experiences have you participated in that were of high quality? Why did you feel that way? (Probe for examples) 389 APPENDIX L MSSE STUDENT PROFESSIONAL LEARNING PORTFOLIO 390 MSSE Student Professional Learning Portfolio The goal of the MSSE program in to enhance a teachers’ professional learning and improve the teaching skills that are specific to science disciplines. Through participation in the MSSE program, current students and graduates, have multiple opportunities to collaborate with other students and faculty, examine current research and best practice, apply new strategies and learning’s in the classroom and reflect on one’s teaching practices. A portfolio can provide a lens in which the MSSE program can be examined by documenting professional learning. The portfolio can also provide a documentation of your own personal and professional learning. The portfolio will contain a variety of items chosen to represent what is happening in your classroom that can be connected to your experience in the MSSE program. Portfolio items will include the following items for one (1) lesson. 1. Instructional Materials like lesson plans, lab design, PowerPoint presentations, handouts, materials generated in class, etc. that includes a brief cover page with an overview of the lesson and the intended outcomes. 2. Two or three student work samples from the lesson that are part of the instruction such as in-class assignments, assessments, journal entries, projects, homework, etc. [OPTIONAL, IF SAMPLES ARE AVAILABLE] 3. A reflection page for each student work sample [OPTIONAL, IF SAMPLES ARE AVAILABLE] 4. An overall reflection about your portfolio entry and your teaching. 5. A reflection about your professional learning in the MSSE program. The graphic organize shown below outlines the structure of the portfolio. It was created by the researcher to provide clarity of the expectations of the portfolio. 391 Instructional Materials Cover Sheet 1. Briefly identify:  The type of school/program in which you teach, and the grade/subject configuration. (single grade, departmentalized, interdisciplinary teams etc.)  The grade(s), age levels, course, number of students taught daily, and the average number in each class: Grades Age Levels Number of Students Average Number of Students in Each Class Courses 2. Briefly describe the lesson being taught including what you did and what the students did. 3. Briefly state the intended goals, objectives, or intended outcomes of the lesson. 4. Briefly describe how this lesson can be connected to your experience in the MSSE program. (If applicable, include ways that MSSE affected student learning.) 392 Student Work Sample Cover Page (Please complete one page for each student work sample) 1. In examining the student work sample, what are indicators the student met your intended goals, objectives, or outcomes for this lesson? 2. What does this student work sample tell you about the student’s understanding of the content? 3. Briefly describe how your experience in the MSSE program has moved your practice forward in meeting the needs of this student. 393 Professional Learning Dimensions A. Inquiry: Using and responding to different sources of evidence sometimes leading to joint research with colleagues. This dimension may include using research, consultation with colleagues and students, use of the Internet, or examining practice in other schools as part of an inquiry process intended to improve professional practice. B. Building social capital: Learning, working, supporting, and talking with colleagues with the intent of understanding and improving students’ learning. This dimension involves interaction with MSSE colleagues or others. C. Critical and responsive learning: Occurs through reflections, self-evaluation, experimentation, and by responding to feedback from others. This dimension is characterized by a teacher’s responsiveness to ideas, suggestions and feedback on practice from multiple sources. D. Valuing Learning: Deepening understanding and learning through talking about and valuing learning with others. This dimension is characterized by valuing and celebrating the learning of both teachers and students and commitment to the concept that all students are capable of learning. E. Content Knowledge: Developing an understanding of the “big ideas” as well as the intricate details of a particular science discipline. This dimension may also include providing appropriate learning strategies and techniques for understanding science concepts. 394 Portfolio Reflection 1. Reflecting back on the lesson that you chose for this portfolio entry and your participation in the MSSE program, what dimension(s) of professional learning had the greatest influence on the design and development of the chosen lesson? 2. Reflecting back on your teaching of this lesson and your participation in the MSSE program, what dimension(s) of professional learning had the greatest influence on your actual teaching of this lesson? 395 Professional Learning Reflection In what ways, if any, was each of these components of professional learning and practice influenced by MSSE? Give an example when applicable. a. Inquiry b. Building Social Capital c. Critical and Responsive Learning 396 d. Valuing Learning e. Content Knowledge 397 APPENDIX M MSSE STUDENT PROFESSIONAL RUBRIC 398 Professional Learning Reflection Rubric – Portfolio Dimensions of Professional Learning To a great extent To some extent Very little evidence Inquiry  Draw on good practice from other teachers/faculty  Read research [practitioner and journals] to improve practice  Use the Internet as a source of ideas  Consult with students about how they learn  Relate my practice to research and best practices  Modify practice based on research and best practice  Collaborate with others to improve practice Building Social Capital  Collaborate to plan teaching  Collaborate with peers/colleagues if I have problems with my teaching  Suggest ideas for colleagues to try  Discuss with colleague what and how they are learning  Use informal opportunities to discuss how students learn  Offer colleagues reassurance and support Critical and Responsive Learning  Examine how practice in one context may work in another  Reflect on my practice to identify personal learning needs  Experiment with my practice as a strategy for improvement  Modify practice in light of student feedback  Modify my practice in light of self-evaluations  Modify my practice in light of evaluations of my practice from administrators, colleagues, peers Valuing Learning  My students learn as a result of my learning  All students are capable of learning  Encourage students to enjoy learning  Celebrate student success 399 Dimensions of Professional Learning To a great extent To some extent Very little evidence Content Knowledge  Deepen my own content knowledge  Understand student thinking in science  Learning how to use science specific strategies  Learning how to assess student learning in science  Learning how to teach science to meet the needs of diverse learners 400 APPENDIX N MSSE STUDENT PROFESSIONAL LEARNING PORTFOLIO RUBRIC 401 MSSE Student Professional Learning Portfolio Rubric Each artifact submitted is assessed on evidence of each of the dimensions of profession learning: inquiry, building social capital, critical and responsive learning, valuing learning, and a focus on content. For each artifact, check all dimensions that are evident in the artifact. Evidence comes from key words, phrases, illustrations, etc., that connect to a dimension of professional learning. Dimensions of Professional Learning Evident Not evident A. Inquiry: Using and responding to different sources of evidence and carrying out joint research and evaluations with colleagues. B. Building social capital: Learning, working, supporting, and talking with colleagues to discuss how students learn. C. Critical and responsive learning: Occurs through reflections, self-evaluation, experimentation, and by responding to feedback. D. Valuing Learning: Deepening understanding and learning through talking about and valuing learning with others E. Content Knowledge: A deep understanding of “big ideas” as well as intricate details of a particular discipline Student Work Samples 1. Does the student work sample response indicate goals and objectives that support the dimensions of valuing learning? To a great extent To some extent Very little evidence 3 2 1 2. Does the student work sample response support the dimension of content knowledge? To a great extent To some extent Very little evidence 3 2 1 Portfolio Reflection 3. Does the portfolio reflection support the dimensions of professional learning in the design of the lesson? To a great extent To some extent Very little evidence 3 2 1 4. Does the portfolio reflection support the dimensions of professional learning in the teaching of the lesson? To a great extent To some extent Very little evidence 3 2 1