Theses and Dissertations at Montana State University (MSU)
Permanent URI for this collectionhttps://scholarworks.montana.edu/handle/1/733
Browse
6 results
Search Results
Item Automated techniques for prioritization of metamorphic relations for effective metamorphic testing(Montana State University - Bozeman, College of Engineering, 2022) Srinivasan, Madhusudan; Chairperson, Graduate Committee: John Paxton and Upulee Kanewala (co-chair)An oracle is a mechanism to decide whether the outputs of the program for the executed test cases are correct. In many situations, the oracle is not available or too difficult to implement. Metamorphic testing is a testing approach that uses metamorphic relations (MRs), properties of the software under test represented in the form of relations among inputs and outputs of multiple executions, to help verify the correctness of a program. Typically, MRs vary in their ability to detect faults in the program under test, and some MRs tend to detect the same set of faults. In this work, we aim to prioritize MRs to improve the efficiency and effectiveness of MT. We present five MR prioritization approaches: (1) Fault-based, (2) Coverage-based, (3) Statement Centrality-based, (4) Variable-based, and (5) Data Diversity-based. To evaluate these MR prioritization approaches, we conducted experiments on complex open- source software systems and machine learning programs. Our results suggest that the proposed MR prioritization approaches outperform the current practice of executing the source and follow-up test cases of the MRs randomly. Further, our results show that Statement Centrality-based and Variable-based approaches outperform Code Coverage and random-based approaches. Also, the proposed approaches show 21% higher rate of fault detection over random-based prioritization. For machine learning programs, the proposed Data Diversity-based MR prioritization approach increases the fault detection effectiveness by up to 40% when compared to the Code Coverage- based approach and reduces the time taken to detect a fault by 29% when compared to random execution of MRs. Further, all the proposed approaches lead to reducing the number of MRs that needs to be executed. Overall, our work would result in saving time and cost during the metamorphic testing process.Item Analyzing the security of C# source code using a hierarchical quality model(Montana State University - Bozeman, College of Engineering, 2022) Harrison, Payton Rae; Chairperson, Graduate Committee: Clemente IzurietaIn software engineering, both in government and in industry, there are no universal standards or guidelines for security or quality. There is an increased need for evaluating the security of source code projects, which is made apparent by the number of real-world cyber attacks that have taken place recently. Our research goal is to design and develop a security quality model that helps stakeholders assess the security of C# source code projects. While there are many analysis tools that can be used to identity security vulnerabilities, the use of a model is beneficial in integrating multiple analysis tools to have better coverage over the number of security vulnerabilities detected (compared to the use of a single tool) and to aggregate these vulnerabilities upward into a broader security quality context. We accomplished our goal by developing and validating a hierarchical security quality model (PIQUE-C#-Sec) to evaluate the security quality of software written in C#. This model is an operationalized model using PIQUE, or the Platform for Investigative software Quality Understanding and Evaluation. PIQUE-C#-Sec improves upon previous security quality models and quality models that precede it by focusing on being specific, flexible, and extensible. This thesis introduces the model design for PIQUE-C#-Sec and examines the results from the efforts of validating the PIQUE-C#-Sec model. This model was validated using sensitivity analysis, which consisted of collecting data on benchmark repositories and observing if and how the PIQUE-C#-Sec model output varied as a function of these repository attributes. Additionally, the model was analyzed by testing to see how the PIQUE-C#-Sec model node values changed because of the tools reporting additional vulnerabilities. Based on these results, we conclude that the PIQUE-C#-Sec model is effective for stakeholders to use when evaluating C# source code, and the model can be used as a security quality gate for evaluating these projects.Item Surveying middle school computer science throughout the United States: a needs assessment for teachers(Montana State University - Bozeman, College of Engineering, 2022) Firth, Olivia Rose; Chairperson, Graduate Committee: Brittany FasyMiddle school computer science plays an important role in increasing student interest and engagement, as well as leveling the playing field for students as they enter high school. Computer science courses or the introduction of CS into other STEM courses is also a valuable opportunity to build higher order thinking and problem-solving skills that will serve students throughout their education. However, the state of middle school computer science (CS) has yet to be studied in depth. Particularly, research is needed to understand the challenges facing a diverse population of middle school CS teachers. With the goal of learning to better support CS teachers, we have performed interviews and surveys amongst teachers throughout the country to illustrate the state of middle school CS and CS teacher self-efficacy. This project fills in this gap by serving as a needs assessment for a difficult to reach population because the track to become a middle school CS teacher is not well-defined, and many of these teachers are new to the role or have taken it on as an additional responsibility.Item Design pattern decay: a study of design pattern grime and its impact on quality and technical debt(Montana State University - Bozeman, College of Engineering, 2021) Griffith, Isaac Daniel; Chairperson, Graduate Committee: Clemente IzurietaTechnical debt is a financial metaphor describing the trade-off between the short-term benefits gained and long-term consequences of design and implementation shortcuts taken over the evolution of a software product. These shortcuts typically manifest as design disharmonies such as code smells, anti-patterns, or design pattern grime. Design pattern grime, which manifests as the accumulation of unnecessary or unrelated software artifacts within design pattern instance classes is of serious concern. Design patterns represent agreed upon methods to solve common problems and are based upon sound principles of good design; thus, these pattern instances' decay implies an evolution away from good design. Currently, little is known about the causal nature of design pattern grime on technical debt and quality or how these three issues interrelate. What is the nature of the relationships between structural design pattern grime, software maintainability, and technical debt measurement? To better understand design pattern grime, we have extended the structural grime taxonomy. We developed an approach to generate design pattern grime instances and inject them with design pattern grime. Using this approach, we conducted 7 experiments evaluating the effects of 26 forms of grime, at 6 severity levels within 16 design pattern types, on software maintainability and technical debt. The results showed that depending upon grime type, grime severity, and pattern type, grime does significantly affect both maintainability and technical debt. We also conducted a verification study on pairs of pattern instances from open-source software systems to evaluate how well the injection process represents the real effects of grime and to verify the results of the experiments. The results of this study showed that there is a disconnect between the injection process and reality, indicating that refinements are still needed. However, the verification study worked as expected in indicating where issues may exist in the process.Item The analysis of binary file security using a hierarchical quality model(Montana State University - Bozeman, College of Engineering, 2022) Johnson, Andrew Lucas; Chairperson, Graduate Committee: Clemente IzurietaSoftware security is commanding significant attention from practitioners. In many organizations, security assessment has been integrated into the software development lifecycle, which allows for continuous monitoring of software weaknesses and vulnerabilities throughout the development process. One often overlooked aspect of the software development lifecycle is the end of the lifecycle. Prior to delivering software to customers, many vendors digitally sign and compile source code into a binary. In binary form, analysis may be done to reveal security flaws that were not present in the original code or that were injected at some point between the code being written and the code being compiled. Our research goal is to improve our ability to assess the security quality of a binary from different stakeholders' perspectives. While many analysis tools exist that identify security flaws, there is little work done to enable the use of multiple tools, which is necessary to identify different types of security flaws. To accomplish our goal, we approach the problem from the perspective of quality modeling. We have designed and developed a software quality model for assessing security quality in binaries (PIQUE-Bin) and operationalized the model by using PIQUE, the Platform for Investigative software Quality Understanding and Evaluation. The design of our model is based on the Microsoft STRIDE model and the software development view of the Common Weakness Enumeration (CWE). The model produces a relative and subjective security score for a binary file. An informal literature review reveals a lack of model-based security metrics targeting binary files, which helped motivate this research. To enhance the validity of this work, a sensitivity analysis assessment based on a benchmark repository of 700 binary files was performed. Model output is validated by measuring tool output sensitivity and calibrated against the presence of injected vulnerabilities. We find that our model is able to measure the security quality of binaries relative to the benchmark repository.Item An extensible, hierarchical architecture for analysis of software quality assurance(Montana State University - Bozeman, College of Engineering, 2021) Rice, David Mark; Chairperson, Graduate Committee: Clemente IzurietaAs software becomes integrated into most aspects of life, a need to assess and guarantee the quality of a software product is paramount. Poor software quality can lead to traffic accidents, failure of life-saving devices, government destabilization, and economic ruin. To assess software quality, quality researchers design quality models. A common quality model will decompose quality concepts such as 'total quality', 'maintainability', and 'confidentiality' into a hierarchy that can eventually be linked to specific lines of code in a software system. However, a problem persists in the domain of quality modeling: quality assessment through use of quality models is not finding acceptance by industry practitioners. This thesis reviews the weaknesses of modern modeling attempts and aims to improve the processes surrounding quality assessment from the perspective of both researchers and academic practitioners. The analysis uses the Goal/Question/Metric paradigm. Two closely related goals are presented that aim to analyze a process of generating, validating, and operationalizing quality models for the purpose of improvement with respect to cost, experimentative capability, collaborative opportunity, and acceptability. A system is designed, PIQUE, that provides functionality to generate experimental quality models. Test cases and exercises are run on the models generated by PIQUE to supply metric data used to answer the questions and goals. The results show that-in the context of a PIQUE-generated quality model compared to a similar non-PIQUE quality model-improvement can be achieved with respect to development cost and experimentative capability. Clear improvement was not found in the context of model operationalization difficulty and output acceptability. Ultimately, partial achievement of both goals is realized. The work concludes that the current problems in the domain of quality modeling can be improved upon, and systems like PIQUE are a valuable approach toward that goal.