topics for research papers software engineering

For enquiries call:

+1-469-442-0620

Programming

What are Software Engineer Research Topics?

Software engineer research topics are areas of exploration and study in the rapidly evolving field of software engineering. These research topics include various software development approaches, quality of software, testing of software, maintenance of software, security measures for software, machine learning models in software engineering, DevOps, and architecture of software. Each of these software engineer research topics has distinct problems and opportunities for software engineers to investigate and make major contributions to the field. In short, research topics for software engineering provide possibilities for software engineers to investigate new technologies, approaches, and strategies for developing and managing complex software systems.

For example, research on agile software development could identify the benefits and drawbacks of using agile methodology, as well as develop new techniques for effectively implementing agile practices. Software testing research may explore new testing procedures and tools, as well as assess the efficacy of existing ones. Software quality research may investigate the elements that influence software quality and develop approaches for enhancing software system quality and minimizing the faults and errors. Software metrics are quantitative measures that are used to assess the quality, maintainability, and performance of software.

The research papers on software engineering topics in this specific area could identify novel measures for evaluating software systems or techniques for using metrics to improve the quality of software. The practice of integrating code changes into a common repository and pushing code changes to production in small, periodic batches is known as continuous integration and deployment (CI/CD). This research could investigate the best practices for establishing CI/CD or developing tools and approaches for automating the entire CI/CD process.

2. Natural Language Processing

Multimodality

Multimodality in Natural Language Processing (NLP) is one of the appealing research ideas for software engineering at the nexus of computer vision, speech recognition, and NLP. The ability of machines to comprehend and generate language from many modalities, such as text, speech, pictures, and video, is referred to as multimodal NLP. The goal of multimodal NLP is to develop systems that can learn from and interpret human communication across several modalities, allowing them to engage with humans in more organic and intuitive ways.

The building of conversational agents or chatbots that can understand and create responses using several modalities is one example of multimodal NLP in action. These agents can analyze text input, voice input, and visual clues to provide more precise and relevant responses, allowing users to have a more natural and seamless conversational experience. Furthermore, multimodal NLP can be used to enhance language translation systems, allowing them to more accurately and effectively translate text, speech, and visual content.

The development of multimodal NLP systems must take efficiency into account. as multimodal NLP systems require significant computing power to process and integrate information from multiple modalities, optimizing their efficiency is critical to ensuring that they can operate in real-time and provide users with accurate and timely responses. Developing algorithms that can efficiently evaluate and integrate input from several modalities is one method for improving the efficiency of multimodal NLP systems.

Overall, efficiency is a critical factor in the design of multimodal NLP systems. Researchers can increase the speed, precision, and scalability of these systems by inventing efficient algorithms, pre-processing approaches, and hardware architectures, allowing them to run successfully and offer real-time replies to consumers. Software Engineering training will help you level up your career and gear up to land you a job in the top product companies as a skilled Software Engineer.

3. Applications of Data Mining in Software Engineering

Mining Software Engineering Data

The mining of software engineering data is one of the significant research paper topics for software engineering, involving the application of data mining techniques to extract insights from enormous datasets that are generated during software development processes. The purpose of mining software engineering data is to uncover patterns, trends, and various relationships that can inform software development practices, increase software product quality, and improve software development process efficiency.

Mining software engineering data, despite its potential benefits, has various obstacles, including the quality of data, scalability, and privacy of data. Continuous research in this area is required to develop more effective data mining techniques and tools, as well as methods for ensuring data privacy and security, to address these challenges. By tackling these issues, mining software engineering data can continue to promote many positive aspects in software development practices and the overall quality of product.

Clustering and Text Mining

Clustering is a data mining approach that is used to group comparable items or data points based on their features or characteristics. Clustering can be used to detect patterns and correlations between different components of software, such as classes, methods, and modules, in the context of software engineering data.

On the other hand, text mining is a method of data mining that is used to extract valuable information from unstructured text data such as software manuals, code comments, and bug reports. Text mining can be applied in the context of software engineering data to find patterns and trends in software development processes

4. Data Modeling

Data modeling is an important area of research paper topics in software engineering study, especially in the context of the design of databases and their management. It involves developing a conceptual model of the data that a system will need to store, organize, and manage, as well as establishing the relationships between various data pieces. One important goal of data modeling in software engineering research is to make sure that the database schema precisely matches the system's and its users' requirements. Working closely with stakeholders to understand their needs and identify the data items that are most essential to them is necessary.

5. Verification and Validation

Verification and validation are significant research project ideas for software engineering research because they help us to ensure that software systems are correctly built and suit the needs of their users. While most of the time, these terms are frequently used interchangeably, they refer to distinct stages of the software development process. The process of ensuring that a software system fits its specifications and needs is referred to as verification. This involves testing the system to confirm that it behaves as planned and satisfies the functional and performance specifications. In contrast, validation is the process of ensuring that a software system fulfils the needs of its users and stakeholders.

This includes ensuring that the system serves its intended function and meets the requirements of its users. Verification and validation are key components of the software development process in software engineering research. Researchers can help to improve the functionality and dependability of software systems, minimize the chance of faults and mistakes, and ultimately develop better software products for their consumers by verifying that software systems are designed correctly and that they satisfy the needs of their users.

6. Software Project Management

Software project management is an important component of software engineering research because it comprises the planning, organization, and control of resources and activities to guarantee that software projects are finished on time, within budget, and to the needed quality standards. One of the key purposes of software project management in research is to guarantee that the project's stakeholders, such as users, clients, and sponsors, are satisfied with their needs. This includes defining the project's requirements, scope, and goals, as well as identifying potential risks and restrictions to the project's success.

7. Software Quality

The quality of a software product is defined as how well it fits in with its criteria, how well it performs its intended functions, and meets the needs of its consumers. It includes features such as dependability, usability, maintainability, effectiveness, and security, among others. Software quality is a prominent and essential research topic in software engineering. Researchers are working to provide methodologies, strategies, and tools for evaluating and improving software quality, as well as forecasting and preventing software faults and defects. Overall, software quality research is a large and interdisciplinary field that combines computer science, engineering, and statistics. Its mission is to increase the reliability, accessibility, and overall quality of software products and systems, thereby benefiting both software developers and end consumers.

8. Ontology

Ontology is a formal specification of a conception of a domain used in computer science to allow knowledge sharing and reuse. Ontology is a popular and essential area of study in the context of software engineering research. The construction of ontologies for specific domains or application areas could be a research topic in ontology for software engineering. For example, a researcher may create an ontology for the field of e-commerce to give common knowledge and terminology to software developers as well as stakeholders in that domain. The integration of several ontologies is another intriguing study topic in ontology for software engineering. As the number of ontologies generated for various domains and applications grows, there is an increasing need to integrate them in order to enable interoperability and reuse.

9. Software Models

In general, a software model acts as an abstract representation of a software system or its components. Software models can be used to help software developers, different stakeholders, and users communicate more effectively, as well as to properly evaluate, design, test, and maintain software systems. The development and evaluation of modeling languages and notations is one research example connected to software models. Researchers, for example, may evaluate the usefulness and efficiency of various modeling languages, such as UML or BPMN, for various software development activities or domains.

Researchers could also look into using software models for software testing and verification. They may investigate how models might be used to produce test cases or to do model checking, a formal technique for ensuring the correctness of software systems. They may also examine the use of models for monitoring at runtime and software system adaptation.

The Software Development Life Cycle (SDLC) is a software engineering process for planning, designing, developing, testing, and deploying software systems. SDLC is an important research issue in software engineering since it is used to manage software projects and ensure the quality of the resultant software products by software developers and project managers. The development and evaluation of novel software development processes is one SDLC-related research topic. SDLC research also includes the creation and evaluation of different software project management tools and practices.

Researchers may also check the implementation of SDLC in specific sectors or applications. They may, for example, investigate the use of SDLC in the development of systems that are more safety-critical, such as medical equipment or aviation systems, and develop new processes or tools to ensure the safety and reliability of these systems. They may also look into using SDLC to design software systems in new sectors like the Internet of Things or in blockchain technology.

Why is Software Engineering Required?

Software engineering is necessary because it gives a systematic way to developing, designing, and maintaining reliable, efficient, and scalable software. As software systems have become more complicated over time, software engineering has become a vital discipline to ensure that software is produced in a way that is fully compatible with end-user needs, reliable, and long-term maintainable.

When the cost of software development is considered, software engineering becomes even more important. Without a disciplined strategy, developing software can result in overinflated costs, delays, and a higher probability of errors that require costly adjustments later. Furthermore, software engineering can help reduce the long-term maintenance costs that occur by ensuring that software is designed to be easy to maintain and modify. This can save money in the long run by lowering the number of resources and time needed to make software changes as needed.

2. Scalability

Scalability is an essential factor in software development, especially for programs that have to manage enormous amounts of data or an increasing number of users. Software engineering provides a foundation for creating scalable software that can evolve over time. The capacity to deploy software to diverse contexts, such as cloud-based platforms or distributed systems, is another facet of scalability. Software engineering can assist in ensuring that software is built to be readily deployed and adjusted for various environments, resulting in increased flexibility and scalability.

3. Large Software

Developers can break down huge software systems into smaller, simpler parts using software engineering concepts, making the whole system easier to maintain. This can help to reduce the software's complexity and makes it easier to maintain the system over time. Furthermore, software engineering can aid in the development of large software systems in a modular fashion, with each module doing a specific function or set of functions. This makes it easier to push new features or functionality to the product without causing disruptions to the existing codebase.

4. Dynamic Nature

Developers can utilize software engineering techniques to create dynamic content that is modular and easily modifiable when user requirements change. This can enable adding new features or functionality to dynamic content easier without disturbing the existing codebase. Another factor to consider for dynamic content is security. Software engineering can assist in ensuring that dynamic content is generated in a secure manner that protects user data and information.

5. Better Quality Management

An organized method of quality management in software development is provided by software engineering. Developers may ensure that software is conceived, produced, and maintained in a way that fulfills quality requirements and provides value to users by adhering to software engineering principles. Requirement management is one component of quality management in software engineering. Testing and validation are another part of quality control in software engineering. Developers may verify that their software satisfies its requirements and is error-free by using an organized approach to testing.

In conclusion, the subject of software engineering provides a diverse set of research topics with the ability to progress the discipline while enhancing software development and maintenance procedures. This article has dived deep into various research topics in software engineering for masters and research topics for software engineering students such as software testing and validation, software security, artificial intelligence, Natural Language Processing, software project management, machine learning, Data Mining, etc. as research subjects. Software engineering researchers have an interesting chance to explore these and other research subjects and contribute to the development of creative solutions that can improve software quality, dependability, security, and scalability.

Researchers may make important contributions to the area of software engineering and help tackle some of the most serious difficulties confronting software development and maintenance by staying updated with the latest research trends and technologies. As software grows more important in business and daily life, there is a greater demand for current research topics in software engineering into new software engineering processes and techniques. Software engineering researchers can assist in shaping the future of software creation and maintenance through their research, ensuring that software stays dependable, safe, reliable and efficient in an ever-changing technological context. KnowledgeHut’s top Programming certification course will help you leverage online programming courses from expert trainers.

Frequently Asked Questions (FAQs)

Ans: To find a research topic in software engineering, you can review recent papers and conference proceedings, talk to different experts in the field, and evaluate your own interests and experience. You can use a combination of these approaches.

Ans: You should study software development processes, various programming languages and their frameworks, software testing and quality assurance, software architecture, various design patterns that are currently being used, and software project management as a software engineering student.

Ans: Empirical research, experimental research, surveys, case studies, and literature reviews are all types of research in software engineering. Each sort of study has advantages and disadvantages, and the research method chosen is determined by the research objective, resources, and available data.

Eshaan Pandey

Eshaan is a Full Stack web developer skilled in MERN stack. He is a quick learner and has the ability to adapt quickly with respect to projects and technologies assigned to him. He has also worked previously on UI/UX web projects and delivered successfully. Eshaan has worked as an SDE Intern at Frazor for a span of 2 months. He has also worked as a Technical Blog Writer at KnowledgeHut upGrad writing articles on various technical topics.

Avail your free 1:1 mentorship session.

Something went wrong

Upcoming Programming Batches & Dates

software engineering Recently Published Documents

Total documents.

Latest Documents
Most Cited Documents
Contributed Authors
Related Sources
Related Keywords

Identifying Non-Technical Skill Gaps in Software Engineering Education: What Experts Expect But Students Don’t Learn

As the importance of non-technical skills in the software engineering industry increases, the skill sets of graduates match less and less with industry expectations. A growing body of research exists that attempts to identify this skill gap. However, only few so far explicitly compare opinions of the industry with what is currently being taught in academia. By aggregating data from three previous works, we identify the three biggest non-technical skill gaps between industry and academia for the field of software engineering: devoting oneself to continuous learning , being creative by approaching a problem from different angles , and thinking in a solution-oriented way by favoring outcome over ego . Eight follow-up interviews were conducted to further explore how the industry perceives these skill gaps, yielding 26 sub-themes grouped into six bigger themes: stimulating continuous learning , stimulating creativity , creative techniques , addressing the gap in education , skill requirements in industry , and the industry selection process . With this work, we hope to inspire educators to give the necessary attention to the uncovered skills, further mitigating the gap between the industry and the academic world.

Opportunities and Challenges in Code Search Tools

Code search is a core software engineering task. Effective code search tools can help developers substantially improve their software development efficiency and effectiveness. In recent years, many code search studies have leveraged different techniques, such as deep learning and information retrieval approaches, to retrieve expected code from a large-scale codebase. However, there is a lack of a comprehensive comparative summary of existing code search approaches. To understand the research trends in existing code search studies, we systematically reviewed 81 relevant studies. We investigated the publication trends of code search studies, analyzed key components, such as codebase, query, and modeling technique used to build code search tools, and classified existing tools into focusing on supporting seven different search tasks. Based on our findings, we identified a set of outstanding challenges in existing studies and a research roadmap for future code search research.

Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this article, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.

Towards an Anatomy of Software Craftsmanship

Context: The concept of software craftsmanship has early roots in computing, and in 2009, the Manifesto for Software Craftsmanship was formulated as a reaction to how the Agile methods were practiced and taught. But software craftsmanship has seldom been studied from a software engineering perspective. Objective: The objective of this article is to systematize an anatomy of software craftsmanship through literature studies and a longitudinal case study. Method: We performed a snowballing literature review based on an initial set of nine papers, resulting in 18 papers and 11 books. We also performed a case study following seven years of software development of a product for the financial market, eliciting qualitative, and quantitative results. We used thematic coding to synthesize the results into categories. Results: The resulting anatomy is centered around four themes, containing 17 principles and 47 hierarchical practices connected to the principles. We present the identified practices based on the experiences gathered from the case study, triangulating with the literature results. Conclusion: We provide our systematically derived anatomy of software craftsmanship with the goal of inspiring more research into the principles and practices of software craftsmanship and how these relate to other principles within software engineering in general.

On the Reproducibility and Replicability of Deep Learning in Software Engineering

Context: Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges without enormous manual feature engineering effort and complex domain knowledge. Objective: Although many DL studies have reported substantial advantages over other state-of-the-art models on effectiveness, they often ignore two factors: (1) reproducibility —whether the reported experimental results can be obtained by other researchers using authors’ artifacts (i.e., source code and datasets) with the same experimental setup; and (2) replicability —whether the reported experimental result can be obtained by other researchers using their re-implemented artifacts with a different experimental setup. We observed that DL studies commonly overlook these two factors and declare them as minor threats or leave them for future work. This is mainly due to high model complexity with many manually set parameters and the time-consuming optimization process, unlike classical supervised machine learning (ML) methods (e.g., random forest). This study aims to investigate the urgency and importance of reproducibility and replicability for DL studies on SE tasks. Method: In this study, we conducted a literature review on 147 DL studies recently published in 20 SE venues and 20 AI (Artificial Intelligence) venues to investigate these issues. We also re-ran four representative DL models in SE to investigate important factors that may strongly affect the reproducibility and replicability of a study. Results: Our statistics show the urgency of investigating these two factors in SE, where only 10.2% of the studies investigate any research question to show that their models can address at least one issue of replicability and/or reproducibility. More than 62.6% of the studies do not even share high-quality source code or complete data to support the reproducibility of their complex models. Meanwhile, our experimental results show the importance of reproducibility and replicability, where the reported performance of a DL model could not be reproduced for an unstable optimization process. Replicability could be substantially compromised if the model training is not convergent, or if performance is sensitive to the size of vocabulary and testing data. Conclusion: It is urgent for the SE community to provide a long-lasting link to a high-quality reproduction package, enhance DL-based solution stability and convergence, and avoid performance sensitivity on different sampled data.

Predictive Software Engineering: Transform Custom Software Development into Effective Business Solutions

The paper examines the principles of the Predictive Software Engineering (PSE) framework. The authors examine how PSE enables custom software development companies to offer transparent services and products while staying within the intended budget and a guaranteed budget. The paper will cover all 7 principles of PSE: (1) Meaningful Customer Care, (2) Transparent End-to-End Control, (3) Proven Productivity, (4) Efficient Distributed Teams, (5) Disciplined Agile Delivery Process, (6) Measurable Quality Management and Technical Debt Reduction, and (7) Sound Human Development.

Software—A New Open Access Journal on Software Engineering

Software (ISSN: 2674-113X) [...]

Improving bioinformatics software quality through incorporation of software engineering practices

Background Bioinformatics software is developed for collecting, analyzing, integrating, and interpreting life science datasets that are often enormous. Bioinformatics engineers often lack the software engineering skills necessary for developing robust, maintainable, reusable software. This study presents review and discussion of the findings and efforts made to improve the quality of bioinformatics software. Methodology A systematic review was conducted of related literature that identifies core software engineering concepts for improving bioinformatics software development: requirements gathering, documentation, testing, and integration. The findings are presented with the aim of illuminating trends within the research that could lead to viable solutions to the struggles faced by bioinformatics engineers when developing scientific software. Results The findings suggest that bioinformatics engineers could significantly benefit from the incorporation of software engineering principles into their development efforts. This leads to suggestion of both cultural changes within bioinformatics research communities as well as adoption of software engineering disciplines into the formal education of bioinformatics engineers. Open management of scientific bioinformatics development projects can result in improved software quality through collaboration amongst both bioinformatics engineers and software engineers. Conclusions While strides have been made both in identification and solution of issues of particular import to bioinformatics software development, there is still room for improvement in terms of shifts in both the formal education of bioinformatics engineers as well as the culture and approaches of managing scientific bioinformatics research and development efforts.

Inter-team communication in large-scale co-located software engineering: a case study

AbstractLarge-scale software engineering is a collaborative effort where teams need to communicate to develop software products. Managers face the challenge of how to organise work to facilitate necessary communication between teams and individuals. This includes a range of decisions from distributing work over teams located in multiple buildings and sites, through work processes and tools for coordinating work, to softer issues including ensuring well-functioning teams. In this case study, we focus on inter-team communication by considering geographical, cognitive and psychological distances between teams, and factors and strategies that can affect this communication. Data was collected for ten test teams within a large development organisation, in two main phases: (1) measuring cognitive and psychological distance between teams using interactive posters, and (2) five focus group sessions where the obtained distance measurements were discussed. We present ten factors and five strategies, and how these relate to inter-team communication. We see three types of arenas that facilitate inter-team communication, namely physical, virtual and organisational arenas. Our findings can support managers in assessing and improving communication within large development organisations. In addition, the findings can provide insights into factors that may explain the challenges of scaling development organisations, in particular agile organisations that place a large emphasis on direct communication over written documentation.

Aligning Software Engineering and Artificial Intelligence With Transdisciplinary

Study examined AI and SE transdisciplinarity to find ways of aligning them to enable development of AI-SE transdisciplinary theory. Literature review and analysis method was used. The findings are AI and SE transdisciplinarity is tacit with islands within and between them that can be linked to accelerate their transdisciplinary orientation by codification, internally developing and externally borrowing and adapting transdisciplinary theories. Lack of theory has been identified as the major barrier toward towards maturing the two disciplines as engineering disciplines. Creating AI and SE transdisciplinary theory would contribute to maturing AI and SE engineering disciplines. Implications of study are transdisciplinary theory can support mode 2 and 3 AI and SE innovations; provide an alternative for maturing two disciplines as engineering disciplines. Study’s originality it’s first in SE, AI or their intersections.

Export Citation Format

Share document.

Journal of Software Engineering Research and Development Cover Image

Search by keyword
Search by citation

Page 1 of 2

Metric-centered and technology-independent architectural views for software comprehension

The maintenance of applications is a crucial activity in the software industry. The high cost of this process is due to the effort invested on software comprehension since, in most of cases, there is no up-to-...

View Full Text

Back to the future: origins and directions of the “Agile Manifesto” – views of the originators

In 2001, seventeen professionals set up the manifesto for agile software development. They wanted to define values and basic principles for better software development. On top of being brought into focus, the ...

Investigating the effectiveness of peer code review in distributed software development based on objective and subjective data

Code review is a potential means of improving software quality. To be effective, it depends on different factors, and many have been investigated in the literature to identify the scenarios in which it adds qu...

On the benefits and challenges of using kanban in software engineering: a structured synthesis study

Kanban is increasingly being used in diverse software organizations. There is extensive research regarding its benefits and challenges in Software Engineering, reported in both primary and secondary studies. H...

Challenges on applying genetic improvement in JavaScript using a high-performance computer

Genetic Improvement is an area of Search Based Software Engineering that aims to apply evolutionary computing operators to the software source code to improve it according to one or more quality metrics. This ...

Actor’s social complexity: a proposal for managing the iStar model

Complex systems are inherent to modern society, in which individuals, organizations, and computational elements relate with each other to achieve a predefined purpose, which transcends individual goals. In thi...

Investigating measures for applying statistical process control in software organizations

The growing interest in improving software processes has led organizations to aim for high maturity, where statistical process control (SPC) is required. SPC makes it possible to analyze process behavior, pred...

An approach for applying Test-Driven Development (TDD) in the development of randomized algorithms

TDD is a technique traditionally applied in applications with deterministic algorithms, in which the input and the expected result are known. However, the application of TDD with randomized algorithms have bee...

Supporting governance of mobile application developers from mining and analyzing technical questions in stack overflow

There is a need to improve the direct communication between large organizations that maintain mobile platforms (e.g. Apple, Google, and Microsoft) and third-party developers to solve technical questions that e...

Working software over comprehensive documentation – Rationales of agile teams for artefacts usage

Agile software development (ASD) promotes working software over comprehensive documentation. Still, recent research has shown agile teams to use quite a number of artefacts. Whereas some artefacts may be adopt...

Development as a journey: factors supporting the adoption and use of software frameworks

From the point of view of the software framework owner, attracting new and supporting existing application developers is crucial for the long-term success of the framework. This mixed-methods study explores th...

Applying user-centered techniques to analyze and design a mobile application

Techniques that help in understanding and designing user needs are increasingly being used in Software Engineering to improve the acceptance of applications. Among these techniques we can cite personas, scenar...

A measurement model to analyze the effect of agile enterprise architecture on geographically distributed agile development

Efficient and effective communication (active communication) among stakeholders is thought to be central to agile development. However, in geographically distributed agile development (GDAD) environments, it c...

A survey of search-based refactoring for software maintenance

This survey reviews published materials related to the specific area of Search-Based Software Engineering that concerns software maintenance and, in particular, refactoring. The survey aims to give a comprehen...

Guest editorial foreword for the special issue on automated software testing: trends and evidence

Similarity testing for role-based access control systems.

Access control systems demand rigorous verification and validation approaches, otherwise, they can end up with security breaches. Finite state machines based testing has been successfully applied to RBAC syste...

An algorithm for combinatorial interaction testing: definitions and rigorous evaluations

Combinatorial Interaction Testing (CIT) approaches have drawn attention of the software testing community to generate sets of smaller, efficient, and effective test cases where they have been successful in det...

How diverse is your team? Investigating gender and nationality diversity in GitHub teams

Building an effective team of developers is a complex task faced by both software companies and open source communities. The problem of forming a “dream”

Investigating factors that affect the human perception on god class detection: an analysis based on a family of four controlled experiments

Evaluation of design problems in object oriented systems, which we call code smells, is mostly a human-based task. Several studies have investigated the impact of code smells in practice. Studies focusing on h...

On the evaluation of code smells and detection tools

Code smells refer to any symptom in the source code of a program that possibly indicates a deeper problem, hindering software maintenance and evolution. Detection of code smells is challenging for developers a...

On the influence of program constructs on bug localization effectiveness

Software projects often reach hundreds or thousands of files. Therefore, manually searching for code elements that should be changed to fix a failure is a difficult task. Static bug localization techniques pro...

DyeVC: an approach for monitoring and visualizing distributed repositories

Software development using distributed version control systems has become more frequent recently. Such systems bring more flexibility, but also greater complexity to manage and monitor multiple existing reposi...

A genetic algorithm based framework for software effort prediction

Several prediction models have been proposed in the literature using different techniques obtaining different results in different contexts. The need for accurate effort predictions for projects is one of the ...

Elaboration of software requirements documents by means of patterns instantiation

Studies show that problems associated with the requirements specifications are widely recognized for affecting software quality and impacting effectiveness of its development process. The reuse of knowledge ob...

ArchReco: a software tool to assist software design based on context aware recommendations of design patterns

This work describes the design, development and evaluation of a software Prototype, named ArchReco, an educational tool that employs two types of Context-aware Recommendations of Design Patterns, to support us...

On multi-language software development, cross-language links and accompanying tools: a survey of professional software developers

Non-trivial software systems are written using multiple (programming) languages, which are connected by cross-language links. The existence of such links may lead to various problems during software developmen...

SoftCoDeR approach: promoting Software Engineering Academia-Industry partnership using CMD, DSR and ESE

The Academia-Industry partnership has been increasingly encouraged in the software development field. The main focus of the initiatives is driven by the collaborative work where the scientific research work me...

Issues on developing interoperable cloud applications: definitions, concepts, approaches, requirements, characteristics and evaluation models

Among research opportunities in software engineering for cloud computing model, interoperability stands out. We found that the dynamic nature of cloud technologies and the battle for market domination make clo...

Game development software engineering process life cycle: a systematic review

Software game is a kind of application that is used not only for entertainment, but also for serious purposes that can be applicable to different domains such as education, business, and health care. Multidisc...

Correlating automatic static analysis and mutation testing: towards incremental strategies

Traditionally, mutation testing is used as test set generation and/or test evaluation criteria once it is considered a good fault model. This paper uses mutation testing for evaluating an automated static anal...

A multi-objective test data generation approach for mutation testing of feature models

Mutation approaches have been recently applied for feature testing of Software Product Lines (SPLs). The idea is to select products, associated to mutation operators that describe possible faults in the Featur...

An extended global software engineering taxonomy

In Global Software Engineering (GSE), the need for a common terminology and knowledge classification has been identified to facilitate the sharing and combination of knowledge by GSE researchers and practition...

A systematic process for obtaining the behavior of context-sensitive systems

Context-sensitive systems use contextual information in order to adapt to the user’s current needs or requirements failure. Therefore, they need to dynamically adapt their behavior. It is of paramount importan...

Distinguishing extended finite state machine configurations using predicate abstraction

Extended Finite State Machines (EFSMs) provide a powerful model for the derivation of functional tests for software systems and protocols. Many EFSM based testing problems, such as mutation testing, fault diag...

Extending statecharts to model system interactions

Statecharts are diagrams comprised of visual elements that can improve the modeling of reactive system behaviors. They extend conventional state diagrams with the notions of hierarchy, concurrency and communic...

On the relationship of code-anomaly agglomerations and architectural problems

Several projects have been discontinued in the history of the software industry due to the presence of software architecture problems. The identification of such problems in source code is often required in re...

An approach based on feature models and quality criteria for adapting component-based systems

Feature modeling has been widely used in domain engineering for the development and configuration of software product lines. A feature model represents the set of possible products or configurations to apply i...

Patch rejection in Firefox: negative reviews, backouts, and issue reopening

Writing patches to fix bugs or implement new features is an important software development task, as it contributes to raise the quality of a software system. Not all patches are accepted in the first attempt, ...

Investigating probabilistic sampling approaches for large-scale surveys in software engineering

Establishing representative samples for Software Engineering surveys is still considered a challenge. Specialized literature often presents limitations on interpreting surveys’ results, mainly due to the use o...

Characterising the state of the practice in software testing through a TMMi-based process

The software testing phase, despite its importance, is usually compromised by the lack of planning and resources in industry. This can risk the quality of the derived products. The identification of mandatory ...

Self-adaptation by coordination-targeted reconfigurations

A software system is self-adaptive when it is able to dynamically and autonomously respond to changes detected either in its internal components or in its deployment environment. This response is expected to ensu...

Templates for textual use cases of software product lines: results from a systematic mapping study and a controlled experiment

Use case templates can be used to describe functional requirements of a Software Product Line. However, to the best of our knowledge, no efforts have been made to collect and summarize these existing templates...

F3T: a tool to support the F3 approach on the development and reuse of frameworks

Frameworks are used to enhance the quality of applications and the productivity of the development process, since applications may be designed and implemented by reusing framework classes. However, frameworks ...

NextBug: a Bugzilla extension for recommending similar bugs

Due to the characteristics of the maintenance process followed in open source systems, developers are usually overwhelmed with a great amount of bugs. For instance, in 2012, approximately 7,600 bugs/month were...

Assessing the benefits of search-based approaches when designing self-adaptive systems: a controlled experiment

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive syst...

Revealing influence of model structure and test case profile on the prioritization of test cases in the context of model-based testing

Test case prioritization techniques aim at defining an order of test cases that favor the achievement of a goal during test execution, such as revealing failures as earlier as possible. A number of techniques ...

A metrics suite for JUnit test code: a multiple case study on open source software

The code of JUnit test cases is commonly used to characterize software testing effort. Different metrics have been proposed in literature to measure various perspectives of the size of JUnit test cases. Unfort...

Designing fault-tolerant SOA based on design diversity

Over recent years, software developers have been evaluating the benefits of both Service-Oriented Architecture (SOA) and software fault tolerance techniques based on design diversity. This is achieved by creat...

Method-level code clone detection through LWH (Light Weight Hybrid) approach

Many researchers have investigated different techniques to automatically detect duplicate code in programs exceeding thousand lines of code. These techniques have limitations in finding either the structural o...

The problem of conceptualization in god class detection: agreement, strategies and decision drivers

The concept of code smells is widespread in Software Engineering. Despite the empirical studies addressing the topic, the set of context-dependent issues that impacts the human perception of what is a code sme...

Editorial Board
Sign up for article alerts and news from this journal

Software Engineering

At Google, we pride ourselves on our ability to develop and launch new products and features at a very fast pace. This is made possible in part by our world-class engineers, but our approach to software development enables us to balance speed and quality, and is integral to our success. Our obsession for speed and scale is evident in our developer infrastructure and tools. Developers across the world continually write, build, test and release code in multiple programming languages like C++, Java, Python, Javascript and others, and the Engineering Tools team, for example, is challenged to keep this development ecosystem running smoothly. Our engineers leverage these tools and infrastructure to produce clean code and keep software development running at an ever-increasing scale. In our publications, we share associated technical challenges and lessons learned along the way.

Recent Publications

Some of our teams.

Climate and sustainability

Software engineering and programming languages

We're always looking for more talented, passionate people.

Publications
News and Events
Education and Outreach

Software Engineering Institute

Research review 2022.

At the 2022 Research Review, our researchers detail how they are forging a new path for software engineering by executing the SEI’s technical strategy to deliver tangible results.

Researchers highlight methods, prototypes, and tools aimed at the most important problems facing the DoD, industry, and academia, including AI engineering, computing at the tactical edge, threat hunting, continuous integration/continuous delivery, and machine learning trustworthiness.

Learn how our researchers' work in areas such as model-based systems engineering, DevSecOps, automated design conformance, software/cyber/AI integration, and AI network defense—to name a few—has produced value for the U.S. Department of Defense (DoD) and advanced the state of the practice.

Monday, November 14, 2022

Tuesday, november 15, 2022, wednesday, november 16, 2022.

Frontiers in Computer Science
Research Topics

Machine Learning for Software Engineering

Total Downloads

Total Views and Downloads

About this Research Topic

The complexity and size of software systems has increased to the extent that traditional manual development and maintenance techniques are no longer adequate for the management of these systems. At the same time, the capabilities of machine learning (ML) systems to operate with code - to analyze, generate, and transform software - have increased to the level that specifically trained ML systems can effectively function as programming assistants to produce or improve code. There is therefore significant potential for utilizing ML approaches to address the problem of increasing software application complexity and scale. This Research Topic will concern the application of AI techniques such as machine learning to software engineering: the application of AI techniques to accelerate software development and to improve software quality through specification and programming assistance, and to support software specification, design, implementation, maintenance, and related activities such as program translation and re-engineering for software modernization. Submissions should address the theme of AI and ML assistance for software engineering processes, including topics such as: * Machine learning approaches relevant for software engineering, including large language models (LLMs) and symbolic ML approaches such as program synthesis from examples. * Natural language processing (NLP) and image processing techniques. Papers should consider the practical application of ML and AI techniques to reduce manual workload and accelerate development for software processes such as: - Requirements formalization - Architectural design and the selection of architectural styles - Software design and the selection of design patterns - Test case construction and test suite optimization - Software modeling and the construction of digital twins - Program comprehension/documentation, program translation, and software reverse and re-engineering - Low-code and no-code software development.

Keywords : Artificial Intelligence, Machine Learning, Software Engineering, Software Development

Important Note : All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic Editors

Topic coordinators, submission deadlines, participating journals.

Manuscripts can be submitted to this Research Topic via the following journals:

total views

Demographics

No records found

total views article views downloads topic views

Top countries

Top referring sites, about frontiers research topics.

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

Google Meet
Mobile Dialer

Resent Search

Management Assignment Writing

Technical Assignment Writing

Finance Assignment Writing

Medical Nursing Writing

Resume Writing

Civil engineering writing

Mathematics and Statistics Projects

CV Writing Service

Essay Writing Service

Online Dissertation Help

Thesis Writing Help

RESEARCH PAPER WRITING SERVICE

Case Study Writing Service

Electrical Engineering Assignment Help

IT Assignment Help

Mechanical Engineering Assignment Help

Homework Writing Help

Science Assignment Writing

Arts Architecture Assignment Help

Chemical Engineering Assignment Help

Computer Network Assignment Help

Arts Assignment Help

Coursework Writing Help

Custom Paper Writing Services

Personal Statement Writing

Biotechnology Assignment Help

C Programming Assignment Help

MBA Assignment Help

English Essay Writing

MATLAB Assignment Help

Narrative Writing Help

Report Writing Help

Get Top Quality Assignment Assistance

Online Exam Help

Macroeconomics Homework Help

Change Management Assignment Help

Operation management Assignment Help

Strategy Assignment Help

Human Resource Management Assignment Help

Psychology Assignment Writing Help

Algebra Homework Help

Best Assignment Writing Tips

Statistics Homework Help

CDR Writing Services

TAFE Assignment Help

Auditing Assignment Help

Literature Essay Help

Online University Assignment Writing

Economics Assignment Help

Programming Language Assignment Help

Political Science Assignment Help

Marketing Assignment Help

Project Management Assignment Help

Geography Assignment Help

Do My Assignment For Me

Business Ethics Assignment Help

Pricing Strategy Assignment Help

The Best Taxation Assignment Help

Finance Planning Assignment Help

Solve My Accounting Paper Online

Market Analysis Assignment

4p Marketing Assignment Help

Corporate Strategy Assignment Help

Project Risk Management Assignment Help

Environmental Law Assignment Help

History Assignment Help

Geometry Assignment Help

Physics Assignment Help

Clinical Reasoning Cycle

Forex Assignment Help

Python Assignment Help

Behavioural Finance Assignment Help

PHP Assignment Help

Social Science Assignment Help

Capital Budgeting Assignment Help

Trigonometry Assignment Help

Java Programming Assignment Help

Corporate Finance Planning Help

Sports Science Assignment Help

Accounting For Financial Statements Assignment Help

Robotics Assignment Help

Cost Accounting Assignment Help

Business Accounting Assignment Help

Activity Based Accounting Assignment Help

Econometrics Assignment Help

Managerial Accounting Assignment Help

R Studio Assignment Help

Cookery Assignment Help

Solidworks assignment Help

UML Diagram Assignment Help

Data Flow Diagram Assignment Help

Employment Law Assignment Help

Calculus Assignment Help

Arithmetic Assignment Help

Write My Assignment

Business Intelligence Assignment Help

Database Assignment Help

Fluid Mechanics Assignment Help

Web Design Assignment Help

Student Assignment Help

Online CPM Homework Help

Chemistry Assignment Help

Biology Assignment Help

Corporate Governance Law Assignment Help

Auto CAD Assignment Help

Public Relations Assignment Help

Bioinformatics Assignment Help

Engineering Assignment Help

Computer Science Assignment Help

C++ Programming Assignment Help

Aerospace Engineering Assignment Help

Agroecology Assignment Help

Finance Assignment Help

Conflict Management Assignment Help

Paleontology Assignment Help

Commercial Law Assignment Help

Criminal Law Assignment Help

Anthropology Assignment Help

Biochemistry Assignment Help

Get the best cheap assignment Help

Online Pharmacology Course Help

Urgent Assignment Help

Paying For Assignment Help

HND Assignment Help

Legitimate Essay Writing Help

Best Online Proofreading Services

Need Help With Your Academic Assignment

Assignment Writing Help In Canada

Assignment Writing Help In UAE

Online Assignment Writing Help in the USA

Assignment Writing Help In Australia

Assignment Writing Help In the UK

Scholarship Essay Writing Help

University of Huddersfield Assignment Help

Ph.D. Assignment Writing Help

Law Assignment Writing Help

Website Design and Development Assignment Help

topics for research papers software engineering

150 Best Research Paper Topics For Software Engineering

Software Engineering is a branch which deals with the creation and improvement of software applications using specific methodologies and clearly defined scientific principles. When developing software products, certain procedures must be followed, the outcome of which is a reliable and reliable software product. Software is a collection of executable code for programs with associated libraries. Software that is designed to meet certain requirements is referred to as a Software Product . This is an excellent subject for a master's thesis, research, or project. There are a variety of topics within Software Engineering which will be useful to M.Tech and other students studying for their masters to write their software thesis.

What is the reason Software Engineering is required?

Software Engineering is necessary due to the frequent shifts in the requirements of users as well as the environment. Through yourch and thesis, you will learn more about the significance of Software Engineering. Here are some other areas in software engineering that are needed:

Big Software: The massive dimension of software makes it necessary for the requirements in software engineering .
Scalability The concept of scaling Software Engineering makes it possible to increase the size of existing software rather than develop brand-new software.
Cost Price Software Engineering also cuts down the manufacturing cost that is incurred during software development.
The dynamic nature of Software - Software Engineering is a crucial factor when the need for new features is to be made in software in place, in the event that the nature of software is fluid.
Better Quality Management - Software Engineering can provide more efficient software development processes to provide superior-high-quality services .

Best Research Paper Topics on Software

Software Engineering Management Unified Software Development Process and Extreme ProgrammingThere are a lot of difficulties with managing the development of software for web-based applications and projects for systems integration that were completed in recent times.
The Blue Sky Software Consulting Company Analysis
Blue Sky Software Consulting Blue Sky Software Consulting company has seen great success over 15 years. The company is not as well-equipped for the current market.
LabVIEW Software: Design Systems of Measurement
LabVIEW is a software program that was created to design systems for measurement. LabVIEW gives you a range of instruments to control the process in an experiment.
Software-producing Firm Reducing Inventory
The link between the reduction in inventory levels and the number of orders is evident. An organization that produces software may think of increasing the amount of software to a lower level.
Moet Hennessy - Louis Vuitton: Enterprise Software
The report will demonstrate how the introduction of ERP will help LVHM Group improve its results by improving its inventories, logistics and accounting.
Virtualization and Software-Defined Networking
The goal of this paper is to analyze the developments in the field of virtualization, software-defined networks and security for networks in the last three years.
Computer Hardware and Software Components
Computers that were developed at the time of the 40s of 1940 have evolved into complex machines that require software and hardware for their operation.
Applications, Software and System Development
The usage the Microsoft Office applications greatly enhance productivity in the classroom as well as at work and during everyday activities at home.
PeopleSoft Inc.'s Software Architecture and Design
With the PIA architecture, any company with an ERP application can access all of its operations through a Web browser.
Co-operative Banking Group's Enterprise Software
The report demonstrates how the implementation of the ERP system within the Co-operative Banking Group will help in improving the company's accounting, inventory and accounting practices as well as logistics processes.
Software Testing: Manual and Automated Web-Application Testing Tools
This research is an empirical study of automated and manual web-based application testing tools to determine the best tool for testing software.
JDA Software Company's Services
JDA Software is a company that has proven its worth in the development of services in areas like manufacturing, wholesale distribution, retailing and travel.
Data Management, Networking and Enterprise Software
Enterprise software is typically developed "in-house" and thus has an inflated cost when contrasted to purchasing the software from another firm.
Software Workshops and Seminars Reflections
Most seminars inspire participants to use their potential as they strive to attain their goals.
The Various Enterprise Resource Planning Software Packages
This paper's purpose is to provide an overview of the various Enterprise Resource Planning (ERP) software applications that are widely employed by companies to manage their business operations.
Explore Factors in IBM SPSS Statistical Software
The "Explore" or "Explore" command in IBM SPSS generates an output with a variety of stats for a single variable, across the entire sample or in sections of the sample.
Split Variables in IBM SPSS Statistical Software
It is the IBM SPSS software provides an option to split files into groups. The members of cases within groups can be determined by the values of split variables in this particular instance.
Syntax Code Writing in Statistical Software
The process of analyzing quantitative data by using IBM SPSS software package IBM SPSS software package often involves performing a variety of operations to calculate the statistical data for the information.
Data Coding in Statistical Software
Data coding is of utmost importance when a proper analysis of this data has to be conducted. Data coding plays an important function when you need to make use of statistical software.
Software Piracy at Kaspersky Cybersecurity Company
Software piracy is a pressing current issue that is manifested both locally with respect to an individual company and also globally.
Hotjar: Web Analytics Software Difference
This report examines Hotjar, which is a web-based analytics tool that comes with a full set of tools to evaluate. This paper examines its strengths and advantages, as well showing how it can aid in the management of decision-making.
Avast Software: Company Analysis
Avast Software is a globally well-known multinational company that is an industry leader in providing security solutions for both business and individual customers.
Project Failure, Project Planning Fundamentals, and Software Tools and Techniques for Alternative Scheduling
From lack of communication to generally unfavourable working conditions, Projects may fail when managers fail to prepare for their implementation.
Computer Elements such as Hardware and Software
Personal computers are usually different from computers used for business in terms of capabilities and the extent of technology used within the equipment.
Review of a New Framework for Software Reliability Measurement
This study draws upon the in-depth study of the software reliability measurement methods and the suggestion of a fresh foundation for reliability measurement built on the software metrics studied in the work of Amar as well as Rabai.

Good Software Research Topics & Essay Examples

Task Management Software in Organization
The goal of the plan for managing projects is to present the process of creating task management software that can be integrated into the context of the company.
A task management software plan's risk management strategy
The present study introduces us to the techniques for risk identification as well as quality assurance and a control plan and explains their significance.
Computer Software Development and Reality Shows
The growth of software in computers has been at such a fast rate over the last 10 years that it has impacted all aspects of our lives and every fibre of our being.
Scrum - Software Development Process
Digital systems and computerized systems have brought life to many areas. Scrum is a process for software development that guarantees high quality and efficiency.
Distribution of Anti-Virus Software
Numerous new threats are reported every fortnight. Cyberattacks, viruses, and other cyber-related threats are becoming an issue.
Marketing Plan: Innovative Type of Software Product
This paper will create an advertisement plan for the new kind of software, which will help to define the segment of clients and the price and communications platform.
Marketing System of Sakhr Software Co
The principal objective of this paper is to examine the marketing process in the same type of organization, like Sakhr Software Co.
Managing Information of Sakhr Software Co
This paper will examine the ideas of managing information for Sakhr Software, which is a well-known language software firm.
CRM Software in Amazon: Gains
The software for managing customers that Amazon.com developed is, from the beginning, one of the latest technology.
Neurofeedback Software and Technology Comparison
MIDI technology helps make the making of, learning or playing more enjoyable. Mobile phones and computer keyboards for music, computers etc., utilize MIDI.
PeopleSoft Software and HR.net Enterprise Software
With the help of HRIS software, HR employees are able to manage their own benefits updates and make changes, allowing them to take more time to focus on other important tasks.
Business Applications: Revelation HelpDesk by Yellow Fish Software
"Revelation HelpDesk" is an online Tracking and Support Software that facilitates seamless coordination to occur between the most important divisions within an organization.
3D signal editing methods and editing software for stereoscopic movies
3D editing for movies is one of the newest trends and is among the most complex processes in the modern film industry.
ERP Software in Inventory Management
Management of inventory ERP applications will be useful when a business has to manage the manner in which it gets goods and cleans up the merchandise.
The Capabilities of Compiere Software and How Well It Fits Into Different Industries
It is the ERP software Compiere can be used by a wide variety of users, including governments, businesses as well as non-governmental organizations (NGOs).
Software Tools for Qualitative Research
This paper reviews software tools to solve complicated tasks in the analysis of data. The paper compares NVivo, HyperRESEARCH, and Dedoose.
Data Scientist and Software Development
Data scientists convert data into insights, giving elaborate guidance to those who use the data to make educated decisions and take action.
IPR Violations in Software Development
The copyright law protects only the declaration but not the software concept. It prohibits copying code from the source without asking permission.
Health IT: Epic Software Analysis
Implementation and adoption of Health IT systems are crucial to improve the efficiency of medical practices, efficiency of workflow as well as patient outcomes.
Agile Software Development Process
The agile process for software development offers numerous benefits, such as the speedy and continuous execution of your project.
Project Management Software and Tools Comparison
The software is used by managers to ensure that there isn't any worker who is receiving more work than others and also to ensure that no worker is falling behind in their job.
Visually impaired people: challenges in Assistive Technology Software
Blind people suffer from a number of disadvantages each day while using digital technology. The various types of software and software discussed in this paper have been specifically designed to help improve the lives of blind people.
WBS completion and software project management
The PERT's results resulted in the development of The Gantt chart. This essay provides an account of the method of working with the Gantt chart.
International Software Development's Ethical Challenges: User-Useful Software
The importance of ethics is when it comes to software development. It helps the creator to create software that will be useful for the user as well as the management.
Achieving the Optimal Process. Software Development
The industry of software development is growing rapidly as the requirements of users change. This requires applications to meet these needs.

Innovative Software to Blog About

System Software: Analysis of Various Types of System Software
The paper provides opinions on the various system softwares using their strengths and weaknesses from the personal experiences of the creator.
Sakhr Software Co.'s Marketing System
The principal goal of this paper is to study the uniqueness of the system of marketing in such an organization as Sakhr Software Co from Kuwait, which specializes in NLP.
Program Code in Assembly Language Using Easy68K Software
A typical scenario is described in the report to write program code in assembly language with Easy68K software. The appropriate tests were carried out with success and outputs.
Benefits and Drawbacks of Agile Software Development Techniques
The use of agile methodologies in the software development process contributes to the improvement of work as well as the effectiveness of performance.
The use of agile methodologies in the development of software contributes to the efficiency of work and efficiency of performance.
Large Scale Software Development
This report gives information on this Resource Scheduling project. It can be useful to an advisory firm that offers various types of resources.
Penguin Sleuth, a Forensic Software Tool
The primary goal of this paper is to examine the various tools for forensic analysis and also provide a comprehensive overview of the functions available for each tool or tool pack.
System Software: Computer System Management
Computer software comprises precise preprogrammed instructions that regulate and coordinate hardware components of the computer.
Ethical Issues Involved in Software Project Management
Ethics within IT have been proven to be very different from other areas of ethics. Ethics issues in IT are usually described as having little.
Advantages and Disadvantages of Software Suites
Computer software comprises specific preprogrammed commands that control and coordinate computer hardware components of an info system.
Descriptive Statistics Using SPSS Software Suite
This paper focuses on the process of producing the descriptive statistical analysis by using SPSS. The purpose of this article is to make use of SPSS to perform an analysis of descriptive data.
Software Development: Creating a Prototype
The aim of this article is to develop an experimental software program that can be utilized to aid breast cancer patients.
Software Engineering and Methodologies
The paper explains how the author learned the software engineering process and methods as an outcome of his experiences at BTR IT Consulting Company.
Information System Hardware and Software
Information technology covers a wide variety of applications in which computer software, along with hardware, is employed.
Software Development Project Using Agile Methods
The report will provide reasons behind why the agile methodology was chosen, the method used, how the team applied this methodology, and also the lessons learned from the massive project of software development.
Flight Planning Software and Aircraft Incidents
Software for flight planning refers to programs utilized to control and manage flights and other procedures while the plane is in flight.
Hardware and Software Systems and Criminal Justice
One of the primary techniques used to decrease the chance of criminal activity is crime mapping. This involves collecting information on crimes and their causes and then analyzing it in order to identify issues.
Why Open-Source Software Will (Or Will Not) Soon Dominate the Field of Database Management Tools
The research aims to determine whether open-source software will rule the field of the database since there is an evolution in the market for business.
Business HRM Software and the Affordable Care Act
The Affordable Care Act has its strengths but also flaws. The reason is the complex nature of the law that creates a variety of challenges.
Antivirus Software Ensuring Security Online
Although it's not perfect and fragmentary, it can be seen as a supplement and not the sole instrument; antivirus software will help protect one's privacy online.
Evaluating Teaching Instructional Software for 21st-Century Technology Resources
The software for teaching Joe Rock and Friends Book 2 is designed for third-grade students who are studying English as an additional language to read and learn new vocabulary.
Britam Insurance Company's Sales and Marketing Management Software
Britam Insurance Company needs to implement the latest marketing and management software in order to keep its place at the forefront of the extremely competitive insurance market.
Software Programs: Adobe Illustrator
With Adobe Illustrator, users can quickly and precisely create various products, like logos, icons as well as drawings.
Strawberry Business: Software Project Management
Although the company has an established management strategy as well as a team of employees and efficient information systems, it lacks a standardized workplace culture and customer relations systems.
Value of Salesforce Software Using VRIO Model
Salesforce CRM software is created to help managers manage their businesses effectively. It connects all teams and managers and collects and manages customer information.
Agile software development, as well as popular variations like Scrum, are the foundation for the work of a variety of testers and developers. No matter what team or method you're currently using, you can get expert guidance on process structure and the skills required to use Lean, Agile, DevOps, Waterfall and more to help you implement it for your business.

Most Interesting Software Research Titles

What Are the Essential Attributes of Good Software?
How Computer Software Can Be Used as a Tool for Education
Accounting Software and Application Software
Online National Polling Software Requirements Specification
Building Their Software for a Company's Success
The Role of Antivirus Software in Protecting Your Computer Data
Intellectual Property Rights, Innovation and Software Technologies
Software Piracy and the Canadian Piracy Act
For the development of software projects, agile methodologies and their Waterscrumfall derivative are used.
Software Tools for Improving Underground Mine Access Layouts
How Software Can Support Academic Librarians' Changing Role
Using the Untangle Software to Overcome Obstacles for Small Businesses
By employing travel portal software, online booking sales will increase.
Analysis of Network Externality and Commercial Software Piracy
Accounting Software and Business Solutions
Analysis of Key Issues and Effects Relating to International Software Piracy
The Distinction Between Computer Science and Software Engineering
Modulation: Computer Software and Unknown Music Virus
Math Software for High School Students with Disabilities
Keyboarding Software Packages: Analysis and Purchase Recommended
Basic Software Development Life Cycle
India's Problems with Software Patents, Copyright, and Piracy
Why Has India Been Able to Build a Thriving Software Industry
Does Social Software Increase Labour Productivity
The Role of Open Source Software for Database Servers

Simple Software Essay Ideas

Human Capital and the Indian Software Industry
Input-Output Computer Windows Software
Business Software Development and Its Implementation
Evaluating Financial Management Software: Quicken Software
Which governance tools are important in Africa for combating software piracy?
Distinguish Between Proprietary Software and Off-The-Shelf
Does Social Software Support Service Innovation
Ambulatory Revenue Management Software
Difference Between Operating Systems and Application Software
Leading a Global Insurgency in the Software Sector are China and India
Call Accounting Software for Every Enterprise
Technology Standards for Software Outsourcing
The Importance of the Agile Approach for Software Development
Application Software: Publisher, Word, and Excel
Employee Monitoring Through Computer Software
Software Development Lifecycle and Testing's Importance
Tools for Global Conditional Policy to Combat Software Piracy
Software for Designing Solar Water Heating Systems
Open Source Software, Competition, and Potential Entry
Indian Software Industry: Gains are distorted and consolidated
Software Programs for Disabled Computer Users and Assistive Technology
Agile Software Architecture, Written by Christine Miyachi
Software Development: The Disadvantages of Agile Methods
Computer Software Technology for Early Childhood
Developing Test Automation Software Development

Easy Software Essay Topics

Growth Trends, Barriers, and Government Initiatives in the Indian Software Industry
How Does Enterprise Software Enable a Business to Use
Integrated Management Software the Processing of Information
Computer Software Training for Doctor's Office
Software Intellectual Property Rights and Venture Capitalist Access
Computer Science Software Specification
Software Projects and Student Software Risk Exposure
Why It Is Difficult to Create Software for Wireless Devices
Affiliate Tracking Software Your Payment Options
How Can Volkswagen Recover From the Cheating Issues It Had Because Illegal Software Was Installed?
Principles of Best Forensic Software Tool
The American Software Industry: A Historical Analysis
How Peripheral Developers Contribute to the Development of Open-Source Software
Agile Methodologies for Software Development
Key Macroeconomic Factors That Affect Software Industry
The Software Industry and India's Economic Development
Improving Customer Service Through Help Desk Software
Enterprise Resource Planning and Sap Software
Antivirus Software and Its Importance
Hardware and Software Used in Public Bank
The Effects of Computer Software Piracy on the Global Economy
Using the Winqsb Software in Critical Path Analysis
General Information About Interactive Multimedia-Based Educational Software
How Affiliate Tracking Software Can Benefit You
Computer Software and Recent Technologies

Frequently asked questions

What are the main topics of software engineering .

software development.

Introduction
Models and architecture for software development
Project management for software (SPM)
Software prerequisites
Testing and debugging software

What makes good research in software engineering ?

The most typical research strategy in software engineering is coming up with a novel method or methodology, validating it through analysis, or demonstrating its application through a case study;

What projects are good for software engineering ?

monitoring of Android tasks.
Analyzing attitudes to rate products
ATM with a fingerprint-based method.
a modern system for managing employees.
Using the AES technique for image encryption.
vote-by-fingerprint technology.
system for predicting the weather

What are the research methods in software engineering ?

We list and contrast the five categories of research methodology that, in our opinion, are most pertinent to software engineering: controlled experiments (including quasi-experiments); case studies (both exploratory and confirmatory); survey research; ethnographies; action research; and controlled experiments.

Is software engineering a research area ?

A relatively recent area of research, software engineering is derived from computer science. Its significance has been generally acknowledged by more and more academics in the field of computers throughout the course of six decades, from 1948 to the present, and it has developed into a vibrant and promising division of the computing profession.

Is software engineering easy ?

Yes, learning software engineering can be challenging at first, especially for those without programming or coding experience or any background in technology. However, numerous courses, tools, and other resources are available to assist with learning how to become a software engineer.

Who is the father of software engineering ?

The "father of software quality," Watts S. Humphrey, was an American software engineering pioneer who lived in Battle Creek, Michigan (U.S.) from July 4, 1927, to October 28, 2010.

What do you do in software engineering ?

roles and tasks for software engineers
creating and keeping up software systems.
testing and evaluating new software applications.
software speed and scalability optimization.
code creation and testing.
consulting with stakeholders such as clients, engineers, security experts, and others.

Which is better it or software engineering ?

IT support engineers cannot build sophisticated solutions, while software engineers can. In a word, they are in charge of creating and putting into use software. Knowing the distinctions makes it easier to choose the right individual to handle our tech-related problems.

Are junior software engineers in demand ?

Yes, there is a need for young coders.

Is software engineering going down ?

Software experts and software goods are oversaturating the job market for software engineers.

What degree do I need to be a software engineer ?

undergraduate degree

Can I be a software engineer without a degree ?

Many software developers lack a degree from a reputable university (or, in some circumstances, none at all).

How many years can a software engineer work ?

An engineer who wants to work in IT has a 15–20 year window.

How many hours do software engineers work ?

Software developers put in 8 to 9 hours each day, or 40 to 45 hours per week.

Top 10 Best Universities Ranking list in India 2022

Generic Conventions: Assignment Help Services

Research Paper Topics For Medical

Top 5 Resources for Writing Excellent Academic Assignments

How to Write a Literature Review for Academic Purposes

Tips for Writing a killer introduction to your assignment

How To Write A Compelling Conclusion For Your University Assignment

Research Papers Topics For Social Science

Best 150 New Research Paper Ideas For Students

7 Best Plagiarism Checkers for Students And Teachers in 2024

Enquiry form.

Search Search for:
Architecture
Military Tech
DIY Projects

Software Engineer Research Paper Topics 2021: Top 5

Whether you’re studying in advance or you’re close to getting that Software Engineering degree, it’s crucial that you look for possible research paper topics in advance. This will help you have an advantage in your course.

First off, remember that software engineering revolves around tech development and improvement.

Hence, your research paper should have the same goal. It shouldn’t be too complex so that you can go through it smoothly. At the same time, it shouldn’t be too easy to the point that it can be looked up online.

Choosing can be a difficult task. Students are often choosing buy assignment from a professional writer because of the wrong topic choice. Thus, to help you land on the best topic for your needs, we have listed the top 5 software engineer research paper topics in the next sections.

Machine Learning

Machine learning is one of the most used research topics of software engineers. If you’re not yet familiar with this, it’s a field that revolves around producing programs that improve its algorithm on its own just by the use of existing data and experience.

Basically, the art of machine learning aims to make intelligent tools. Here, you will need to use various statistical methods for your computers’ algorithms. This somehow makes it a complex and long topic.

Even so, the good thing about the said field is it covers a lot of subtopics. These can include using machine learning for face spoof detection, iris detection, sentiment analysis technique, and likes. Usually, though, machine learning will go hand in hand with certain detection systems.

Artificial Intelligence

Artificial Intelligence is a much easier concept than machine learning. Note, though, that the latter is just another type of AI tool.

AI refers to the human-like intelligence integrated into machines and computer programs. Focusing on this will give you much more topics to write about. Since it’s present in a lot of fields like gaming, marketing, and even random automated tasks, you will have more materials to refer to.

Some things that you can write about in your paper include AI’s relationship with software engineering, robotics, and natural processing. You can also write about the different types of artificial intelligence tools for a more guided research paper.

Internet Of Things

Another topic that you can write about is the Internet of Things, or more commonly known as IoT . This refers to interconnected devices, machines, or even living beings as long as a network exists.

Writing about IoT will open a huge array of possibilities to write about. You can talk about whether the topic is a problem that needs additional solutions or improvements. At the same time, you will be able to talk about specific machine requirements since IoT works mainly with communication servers.

In addition, the concept of the Internet of Things is also used in several fields like agriculture, e-commerce, and medicine. Because of this, you can rest assured that you won’t run out of things to talk about or refer to.

Software Development Models

Next up, we have software development models. If you want to write about a research paper(or maybe you decided to purchase custom research paper ?) relating to how one can start building an app or software, then using software development models as a topic is a good choice.

Here, you can choose to write about what the concept is or delve deeper into its different types. You can look into the Waterfall Model, V-Model, Incremental, RAD, Agile, Iterative, Spiral, and Prototype. You can choose either one or all of the models and then relate them to software engineering.

Clone Management

One of the most important elements in software engineering is the clone base. Hence, using this as a research topic will help you stay relevant to your course and its needs. In particular, you can focus on clone management.

Clone management is a task that revolves around ensuring that a database is free from error and duplicated codes. What makes this a good topic is its materials are still limited in the field of software engineering. This is compared to other clone-related topics. Hence, you can ensure a distinct topic for your paper.

To land on the best topic, take your interest into account. Look for the field that makes you curious and entertained. In this way, you can build motivation to actually know more about it, and not just for the sake of submitting.

Another good tip is to choose a unique topic. The ones we discussed above can be considered unique since they are some of the latest software-related topics. If you’re going to use a common one, then make sure that you put your own little twist to it. You can also consider seeing the topic in a different light.

Anyhow, your research paper, its grade, and overall quality will greatly depend on what you choose to write about.

Software Engineering Institute

Technical papers.

The SEI Digital Library houses thousands of technical papers and other documents, ranging from SEI Technical Reports on groundbreaking research to conference proceedings, survey results, and source code.

Explainable Verification: Survey, Situations, and New Ideas

April 16, 2024 • white paper, by bjorn andersson, mark h. klein, dionisio de niz.

This report focuses on potential changes in software development practice and research that would help tools used for formal methods explain their output, making software practitioners more likely to trust …

Zero Trust Industry Days 2024 Scenario: Secluded Semiconductors, Inc.

February 27, 2024 • white paper, by rhonda brown.

This scenario guides discussions of solutions submitted to address the challenges of implementing zero trust.

Considerations for Evaluating Large Language Models for Cybersecurity Tasks

February 20, 2024 • white paper, by jeff gennari, shing-hon lau, samuel j. perl, joel parish (openai), girish sastry (openai).

In this paper, researchers from the SEI and OpenAI explore the opportunities and risks associated with using large language models (LLMs) for cybersecurity tasks.

Navigating Capability-Based Planning: The Benefits, Challenges, and Implementation Essentials

February 7, 2024 • white paper, by anandi hira, william nichols.

Based on industry and government sources, this paper summarizes the benefits and challenges of implementing Capability-Based Planning (CBP).

Encoding Verification Arguments to Analyze High-Level Design Certification Claims: Experiment Zero (E0)

January 18, 2024 • white paper, by bjorn andersson, mark h. klein, dionisio de niz, douglas schmidt (vanderbilt university), ronald koontz (boeing company), john lehoczky (carnegie mellon university), george romanski (federal aviation administration), jonathan preston (lockheed martin corporation), daniel shapiro (institute of defense analysis), floyd fazi (lockheed martin corporation), david tate (institute of defense analysis), gordon putsche (the boeing company), hyoseung kim (university of california, riverside).

This paper discusses whether automation of certification arguments can identify problems that occur in real systems.

The Measurement Challenges in Software Assurance and Supply Chain Risk Management

December 22, 2023 • white paper, by nancy r. mead, carol woody, scott hissam.

This paper recommends an approach for developing and evaluating cybersecurity metrics for open source and other software in the supply chain.

Report to the Congressional Defense Committees on National Defense Authorization Act (NDAA) for Fiscal Year 2022 Section 835 Independent Study on Technical Debt in Software-Intensive Systems

December 7, 2023 • technical report, by ipek ozkaya, brigid o'hearn, julie b. cohen, forrest shull.

This independent study of technical debt in software-intensive systems was sent to Congress in December 2023 to satisfy the requirements of NDAA Section 835.

Assessing Opportunities for LLMs in Software Engineering and Acquisition

November 1, 2023 • white paper, by julie b. cohen, james ivers, ipek ozkaya, stephany bellomo, shen zhang.

This white paper examines how decision makers, such as technical leads and program managers, can assess the fitness of large language models (LLMs) to address software engineering and acquisition needs.

Acquisition Security Framework (ASF): Managing Systems Cybersecurity Risk (Expanded Set of Practices)

October 2, 2023 • technical note, by michael s. bandor, charles m. wallen, carol woody, christopher j. alberts.

This framework of practices helps programs coordinate their management of engineering and supply chain risks across the systems lifecycle.

Simulating Realistic Human Activity Using Large Language Model Directives

October 2, 2023 • technical report, by sean huff, thomas g. podnar, dustin d. updyke.

The authors explore how activities generated from the GHOSTS Framework’s NPC client compare to activities produced by GHOSTS’ default behavior and LLMs.

Why Your Software Cost Estimates Change Over Time and How DevSecOps Data Can Help Reduce Cost Risk

September 29, 2023 • white paper, by julie b. cohen.

Early software cost estimates are often off by over 40%; this paper discusses how programs must continually update estimates as more information becomes available.

A Retrospective in Engineering Large Language Models for National Security

By andrew o. mellinger, tyler brooks, shannon gallagher, bryan brown, eric heim, hollen barmer, william nichols, nick winski, nathan m. vanhoudnos, jasmine ratchford, angelique mcdowell, swati rallapalli.

This document discusses the findings, recommendations, and lessons learned from engineering a large language model for national security use cases.

U.S. Leadership in Software Engineering & AI Engineering: Critical Needs & Priorities Workshop - Executive Summary

August 25, 2023 • white paper, by ipek ozkaya, douglas schmidt (vanderbilt university), forrest shull, john e. robert, erin harper, anita carleton.

A joint SEI/NITRD workshop will advance U.S. national interests through software and AI engineering and accelerate progress across virtually all scientific domains.

A Holistic View of Architecture Definition, Evolution, and Analysis

August 24, 2023 • technical report, by james ivers, sebastián echeverría, rick kazman.

This report focuses on performing architectural decisions and architectural analysis, spanning multiple quality attributes, in a sustainable and ongoing way.

Emerging Technologies: Seven Themes Changing the Future of Software in the DoD

August 24, 2023 • white paper, by scott hissam, shen zhang, michael abad-santos.

This report summarizes the SEI's Emerging Technologies Study (ETS) and identifies seven emerging technologies to watch in software engineering practices and technology.

Demonstrating the Practical Utility and Limitations of ChatGPT Through Case Studies

August 23, 2023 • white paper, by clarence worrell, matthew walsh, alejandro gomez, dominic a. ross.

In this study, SEI researchers conducted four case studies using GPT-3.5 to assess the practical utility of large language models such as ChatGPT.

Software Excellence Through the Agile High Velocity Development℠ Process

July 17, 2023 • technical report, by barti k. perini (ishpi information technologies, inc.), stephen shook (ishpi information technologies, inc.).

The High Velocity Development℠ process earned Ishpi Information Technologies, Inc. the 2023 Watts Humphrey Software Quality Award.

Coding the Future: Recommendations for Defense Software R&D

July 13, 2023 • white paper, by software engineering institute.

This report outlines the key recommendations from the November 2022 workshop "Software as a Modernization Priority."

Engineering of Edge Software Systems: A Report from the November 2022 SEI Workshop on Software Systems at the Edge

June 30, 2023 • white paper, by ipek ozkaya, grace lewis, kevin a. pitstick.

Based on a workshop with thought leaders in the field, this report identifies recommended areas of focus for engineering software systems at the edge.

Software Bill of Materials Framework: Leveraging SBOMs for Risk Reduction

June 14, 2023 • white paper, by carol woody, christopher j. alberts, michael s. bandor, charles m. wallen.

This paper is a Software Bill of Materials (SBOM) Framework that is a starting point for expanding the use of SBOMs for managing software and systems risk.

Generative AI: Key Opportunities and Research Challenges

June 9, 2023 • white paper.

This 2023 workshop report identifies DoD use cases for generative AI and discusses meeting challenges and needs such as investing in guardrails and responsible AI amid a race to capability.

Securing UEFI: An Underpinning Technology for Computing

May 30, 2023 • white paper, by vijay s. sarvepalli.

This paper highlights the technical efforts to secure the UEFI-based firmware that serves as a foundational piece of modern computing environments.

Using Model-Based Systems Engineering (MBSE) to Assure a DevSecOps Pipeline is Sufficiently Secure

May 23, 2023 • technical report, by timothy a. chick, nataliya shevchenko, scott pavetti.

This report describes how analysts can use a model-based systems engineering (MBSE) approach to detect and mitigate cybersecurity risks to a DevSecOps pipeline.

A Strategy for Component Product Lines: Report 2: Specification Modeling for Components in a Component Product Line

May 17, 2023 • special report, by john mcgregor, john j. hudak, sholom g. cohen.

This report introduces the “model chain” concept for specifying a component product line and realizing architecture requirements through the creation–evolution process.

A Strategy for Component Product Lines: Report 3: Component Product Line Governance

May 4, 2023 • special report, by sholom g. cohen, alfred schenker.

This report provides guidance for the community involved with developing and sustaining product lines of components used by the U.S. government.

Program Managers—The DevSecOps Pipeline Can Provide Actionable Data

April 24, 2023 • white paper, by julie b. cohen, bill nichols.

This paper describes the Automated Continuous Estimation for a Pipeline of Pipelines research project, which automates data collection to track program progress.

Zero Trust Industry Day 2022: Areas of Future Research

January 25, 2023 • white paper, by timothy morrow, trista polaski, matthew nicolai.

This paper describes the future research discussed at the 2022 Zero Trust Industry Day event.

Industry Best Practices for Zero Trust Architecture

December 13, 2022 • white paper, by timothy morrow, nathaniel richmond, matthew nicolai.

This paper describes best practices identified during the SEI’s Zero Trust Industry Day 2022, and provides ways to help organizations shift to zero trust.

A Strategy for Component Product Lines: Report 1: Scoping, Objectives, and Rationale

December 8, 2022 • special report, by gabriel moreno, john j. hudak, sholom g. cohen, alfred schenker, john mcgregor.

This report establishes a Component Product Line Strategy to address problems in systematically reusing and integrating components built to conform to component specification models.

Acquisition Security Framework (ASF): Managing Systems Cybersecurity Risk

November 11, 2022 • technical note.

This report provides an overview of the Acquisition Security Framework (ASF), a description of the practices developed thus far, and a plan for completing the ASF body of work.

Zero Trust Industry Day Experience Paper

October 31, 2022 • white paper, by rhonda brown, mary popeck, timothy morrow.

This paper describes the results of the 2022 Zero Trust Industry Day event.

Challenge Development Guidelines for Cybersecurity Competitions

October 27, 2022 • technical report, by dennis m. allen, leena arora, joseph vessella, josh hammerstein, matt kaar, jarrett booz.

This paper draws on the SEI’s experience to provide general-purpose guidelines and best practices for developing effective cybersecurity challenges.

Acquisition Security Framework (ASF): An Acquisition and Supplier Perspective on Managing Software-Intensive Systems’ Cybersecurity Risk

October 4, 2022 • white paper, by carol woody, christopher j. alberts, charles m. wallen, michael s. bandor.

The Acquisition Security Framework (ASF) contains practices that support programs acquiring/building a secure, resilient software-reliant system to manage risks.

Designing Vultron: A Protocol for Multi-Party Coordinated Vulnerability Disclosure (MPCVD)

September 15, 2022 • special report, by allen d. householder.

This report proposes a formal protocol specification for MPCVD to improve the interoperability of both CVD and MPCVD processes.

Common Sense Guide to Mitigating Insider Threats, Seventh Edition

September 7, 2022 • technical report.

The guide describes 22 best practices for mitigating insider threat based on the CERT Division's continued research and analysis of more than 3,000 insider threat cases.

Coordinated Vulnerability Disclosure User Stories

August 25, 2022 • white paper, by art manion, timur d. snoke, vijay s. sarvepalli, jonathan spring, allen d. householder, laurie tyzenhaus, brad runyon, eric hatleback, charles g. yarbrough.

This paper provides user stories to guide the development of a technical protocol and application programming interface for Coordinated Vulnerability Disclosure.

LLVM Intermediate Representation for Code Weakness Identification

July 8, 2022 • white paper, by shannon gallagher, william klieber, david svoboda.

This paper examines whether intermediate representation used in Large Language Models can be useful to indicate the presence of software vulnerabilities.

Digital Engineering Effectiveness

May 19, 2022 • white paper, by alfred schenker, bill nichols, tyler smith (adventium labs, inc.).

This paper explores the reluctance of developers of cyber-physical systems to embrace digital engineering (DE), how DE methods should be tailored to achieve their stakeholders' goals, and how to measure …

A Brief Introduction to the Evaluation of Learned Models for Aerial Object Detection

May 2, 2022 • white paper, by eric heim.

The SEI AI Division assembled guidance on the design, production, and evaluation of machine-learning models for aerial object detection.

Guidance for Tailoring DoD Request for Proposals (RFPs) to Include Modeling

April 27, 2022 • special report, by tom merendino, robert wojcik, julie b. cohen.

This report provides guidance for government program offices that are including digital engineering/modeling requirements into a request for proposal.

Modeling to Support DoD Acquisition Lifecycle Events (Version 1.4)

April 26, 2022 • white paper, by tom merendino, julie b. cohen, robert wojcik.

This document provides suggestions for producing requirement, system, and software models that will be used to support various DoD system acquisition lifecycle events.

Experiences with Deploying Mothra in Amazon Web Services (AWS)

April 26, 2022 • technical report, by daniel ruef, john stogoski, brad powell.

The authors describe development of an at-scale prototype of an on-premises system to test the performance of Mothra in the cloud and provide recommendations for similar deployments.

Extensibility

April 6, 2022 • technical report.

This report summarizes how to systematically analyze a software architecture with respect to a quality attribute requirement for extensibility.

TwinOps: Digital Twins Meets DevOps

March 24, 2022 • technical report, by joe yankel, jerome hugues, anton hristozov, john j. hudak.

This report describes ModDevOps, an approach that bridges model-based engineering and software engineering using DevOps concepts and code generation from models, and TwinOps, a specific ModDevOps pipeline.

March 16, 2022 • Technical Report

By philip bianco, james ivers, sebastián echeverría, rick kazman.

This report summarizes how to systematically analyze a software architecture with respect to a quality attribute requirement for robustness.

An Analysis of How Many Undiscovered Vulnerabilities Remain in Information Systems

March 9, 2022 • white paper, by jonathan spring.

This paper examines the paradigm that the number of undiscovered vulnerabilities is manageably small through the lens of mathematical concepts from the theory of computing.

Using XML to Exchange Floating Point Data

February 10, 2022 • white paper, by john klein.

This paper explains issues of using XML to exchange floating point values, how to address them, and the limits of technology to enforce a correct implementation.

Using Machine Learning to Increase NPC Fidelity

December 1, 2021 • technical report, by dustin d. updyke, thomas g. podnar, geoffrey b. dobson, john yarger.

The authors describe how they used machine learning (ML) modeling to create decision-making preferences for non-player characters (NPCs).

A Prototype Set of Cloud Adoption Risk Factors

October 27, 2021 • white paper, by christopher j. alberts.

Alberts discusses the results of a study to identify a prototype set of risk factors for adopting cloud technologies.

Cloud Security Best Practices Derived from Mission Thread Analysis

September 2, 2021 • technical report, by timothy morrow, donald faatz, nathaniel richmond, angel luis hueca, vincent lapiana.

This report presents practices for secure, effective use of cloud computing and risk reduction in transitioning applications and data to the cloud, and considers the needs of limited-resource businesses.

Accenture: An Automation Maturity Journey

July 29, 2021 • technical report, by rajendra t. prasad (accenture).

This paper describes work in the area of automation that netted Accenture the 2020 Watts Humphrey Software Process Achievement Award.

Planning and Design Considerations for Data Centers

July 19, 2021 • technical note, by lyndsi a. hughes, david sweeney, mark kasunic.

This report shares important lessons learned from establishing small- to mid-size data centers.

Integrating Zero Trust and DevSecOps

July 5, 2021 • white paper, by timothy morrow, geoff sanders, nathaniel richmond, carol woody.

This paper discusses the interdependent strategies of zero trust and DevSecOps in the context of application development.

A State-Based Model for Multi-Party Coordinated Vulnerability Disclosure (MPCVD)

July 1, 2021 • special report, by allen d. householder, jonathan spring.

This report discusses performance indicators that stakeholders in Coordinated Vulnerability Disclosure (CVD) can use to measure its effectiveness.

Human-Centered AI

June 25, 2021 • white paper, by jay palat, matt gaston, frank redner, carol j. smith, tanisha smith, hollen barmer, rachel dzombak.

This white paper discusses Human-Centered AI: systems that are designed to work with, and for, people.

Robust and Secure AI

By rachel dzombak, hollen barmer, eric heim, nathan m. vanhoudnos, tanisha smith, frank redner, matt gaston, jay palat.

This white paper discusses Robust and Secure AI systems: AI systems that reliably operate at expected levels of performance, even when faced with uncertainty and in the presence of danger …

Scalable AI

By jay palat, matt gaston, frank redner, tanisha smith, hollen barmer, rachel dzombak, john wohlbier.

This white paper discusses Scalable AI: the ability of AI algorithms, data, models, and infrastructure to operate at the size, speed, and complexity required for the mission.

The Sector CSIRT Framework: Developing Sector-Based Incident Response Capabilities

June 8, 2021 • technical report, by tracy bills, sharon mudd, justin novak, brittany manley, angel luis hueca, david mcintire.

This framework guides the development and implementation of a sector CSIRT.

Foundation of Cyber Ranges

May 19, 2021 • technical report, by bill reed, dustin d. updyke, geoffrey b. dobson, thomas g. podnar.

This report details the design considerations and execution plan for building high-fidelity, realistic virtual cyber ranges that deliver maximum training and exercise value for cyberwarfare participants.

Software Assurance Guidance and Evaluation (SAGE) Tool

May 3, 2021 • white paper, by robert schiela, ebonie mcneil, luiz antunes, hasan yasar.

The Software Assurance Guidance and Evaluation (SAGE) tool helps an organization assess the security of its systems development and operations practices.

Prioritizing Vulnerability Response: A Stakeholder-Specific Vulnerability Categorization (Version 2.0)

April 30, 2021 • white paper, by jonathan spring, allen d. householder, art manion, vijay s. sarvepalli, eric hatleback, laurie tyzenhaus, madison oliver, charles g. yarbrough.

This paper presents version 2.0 of a testable Stakeholder-Specific Vulnerability Categorization (SSVC) that takes the form of decision trees and that avoids some problems with the Common Vulnerability Scoring System …

Modeling and Validating Security and Confidentiality in System Architectures

March 19, 2021 • technical report, by aaron greenhouse, lutz wrage, jörgen hansson (university of skovde).

This report presents an approach for modeling and validating conﬁdentiality using the Bell–LaPadula security model and the Architecture Analysis & Design Language.

Overview of Practices and Processes of the CMMC 1.0 Assessment Guides (CMMC 1.0)

March 3, 2021 • white paper, by douglas gardner.

This document is intended to help anyone unfamiliar with cybersecurity standards get started with the Department of Defense (DoD) Cybersecurity Maturity Model Certification (CMMC).

Zero Trust: Risks and Research Opportunities

March 1, 2021 • white paper, by geoff sanders, timothy morrow.

This paper describes a zero trust vignette and three mission threads that highlight risks and research areas to consider for zero trust environments.

Artificial Intelligence (AI) and Machine Learning (ML) Acquisition and Policy Implications

February 26, 2021 • white paper, by william e. novak.

This paper reports on a high-level survey of a set of both actual and potential acquisition and policy implications of the use of Artificial Intelligence (AI) and Machine Learning (ML) …

Security Engineering Risk Analysis (SERA) Threat Archetypes

December 16, 2020 • white paper, by carol woody, christopher j. alberts.

This report examines the concept of threat archetypes and how analysts can use them during scenario development.

Loss Magnitude Estimation in Support of Business Impact Analysis

December 15, 2020 • technical report, by brett tucker, daniel j. kambic, david tobar, andrew p. moore.

The authors describe a project to develop an estimation method that yields greater confidence in and improved ranges for estimates of potential cyber loss magnitude.

Emerging Technologies 2020: Six Areas of Opportunity

December 14, 2020 • white paper.

This study seeks to understand what the software engineering community perceives to be key emerging technologies. The six technologies described hold great promise and, in some cases, have already attracted …

Maintainability

December 1, 2020 • technical report, by rick kazman, john klein, james ivers, philip bianco.

This report summarizes how to systematically analyze a software architecture with respect to a quality attribute requirement for maintainability.

Advancing Risk Management Capability Using the OCTAVE FORTE Process

November 17, 2020 • technical note, by brett tucker.

OCTAVE FORTE is a process model that helps organizations evaluate their security risks and use ERM principles to bridge the gap between executives and practitioners.

Analytic Capabilities for Improved Software Program Management

November 2, 2020 • white paper, by christopher miller, david zubrow.

This white paper describes an update to the SEI Quantifying Uncertainty in Early Lifecycle Cost Estimation approach.

AI Engineering for Defense and National Security: A Report from the October 2019 Community of Interest Workshop

October 29, 2020 • special report.

Based on a workshop with thought leaders in the field, this report identifies recommended areas of focus for AI Engineering for Defense and National Security.

NICE Framework Cybersecurity Evaluator

August 20, 2020 • white paper, by christopher herr.

This cybersecurity evaluator is designed to assess members of the cyber workforce within the scope of the NICE Cybersecurity Workforce Framework.

Current Ransomware Threats

August 19, 2020 • white paper, by marisa midler, kyle o'meara.

This report by Marisa Midler, Kyle O'Meara, and Alexandra Parisi discusses ransomware, including an explanation of its design, distribution, execution, and business model.

An Updated Framework of Defenses Against Ransomware

August 18, 2020 • white paper, by timur d. snoke, timothy j. shimeall.

This report, loosely structured around the NIST Cybersecurity Framework, seeks to frame an approach for defending against Ransomware-as-a-Service (RaaS) as well as direct ransomware attacks.

Historical Analysis of Exploit Availability Timelines

August 13, 2020 • white paper, by david warren, jeff chrabaszcz (govini), trent novelly, allen d. householder, jonathan spring.

This paper analyzes when and how known exploits become associated with the vulnerabilities that made them possible.

Architecture Evaluation for Universal Command and Control

August 3, 2020 • white paper, by john klein, harry l. levinson, reed little, jason popowski, philip bianco, patrick donohoe.

The SEI developed an analysis method to assess function allocations in existing C2 systems and reason about design choices and tradeoffs during the design of new C2 systems.

A Risk Management Perspective for AI Engineering

June 10, 2020 • white paper.

This paper describes several steps of OCTAVE FORTE in the context of adopting AI technology.

Attack Surface Analysis - Reduce System and Organizational Risk

June 8, 2020 • white paper, by robert j. ellison, carol woody.

This paper offers system defenders an overview of how threat modeling can provide a systematic way to identify potential threats and prioritize mitigations.

Guide to Implementing DevSecOps for a System of Systems in Highly Regulated Environments

April 8, 2020 • technical report, by jose a. morales, peter capell, david james shepard, richard turner, patrick r. place, suzanne miller.

This Technical Report provides guidance to projects interested in implementing DevSecOps (DSO) in defense or other highly regulated environments, including those involving systems of systems.

Integrability

February 7, 2020 • technical report, by rick kazman, john klein, philip bianco, james ivers.

This report summarizes how to systematically analyze a software architecture with respect to a quality attribute requirement for integrability.

Comments on NISTIR 8269 (A Taxonomy and Terminology of Adversarial Machine Learning)

February 4, 2020 • white paper, by jonathan spring, april galyardt, nathan m. vanhoudnos.

Feedback to the U.S. National Institute of Standards and Technology (NIST) about NIST IR 8269, a draft report detailing the proposed taxonomy and terminology of Adversarial Machine Learning (AML).

Penetration Tests Are The Check Engine Light On Your Security Operations

January 7, 2020 • white paper, by dan j. klinedinst, allen d. householder.

A penetration test serves as a lagging indicator of a network security operations problem. Organizations should implement and document several security controls before a penetration test can be useful.

Prioritizing Vulnerability Response: A Stakeholder-Specific Vulnerability Categorization

December 4, 2019 • white paper, by allen d. householder, jonathan spring, art manion, deana shick, eric hatleback.

This paper presents a testable Stakeholder-Specific Vulnerability Categorization (SSVC) that takes the form of decision trees and that avoids some problems with the Common Vulnerability Scoring System (CVSS).

AI Engineering: 11 Foundational Practices

September 12, 2019 • white paper.

This initial set of recommendations can help organizations that are beginning to build, acquire, and integrate artificial intelligence capabilities into business and mission systems.

Machine Learning in Cybersecurity: A Guide

September 5, 2019 • technical report, by ed stoner, joshua fallon, april galyardt, jonathan spring, leigh b. metcalf, angela horneman.

This report suggests seven key questions that managers and decision makers should ask about machine learning tools to effectively use those tools to solve cybersecurity problems.

Operational Test & Evaluation (OT&E) Roadmap for Cloud-Based Systems

September 2, 2019 • white paper, by john klein, christopher j. alberts, carol woody, charles m. wallen.

This paper provides an overview of the preparation and work that the AEC needs to perform to successfully transition the Army to cloud computing.

IEEE Computer Society/Software Engineering Institute Watts S. Humphrey Software Process Achievement Award 2018: U.S. Army Combat Capabilities Development Command Armaments Center, Fire Control Systems and Technology Directorate

August 1, 2019 • technical report, by victor a. elias (u.s. army ccdc armaments center, fire control systems and technology directorate).

This report presents a systemic approach to software development process improvement and its impact for the U.S. Army Combat Capabilities Development Command Armaments Center, Fire Control Systems and Technology Directorate …

Overview of Risks, Threats, and Vulnerabilities Faced in Moving to the Cloud

July 11, 2019 • technical report, by kelwyn pender, carrie lee (u.s. department of veteran affairs), donald faatz, timothy morrow.

This report, updated in October 2020, examines the changes to risks, threats, and vulnerabilities when applications are deployed to cloud services.

Automatically Detecting Technical Debt Discussions

June 24, 2019 • white paper, by robert nord, ipek ozkaya, zachary kurtz, raghvinder sangwan.

This study introduces (1) a dataset of expert labels of technical debt in developer comments and (2) a classifier trained on those labels.

Multi-Method Modeling and Analysis of the Cybersecurity Vulnerability Management Ecosystem

By allen d. householder, andrew p. moore.

This paper presents modeling and analysis of two critical foundational processes of the cybersecurity vulnerability management ecosystem using a combination of system dynamics and agent-based modeling techniques.

SCAIFE API Definition Beta Version 0.0.2 for Developers

June 14, 2019 • white paper, by ebonie mcneil, lori flynn.

This paper provides the SCAIFE API definition for beta version 0.0.2. SCAIFE is an architecture that supports static analysis alert classification and prioritization.

Creating xBD: A Dataset for Assessing Building Damage from Satellite Imagery

May 21, 2019 • white paper.

We present a preliminary report for xBD, a new large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research.

Integration of Automated Static Analysis Alert Classification and Prioritization with Auditing Tools: Special Focus on SCALe

May 13, 2019 • technical report, by lori flynn, david svoboda, ebonie mcneil, zachary kurtz, derek leung, jiyeon lee (carnegie mellon university).

This report summarizes progress and plans for developing a system to perform automated classification and advanced prioritization of static analysis alerts.

Cybersecurity Career Paths and Progression

May 7, 2019 • white paper, by nicholas giruzzi, marie baker, dennis m. allen, melissa burns.

This paper explores the current state of cybersecurity careers, from the importance of early exposure, to methods of entry into the field, to career progression.

Cybersecurity Talent Identification and Assessment

By dennis m. allen, marie baker, christopher herr.

To help fill cybersecurity roles, this paper explores how organizations identify talent, discusses assessment capabilities, and provides recommendations on recruitment and talent evaluations.

Cybersecurity Careers of the Future

By dennis m. allen.

Using workforce data analysis, this paper identifies key cybersecurity skills the workforce needs to close the cybersecurity workforce gap.

A Targeted Improvement Plan for Service Continuity

April 8, 2019 • technical note, by philip a. scolieri, jeffrey pinckard, robert a. vrtis, andrew f. hoover, gavin jurecko.

Describes how an organization can leverage the results of a Cyber Resilience Review to create a Targeted Improvement Plan for its service continuity management.

Exploring the Use of Metrics for Software Assurance

March 7, 2019 • technical note, by carol woody, robert j. ellison, charlie ryan.

This report proposes measurements for each Software Assurance Framework (SAF) practice that a program can select to monitor and manage the progress it's making toward software assurance.

Common Sense Guide to Mitigating Insider Threats, Sixth Edition

February 27, 2019 • technical report, by sarah miller, tracy cassidy, michael c. theis, daniel l. costa, william r. claycomb, andrew p. moore, randall f. trzeciak.

The guide presents recommendations for mitigating insider threat based on the CERT Division's continued research and analysis of more than 1,500 insider threat cases.

An Approach for Integrating the Security Engineering Risk Analysis (SERA) Method with Threat Modeling

February 6, 2019 • white paper.

This report examines how cybersecurity data generated by a threat modeling method can be integrated into a mission assurance context using the SERA Method.

Infrastructure as Code: Final Report

January 28, 2019 • white paper, by doug reynolds, john klein.

This project explored the feasibility of infrastructure as code, developed prototype tools, populated a model of the deployment architecture, and automatically generated IaC scripts from the model.

Incident Management Capability Assessment

December 19, 2018 • technical report, by samuel j. perl, mark zajicek, robin ruefle, christopher j. alberts, pennie walters, carly l. huth, audrey j. dorofee, david mcintire.

The capabilities presented in this report provide a benchmark of incident management practices.

Program Manager's Guidebook for Software Assurance

December 14, 2018 • special report, by carol woody, timothy a. chick, kenneth nidiffer.

This guidebook helps program managers address the software assurance responsibilities critical in defending software-intensive systems, including mission threads and cybersecurity.

DoD Developer’s Guidebook for Software Assurance

By bill nichols, tom scanlon.

This guidebook helps software developers for DoD programs understand expectations for software assurance and standards and requirements that affect assurance.

Towards Improving CVSS

December 4, 2018 • white paper, by allen d. householder, jonathan spring, deana shick, art manion, eric hatleback.

This paper outlines challenges with the Common Vulnerability Scoring System (CVSS).

GHOSTS in the Machine: A Framework for Cyber-Warfare Exercise NPC Simulation

December 3, 2018 • technical report, by adam d. cerini, benjamin l. earl, thomas g. podnar, geoffrey b. dobson, luke j. osterritter, dustin d. updyke.

This report outlines how the GHOSTS (General HOSTS) framework helps create realism in cyber-warfare simulations and discusses how it was used in a case study.

Composing Effective Software Security Assurance Workflows

October 18, 2018 • technical report, by bill nichols, jim mchale, aaron volkmann, david sweeney, william snavely.

In an effort to determine how to make secure software development more cost effective, the SEI conducted a research study to empirically measure the effects that security tools—primarily automated static …

FedCLASS: A Case Study of Agile and Lean Practices in the Federal Government

October 5, 2018 • special report, by jeff davenport, tamara marshall-keim, linda parker gates, nanette brown.

This study reports the successes and challenges of using Agile and Lean methods and cloud-based technologies in a government software development environment.

Threat Modeling for Cyber-Physical System-of-Systems: Methods Evaluation

September 25, 2018 • white paper, by nataliya shevchenko, carol woody, brent frye.

This paper compares threat modeling methods for cyber-physical systems and recommends which methods (and combinations of methods) to use.

Software Architecture Publications

September 17, 2018 • white paper.

The SEI compiled this bibliography of publications about software architecture as a resource for information about system architecture throughout its lifecycle.

Practical Precise Taint-flow Static Analysis for Android App Sets

August 27, 2018 • white paper, by william klieber, lori flynn, william snavely, michael zheng.

This paper describes how to detect taint flow in Android app sets with a static analysis method that is fast and uses little disk and memory space.

Threat Modeling: A Summary of Available Methods

August 9, 2018 • white paper, by carol woody, nataliya shevchenko, tom scanlon, timothy a. chick, paige o'riordan.

This paper discusses twelve threat modeling methods from a variety of sources that target different parts of the development process.

Navigating the Insider Threat Tool Landscape: Low-Cost Technical Solutions to Jump-Start an Insider Threat Program

July 3, 2018 • white paper, by michael j. albrethsen, derrick spooner, daniel l. costa, george silowash.

This paper explores low cost technical solutions that can help organizations prevent, detect, and respond to insider incidents.

Blacklist Ecosystem Analysis: July - December 2017

April 19, 2018 • white paper, by leigh b. metcalf, eric hatleback.

This short report provides a summary of the various analyses of the blacklist ecosystem performed from July 1, 2017, through December 31, 2017.

ROI Analysis of the System Architecture Virtual Integration Initiative

April 12, 2018 • technical report, by jörgen hansson (university of skovde), steve helton (the boeing company), peter h. feiler.

This report presents an analysis of the economic effects of the System Architecture Virtual Integration approach on the development of software-reliant systems for aircraft compared to existing development paradigms.

Implementing DevOps Practices in Highly Regulated Environments

April 2, 2018 • white paper, by jose a. morales, aaron volkmann, hasan yasar.

In this paper, the authors layout the process with insights on performing a DevOps assessment in a highly regulated environment.

A Mapping of the Health Insurance Portability and Accountability Act (HIPAA) Security Rule to the Cyber Resilience Review (CRR)

March 29, 2018 • technical note, by robert a. vrtis, matthew trevors, greg porter (heinz college at carnegie mellon university).

This technical note describes mapping of HIPAA Security Rule requirements to practice questions found in the CERT Cyber Resilience Review for organizations' use in HIPAA compliance.

A Hybrid Threat Modeling Method

March 27, 2018 • technical note, by krishnamurthy vemuru (university of virginia), ole villadsen (carnegie mellon university), nancy r. mead, forrest shull.

Presents a hybrid method of threat modeling that attempts to meld the desirable features of three methods: Security Cards, Persona non Grata, and STRIDE.

Cyber Mutual Assistance Workshop Report

February 13, 2018 • special report, by katie c. stewart, jonathon monken (pjm interconnection), fernando maymi, phd (army cyber institute), dan bennett, phd (army cyber institute), dan huynh (army cyber institute), blake rhoades (army cyber institute), matt hutchison (army cyber institute), judy esquibel (army cyber institute), bill lawrence (north american electric reliability corporation).

The Army Cyber Institute hosted a Cyber Mutual Assistance Workshop to identify challenges in defining cyber requirements for Regional Mutual Assistance Groups.

Embedded Device Vulnerability Analysis Case Study Using Trommel

December 6, 2017 • white paper, by kyle o'meara, madison oliver.

This document provides security researchers with a repeatable methodology to produce more thorough and actionable results when analyzing embedded devices for vulnerabilities.

2017 Emerging Technology Domains Risk Survey

October 5, 2017 • technical report, by kyle o'meara, dan j. klinedinst, joel land.

This report describes our understanding of future technologies and helps US-CERT identify vulnerabilities, promote security practices, and understand vulnerability risk.

R-EACTR: A Framework for Designing Realistic Cyber Warfare Exercises

September 29, 2017 • technical report, by adam d. cerini, thomas g. podnar, geoffrey b. dobson, luke j. osterritter.

R-EACTR is a design framework for cyber warfare exercises. It ensures that designs of team-based exercises factor realism into all aspects of the participant experience.

Architecture Practices for Complex Contexts

September 26, 2017 • white paper.

This doctoral thesis, completed at Vrije Universiteit Amsterdam, focuses on software architecture practices for systems of systems, including data-intensive systems.

Defining a Progress Metric for CERT-RMM Improvement

September 8, 2017 • technical note, by david tobar, nader mehravari, gregory crabb (united states postal service).

Describes the Cybersecurity Program Progress Metric and how its implementation in a large, diverse U.S. national organization can serve to indicate progress toward improving cybersecurity and resilience capabilities.

Blacklist Ecosystem Analysis: January - June, 2017

August 22, 2017 • white paper.

This short report provides a summary of the various analyses of the blacklist ecosystem performed to date. It also appends the latest additional data to those analyses; the added data …

The CERT Guide to Coordinated Vulnerability Disclosure

August 15, 2017 • special report, by allen d. householder, art manion, christopher king, garret wassermann.

This guide provides an introduction to the key concepts, principles, and roles necessary to establish a successful Coordinated Vulnerability Disclosure process. It also provides insights into how CVD can go …

Systemic Vulnerabilities in Customer-Premises Equipment (CPE) Routers

July 11, 2017 • special report, by joel land.

This report describes a test framework that the CERT/CC developed to identify systemic and other vulnerabilities in CPE routers.

Department of Defense Software Factbook

July 11, 2017 • technical report, by david zubrow, christopher miller, rhonda brown, james mccurley, brad clark, mike zuccher (no affiliation).

In this report, the Software Engineering Institute has analyzed data related to DoD software projects and translated it into information that is frequently sought-after across the DoD.

DidFail: Coverage and Precision Enhancement

July 6, 2017 • technical report, by karan dwivedi (no affiliation), hongli yin (no affiliation), pranav bagree (no affiliation), xiaoxiao tang (no affiliation), william snavely, william klieber, lori flynn.

This report describes recent enhancements to Droid Intent Data Flow Analysis for Information Leakage (DidFail), the CERT static taint analyzer for sets of Android apps.

The Hard Choices Game Explained

June 26, 2017 • white paper, by erin lim, philippe kruchten, robert nord, nanette brown, ipek ozkaya.

The Hard Choices game is a simulation of the software development cycle meant to communicate the concepts of uncertainty, risk, and technical debt.

Federal Virtual Training Environment (FedVTE)

June 5, 2017 • white paper, by april galyardt, dominic a. ross, marie baker.

The Federal Virtual Training Environment (FedVTE) is an online, on‐demand training system containing cybersecurity and certification prep courses, at no cost to federal, state, and local government employees.

Blacklist Ecosystem Analysis: July – December 2016

June 1, 2017 • white paper.

This report provides a summary of various analyses of the blacklist ecosystem performed to date. It also appends the latest additional data to those analyses; the added data in this …

Guide to Software Architecture Tools

May 22, 2017 • white paper.

This document discusses tools and methods for analyzing the architecture, establishing requirements, evaluating the architecture, and defining the architecture.

System-of-Systems Software Architecture Evaluation

May 15, 2017 • white paper.

The SoS Architecture Evaluation Method provides an initial identification of SoS architectural risks and quality attribute inconsistencies across the constituent systems.

IEEE Computer Society/Software Engineering Institute Watts S. Humphrey Software Process Achievement Award

SEI-Certified PSP Developer Examination: Sample Questions

This page contains sample questions similar to those found on the PSP Developer examination.

IEEE Computer Society/Software Engineering Institute Watts S. Humphrey Software Process Achievement Award 2016: Raytheon Integrated Defense Systems

April 28, 2017 • technical report, by neal mackertich (raytheon), peter kraus (raytheon), kurt mittelstaedt (raytheon), brian foley (raytheon), dan bardsley (raytheon), kelli grimes (raytheon), mike nolan (raytheon).

The Raytheon Integrated Defense Systems DFSS team has been recognized with the 2016 Watts Humphrey Software Process Achievement Award.

IEEE Computer Society/Software Engineering Institute Watts S. Humphrey Software Process Achievement (SPA) Award 2016: Nationwide

April 13, 2017 • technical report, by will j.m. pohlman (nationwide it).

This report describes the 10-year history of Nationwide's software process improvement journey. Nationwide received the 2016 Watts Humphrey Software Process Achievement Award from the SEI and IEEE.

Prototype Software Assurance Framework (SAF): Introduction and Overview

April 6, 2017 • technical note.

In this report, the authors discuss the Software Assurance Framework (SAF), a collection of cybersecurity practices that programs can apply across the acquisition lifecycle and supply chain.

15 Tips for Preparing and Delivering a Great Presentation at SATURN

March 14, 2017 • white paper.

You submitted a proposal to SATURN, and it got accepted. Congratulations! Here are 15 tips for creating and giving a great presentation at SATURN.

The CISO Academy

February 23, 2017 • white paper, by pamela d. curtis, summer c. fowler, david tobar, david ulicne.

In this paper, the authors describe the project that led to the creation of the U.S. Postal Service's CISO Academy.

Agile Acquisition and Milestone Reviews

February 15, 2017 • white paper.

Acquisition & Management Concerns for Agile Use in Government Series - 4

Management and Contracting Practices for Agile Programs

Acquisition & Management Concerns for Agile Use in Government Series - 3

Estimating in Agile Acquisition

Acquisition & Management Concerns for Agile Use in Government Series - 5

Agile Development and DoD Acquisitions

Acquisition & Management Concerns for Agile Use in Government Series - 1

Agile Culture in the DoD

Acquisition & Management Concerns for Agile Use in Government Series - 2

Adopting Agile in DoD IT Acquisitions

Acquisition & Management Concerns for Agile Use in Government Series - 6

Supply Chain and Commercial-off-the-Shelf (COTS) Assurance

January 24, 2017 • white paper.

The Software Engineering Institute can help your organization apply techniques to reduce software supply chain risk.

COTS-Based Systems

This paper presents a summary of SEI commercial off-the-shelf (COTS) software documents and COTS tools.

Create a CSIRT

January 18, 2017 • white paper.

This white paper discusses the issues and decisions organizations should address when planning, implementing, and building a CSIRT.

Skills Needed When Staffing Your CSIRT

This white paper describes a set of skills that CSIRT staff members should have to provide basic incident-handling services.

CSIRT Frequently Asked Questions (FAQ)

This FAQ addresses CSIRTS, organizations responsible for receiving, reviewing, and responding to computer security incident reports and activity.

CERT-RMM Capability Appraisals

January 17, 2017 • white paper.

The white paper describe CERT-RMM appraisals and the benefits they offer organizations.

A Technical History of the SEI

January 6, 2017 • special report, by larry druffel.

This report chronicles the technical accomplishments of the Software Engineering Institute and its impact on the Department of Defense software community, as well as on the broader software engineering community.

SQUARE Frequently Asked Questions (FAQ)

January 5, 2017 • white paper.

This paper contains information about SQUARE, a process that helps organizations build security into the early stages of the software production lifecycle.

Common Sense Guide to Mitigating Insider Threats, Fifth Edition

December 21, 2016 • technical report, by tracy cassidy, michael j. albrethsen, michael c. theis, daniel l. costa, jason w. clark, andrew p. moore, randall f. trzeciak, matthew l. collins, jeremy r. strozer.

Presents recommendations for mitigating insider threat based on CERT's continued research and analysis of over 1,000 cases.

Architecture-Led Safety Process

By david p. gluch, julien delange, peter h. feiler, john mcgregor.

Architecture-Led Safety Analysis (ALSA) is a safety analysis method that uses early architecture knowledge to supplement traditional safety analysis techniques to identify faults as early as possible.

The Critical Role of Positive Incentives for Reducing Insider Threats

December 15, 2016 • technical report, by palma buttles-valdez, nathan m. vanhoudnos, samuel j. perl, tracy cassidy, andrew p. moore, daniel bauer, jennifer cowley, jeff savinda, allison parshall, matthew l. collins, elizabeth a. monaco, jamie l. moyes, denise m. rousseau (carnegie mellon university).

This report describes how positive incentives complement traditional practices to provide a better balance for organizations' insider threat programs.

Update 2016: Considerations for Using Agile in DoD Acquisition

December 14, 2016 • technical note, by alfred schenker, mary ann lapham, suzanne miller, ray c. williams, charles (bud) hammons, dan ward (dan ward consulting), daniel burton.

This report updates a 2010 technical note, addressing developments in commercial Agile practices as well as the Department of Defense (DoD) acquisition environment.

Scaling Agile Methods for Department of Defense Programs

December 13, 2016 • technical note, by suzanne miller, mary ann lapham, peter capell, eileen wrubel, will hayes.

This report discusses methods for scaling Agile processes to larger software development programs in the Department of Defense.

Low Cost Technical Solutions to Jump Start an Insider Threat Program

December 12, 2016 • technical note.

This technical note explores free and low cost technical solutions to help organizations prevent, detect, and respond to malicious insiders.

RFP Patterns and Techniques for Successful Agile Contracting

December 2, 2016 • special report, by larri ann rosser (raytheon intelligence information and services), steven martin (space and missile systems center), thomas e. friend (agile on target), greg howard (mitre), michael ryan (btas), john h. norton iii (raytheon integrated defense systems), keith korzec, peter capell, mary ann lapham.

This report discusses request-for-proposal patterns and techniques for successfully contracting a federal Agile project.

Ultra-Large-Scale Systems: Socio-adaptive Systems

December 1, 2016 • white paper, by mark h. klein, gabriel moreno, linda m. northrop, scott hissam, lutz wrage.

Ultra-large-scale systems are interdependent webs of software, people, policies, and economics. In socio-adaptive systems, humans and software interact as peers.

Cyber-Physical Systems

By david kyle, scott hissam, gabriel moreno, jeffrey hansen, john j. hudak, bjorn andersson, mark h. klein, dionisio de niz, sagar chaki.

Cyber-physical systems (CPS) integrate computational algorithms and physical components. SEI promotes the efficient development of high-confidence, distributed CPS.

Pervasive Mobile Computing

By edwin j. morris, grace lewis, james edmondson, william anderson, marc novakouski, jeff boleng, ben w. bradshaw, james root.

Pervasive mobile computing focuses on how soldiers and first responders can use smartphones, tablets, and other mobile/wearable devices at the tactical edge.

Predictability by Construction

By scott hissam, gabriel moreno, linda m. northrop, kurt c. wallnau, sagar chaki.

Predictability by construction (PBC) makes the behavior of a component-based system predictable before implementation, based on known properties of components.

Blacklist Ecosystem Analysis: January – June, 2016

Faa research project on system complexity effects on aircraft safety: testing the identified metrics, november 30, 2016 • white paper, by bill nichols, sarah sheard, michael d. konrad, charles weinstock.

This report describes a test of an algorithm for estimating the complexity of a safety argument.

FAA Research Project on System Complexity Effects on Aircraft Safety: Estimating Complexity of a Safety Argument

By charles weinstock, michael d. konrad, sarah sheard, bill nichols.

This report presents a formula for estimating the complexity of an avionics system and directly connects that complexity to the size of its safety argument.

FAA Research Project on System Complexity Effects on Aircraft Safety: Identifying the Impact of Complexity on Safety

By donald firesmith, sarah sheard, michael d. konrad, charles weinstock.

This report organizes our work on the impact of software complexity on aircraft safety by asking, “How can complexity complicate safety and, thus, certification?”

FAA Research Project on System Complexity Effects on Aircraft Safety: Candidate Complexity Metrics

By sarah sheard, bill nichols.

This special report identifies candidate measures of complexity for systems with embedded software that relate to safety, assurance, or both.

FAA Research Project on System Complexity Effects on Aircraft Safety: Literature Search to Define Complexity for Avionics Systems

By sarah sheard, michael d. konrad.

This special report describes the results of a literature review sampling what is known about complexity for application in the context of safety and assurance.

Seven Proposal-Writing Tips That Make Conference Program Committees Smile

By mike petock, bill pollak.

Writing a great session proposal for a conference is difficult. Here are seven tips for writing a session proposal that will make reviewers go from frown to smile.

Definition and Measurement of Complexity in the Context of Safety Assurance

October 27, 2016 • technical report, by bill nichols, charles weinstock, michael d. konrad, sarah sheard.

This report describes research to define complexity measures for avionics systems to help the FAA identify when systems are too complex to assure their safety.

Establishing Trusted Identities in Disconnected Edge Environments

October 27, 2016 • white paper, by dan j. klinedinst, sebastián echeverría, keegan m. williams.

he goal of this paper is to present a solution for establishing trusted identities in disconnected environments based on secure key generation and exchange in the field.

A Mapping of the Federal Financial Institutions Examination Council (FFIEC) Cybersecurity Assessment Tool (CAT) to the Cyber Resilience Review (CRR)

October 25, 2016 • technical note, by jeffrey pinckard, robert a. vrtis, michael rattigan.

To help financial organizations assess cyber resilience, we map FFIEC Cybersecurity Assessment Tool (CAT) statements to Cyber Resilience Review (CRR) questions.

Managing Third Party Risk in Financial Services Organizations: A Resilience-Based Approach

September 27, 2016 • white paper, by john haller, charles m. wallen.

A resilience-based approach can help financial services organizations to manage cybersecurity risks from outsourcing and comply with federal regulations.

Agile Development in Government: Myths, Monsters, and Fables

September 21, 2016 • white paper, by mary ann lapham, suzanne miller, david j. carney.

This volume is a reflection on attitudes toward Agile software development now current in the government workplace.

Striving for Effective Cyber Workforce Development

September 12, 2016 • white paper, by marie baker.

This paper reviews the issue of cyber awareness and identify efforts to combat this deficiency and concludes with strategies moving forward.

Segment-Fixed Priority Scheduling for Self-Suspending Real-Time Tasks

August 18, 2016 • technical report, by ragunathan (raj) rajkumar, junsung kim, jian-jia chen, wen-hung huang, geoffrey nelissen, bjorn andersson, dionisio de niz.

This report describes schedulability analyses and proposes segment-fixed priority scheduling for self-suspending tasks.

Creating Centralized Reporting for Microsoft Host Protection Technologies: The Enhanced Mitigation Experience Toolkit (EMET)

August 18, 2016 • technical note, by joseph tammariello, craig lewis.

This report describes how to set up a centralized reporting console for the Windows Enhanced Mitigation Experience Toolkit.

The QUELCE Method: Using Change Drivers to Estimate Program Costs

August 17, 2016 • technical note, by sarah sheard.

This technical note introduces Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE), a method for estimating program costs early in development.

Blacklist Ecosystem Analysis: 2016 Update

August 15, 2016 • white paper, by eric hatleback, leigh b. metcalf, jonathan spring.

This white paper, which is the latest in a series of regular updates, builds upon the analysis of blacklists presented in our 2013 and 2014 reports.

Architecture Fault Modeling and Analysis with the Error Model Annex, Version 2

June 22, 2016 • technical report, by peter h. feiler, julien delange, john j. hudak, david p. gluch.

This report describes the Error Model Annex, Version 2 (EMV2), notation for architecture fault modeling, which supports safety, reliability, and security analyses.

A Requirement Specification Language for AADL

By lutz wrage, julien delange, peter h. feiler.

This report describes a textual requirement specification language, called ReqSpec, for the Architecture Analysis & Design Language (AADL) and demonstrates its use.

DMPL: Programming and Verifying Distributed Mixed-Synchrony and Mixed-Critical Software

June 21, 2016 • technical report, by sagar chaki, david kyle.

DMPL is a language for programming distributed real-time, mixed-criticality software. It supports distributed systems in which each node executes a set of periodic real-time threads that are scheduled by priority …

Wireless Emergency Alerts Commercial Mobile Service Provider (CMSP) Cybersecurity Guidelines

June 9, 2016 • special report, by christopher j. alberts, carol woody, audrey j. dorofee.

This report provides members of the Commercial Mobile Service Provider (CMSP) community with practical guidance for better managing cybersecurity risk exposure, based on an SEI study of the CMSP element …

Report Writer and Security Requirements Finder: User and Admin Manuals

June 7, 2016 • special report, by anand sankalp (carnegie mellon university), gupta anurag (carnegie mellon), priyam swati (carnegie mellon university), yaobin wen (carnegie mellon university), walid el baroni (carnegie mellon university), nancy r. mead.

This report presents instructions for using the Malware-driven Overlooked Requirements (MORE) website applications.

Applying the Goal-Question-Indicator-Metric (GQIM) Method to Perform Military Situational Analysis

May 23, 2016 • technical note, by douglas gray.

This report describes how to use the goal-question-indicator-metric method in tandem with the military METT-TC method (mission, enemy, time, terrain, troops available, and civil-military considerations).

An Insider Threat Indicator Ontology

May 10, 2016 • technical report, by matthew l. collins, samuel j. perl, michael j. albrethsen, derrick spooner, daniel l. costa, george silowash.

This report presents an ontology for insider threat indicators, describes how the ontology was developed, and outlines the process by which it was validated.

Using Honeynets and the Diamond Model for ICS Threat Analysis

May 6, 2016 • technical report, by deana shick, kyle o'meara, john kotheimer.

This report presents an approach to analyzing approximately 16 gigabytes of full packet capture data collected from an industrial control system honeynet—a network of seemingly vulnerable machines designed to lure …

2016 State of Cybercrime Survey

May 2, 2016 • white paper.

This paper examines the current state of cybercrime and explores how organizations and individuals respond to cybercrime threats.

April 19, 2016 • White Paper

This report introduces the Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE) method for estimating program costs early in a development lifecycle.

A Unique Approach to Threat Analysis Mapping: A Malware-Centric Methodology

April 19, 2016 • technical report, by kyle o'meara, deana shick.

As they constantly change network infrastructure, adversaries consistently use and update their tools. This report presents a way for researchers to begin threat analysis with those tools rather than with …

On Board Diagnostics: Risks and Vulnerabilities of the Connected Vehicle

April 13, 2016 • white paper, by christopher king, dan j. klinedinst.

This report describes cybersecurity risks and vulnerabilities in modern connected vehicles.

2016 Emerging Technology Domains Risk Survey

April 8, 2016 • technical report, by todd lewellen, dan j. klinedinst, christopher king, garret wassermann.

This 2016 report provides a snapshot of our current understanding of future technologies.

Malware Capability Development Patterns Respond to Defenses: Two Case Studies

March 7, 2016 • white paper, by ed stoner, deana shick, jonathan spring, kyle o'meara.

In this paper, the authors describe their analysis of two case studies to outline the relationship between adversaries and network defenders.

Cyber-Foraging for Improving Survivability of Mobile Systems

February 18, 2016 • technical report, by sebastián echeverría, grace lewis, james root, ben w. bradshaw.

This report presents an architecture and experimental results that demonstrate that cyber-foraging using tactical cloudlets increases the survivability of mobile systems.

CERT-RMM Version 1.2 Release Notes

February 14, 2016 • white paper.

This document contains the release notes for CERT-RMM Version 1.2, released February 2014.

DoD Software Factbook

December 31, 2015 • white paper, by david zubrow, james mccurley, brad clark.

This DoD Factbook is an initial analysis of software engineering data from the perspective of policy and management questions about software projects.

Architecture-Led Safety Analysis of the Joint Multi-Role (JMR) Joint Common Architecture (JCA) Demonstration System

December 31, 2015 • special report, by peter h. feiler.

This report summarizes an architecture-led safety analysis of the aircraft-survivability situation-awareness system for the Joint Multi-Role vertical lift program.

Requirements and Architecture Specification of the Joint Multi-Role (JMR) Joint Common Architecture (JCA) Demonstration System

This report describes a method for capturing information from requirements documents in AADL and the draft Requirement Definition & Analysis Language Annex.

Potential System Integration Issues in the Joint Multi-Role (JMR) Joint Common Architecture (JCA) Demonstration System

By john j. hudak, peter h. feiler.

This report describes a method for capturing information from requirements documents in AADL to identify potential integration problems early in system development.

Extending AADL for Security Design Assurance of Cyber-Physical Systems

December 16, 2015 • technical report, by allen d. householder, rick kazman, john j. hudak, robert j. ellison, carol woody.

This report demonstrates the viability and limitations of using the Architecture Analysis and Design Language (AADL) through an extended example that allows for specifying and analyzing the security properties of …

Cybersecurity Considerations for Vehicles

December 10, 2015 • white paper, by mark sherman, jens palluch (method park).

In this paper the authors discuss the number of ECUs and software in modern vehicles and the need for cybersecurity to include vehicles.

Analytic Approaches to Detect Insider Threats

December 9, 2015 • white paper.

This paper identifies steps that organizations can use to enhance their security posture to detect potential insider threats.

Intelligence Preparation for Operational Resilience (IPOR)

December 7, 2015 • special report.

The author describes Intelligence Preparation for Operational Resilience (IPOR), a framework for preparing intelligence that complements commonly used intelligence frameworks such as Intelligence Preparation of the Battlefield (IPB).

Evaluating and Mitigating the Impact of Complexity in Software Models

December 3, 2015 • technical report, by min-young nam, john j. hudak, julien delange, jim mchale, bill nichols.

This report defines software complexity, metrics for complexity, and the effects of complexity on cost and presents an analysis tool to measure complexity in models.

Cyber + Culture Early Warning Study

November 30, 2015 • special report, by char sample.

This study was designed to profile cyber actors, and to examine the time interval between cyber and kinetic events in order to gain greater insights into nation-state cyber responses to …

Effective Insider Threat Programs: Understanding and Avoiding Potential Pitfalls

October 16, 2015 • white paper, by matthew l. collins, randall f. trzeciak, andrew p. moore, william e. novak, michael c. theis.

In this paper, the authors describe the potential ways an insider threat program (InTP) could go wrong and engage the community to discuss its concerns.

Structuring the Chief Information Security Officer Organization

October 6, 2015 • technical note, by pamela d. curtis, gregory crabb (united states postal service), brendan fitzpatrick, david tobar, nader mehravari, julia h. allen.

The authors describe how they defined a CISO team structure and functions for a national organization using sources such as CISOs, policies, and lessons learned from cybersecurity incidents.

Improving Federal Cybersecurity Governance Through Data-Driven Decision Making and Execution

September 16, 2015 • technical report, by robert w. stoddard, julia h. allen, anne connell, c. aaron cois, douglas gray, michael riley (veris group), brian d. wisniewski, erik ebel (veris group), william gulley (veris group), marie vaughn (veris group).

This technical report focuses on cybersecurity at the indirect, strategic level. It discusses how cybersecurity decision makers at the tactical or implementation level can establish a supportive contextual environment to …

Secure Coding Analysis of an AADL Code Generator's Runtime System

September 12, 2015 • white paper, by david keaton.

This paper describes a secure coding analysis of the PolyORB-HI-C runtime system used by C language code output from the Ocarina AADL code generator.

Contracting for Agile Software Development in the Department of Defense: An Introduction

August 18, 2015 • technical note, by eileen wrubel, jon gross.

This technical note addresses effective contracting for Agile software development and offers a primer on Agile based on a contracting officer's goals.

CND Equities Strategy

July 22, 2015 • white paper, by jonathan spring, ed stoner.

In this paper, the authors discuss strategies for successful computer network defense (CND) based on considering the adversaries' responses.

Comments on Bureau of Industry and Security (BIS) Proposed Rule Regarding Wassenaar Arrangement 2013 Plenary Agreements Implementation for Intrusion and Surveillance Items

By art manion, allen d. householder.

In this paper, CERT researchers comment on the proposed rule, Wassenaar Arrangement 2013 Plenary Agreements Implementation: Intrusion and Surveillance Items.

Enabling Incremental Iterative Development at Scale: Quality Attribute Refinement and Allocation in Practice

June 4, 2015 • technical report, by neil ernst, robert nord, stephany bellomo, ipek ozkaya.

This report describes industry practices used to develop business capabilities and suggests approaches to enable large-scale iterative development, or agile at scale.

State of Practice Report: Essential Technical and Nontechnical Issues Related to Designing SoS Platform Architectures

May 13, 2015 • technical report, by john klein, sholom g. cohen.

This report analyzes the state of the practice in system-of-systems (SoS) development, based on 12 interviews of leading SoS developers in the DoD and industry.

Emerging Technology Domains Risk Survey

April 30, 2015 • technical note, by andrew o. mellinger, christopher king, jonathan chu.

This report provides a snapshot in time of our current understanding of future technologies.

SCALe Analysis of JasPer Codebase

April 1, 2015 • white paper, by david svoboda.

In this paper, David Svoboda provides the findings of a SCALe audit on a codebase.

Model-Driven Engineering: Automatic Code Generation and Beyond

March 25, 2015 • technical note, by harry l. levinson, john klein, jay marchetti.

This report offers guidance on selecting, analyzing, and evaluating model-driven engineering tools for automatic code generation in acquired systems.

Defining a Maturity Scale for Governing Operational Resilience

March 19, 2015 • technical note, by julia h. allen, katie c. stewart, lisa r. young, michelle a. valdez, audrey j. dorofee.

Governing operational resilience requires the appropriate level of sponsorship, a commitment to strategic planning that includes resilience objectives, and proper oversight of operational resilience activities.

SEI SPRUCE Project: Curating Recommended Practices for Software Producibility

March 16, 2015 • white paper, by bill pollak, michael d. konrad, mike petock, tamara marshall-keim, b. craig meyers, gerald w. miller.

This paper describes the Systems and Software Producibility Collaboration Environment (SPRUCE) project and the resulting recommended practices on five software topics.

Improving Quality Using Architecture Fault Analysis with Confidence Arguments

March 10, 2015 • technical report, by peter h. feiler, julien delange, charles weinstock, john b. goodenough, neil ernst, ari z. klein.

The case study shows that by combining an analytical approach with confidence maps, we can present a structured argument that system requirements have been met and problems in the design …

Making DidFail Succeed: Enhancing the CERT Static Taint Analyzer for Android App Sets

March 4, 2015 • technical report, by william snavely, jonathan burket, jonathan lim, wei shen, lori flynn, william klieber.

In this report, the authors describe how the DidFail tool was enhanced to improve its effectiveness.

Eliminative Argumentation: A Basis for Arguing Confidence in System Properties

February 25, 2015 • technical report, by john b. goodenough, charles weinstock, ari z. klein.

This report defines the concept of eliminative argumentation and provides a basis for assessing how much confidence one should have in an assurance case argument.

A Proven Method for Meeting Export Control Objectives in Postal and Shipping Sectors

February 10, 2015 • technical note, by gregory crabb (united states postal service), pamela d. curtis, julia h. allen, nader mehravari.

This report describes how the CERT-RMM enabled the USPIS to implement an innovative approach for achieving complex international mail export control objectives.

Measuring What Matters Workshop Report

February 9, 2015 • technical note, by katie c. stewart, julia h. allen, lisa r. young, michelle a. valdez.

This report describes the inaugural Measuring What Matters Workshop conducted in November 2014, and the team's experiences in planning and executing the workshop and identifying improvements for future offerings.

A Dynamic Model of Sustainment Investment

February 5, 2015 • technical report, by sarah sheard, mike phillips, andrew p. moore, robert ferguson.

This paper describes a dynamic sustainment model that shows how budgeting, allocation of resources, mission performance, and strategic planning are interrelated and how they affect each other over time.

Cybersecurity Assurance

January 15, 2015 • white paper.

This paper describes the SEI research and solutions that help organizations gain justified confidence in their cybersecurity posture.

Blacklist Ecosystem Analysis Update: 2014

January 7, 2015 • white paper, by leigh b. metcalf, jonathan spring.

This white paper compares the contents of 85 different Internet blacklists to discover patterns in shared entries.

Predicting Software Assurance Using Quality and Reliability Measures

December 22, 2014 • technical note, by bill nichols, carol woody, robert j. ellison.

In this report, the authors discuss how a combination of software development and quality techniques can improve software security.

Regional Use of Social Networking Tools

December 17, 2014 • technical report, by kate meeuf.

This paper explores the regional use of social networking services (SNSs) to determine if participation with a subset of SNSs can be applied to identify a user's country of origin.

Domain Parking: Not as Malicious as Expected

December 10, 2014 • white paper, by jonathan spring, leigh b. metcalf.

In this paper we discuss scalable detection methods for domain names parking on reserved IP address space, and then using this data set, evaluate whether this behavior appears to be …

Pattern-Based Design of Insider Threat Programs

December 9, 2014 • technical note, by robin ruefle, dave mundie, andrew p. moore, david mcintire, matthew l. collins.

In this report, the authors describe a pattern-based approach to designing insider threat programs that could provide a better defense against insider threats.

Introduction to the Security Engineering Risk Analysis (SERA) Framework

December 4, 2014 • technical note, by audrey j. dorofee, christopher j. alberts, carol woody.

This report introduces the SERA Framework, a model-based approach for analyzing complex security risks in software-reliant systems and systems of systems early in the lifecycle.

Using Malware Analysis to Tailor SQUARE for Mobile Platforms

November 18, 2014 • technical note, by nancy r. mead, gregory paul alice.

This technical note explores the development of security requirements for the K-9 Mail application, an open source email client for the Android operating system.

A Method for Aligning Acquisition Strategies and Software Architectures

October 29, 2014 • technical note, by david j. carney, cecilia albert, patrick r. place, lisa brownsword.

This report describes the third year of the SEI's research into aligning acquisition strategies and software architecture.

Agile Methods in Air Force Sustainment: Status and Outlook

October 23, 2014 • technical note, by mary ann lapham, eileen wrubel, stephen beck, michael s. bandor, colleen regan.

This paper examines using Agile techniques in the software sustainment arena—specifically Air Force programs. The intended audience is the staff of DoD programs and related personnel who intend to use …

Development of an Intellectual Property Strategy: Research Notes to Support Department of Defense Programs

October 14, 2014 • special report, by charlene gross.

This report is intended to help program managers understand categories of intellectual property, various intellectual property challenges, and approaches to assessing the license rights that the program needs for long-term …

AADL Fault Modeling and Analysis Within an ARP4761 Safety Assessment

October 10, 2014 • technical report, by david p. gluch, peter h. feiler, julien delange, john j. hudak.

This report describes how the Architecture Analysis and Design Language (AADL) Error Model Annex supports the safety-assessment methods in SAE Standard ARP4761.

CERT Resilience Management Model—Mail-Specific Process Areas: International Mail Transportation (Version 1.0)

September 18, 2014 • technical note, by pamela d. curtis, gregory crabb (united states postal service), sam lin, dawn wilkes, nader mehravari, julia h. allen.

This report describes a new process area that ensures that international mail is transported according to Universal Postal Union standards.

CERT Resilience Management Model—Mail-Specific Process Areas: Mail Revenue Assurance (Version 1.0)

By julia h. allen, nader mehravari, david w. white, gregory crabb (united states postal service), pamela d. curtis.

This report describes a new process area that ensures that the USPS is compensated for mail that is accepted, transported, and delivered.

CERT Resilience Management Model—Mail-Specific Process Areas: Mail Induction (Version 1.0)

By pamela d. curtis, gregory crabb (united states postal service), david w. white, nader mehravari, julia h. allen.

This report describes a new process area that ensures that mail is inducted into the U.S. domestic mail stream according to USPS standards and requirements.

Smart Collection and Storage Method for Network Traffic Data

September 15, 2014 • technical report, by angela horneman, nathan dell.

This report discusses considerations and decisions to be made when designing a tiered network data storage solution.

A Systematic Approach for Assessing Workforce Readiness

August 18, 2014 • technical report, by david mcintire, christopher j. alberts.

In this report, the authors present the Competency Lifecycle Roadmap and the readiness test development method, both used to maintain workforce readiness.

Assuring Software Reliability

August 15, 2014 • special report, by robert j. ellison.

This report describes ways to incorporate the analysis of the potential impact of software failures--regardless of their cause--into development and acquisition practices through the use of software assurance.

Patterns and Practices for Future Architectures

August 15, 2014 • technical note, by eric werner, scott mcmillan, jonathan chu.

This report discusses best practices and patterns that will make high-performance graph analytics on new and emerging architectures more accessible to users.

Abuse of Customer Premise Equipment and Recommended Actions

August 7, 2014 • white paper, by jonathan spring, paul vixie, chris hallenbeck.

In this paper, the authors provide recommendations for addressing problems related to poor management of Consumer Premise Equipment (CPE).

Performance of Compiler-Assisted Memory Safety Checking

July 31, 2014 • technical note, by david keaton, robert c. seacord.

This technical note describes the criteria for deploying a compiler-based memory safety checking tool and the performance that can be achieved with two such tools whose source code is freely …

Unintentional Insider Threats: A Review of Phishing and Malware Incidents by Economic Sector

July 18, 2014 • technical note, by cert insider threat team.

This report analyzes unintentional insider threat cases of phishing and other social engineering attacks involving malware.

Evaluation of the Applicability of HTML5 for Mobile Applications in Resource-Constrained Edge Environments

July 2, 2014 • technical note, by grace lewis, bryan yan (carnegie mellon university – institute for software research).

This technical note presents an analysis of the feasibility of using HTML5 for developing mobile applications, for "edge" environments where resources and connectivity are uncertain, such as in battlefield or …

Agile Software Teams: How They Engage with Systems Engineering on DoD Acquisition Programs

July 1, 2014 • technical note, by mary ann lapham, suzanne miller, timothy a. chick, eileen wrubel.

This technical note addresses issues with Agile software teams engaging systems engineering functions in developing and acquiring software-reliant systems.

Improving the Automated Detection and Analysis of Secure Coding Violations

June 27, 2014 • technical note, by daniel plakosh, robert c. seacord, robert w. stoddard, david svoboda, david zubrow.

This technical note describes the accuracy analysis of the Source Code Analysis Laboratory (SCALe) tools and the characteristics of flagged coding violations.

CERT® Resilience Management Model (CERT®-RMM) V1.1: NIST Special Publication Crosswalk Version 2

June 11, 2014 • technical note, by lisa r. young, kevin g. partridge, mary popeck.

This update to Version 1 of this same title (CMU/SEI-2011-TN-028) maps CERT-RMM process areas to certain NIST 800-series special publications.

The Business Case for Systems Engineering: Comparison of Defense Domain and Non-defense Projects

June 10, 2014 • special report, by dennis goldenson, joseph p. elm.

This report analyzes differences in systems-engineering activities for defense and non-defense projects and finds differences in both deployment and effectiveness.

Job Analysis Results for Malicious-Code Reverse Engineers: A Case Study

June 3, 2014 • technical report, by jennifer cowley.

This report describes individual and team factors that enable, encumber, or halt the development of malicious-code reverse engineering expertise.

An Introduction to the Mission Risk Diagnostic for Incident Management Capabilities (MRD-IMC)

May 30, 2014 • technical note, by christopher j. alberts, robin ruefle, mark zajicek, audrey j. dorofee.

The Mission Risk Diagnostic for Incident Management Capabilities revises the Incident Management Mission Diagnostic Method with updated and expanded drivers.

A Taxonomy of Operational Cyber Security Risks Version 2

May 21, 2014 • technical note, by lisa r. young, mary popeck, james j. cebula.

This second version of the 2010 report presents a taxonomy of operational cyber security risks and harmonizes it with other risk and security activities.

An Evaluation of A-SQUARE for COTS Acquisition

May 13, 2014 • technical note, by nancy r. mead, sidhartha mani.

An evaluation of the effectiveness of Software Quality Requirements Engineering for Acquisition (A-SQUARE) in a project to select a COTS product for the advanced metering infrastructure of a smart grid.

Investigating Advanced Persistent Threat 1 (APT1)

May 12, 2014 • technical report, by deana shick, angela horneman.

This report analyzes unclassified data sets in an attempt to understand APT1's middle infrastructure.

Precise Static Analysis of Taint Flow for Android Application Sets

May 9, 2014 • white paper, by amar s. bhosale (no affiliation).

This thesis describes a static taint analysis for Android that combines the FlowDroid and Epicc analyses to track inter- and intra-component data flow.

Data-Driven Software Assurance: A Research Study

May 9, 2014 • technical report, by julia l. mullaney, michael f. orlando, erin harper, michael d. konrad, art manion, bill nichols, andrew p. moore.

In 2012, Software Engineering Institute (SEI) researchers began investigating vulnerabilities reported to the SEI's CERT Division. A research project was launched to investigate design-related vulnerabilities and quantify their effects.

ALTernatives to Signatures (ALTS)

April 30, 2014 • white paper, by george jones, john stogoski.

This paper presents the results of a study of non-signature-based approaches to detecting malicious activity in computer network traffic.

Potential Use of Agile Methods in Selected DoD Acquisitions: Requirements Development and Management

April 29, 2014 • technical note, by david j. carney, kenneth nidiffer, suzanne miller.

This report explores issues that practitioners in the field who are actively adopting Agile methods have identified in our interviews about their experience in defining and managing requirements.

The Readiness & Fit Analysis: Is Your Organization Ready for Agile?

April 28, 2014 • white paper, by suzanne miller.

This paper summarizes the Readiness & Fit Analysis and describes its extension to support risk identification for organizations that are adopting agile methods.

International Implementation of Best Practices for Mitigating Insider Threat: Analyses for India and Germany

April 16, 2014 • technical report, by randall f. trzeciak, george silowash, lori flynn, michael c. theis, tracy cassidy, palma buttles-valdez, carly l. huth, travis wright (carnegie mellon university, master of science in information security policy and management program).

This report analyzes insider threat mitigation in India and Germany, using the new framework for international cybersecurity analysis described in the paper titled “Best Practices Against Insider Threats in All …

Wireless Emergency Alerts (WEA) Cybersecurity Risk Management Strategy for Alert Originators

March 31, 2014 • special report, by the wea project team.

In this report, the authors describe a cybersecurity risk management (CSRM) strategy that alert originators can use throughout WEA adoption, operations, and sustainment, as well as a set of governance …

Maximizing Trust in the Wireless Emergency Alerts (WEA) Service

February 28, 2014 • special report, by carol woody, robert j. ellison.

This 2014 report presents recommendations for stakeholders of the Wireless Emergency Alerts (WEA) service that resulted from the development of two trust models, focusing on how to increase both alert …

Wireless Emergency Alerts: Trust Model Simulations

February 26, 2014 • special report, by timothy morrow, joseph p. elm, robert w. stoddard.

This report presents four types of simulations run on the public trust model and the alert originator trust model developed for the Wireless Emergency Alerts (WEA) service, focusing on how …

Commercial Mobile Alert Service (CMAS) Alerting Pipeline Taxonomy

February 24, 2014 • technical report.

This report presents the Commercial Mobile Alert Service (CMAS) Alerting Pipeline Taxonomy, a hierarchical classification that encompasses four elements of the alerting pipeline, to help stakeholders understand and reason about …

Best Practices in Wireless Emergency Alerts

February 19, 2014 • special report, by elizabeth trocki stark (sra international, inc.), jennifer lavan (sra international, inc.), robert j. ellison, john mcgregor, tamara marshall-keim, rita c. creel, carol woody, christopher j. alberts, joseph p. elm.

This report presents four best practices for the Wireless Emergency Alerts (WEA) service, including implementing WEA in a local jurisdiction, training emergency staff in using WEA, cross-jurisdictional governance of WEA, …

Study of Integration Strategy Considerations for Wireless Emergency Alerts

This report identifies key challenges and offers recommendations for alert originators navigating the process of adopting and integrating the Wireless Emergency Alerts (WEA) service into their emergency management systems.

Results in Relating Quality Attributes to Acquisition Strategies

February 4, 2014 • technical note, by lisa brownsword, cecilia albert, patrick r. place, david j. carney.

This technical note describes the second phase of a study that focuses on the relationships between software architecture and acquisition strategy -- more specifically, their alignment or misalignment.

Agile Metrics: Progress Monitoring of Agile Contractors

January 27, 2014 • technical note, by timothy a. chick, eileen wrubel, will hayes, mary ann lapham, suzanne miller.

This technical note offers a reference for those working to oversee software development on the acquisition of major systems from developers using Agile methods.

Agile Methods and Request for Change (RFC): Observations from DoD Acquisition Programs

January 24, 2014 • technical note, by mary ann lapham, eileen wrubel, michael s. bandor.

This technical note looks at the evaluation and negotiation of technical proposals that reflect iterative development approaches that in turn leverage Agile methods.

Unintentional Insider Threats: Social Engineering

January 21, 2014 • technical note, by cert insider threat center.

In this report, the authors explore the unintentional insider threat (UIT) that derives from social engineering.

Improving the Security and Resilience of U.S. Postal Service Mail Products and Services Using the CERT® Resilience Management Model

January 17, 2014 • technical note.

In this report, the authors describe how to improve the resilience of U.S. Postal Service products and services

A Proven Method for Identifying Security Gaps in International Postal and Transportation Critical Infrastructure

By nader mehravari, julia h. allen, pamela d. curtis, gregory crabb (united states postal service).

In this report, the authors describe a method of identifying physical security gaps in international mail processing centers and similar facilities.

Cloud Service Provider Methods for Managing Insider Threats: Analysis Phase II, Expanded Analysis and Recommendations

January 8, 2014 • technical note, by chas difatta (no affiliation), greg porter (heinz college at carnegie mellon university), lori flynn.

In this report, the authors discuss the countermeasures that cloud service providers use and how they understand the risks posed by insiders.

TSP Symposium 2013 Proceedings

January 8, 2014 • special report, by sergio cardona (universidad del quindío), leticia pérez (universidad de la república), rafael rincón (universidad eafit), joão pascoal faria (university of porto), mushtaq raza (university of porto), pedro c. henriques (strongstep – innovation in software quality), diego vallespir (universidad de la república), fernanda grazioli (universidad de la república), silvana moreno (universidad de la república), bill nichols, jim mchale.

This special report contains proceedings of the 2013 TSP Symposium. The conference theme was “When Software Really Matters,” which explored the idea that when product quality is critical, high-quality practices …

Understanding Patterns for System-of-Systems Integration

December 17, 2013 • technical report, by klaus schmid, claus nielsen (no affiliation), rick kazman.

This report discusses how a software architect can address the system-of-systems integration challenge from an architectural perspective.

Foundations for Software Assurance

December 16, 2013 • white paper, by carol woody, nancy r. mead, dan shoemaker (university of detroit mercy).

In this paper, the authors highlight efforts to address the principles of software assurance and its educational curriculum.

The Topological Properties of the Local Clustering Coefficient

December 9, 2013 • white paper, by leigh b. metcalf.

In this paper, Leigh Metcalf examines the local clustering coefficient for and provides a new formula to generate the local clustering coefficient.

Using Software Development Tools and Practices in Acquisition

December 3, 2013 • technical note, by harry l. levinson, richard librizzi.

This technical note provides an introduction to key automation and analysis techniques.

Spotlight On: Programmers as Malicious Insiders–Updated and Revised

December 2, 2013 • white paper, by andrew p. moore, randall f. trzeciak, dawn cappelli, matthew l. collins, thomas c. caron (john heinz iii college, school of information systems management, carnegie mellon university).

In this paper, the authors describe the who, what, when, where, and how of attacks by insiders using programming techniques and includes case examples.

Software Assurance Measurement – State of the Practice

November 29, 2013 • technical note, by dan shoemaker (university of detroit mercy), nancy r. mead.

In this report, the authors describe the current state of the practice and emerging trends in software assurance measurement.

A Defect Prioritization Method Based on the Risk Priority Number

November 26, 2013 • white paper, by will hayes, julie b. cohen, robert ferguson.

This paper describes a technique that helps organizations address and resolve conflicting views and create a better value system for defining releases.

Agile Security - Review of Current Research and Pilot Usage

November 21, 2013 • white paper, by carol woody.

This white paper was produced to focus attention on the opportunities and challenges for embedding information assurance considerations into Agile development and acquisition.

Cloud Service Provider Methods for Managing Insider Threats: Analysis Phase I

November 15, 2013 • technical note, by greg porter (heinz college at carnegie mellon university).

In this report, Greg Porter documents preliminary findings from interviews with cloud service providers on their insider threat controls.

Mobile SCALe: Rules and Analysis for Secure Java and Android Coding

November 8, 2013 • technical report, by david svoboda, dean sutherland, william klieber, lori flynn, limin jia (carnegie mellon university, department of electrical and computer engineering), lujo bauer (carnegie mellon university, department of electrical and computer engineering), fred long.

In this report, the authors describe Android secure coding rules, guidelines, and static analysis developed as part of the Mobile SCALe project.

Advancing Cybersecurity Capability Measurement Using the CERT-RMM Maturity Indicator Level Scale

November 7, 2013 • technical note, by richard a. caralli, matthew j. butkovic.

In this report, the authors review the specific and generic goals and practices in CERT-RMM to determine if a better scale could be developed.

CERT® Resilience Management Model (CERT®-RMM) V1.1: NIST Special Publication 800-66 Crosswalk

October 28, 2013 • technical note, by ma-nyahn kromah (sungard availability services), lisa r. young.

In this report, the authors map CERT-RMM process areas to key activities in NIST Special Publication 800-66 Revision 1.

Passive Detection of Misbehaving Name Servers

October 4, 2013 • technical report.

In this report, the authors explore name-server flux and two types of data that can reveal it.

Insider Threat Control: Using Plagiarism Detection Algorithms to Prevent Data Exfiltration in Near Real Time

October 3, 2013 • technical note, by todd lewellen, daniel l. costa, george silowash.

In this report, the authors describe how an insider threat control can monitor an organization's web request traffic for text-based data exfiltration.

Introduction to the Mission Thread Workshop

October 1, 2013 • technical report, by william wood, michael j. gagliardi, timothy morrow.

This report introduces the Mission Thread Workshop, a method for understanding architectural and engineering considerations for developing and sustaining systems of systems. It describes the three phases of the workshop …

Parallel Worlds: Agile and Waterfall Differences and Similarities

October 1, 2013 • technical note, by ipek ozkaya, suzanne miller, mary ann lapham, timothy a. chick, steve palmquist.

This report helps readers understand Agile. The report assembles terms and concepts from both the traditional world of waterfall-based development and the Agile environment to show the many similarities and …

Everything You Wanted to Know About Blacklists But Were Afraid to Ask

September 30, 2013 • white paper.

This document compares the contents of 25 different common public-internet blacklists in order to discover any patterns in the shared entries.

Roadmap to Software Assurance Competency

September 23, 2013 • white paper.

This white paper describes the Software Assurance (SwA) Core Body of Knowledge and SwA competency levels.

TSP Performance and Capability Evaluation (PACE): Customer Guide

September 1, 2013 • special report, by mark kasunic, bill nichols, timothy a. chick.

This guide describes the evaluation process and lists the steps organizations and programs must complete to earn a TSP-PACE certification.

TSP Performance and Capability Evaluation (PACE): Team Preparedness Guide

By timothy a. chick, bill nichols, mark kasunic.

This document describes the TSP team data that teams normally produce and that are required as input to the TSP-PACE process.

Best Practices Against Insider Threats in All Nations

August 27, 2013 • technical note, by carly l. huth, palma buttles-valdez, lori flynn, randall f. trzeciak.

In this report, the authors summarize best practices for mitigating insider threats in international contexts.

The Role of Computer Security Incident Response Teams in the Software Development Life Cycle

August 20, 2013 • white paper, by robin ruefle.

In this paper, Robin Ruefle describes how an incident management can provide input to the software development process.

State of Cyber Workforce Development

August 15, 2013 • white paper.

This paper summarizes the current posture of the cyber workforce and several initiatives designed to strengthen, grow, and retain cybersecurity professionals.

Training and Awareness

August 7, 2013 • white paper, by carol sledge, ken van wyk (no affiliation).

In this paper, the authors provide guidance on training and awareness opportunities in the field of software security.

Evidence of Assurance: Laying the Foundation for a Credible Security Case

By howard f. lipson, charles weinstock.

In this paper, the authors provide examples of several of the kinds of evidence that can contribute to a security case.

Security and Project Management

August 6, 2013 • white paper.

In this paper, Robert Ellison explains what project managers should consider because they relate to security needs.

An Evaluation of Cost-Benefit Using Security Requirements Prioritization Methods

August 5, 2013 • white paper, by travis christian, nancy r. mead.

In this paper, the authors provide background information on penetration testing processes and practices.

Unintentional Insider Threats: A Foundational Study

August 1, 2013 • technical note.

In this report, the CERT Insider Threat team examines unintentional insider threat (UIT), a largely unrecognized problem.

Teaching Security Requirements Engineering Using SQUARE

July 31, 2013 • white paper, by nancy r. mead, dan shoemaker (university of detroit mercy), jeff ingalsbe (university of detroit mercy).

In this paper, the authors detail the validation of a teaching model for security requirements engineering that ensures that security is built into software.

Trustworthy Composition: The System Is Not Always the Sum of Its Parts

In this paper, Robert Ellison surveys several profound technical problems faced by practitioners assembling and integrating secure and survivable systems.

Development of a Master of Software Assurance Reference Curriculum - 2013 IJSSE

By julia h. allen, nancy r. mead, mark a. ardis (stevens institute of technology), thomas b. hilburn (embry-riddle aeronautical university), andrew j. kornecki (embry-riddle aeronautical university), richard c. linger (oak ridge national laboratory), james mcdonald (monmouth university).

In this paper, the authors present an overview of the Master of Software Assurance curriculum, including its history, student prerequisites, and outcomes

Strengthening Ties Between Process and Security

In this paper, Carol Woody summarizes recent key accomplishments, including harmonizing security practices with CMMI and using assurance cases.

Estimating Benefits from Investing in Secure Software Development

By ashish arora, rahul telang, steven frank.

In this paper, the authors discuss the costs and benefits of incorporating security in software development and presents formulas for calculating security costs and security benefits.

What Measures Do Vendors Use for Software Assurance?

By jeremy epstein.

In this paper, Jeremy Epstein examines what real vendors do to ensure that their products are reasonably secure.

The Development of a Graduate Curriculum for Software Assurance

By nancy r. mead, mark a. ardis (stevens institute of technology).

In this paper, the authors describe the work of the Master of Software Assurance curriculum project, including sources, process, products, and more.

Secure Software Development Life Cycle Processes

By noopur davis.

In this paper, Noopur Davis presents information about processes, standards, and more that support or could support secure software development.

Applicability of Cultural Markers in Computer Network Attack Attribution

July 11, 2013 • white paper.

In this 2013 white paper, Char Sample discusses whether cultural influences leave traces in computer network attack (CAN) choices and behaviors.

Improving Software Assurance

July 5, 2013 • white paper.

In this paper, the authors discuss what practitioners should know about software assurance, where to look, what to look for, and how to demonstrate improvement.

Scale: System Development Challenges

In this paper, the authors describe software assurance challenges inherent in networked systems development and propose a solution.

Requirements Prioritization Case Study Using AHP

By nancy r. mead.

In this paper, Nancy Mead describes a tradeoff analysis that can select a suitable requirements prioritization method and the results of trying one method.

Arguing Security - Creating Security Assurance Cases

By john b. goodenough, charles weinstock, howard f. lipson.

In this paper, the authors explain an approach to documenting an assurance case for system security.

SQUARE Process

In this paper, Nancy Mead describes the SQUARE process as a means for eliciting, categorizing, and prioritizing security requirements for IT systems.

Requirements Elicitation Case Studies Using IBIS, JAD, and ARM

In this paper, Nancy Mead describes a tradeoff analysis that can be used to select a suitable requirements elicitation method.

The Common Criteria

In this paper, Nancy Mead discusses how Common Criteria is evaluated, it also presents a standard that is related to developing security requirements.

Measures and Measurement for Secure Software Development

July 3, 2013 • white paper, by david zubrow, james mccurley, carol dekkers.

In this paper, the authors discuss how measurement can be applied improve the security characteristics of the software being developed.

Predictive Models for Identifying Software Components Prone to Failure During Security Attacks

By laurie williams, michael gegick, mladan vouk.

In this paper, the authors describes how the presence of security faults correlates strongly with the presence of a more general category of reliability faults.

Measuring the Software Security Requirements Engineering Process

In this paper, Nancy Mead describes a measurement approach to security requirements engineering to analyze projects that were developed with and without SQUARE.

System-of-Systems Influences on Acquisition Strategy Development

July 2, 2013 • white paper, by rita c. creel, robert j. ellison.

In this paper, the authors discuss significant new sources of risk and recommend ways to address them.

Risk-Centered Practices

By julia h. allen.

In this paper, Julia Allen discusses the role that risk management and risk assessment play in choosing which security practices to implement.

Supply-Chain Risk Management: Incorporating Security into Software Development

In this paper, the authors describe practices that address defects and mechanisms for introducing these practices into the acquisition lifecycle.

Prioritizing IT Controls for Effective, Measurable Security

By daniel phelps, kurt milne, gene kim (ip services and itpi).

In this paper, the authors summarize results from the IT Controls Performance Study conducted by the IT Process Institute.

Building Security into the Business Acquisition Process

By dan shoemaker (university of detroit mercy).

In this paper, Dan Shoemaker presents the standard process for acquiring software products and services in business.

Navigating the Security Practice Landscape

In this paper, Julia Allen presents a summary of ten leading sources of security practice definition and implementation guidance.

Assuring Software Systems Security: Life Cycle Considerations for Government Acquisitions

By rita c. creel.

In this paper, Rita Creel identifies acquirer activities and resources necessary to support contractor efforts to build secure software-intensive systems.

Plan, Do, Check, Act

In this paper, Ken van Wyk provides a primer on the most commonly used tools for traditional penetration testing.

Finding a Vendor You Can Trust in the Global Marketplace

By dan shoemaker (university of detroit mercy), art conklin.

In this paper, the authors introduce the concept of standardized third-party certification of supplier process capability.

Results of SEI Line-Funded Exploratory New Starts Projects: FY 2012

July 1, 2013 • technical report, by robert nord, robert w. stoddard, lisa brownsword, dennis goldenson, mary ann lapham, david zubrow, william r. claycomb, lori flynn, peter h. feiler, rick kazman, robert ferguson, stephany bellomo, ipek ozkaya, sagar chaki, arie gurfinkel, julie b. cohen, john j. hudak, jeff havrilla, bjorn andersson, john mcgregor, james mccurley, carly l. huth, david mcintire, david p. gluch, wesley jin, chuck hines, brittany phillips, yuanfang cai (drexel university).

This report describes line-funded exploratory new starts (LENS) projects that were conducted during fiscal year 2012 (October 2011 through September 2012).

Insider Threat Attributes and Mitigation Strategies

July 1, 2013 • technical note, by george silowash.

In this report, George Silowash maps common attributes of insider threat cases to characteristics important for detecting, preventing, or mitigating the threat.

Pointer Ownership Model

June 10, 2013 • white paper.

In this paper, David Svoboda describes the Pointer Ownership Model, which can statically identify classes of errors involving dynamic memory in C/C++ programs.

Common Software Platforms in System-of-Systems Architectures: The State of the Practice

June 6, 2013 • white paper, by rick kazman, sholom g. cohen, john klein.

System-of-systems (SoS) architectures based on common software platforms have been commercially successful, but progress on creating and adopting them has been slow. This study aimed to understand technical issues for …

Software Assurance for Executives: Mapping of Common Topics to Specific Materials

June 3, 2013 • white paper.

In this paper, the authors present common topics, course materials, and resources related to the Software Assurance for Executives course held in June 2013.

Software Assurance for Executives

This legal form was used in the Software Assurance for Executives course that was held in June 2013.

Isolating Patterns of Failure in Department of Defense Acquisition

June 1, 2013 • technical note, by lisa brownsword, patrick r. place, cecilia albert, john j. hudak, charles (bud) hammons, david j. carney.

This report documents an investigation into issues related to aligning acquisition strategies with business and mission goals.

Socio-Adaptive Systems Challenge Problems Workshop Report

June 1, 2013 • special report, by mark h. klein, timothy morrow, scott hissam.

This report presents a summary of the findings of the Socio-Adaptive Systems Challenge Problem Workshop, held in Pittsburgh, PA, on April 12-13, 2012.

Strengths in Security Solutions

May 31, 2013 • white paper, by carol woody, allen d. householder, robert c. seacord, arjuna shunn (microsoft).

In this white paper, the authors map eight CERT tools, services, and processes to Microsoft's Simplified Security Development Lifecycle.

Integrating Software Assurance Knowledge into Conventional Curricula

May 23, 2013 • white paper.

In this paper, the authors discuss the results of comparing the Common Body of Knowledge for Secure Software Assurance with traditional computing disciplines.

Maturity of Practice

In this paper, Julia Allen identifies indicators that organizations are addressing security as a governance and management concern, at the enterprise level.

Integrating Security and IT

May 21, 2013 • white paper.

In this paper, Julia Allen describes the key relationship between IT processes and security controls.

Individual Certification of Security Proficiency for Software Professionals: Where Are We? Where Are We Going?

In this paper, Dan Shoemaker describes existing professional certifications in information assurance and emerging certifications for secure software assurance.

How Much Security Is Enough?

In this paper, Julia Allen provides guidelines for answering this question, including means for determining adequate security based on risk.

Models for Assessing the Cost and Value of Software Assurance

By john bailey, dan shoemaker (university of detroit mercy), antonio drommi, jeff ingalsbe (university of detroit mercy), nancy r. mead.

In this paper, the authors present IT valuation models that represent the most commonly accepted approaches to the valuation of IT and IT processes.

Adapting Penetration Testing for Software Development Purposes

By ken van wyk (no affiliation).

In this paper, Ken van Wyk provides background information on penetration testing processes and practices.

Requirements Engineering Annotated Bibliography

In this paper, Nancy Mead provides a bibliography of sources related to requirements engineering.

Defining the Discipline of Secure Software Assurance: Initial Findings from the National Software Assurance Repository

By nancy r. mead, jeff ingalsbe (university of detroit mercy), dan shoemaker (university of detroit mercy), rita barrios.

In this paper, the authors characterize the current state of secure software assurance work and suggest future directions.

Making the Business Case for Software Assurance

In this paper, Nancy Mead provides an overview of the Business Case content area.

Spotlight On: Insider Theft of Intellectual Property Inside the United States Involving Foreign Governments or Organizations (2013)

May 20, 2013 • technical note, by andrew p. moore, randall f. trzeciak, derrick spooner, dawn cappelli, matthew l. collins.

In this report, the authors provide a snapshot of individuals involved in insider threat cases and recommends how to mitigate the risk of similar incidents.

The Software Assurance Competency Model: A Roadmap to Enhance Individual Professional Capability

May 16, 2013 • white paper, by nancy r. mead, dan shoemaker (university of detroit mercy).

In this paper, the authors describe a software assurance competency model that can be used by professionals to improve their software assurance skills.

Building a Body of Knowledge for ICT Supply Chain Risk Management

In this paper, the authors propose a set of Supply Chain Risk Management (SCRM) activities and practices for Information and Communication Technologies (ICT).

Modeling Tools References

May 15, 2013 • white paper, by samuel t. redwine.

In this paper, Samuel Redwine provides references related to modeling tools.

Software Assurance Education Overview

In this paper, Nancy Mead discusses the growing demand for skilled professionals who can build security and correct functionality into software.

Governance and Management References

May 14, 2013 • white paper.

In this paper, Julia Allen provides references related to governance and management.

Getting Secure Software Assurance Knowledge into Conventional Practice

By linda laird, nancy r. mead, dan shoemaker (university of detroit mercy).

In this paper, the authors describe three educational initiatives in support of software assurance education.

General Modeling Concepts

In this paper, Samuel Redwine introduces several concepts related to the Introduction to Modeling Tools for Software Security article and modeling in general.

A Systemic Approach for Assessing Software Supply-Chain Risk

By robert j. ellison, carol woody, christopher j. alberts, rita c. creel, audrey j. dorofee.

In this paper, the authors highlight the approach being implemented by SEI researchers for assessing and managing software supply-chain risks and provides a summary of the status of this work.

Framing Security as a Governance and Management Concern: Risks and Opportunities

In this paper, Julia Allen describes six "assets" or requirements of being in business that can be compromised by insufficient security investment.

Assembly, Integration, and Evolution Overview

By howard f. lipson.

In this paper, Howard Lipson describes the objective of the Assembly, Integration & Evolution content area.

A Common Sense Way to Make the Business Case for Software Assurance

By dan shoemaker (university of detroit mercy), jeff ingalsbe (university of detroit mercy), antonio drommi, nancy r. mead, john bailey.

In this article, the authors demonstrate how a true cost/benefit for secure software can be derived.

Deployment and Operations References

In this paper, Julia Allen provides a list of references related to deployment and operations.

Deploying and Operating Secure Systems

In this paper, Julia Allen provides a brief overview of deployment and operations security issues and advice for using related practices.

Two Nationally Sponsored Initiatives for Disseminating Assurance Knowledge

In this paper, the authors describe two efforts that support national cybersecurity education goals.

By Dan Shoemaker (University of Detroit Mercy), Nancy R. Mead, Carol Woody

In this paper, the authors highlight efforts underway to address our society's growing dependence on software and the need for effective software assurance.

Assurance Cases Overview

In this paper, Howard Lipson introduces the concepts and benefits of developing and maintaining assurance cases for security.

It’s a Nice Idea but How Do We Get Anyone to Practice It? A Staged Model for Increasing Organizational Capability in Software Assurance

May 13, 2013 • white paper.

In this paper, Dan Shoemaker presents a standard approach to increasing the security capability of a typical IT function.

Software Security Engineering: A Guide for Project Managers (white paper)

By sean barnum, gary mcgraw, julia h. allen, nancy r. mead, robert j. ellison.

In this guide, the authors discuss our reliance on software and systems that use the internet or internet-exposed private networks.

Requirements Elicitation Introduction

In this paper, Nancy Mead discusses elicitation methods and the kind of tradeoff analysis that can be done to select a suitable one.

Requirements Prioritization Introduction

In this paper, Nancy Mead discusses using a systematic prioritization approach to prioritize security requirements.

Optimizing Investments in Security Countermeasures: A Practical Tool for Fixed Budgets

By jonathan caulkins, eric hough, hassan osman, nancy r. mead.

In this paper, the authors introduce a novel method of optimizing using integer programming (IP).

Security Is Not Just a Technical Issue

In this paper, Julia Allen defines the scope of governance concern as they apply to security.

PSP-VDC: An Adaptation of the PSP that Incorporates Verified Design by Contract

May 7, 2013 • technical report, by silvana moreno (universidad de la república), álvaro tasistro (universidad ort uruguay), diego vallespir (universidad de la república), bill nichols.

This paper describes a proposal for integrating Verified Design by Contract into PSP in order to reduce the amount of defects present at the Unit Testing phase, while preserving or …

How You Can Help Your Utility Clients with a Critical Aspect of Smart Grid Transformation They Might be Overlooking

May 1, 2013 • white paper, by the sgmm communications team.

This paper discusses how you can use the Smart Grid Maturity Model (SGMM) to benefit your utility clients.

Five Smart Grid Questions Every Utility Executive Should Ask

This paper recommends the Smart Grid Maturity Model (SGMM), a tool utilities can use to plan and measure smart grid progress.

Application Virtualization as a Strategy for Cyber Foraging in Resource-Constrained Environments

May 1, 2013 • technical note, by dominik messinger, grace lewis.

This technical note explores application virtualization as a more lightweight alternative to VM synthesis for cloudlet provisioning.

The Perils of Treating Software as a Specialty Engineering Discipline

April 30, 2013 • white paper, by keith korzec, tom merendino.

This paper reviews the perils of insufficiently engaging key software domain experts during program development.

Four Pillars for Improving the Quality of Safety-Critical Software-Reliant Systems

April 29, 2013 • white paper, by lutz wrage, charles weinstock, john b. goodenough, arie gurfinkel, peter h. feiler.

This white paper presents an improvement strategy comprising four pillars of an integrate-then-build practice that lead to improved quality through early defect discovery and incremental end-to-end validation and verification.

MERIT Interactive Insider Threat Training Simulator

April 16, 2013 • white paper.

In this paper, the authors describe how state-of-the-art multi-media technologies were used to develop the MERIT InterActive training simulator.

Software Assurance Competency Model

March 11, 2013 • technical note, by thomas b. hilburn (embry-riddle aeronautical university), andrew j. kornecki (embry-riddle aeronautical university), mark a. ardis (stevens institute of technology), glenn johnson ((isc)2), nancy r. mead.

In this report, the authors describe a model that helps create a foundation for assessing and advancing the capability of software assurance professionals.

Detecting and Preventing Data Exfiltration Through Encrypted Web Sessions via Traffic Inspection

March 1, 2013 • technical note, by todd lewellen, daniel l. costa, george silowash, joshua w. burns.

In this report, the authors present methods for detecting and preventing data exfiltration using a Linux-based proxy server in a Microsoft Windows environment.

Justification of a Pattern for Detecting Intellectual Property Theft by Departing Insiders

By dave mundie, david zubrow, andrew p. moore, david mcintire.

In this report, the authors justify applying the pattern “Increased Review for Intellectual Property (IP) Theft by Departing Insiders.”

Quantifying Uncertainty in Expert Judgment: Initial Results

March 1, 2013 • technical report, by robert w. stoddard, dennis goldenson.

The work described in this report, part of a larger SEI research effort on Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE), aims to develop and validate methods for calibrating …

History of CERT-RMM

February 15, 2013 • white paper.

This paper explains the history of how the CERT-RMM came to be.

The MAL: A Malware Analysis Lexicon

February 1, 2013 • technical note, by david mcintire, dave mundie.

In this report, the authors present results of the Malware Analysis Lexicon (MAL) initiative, which developed the first common vocabulary for malware analysis.

Tunisia Case Study

January 24, 2013 • white paper.

This case study describes the experiences of the Tunisia CSIRT in getting its organization up and running.

Columbia CSIRT Case Study

This case study describes the experiences of the Columbia CSIRT in getting its organization up and running.

Insider Threat Control: Using Universal Serial Bus (USB) Device Auditing to Detect Possible Data Exfiltration by Malicious Insiders

January 1, 2013 • technical note, by george silowash, todd lewellen.

In this report, the authors present methods for auditing USB device use in a Microsoft Windows environment.

Cyber Intelligence Tradecraft Project: Summary of Key Findings

January 1, 2013 • white paper, by kate ambrose, troy townsend, andrew o. mellinger, jay mcallister, melissa ludwick.

This study, known as the Cyber Intelligence Tradecraft Project (CITP), seeks to advance the capabilities of organizations performing cyber intelligence by elaborating on best practices and prototyping solutions to shared …

Insider Threat Control: Understanding Data Loss Prevention (DLP) and Detection by Correlating Events from Multiple Sources

By christopher king, george silowash.

In this report, the authors present methods for controlling removable media devices in a MS Windows environment.

SEI Product Line Bibliography

December 31, 2012 • white paper.

This bibliography lists SEI and non-SEI resources that have informed the SEI Product Lines efforts. Examples cover diverse domains and show the kind of improvements you can achieve using a …

A Framework for Software Product Line Practice, Version 5.0

By sholom g. cohen, linda m. northrop, reed little, john mcgregor, paul c. clements, felix bachmann, john k. bergey, gary chastek, patrick donohoe, liam o'brien, lawrence g. jones, robert w. krut, jr..

This document describes the activities and practices in which an organization must be competent before it can benefit from fielding a product line of software systems.

Chronological Examination of Insider Threat Sabotage: Preliminary Observations

December 1, 2012 • white paper, by carly l. huth, david mcintire, william r. claycomb, lori flynn, todd lewellen.

In this paper, the authors examine 15 cases of insider threat sabotage of IT systems to identify points in the attack time-line.

The Business Case for Systems Engineering Study: Assessing Project Performance from Sparse Data

December 1, 2012 • special report, by joseph p. elm.

This report describes the data collection and analysis process used to support the assessment of project performance for the systems engineering (SE) effectiveness study.

Analyzing Cases of Resilience Success and Failure - A Research Study

December 1, 2012 • technical note, by andrew p. moore, randall f. trzeciak, robert w. stoddard, julia h. allen, nader mehravari, pamela d. curtis, kevin g. partridge.

In this report, the authors describe research aimed at helping organizations to know the business value of implementing resilience processes and practices.

Common Sense Guide to Mitigating Insider Threats, Fourth Edition

December 1, 2012 • technical report, by dawn cappelli, timothy j. shimeall, lori flynn, george silowash, andrew p. moore, randall f. trzeciak.

In this report, the authors define insider threats and outline current insider threat patterns and trends.

Arabic Language Translation of CMMI for Services V1.3

November 1, 2012 • white paper, by the cmmi product team.

Arabic translation of CMMI-SVC V1.3

TSP Symposium 2012 Proceedings

November 1, 2012 • special report, by shigeru kusakabe (kyushu university), yoichi omori (kyushu university), keijiro araki (kyushu university), fernanda grazioli (universidad de la república), silvana moreno (universidad de la república), álvaro tasistro (universidad ort uruguay), diego vallespir (universidad de la república), joão pascoal faria (university of porto), mushtaq raza (university of porto), pedro c. henriques (strongstep – innovation in software quality), césar duarte (strongstep – innovation in software quality), elias fallon (cadence design systems, inc.), lee gazlay (cadence design systems, inc.), bill nichols.

The 2012 TSP Symposium was organized by the Software Engineering Institute (SEI) and took place September 18-20 in St. Petersburg, FL. The goal of the TSP Symposium is to bring …

DoD Information Assurance and Agile: Challenges and Recommendations Gathered Through Interviews with Agile Program Managers and DoD Accreditation Reviewers

November 1, 2012 • technical note, by stephany bellomo, carol woody.

This paper discusses the natural tension between rapid fielding and response to change (characterized as agility) and DoD information assurance policy. Data for the paper was gathered through interviews with …

Reliability Improvement and Validation Framework

By peter h. feiler, arie gurfinkel, charles weinstock, john b. goodenough, lutz wrage.

This report discusses the reliability validation and improvement framework developed by the SEI. The purpose of this framework is to provide a foundation for addressing the challenges of qualifying increasingly …

The Business Case for Systems Engineering Study: Results of the Systems Engineering Effectiveness Survey

By joseph p. elm, dennis goldenson.

This report summarizes the results of a survey that had the goal of quantifying the connection between the application of systems engineering (SE) best practices to projects and programs and …

Maturity Models 101: A Primer for Applying Maturity Models to Smart Grid Security, Resilience, and Interoperability

By richard a. caralli, austin montgomery, mark knight (cgi group).

In this paper, the authors explain the history and evolution of and applications for maturity models.

Technical Debt: From Metaphor to Theory and Practice

By robert nord, ipek ozkaya, philippe kruchten.

This article discusses the technical debt metaphor and considers it beyond a "rhetorical concept." The article explores the role of decision making about developmental activities and future changes and the …

Architecture-Driven Semantic Analysis of Embedded Systems (Dagstuhl Seminar 12272)

October 10, 2012 • special report, by peter h. feiler, jerome hugues.

This report documents the program and outcomes of presentations and working groups from Dagstuhl Seminar 12272, "Architecture-Driven Semantic Analysis of Embedded Systems."

Spotlight On: Insider Threat from Trusted Business Partners Version 2: Updated and Revised

October 1, 2012 • white paper, by andrew p. moore, randall f. trzeciak, derrick spooner, todd lewellen, robert weiland (carnegie mellon university), dawn cappelli.

In this article, the authors focus on cases in which the malicious insider was employed by a trusted business partner of the victim organization.

The Role of Standards in Cloud-Computing Interoperability

October 1, 2012 • technical note, by grace lewis.

This report explores the role of standards in cloud-computing interoperability. It covers cloud-computing basics and standard-related efforts, discusses several use cases, and provides recommendations for cloud-computing adoption.

Cloud Computing at the Tactical Edge

By grace lewis, edwin j. morris, soumya simanta, mahadev satyanarayanan (carnegie mellon university), kiryong ha (carnegie mellon school of computer science).

This technical note presents a strategy to overcome the challenges of obtaining sufficient computation power to run applications needed for warfighting and disaster relief missions. It discusses the use of …

Well There’s Your Problem: Isolating the Crash-Inducing Bits in a Fuzzed File

In this 2012 report, Allen Householder describes an algorithm for reverting bits from a fuzzed file to those found in the original seed file to recreate the crash.

Resource Allocation in Dynamic Environments

October 1, 2012 • technical report, by jeffrey hansen, gabriel moreno, daniel plakosh, joe seibel, scott hissam, b. craig meyers, lutz wrage.

When warfighting missions are conducted in a dynamic environment, the allocation of resources needed for mission operation can change from moment to moment. This report addresses two challenges of resource …

Building an Incident Management Body of Knowledge

September 7, 2012 • white paper, by dave mundie, robin ruefle.

In this paper, the authors describe the components of the CERT Incident Management Body of Knowledge (CIMBOK) and how they were constructed.

SEPG Europe 2012 Conference Proceedings

September 1, 2012 • special report, by jose maria garcia (software quality assurance), ana m. moreno (universidad politecnica de madrid), radouane oudrhiri (systonomy), fabrizio pellizzetti (systonomy), alejandro ruiz-robles (university of piura), maria-isabel sanchez-segura (carlos iii university of madrid), prasad m. shrasti (tata consultancy services), aman kumar singhal (infosys), javier garcia-guzman (carlos iii university of madrid), javier garzas (kybele research and kybele consulting), amit arun javadekar (infosys), patrick kirwan, joaquin lasheras (centic), fuensanta medina-dominguez (carlos iii university of madrid), erich meier (method park), arturo mora-soto (carlos iii university of madrid).

This report compiles seven papers based on presentations given at SEPG Europe 2012.

Competency Lifecycle Roadmap: Toward Performance Readiness

September 1, 2012 • technical note, by robin ruefle, christopher j. alberts, sandra behrens.

In this report, the authors describe the Competency Lifecycle Roadmap (CLR), a preliminary roadmap for understanding and building workforce readiness.

Communication Among Incident Responders – A Study

By brett tjaden, robert floodeen.

In this report, the authors describe three factors for helping or hindering the cooperation of incident responders.

Toward a Theory of Assurance Case Confidence

September 1, 2012 • technical report, by ari z. klein, charles weinstock, john b. goodenough.

In this report, the authors present a framework for thinking about confidence in assurance case arguments.

Insider Fraud in Financial Services

August 3, 2012 • white paper.

In this brochure, the authors present the findings of a study that analyzed computer criminal activity in the financial services sector.

Probability-Based Parameter Selection for Black-Box Fuzz Testing

August 1, 2012 • technical note, by allen d. householder, jonathan foote.

In this report, the authors describe an algorithm for automating the selection of seed files and other parameters used in black-box fuzz testing.

Results of SEI Line-Funded Exploratory New Starts Projects

August 1, 2012 • technical report, by bill nichols, robert nord, cory cohen, soumya simanta, rick kazman, nanette brown, william casey, david french, edwin j. morris, arie gurfinkel, sagar chaki, dionisio de niz, ipek ozkaya, brad myers, gene cahill, ofer strichman, raghvinder sangwan, len bass, peppo valetto.

This report describes the line-funded exploratory new starts (LENS) projects that were undertaken during fiscal year 2011. For each project, the report presents a brief description and a recounting of …

Network Profiling Using Flow

By sid faber, austin whisnant.

In this report, the authors provide a step-by-step guide for profiling and discovering public-facing assets on a network using netflow data.

Insider Threats to Cloud Computing: Directions for New Research Challenges

July 16, 2012 • white paper, by william r. claycomb, alex nicoll.

In this paper, the authors explain how cloud computing related insider threats are a serious concern, but that this threat has not been thoroughly explored.

Insider Threat Study: Illicit Cyber Activity Involving Fraud in the U.S. Financial Services Sector

July 1, 2012 • special report, by david mcintire, adam cummings, andrew p. moore, randall f. trzeciak, todd lewellen.

In this report, the authors describe insights and risk indicators of malicious insider activity in the banking and finance sector.

Supporting the Use of CERT Secure Coding Standards in DoD Acquisitions

July 1, 2012 • technical note, by john k. bergey, philip miller, robert c. seacord, timothy morrow.

In this report, the authors provide guidance for helping DoD acquisition programs address software security in acquisitions.

The Evolution of a Science Project: A Preliminary System Dynamics Model of a Recurring Software-Reliant Acquisition Behavior

July 1, 2012 • technical report, by william e. novak, andrew p. moore, christopher j. alberts.

This report uses a preliminary system dynamics model to analyze a specific adverse acquisition dynamic concerning the poorly controlled evolution of small prototype efforts into full-scale systems.

Introduction to System Strategies

June 27, 2012 • white paper.

In this paper, the authors discuss the effects of the changing operational environment on the development of secure systems.

Introduction to Modeling Tools for Software Security

June 24, 2012 • white paper.

In this paper, Samuel Redwine introduces security concepts and tools useful for modeling security properties.

Security-Specific Bibliography

June 22, 2012 • white paper.

In this paper, the authors provide a bibliography of sources related to security.

A Virtual Upgrade Validation Method for Software-Reliant Systems

June 1, 2012 • technical report, by dionisio de niz, peter h. feiler, david p. gluch, lutz wrage.

This report presents the Virtual Upgrade Validation (VUV) method, an approach that uses architecture-centric, model-based analysis to identify system-level problems early in the upgrade process to complement established test qualification …

Report from the First CERT-RMM Users Group Workshop Series

April 1, 2012 • technical note, by lisa r. young, julia h. allen.

In this report, the authors describe the first CERT RMM Users Group (RUG) Workshop Series and the experiences of participating members and CERT staff.

Source Code Analysis Laboratory (SCALe)

By david svoboda, robert w. stoddard, robert c. seacord, will dormann, james mccurley, philip miller, jefferson welch.

In this report, the authors describe the CERT Program's Source Code Analysis Laboratory (SCALe), a conformance test against secure coding standards.

Insider Threat Security Reference Architecture

April 1, 2012 • technical report, by joji montelibano, andrew p. moore.

In this report, the authors describe the Insider Threat Security Reference Architecture (ITSRA), an enterprise-wide solution to the insider threat.

A Pattern for Increased Monitoring for Intellectual Property Theft by Departing Insiders

By andrew p. moore, dave mundie, michael hanley.

In this report, the authors present techniques for helping organizations plan, prepare, and implement means to mitigate insider theft of intellectual property.

The Impact of Passive DNS Collection on End-User Privacy

March 22, 2012 • white paper, by jonathan spring, carly l. huth.

In this paper, the authors discuss whether pDNS allows reconstruction of an end user's DNS behavior and if DNS behavior is personally identifiable information.

Approaches for Edge-Enabled Tactical Systems

March 19, 2012 • white paper.

This booklet contains brief articles about using mobile devices in the areas of edge-enabled systems and cloud computing and a report on cloud offload in hostile environments.

Digital Investigation Workforce Development

March 1, 2012 • white paper.

In this paper, the authors describe an approach for deriving measures of software security from well-established and commonly used standard practices.

What’s New in V2 of the Architecture Analysis & Design Language Standard?

March 1, 2012 • special report, by peter h. feiler, joe seibel, lutz wrage.

This report provides an overview of changes and improvements to the Architecture Analysis & Design Language (AADL) standard for describing both the software architecture and the execution platform architectures of …

Principles of Trust for Embedded Systems

March 1, 2012 • technical note, by david fisher.

In this report, David Fisher provides substance and explicit meaning to the terms trust and trustworthy as they relate to automated systems.

Deriving Software Security Measures from Information Security Standards of Practice

February 16, 2012 • white paper, by robert w. stoddard, julia h. allen, christopher j. alberts.

In this paper, the authors describe an approach for deriving measures of software security from common standard practices for information security.

Risk-Based Measurement and Analysis: Application to Software Security

February 1, 2012 • technical note, by christopher j. alberts, julia h. allen, robert w. stoddard.

In this report, the authors present the concepts of a risk-based approach to software security measurement and analysis and describe the IMAF and MRD.

Mission Risk Diagnostic (MRD) Method Description

By christopher j. alberts, audrey j. dorofee.

In this report, the authors describe the Mission Risk Diagnostic (MRD) method, which is used to assess risk in systems across the lifecycle and supply chain.

Proceedings of the Smart Grid Maturity Model Leadership Workshop

January 31, 2012 • special report.

In January 2012, leaders in the electric power industry collaborated with the SEI to build the future of the Smart Grid Maturity Model at the SGMM Leadership Workshop.

Modifying Lanchester's Equations for Modeling and Evaluating Malicious Domain Name Take-Down

January 6, 2012 • white paper.

In this paper, Jonathan Spring models internet competition on large, decentralized networks using a modification of Lanchester's equations for combat.

January 2, 2012 • White Paper

In this paper, the authors demonstrate that there are name servers that exhibit IP address flux, a behavior that falls outside the prescribed parameters.

Discerning the Intent of Maturity Models from Characterizations of Security Posture

January 1, 2012 • white paper.

In this paper, Rich Caralli discusses how using maturity models and characterizing security posture are activities with different intents, outcomes, and uses.

Communication Among Incident Responders - A Study

By robert floodeen, brett tjaden.

In this paper, the authors describe preliminary results of a study of how effective nine autonomous incident response organizations are.

Best Practices for Artifact Versioning in Service-Oriented Systems

January 1, 2012 • technical note, by william anderson, marc novakouski, grace lewis, jeff davenport.

This report describes some of the challenges of software versioning in an SOA environment and provides guidance on how to meet these challenges by following industry guidelines and recommended practices.

Interoperability in the e-Government Context

By marc novakouski, grace lewis.

This report describes a proposed model through which to understand interoperability in the e-government context.

Spotlight On: Malicious Insiders and Organized Crime Activity

By christopher king.

In this report, Christopher King provides a snapshot of who malicious insiders are, what and how they strike, and why.

A Closer Look at 804: A Summary of Considerations for DoD Program Managers

December 1, 2011 • special report, by stephany bellomo.

The information in this report is intended to help program managers reason about actions they may need to take to adapt and comply with the Section 804 NDAA for 2010 …

Standards-Based Automated Remediation: A Remediation Manager Reference Implementation, 2011 Update

By sagar chaki, mary popeck, rita c. creel, benjamin mccormick, jeff davenport, mike kinney (national security agency).

In this report, the authors describe work to develop standards for automated remediation of vulnerabilities and compliance issues on DoD networked systems.

Using Defined Processes as a Context for Resilience Measures

December 1, 2011 • technical note, by pamela d. curtis, linda parker gates, julia h. allen.

In this report, the authors describe how implementation-level processes can provide context for identifying and defining measures of operational resilience.

Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE)

December 1, 2011 • technical report, by debra anderson, james mccurley, robert w. stoddard, dennis goldenson, david zubrow, robert ferguson.

The method of quantifying uncertainty described in this report synthesizes scenario building, Bayesian Belief Network (BBN) modeling and Monte Carlo simulation into an estimation method that quantifies uncertainties, allows subjective …

An Investigation of Techniques for Detecting Data Anomalies in Earned Value Management Data

By mark kasunic, david zubrow, dennis goldenson, james mccurley.

This research demonstrated the effectiveness of various statistical techniques for discovering quantitative data anomalies.

German language translation of CMMI for Development, V1.3

November 1, 2011 • white paper.

This PDF contains a German language translation of CMMI for Development, V1.3.

Japanese Language Translation of CMMI for Development, V1.3

CERT® Resilience Management Model (CERT®-RMM) V1.1: NIST Special Publication Crosswalk Version 1

November 1, 2011 • technical note, by lisa r. young, kevin g. partridge.

In this report, the authors map CERT-RMM process areas to selected NIST special publications in the 800 series.

Agile Methods: Selected DoD Management and Acquisition Concerns

October 1, 2011 • technical note, by mary ann lapham, suzanne miller, nanette brown, alfred schenker, bart hackemack, linda levine, lorraine adams, charles (bud) hammons.

This technical note addresses some of the key issues that either must be understood to ease the adoption of Agile or are seen as potential barriers to adoption of Agile …

An Acquisition Perspective on Product Evaluation

By harry l. levinson, richard librizzi, grady campbell.

This technical note focuses on software acquisition and development practices related to the evaluation of products before, during, and after implementation.

CERT® Resilience Management Model (RMM) v1.1: Code of Practice Crosswalk Commercial Version 1.1

By kevin g. partridge, lisa r. young.

In this report, the authors explain how CERT-RMM process areas, industry standards, and codes of practice are used by organizations in an operational setting.

Insider Threat Control: Using Centralized Logging to Detect Data Exfiltration Near Insider Termination

By joji montelibano, michael hanley.

In this report, the authors present an insider threat pattern on how organizations can combat insider theft of intellectual property.

CERT® Resilience Management Model Capability Appraisal Method (CAM) Version 1.1

October 1, 2011 • technical report, by resilient enterprise management team.

In this report, the authors demonstrate that the SCAMPI method can be adapted and applied to CERT-RMM V1.1 as the reference model for a process appraisal.

Smart Grid Maturity Model: Matrix, Version 1.2

September 1, 2011 • white paper.

This document shows a matrix related to Smart Grid Maturity Model levels.

Proceedings of the Fourth International Workshop on a Research Agenda for Maintenance and Evolution of Service-Oriented Systems (MESOA 2010)

September 1, 2011 • special report, by dennis b. smith, kostas kontogiannis, grace lewis.

This report summarizes the proceedings from the 2010 MESOA workshop and includes the accepted papers that were the basis for the presentations given during the workshop.

Software Assurance Curriculum Project Volume IV: Community College Education

September 1, 2011 • technical report, by nancy r. mead, mark a. ardis (stevens institute of technology), elizabeth k. hawthorne (union county college).

In this report, the authors focus on community college courses for software assurance.

Understanding and Leveraging a Supplier’s CMMI Efforts: A Guidebook for Acquirers (Revised for V1.3)

By john scibilia, lawrence t. osiecki, mike phillips.

This guidebook helps acquisition organizations formulate questions for their suppliers related to CMMI. It also helps organizations interpret responses to identify and evaluate risks for a given supplier.

Smart Grid Maturity Model, Version 1.2: Model Definition

By the sgmm team.

The Smart Grid Maturity Model (SGMM) is business tool that provides a framework for electric power utilities to help modernize their operations and practices for delivering electricity.

Keeping Your Family Safe in a Highly Connected World

August 10, 2011 • white paper, by jonathan frederick, marie baker.

In this paper, the authors describe the risks of being victims of theft, including becoming involved unknowingly in illegal activities over a networked device.

Which CMMI Model Is for You?

August 1, 2011 • white paper, by mike phillips, sandra shrum.

A short white paper that provides guidance on selecting the best CMMI model for process improvement.

Architecting Service-Oriented Systems

August 1, 2011 • technical note, by philip bianco, grace lewis, paulo merson, soumya simanta.

This report presents guidelines for architecting service-oriented systems and the effect of architectural principles on system quality attributes.

Standards-Based Automated Remediation: A Remediation Manager Reference Implementation

July 1, 2011 • special report, by sagar chaki, mary popeck, rita c. creel, jeff davenport, mike kinney (national security agency), benjamin mccormick.

In this report, the authors describe work to develop standards for vulnerability and compliance remediation on DoD networked systems.

A Decision Framework for Selecting Licensing Rights for Noncommercial Computer Software in the DoD Environment

July 1, 2011 • technical report.

This report describes standard noncommercial software licensing alternatives as defined by U.S. Government and DoD regulations. It suggests an approach for identifying agency needs for license rights and the license …

Measures for Managing Operational Resilience

By pamela d. curtis, julia h. allen.

In this report, the Resilient Enterprise Management (REM) team suggests a set of top ten strategic measures for managing operational resilience.

An Online Learning Approach to Information Systems Security Education

June 13, 2011 • white paper, by robert c. seacord, norman bier (carnegie mellon university), marsha lovett (carnegie mellon university).

In this paper, the authors describe the development of a secure coding module that shows how to capture content, ensure learning, and scale to meet demand.

Monitoring Cloud Computing by Layer, Part 2

June 1, 2011 • white paper.

In this paper, Jonathan Spring presents a set of recommended restrictions and audits to facilitate cloud security.

A Preliminary Model of Insider Theft of Intellectual Property

June 1, 2011 • technical note, by dawn cappelli, thomas c. caron (john heinz iii college, school of information systems management, carnegie mellon university), eric d. shaw, andrew p. moore, randall f. trzeciak, derrick spooner.

In this report, the authors describe general observations about and a preliminary system dynamics model of insider crime based on our empirical data.

Software Assurance for System of Systems

May 1, 2011 • white paper, by john b. goodenough, linda m. northrop.

In this paper, the authors discuss confidence in system and SoS behavior and how theories can be used to make the assurance process more effective.

Architecture Evaluation without an Architecture: Experience with the Smart Grid

April 30, 2011 • white paper, by rick kazman, gabriel moreno, james ivers, len bass.

This paper describes an analysis of some of the challenges facing one portion of the Electrical Smart Grid in the United States - residential Demand Response (DR) systems.

Correlating Domain Registrations and DNS First Activity in General and for Malware

April 11, 2011 • white paper, by ed stoner, jonathan spring, leigh b. metcalf.

In this paper, the authors describe a pattern in the amount of time it takes for that domain to be actively resolved on the Internet.

Architectures for the Cloud: Best Practices for Navy Adoption of Cloud Computing

April 5, 2011 • white paper.

The goal of SEI research is to create best practices for architecture and design of systems that take advantage of the cloud, leading to greater system quality from both a …

Monitoring Cloud Computing by Layer, Part 1

April 1, 2011 • white paper, principles of survivability and information assurance.

In this paper, the authors describe a Security Information and Event Management signature for detecting possible malicious insider activity.

Employing SOA to Achieve Information Dominance

SEI research will enable the Navy to to develop service-oriented systems that address information dominance priority requirements.

Managing Technical Debt in Software-Reliant Systems

By nanette brown.

This whitepaper argues that there is an opportunity to study and improve the “technical debt” metaphor concept and offers software engineers a foundation for managing such trade-offs based on models …

Appraisal Requirements for CMMI Version 1.3 (ARC, V1.3)

April 1, 2011 • technical report, by scampi upgrade team.

The Appraisal Requirements for CMMI, Version 1.3 (ARC, V1.3), defines the requirements for appraisal methods intended for use with Capability Maturity Model Integration (CMMI) and with the People CMM.

Best Practices for National Cyber Security: Building a National Computer Security Incident Management Capability, Version 2.0

By samuel a. merrell, bradford j. willke, john haller, matthew j. butkovic.

In this 2011 report, an update to its 2010 counterpart, the authors provide insight that interested organizations and governments can use to develop a national incident management capability.

Trusted Computing in Embedded Systems Workshop

March 1, 2011 • special report, by archie d. andrews, jonathan m. mccune.

In this report, the authors describe the November 2010 Trusted Computing in Embedded Systems Workshop held at Carnegie Mellon University.

Issues and Opportunities for Improving the Quality and Use of Data in the Department of Defense

By erin harper, mark kasunic, david zubrow.

This report contains the recommendations of an SEI-lead, joint-sponsored workshop by the OSD (AT&L) and DDR&, around the topics of data quality, data analysis, and data use.

IEEE Computer Society/Software Engineering Institute Software Process Achievement (SPA) Award 2009

March 1, 2011 • technical report, by satyendra kumar, ramakrishnan m..

This report describes the work of the 2009 recipient of the IEEE Computer Society Software Process Achievement Award, jointly established by the SEI and IEEE to recognize outstanding achievements in …

CMMI for Acquisition (CMMI-ACQ) Primer, Version 1.3

By mike phillips.

Acquisition practices for the project level that help you get started with CMMI for Acquisition practices without using the whole model.

Software Assurance Curriculum Project Volume III: Master of Software Assurance Course Syllabi

By julia h. allen, nancy r. mead, richard c. linger (oak ridge national laboratory), andrew j. kornecki (embry-riddle aeronautical university), thomas b. hilburn (embry-riddle aeronautical university), mark a. ardis (stevens institute of technology).

In this report, the authors provide sample syllabi for the nine core courses in the Master of Software Assurance Reference Curriculum.

Delivering Software-Reliant Products Faster: Take Action to Help Your Organization Gain Speed Without Sacrificing Quality

February 14, 2011 • white paper.

Learn how to deliver software-reliant products faster and explore ways to use software architecture more effectively.

Delivering Software-Reliant Products Faster: Help Your Organization Gain Speed Without Sacrificing Quality

Learn how to look into the initial steps suggested for delivering software-reliant products faster.

A Framework for Evaluating Common Operating Environments: Piloting, Lessons Learned, and Opportunities

February 1, 2011 • special report, by steve rosemergy, cecilia albert.

This report explores the interdependencies among common language, business goals, and soft-ware architecture as the basis for a common framework for conducting evaluations of software technical solutions.

Integrating the Master of Software Assurance Reference Curriculum into the Model Curriculum and Guidelines for Graduate Degree Programs in Information Systems

February 1, 2011 • technical note, by dan shoemaker (university of detroit mercy), jeff ingalsbe (university of detroit mercy), nancy r. mead.

In this report, the authors examine how the Master of Software Assurance Reference Curriculum can be used for a Master of Science in Information Systems.

An Analysis of Technical Observations in Insider Theft of Intellectual Property Cases

By michael hanley, joji montelibano, tyler dean, will schroeder, matt houy, randall f. trzeciak.

In this report, the authors provide an overview of techniques used by malicious insiders to steal intellectual property.

Results of SEI Independent Research and Development Projects (FY 2010)

February 1, 2011 • technical report, by gabriel moreno, jeffrey hansen, john j. hudak, daniel plakosh, joe seibel, charles weinstock, cory cohen, william anderson, soumya simanta, peter h. feiler, robert nord, dionisio de niz, ipek ozkaya, edwin j. morris, nanette brown, jörgen hansson (university of skovde), lutz wrage, david p. gluch, richard c. linger (oak ridge national laboratory), howard f. lipson, david fisher, onur mutlu, christopher craig, tim daly, andres diaz-pace, ragunathan rajkumar, karthik lakshmanan, mark pleszkoch, archie d. andrews.

This report describes results of independent research and development (IRAD) projects undertaken in fiscal year 2010.

Network Monitoring for Web-Based Threats

By matthew heckathorn.

In this report, Matthew Heckathorn models the approach an attacker would take and provides detection or prevention methods to counter that approach.

Function Extraction (FX) Research for Computation of Software Behavior: 2010 Development and Application of Semantic Reduction Theorems for Behavior Analysis

By tim daly, mark pleszkoch, richard c. linger (oak ridge national laboratory).

In this report, the authors present research to compute the behavior of software with mathematical precision and how this research has been implemented.

FloCon 2011 Proceedings

January 10, 2011 • white paper.

These papers were presented at FloCon 2011, where participants discussed dark space, web servers, spam, and the susceptibility of DNS servers to cache poisoning.

Deriving Candidate Technical Controls and Indicators of Insider Attack from Socio-Technical Models and Data

January 1, 2011 • technical note, by michael hanley.

In this 2011 report, Michael Hanley demonstrates how a method for modeling insider crimes can create candidate technical controls and indicators.

Trust and Trusted Computing Platforms

By archie d. andrews, jonathan m. mccune, david fisher.

This technical note examines the Trusted Platform Module, which arose from work related to the Independent Research and Development project "Trusted Computing in Extreme Adversarial Environments: Using Trusted Hardware as …

Enabling Agility Through Architecture

December 16, 2010 • white paper, by nanette brown, ipek ozkaya, robert nord.

Enabling Agility Through Architecture: A Crosstalk article by Nanette Brown, Rod Nord, and Ipek Ozkaya.

Software Supply Chain Risk Management: From Products to Systems of Systems

December 1, 2010 • technical note, by christopher j. alberts, carol woody, rita c. creel, robert j. ellison, audrey j. dorofee.

In this report, the authors consider current practices in software supply chain analysis and suggest some foundational practices.

A Taxonomy of Operational Cyber Security Risks

By james j. cebula, lisa r. young.

In this report, the authors present a taxonomy of operational cyber security risks and its harmonization with other risk and security activities.

Source Code Analysis Laboratory (SCALe) for Energy Delivery Systems

December 1, 2010 • technical report, by philip miller, jefferson welch, james mccurley, david svoboda, robert w. stoddard, robert c. seacord, will dormann.

In this report, the authors describe the Source Code Analysis Laboratory (SCALe), which tests software for conformance to CERT secure coding standards.

Adaptive Flow Control for Enabling Quality of Service in Tactical Ad Hoc Wireless Networks

By edwin j. morris, soumya simanta, scott hissam, jeffrey hansen, daniel plakosh, b. craig meyers, lutz wrage.

The network infrastructure for users such as emergency responders or warfighters is wireless, ad hoc, mobile, and lacking in sufficient bandwidth. This report documents the results from 18 experiments to …

Combining Architecture-Centric Engineering with the Team Software Process

By robert nord, felix bachmann, jim mchale.

ACE methods and the TSP provides an iterative approach for delivering high quality systems on time and within budget. The combined approach helps organizations that must set an architecture/developer team …

Beyond Technology Readiness Levels for Software: U.S. Army Workshop Report

By suzanne miller, cecilia albert, stephen blanchette, jr..

This report synthesizes presentations, discussions, and outcomes from the "Beyond Technology Readiness Levels for Software" workshop from August 2010.

The CERT Approach to Cybersecurity Workforce Development

By christopher may, josh hammerstein.

This report describes a model commonly used for developing and maintaining a competent cybersecurity workforce, explains some operational limitations associated with that model, and presents a new approach to cybersecurity …

Guide for SCAMPI Appraisals: Accelerated Improvement Method (AIM)

December 1, 2010 • special report.

This document provides guidance to lead appraisers and appraisal teams unfamiliar with TSP+ when conducting Standard CMMI Appraisal Method for Process Improvement (SCAMPI) appraisals within organizations that use the TSP+ …

Implementation Guidance for the Accelerated Improvement Method (AIM)

This 2010 report describes the (AIM which helps an organization to implement high-performance, high-quality CMMI practices much more quickly than industry norms.

Executive Overview: Best Practices for Adoption of Cloud Computing

November 24, 2010 • white paper.

This paper describes the SEI approach to cloud computing research for the DoD.

Executive Overview: Employing SOA to Achieve Information Dominance

The current ability to implement systems in the DoD based on SOA technologies falls short of the DoD's goals. To close the gaps in these areas, research is needed in …

French language translation of CMMI for Development, V1.3

November 1, 2010 • white paper.

This is The French language translation of CMMI for Development, V1.3.

Dutch language translation of CMMI for Development V1.3

This document is the Dutch language translation of CMMI-DEV V1.3.

Spanish Language Translation of CMMI for Development, v1.3

Spanish language translation of CMMI for Development, v1.3

Traditional Chinese Language Translation of CMMI for Development V1.3

CMMI-DEV V1.3 Traditional Chinese Translation

A Workshop on Analysis and Evaluation of Enterprise Architectures

November 1, 2010 • technical note, by john klein, michael j. gagliardi.

This report summarizes a workshop on the analysis and evaluation of enterprise architectures that was held at the SEI in April of 2010.

Performance Analysis of WS-Security Mechanisms in SOAP-Based Web Services

November 1, 2010 • technical report, by gunnar peterson, marc novakouski, soumya simanta, edwin j. morris, grace lewis.

This paper presents the results of a series of experiments targeted at analyzing the performance impact of adding WS-Security, a common security standard used in IdM frameworks, to SOAP-based web …

CMMI for Acquisition, Version 1.3

The CMMI-ACQ model provides guidance for applying CMMI best practices in an acquiring organization. Best practices in the model focus on activities for initiating and managing the acquisition of products …

CMMI for Development, Version 1.3

This 2010 report details CMMI for Development (CMMI-DEV) V.1.3, which provides a comprehensive integrated set of guidelines for developing products and services.

CMMI for Services, Version 1.3

This 2010 report details CMMI for Services (CMMI-SVC) V.1.3, which provides a comprehensive integrated set of guidelines for providing superior services.

Strategic Planning with Critical Success Factors and Future Scenarios: An Integrated Strategic Planning Framework

By linda parker gates.

This report explores the value of enhancing typical strategic planning techniques with the CSF method and scenario planning.

Designing for Incentives: Better Information Sharing for Better Software Engineering

October 31, 2010 • white paper.

This paper outlines a research agenda in bridging to the economic theory of mechanism design, which seeks to align incentives in multi-agent systems with private information and conflicting goals.

Cloud Computing Basics Explained

September 30, 2010 • white paper.

This paper seeks to help organizations understand cloud computing essentials, including drivers for and barriers to adoption, in support of making decisions about adopting the approach.

Primer on SOA Terms

September 1, 2010 • white paper.

This white paper presents basic terminology related to Service- Oriented Architecture (SOA). The goal of the paper is to establish a baseline of terms for service-oriented systems.

T-Check in System-of-Systems Technologies: Cloud Computing

September 1, 2010 • technical note, by grace lewis, harrison d. strowd.

The purpose of this report is to examine a set of claims about cloud computing adoption.

Emerging Technologies for Software-Reliant Systems of Systems

The purpose of this report is to present an informal survey of technologies that are, or are likely to become, important for software-reliant systems of systems in response to current …

Integrated Measurement and Analysis Framework for Software Security

By christopher j. alberts, robert w. stoddard, julia h. allen.

In this report, the authors address how to measure software security in complex environments using the Integrated Measurement and Analysis Framework (IMAF).

Security Requirements Reusability and the SQUARE Methodology

In this report, the authors discuss how security requirements engineering can incorporate reusable requirements.

Measuring Operational Resilience Using the CERT® Resilience Management Model

By noopur davis, julia h. allen.

In this 2010 report, the authors begin a dialogue and establish a foundation for measuring and analyzing operational resilience.

Program Executive Officer Aviation, Major Milestone Reviews: Lessons Learned Report

September 1, 2010 • technical report, by kate ambrose, scott reed.

This report documents ideas and recommendations for improving the overall acquisition process and presents the actions taken by project managers in several programs to develop, staff, and obtain approval for …

Smart Grid Maturity Model, Version 1.1: Model Definition

Success in acquisition: using archetypes to beat the odds, by william e. novak, linda levine.

This report describes key elements in systems thinking, provides an introduction to general systems archetypes, and applies these concepts to the software acquisition domain.

Building Assured Systems Framework

By julia h. allen, nancy r. mead.

This report presents the Building Assured Systems Framework (BASF) that addresses the customer and researcher challenges of selecting security methods and research approaches for building assured systems.

Using TSP Data to Evaluate Your Project Performance

By bill nichols, james mccurley, shigeru sasao.

A set of measures was determined that allow analyses This report discusses the application of a set of measures to a data set of 41 TSP projects from an organization …

Suggestions for Documenting SOA-Based Systems

This report provides suggestions for documenting service-oriented architecture-based systems based on the Views & Beyond (V&B) software documentation approach.

Exploring Acquisition Strategies for Adopting a Software Product Line

August 25, 2010 • white paper, by john k. bergey, lawrence g. jones.

Some basics of software product line practice, the challenges that make product line acquisition unique, and three basic acquisition strategies are all part of this white paper.

YAF: Yet Another Flowmeter

August 23, 2010 • white paper, by chris inacio, brian trammell.

In this paper, the authors describe issues encountered in designing and implementing YAF.

A Continuous Time List Capture Model for Internet Threats

August 4, 2010 • white paper, by rhiannon weaver.

In this paper, Rhiannon Weaver describes a population study of malware files under the CTLC framework and presents a simulation study as well as future work.

Software Assurance Curriculum Project Volume I: Master of Software Assurance Reference Curriculum

August 1, 2010 • technical report, by james mcdonald (monmouth university), richard c. linger (oak ridge national laboratory), thomas b. hilburn (embry-riddle aeronautical university), andrew j. kornecki (embry-riddle aeronautical university), mark a. ardis (stevens institute of technology), julia h. allen, nancy r. mead.

In this report, the authors present a master of software assurance curriculum that educational institutions can use to create a degree program or track.

Risk Management Framework

By audrey j. dorofee, christopher j. alberts.

In this report, the authors specify (1) a framework that documents best practice for risk management and (2) an approach for evaluating a program's risk management practice in relation to …

Software Assurance Curriculum Project Volume II: Undergraduate Course Outlines

By thomas b. hilburn (embry-riddle aeronautical university), richard c. linger (oak ridge national laboratory), nancy r. mead.

In this report, the authors describe seven courses for an undergraduate curriculum specialization for software assurance.

A Framework for Modeling the Software Assurance Ecosystem: Insights from the Software Assurance Landscape Project

By christopher j. alberts, carol woody, lisa brownsword, andrew p. moore.

In this report, the authors describe the SEI Assurance Modeling Framework, piloting to prove its value, and insights gained from that piloting.

COVERT: A Framework for Finding Buffer Overflows in C Programs via Software Verification

By arie gurfinkel, sagar chaki.

In this report, the authors present COVERT, an automated framework for finding buffer overflows in C programs using software verification tools and techniques.

Measurement and Analysis Infrastructure Diagnostic, Version 1.0: Method Definition Document

By mark kasunic.

This 2010 report is a guidebook for conducting a Measurement and Analysis Infrastructure Diagnostic (MAID) evaluation.

Security Requirements Engineering

July 14, 2010 • white paper.

In this paper, Nancy Mead how a systematic approach to security requirements engineering helps to avoid problems.

Adapting the SQUARE Process for Privacy Requirements Engineering

July 1, 2010 • technical note, by nancy r. mead, ashwini bijwe (carnegie mellon university).

In this 2010 report, the authors explore how the SQUARE process can be adapted for privacy requirements engineering in software development.

Team Software Process (TSP) Body of Knowledge (BOK)

July 1, 2010 • technical report, by timothy a. chick, bill nichols, watts s. humphrey, marsha pomeroy-huff.

The TSP BOK helps practitioners and employers assess and improve their skills, and shows academic institutions how to incorporate TSP into their engineering courses.

Programmatic and Constructive Interdependence: Emerging Insights and Predictive Indicators of Development Resource Demand

By mark kasunic, william anderson, david zubrow, paul l. hardin, iii, mary m. brown, robert m. flowe, james mccurley.

This 2010 report describes a series of ongoing research efforts that investigate the role of interdependence in the acquisition of major defense acquisition programs.

Rayon: A Unified Framework for Data Visualization

June 24, 2010 • white paper, by phil groce.

In this paper, Phil Groce describes the Rayon visualization toolkit, developed to augment network analytic information and improve analytic operations.

Finding Malicious Activity in Bulk DNS Data

By ed stoner.

In this paper, Ed Stoner describes techniques for detecting certain types of malicious traffic.

Best Practices for National Cyber Security: Building a National Computer Security Incident Management Capability

June 1, 2010 • special report, by john haller, matthew j. butkovic, samuel a. merrell, bradford j. willke.

In this report, the authors provide insight that interested organizations and governments can use to develop a national incident management capability.

Team Software Process (TSP) Coach Mentoring Program Guidebook, Version 1.1

By jefferson welch, alan willett, marsha pomeroy-huff, robert cannon, timothy a. chick, bill nichols, jim mchale.

This guidebook is designed to explain the steps for becoming an SEI-Certified Team Software Process (TSP) Coach or SEI-Certified TSP Mentor Coach, with emphasis on guiding individuals through the mentoring …

Survivability Analysis Framework

June 1, 2010 • technical note.

In this report, the authors describe the Survivability Analysis Framework, which is used to evaluate critical operational capabilities.

Software Product Lines: Report of the 2010 U.S. Army Software Product Line Workshop

June 1, 2010 • technical report, by linda m. northrop, sholom g. cohen, john k. bergey, gary chastek, patrick donohoe, lawrence g. jones.

This report synthesizes presentations and discussions from a 2010 workshop to discuss product line practices and operational accomplishments.

Performance Effects of Measurement and Analysis: Perspectives from CMMI High Maturity Organizations and Appraisers

By dennis goldenson, james mccurley.

This report describes results from two recent surveys conducted by the Software Engineering Institute (SEI) to collect information about the measurement and analysis activities of software systems development organizations.

Resource Allocation in Distributed Mixed-Criticality Cyber-Physical Systems

May 31, 2010 • white paper, by karthik lakshmanan.

This paper explains a formal overload-resilience metric called ductility.

The Illusion of Certainty - Paper

May 25, 2010 • white paper, by grady campbell.

In this 2010 paper, Grady Campbell - delivered at the 7th Acquisition Research Symposium - argues that a new approach to acquisition is needed that recognizes that hiding uncertainty is …

Edge Enabled Systems

May 19, 2010 • white paper, by zacharie hall (aberdeen proving ground), joseph giampapa, rick kazman, kurt c. wallnau, daniel plakosh.

This paper describes the characteristics of edge systems and the edge organizations in which these systems operate, and make initial recommendations about how such systems and organizations can be created …

Managing Variation in Services in a Software Product Line Context

May 1, 2010 • technical note, by sholom g. cohen, robert w. krut, jr..

This report highlights the mutual benefits of combining systematic reuse approaches from product line development with flexible approaches for implementing business processes in a service oriented architecture.

Evaluating and Mitigating Software Supply Chain Security Risks

By carol woody, robert j. ellison, john b. goodenough, charles weinstock.

In this 2010 report, the authors identify software supply chain security risks and specify evidence to gather to determine if these risks have been mitigated.

Relating Business Goals to Architecturally Significant Requirements for Software Systems

By paul c. clements, len bass.

The purpose of this report is to facilitate better elicitation of high-pedigree quality attribute requirements. Toward this end, we want to be able to elicit business goals reliably and understand …

Case Study: Model-Based Analysis of the Mission Data System Reference Architecture

May 1, 2010 • technical report, by peter h. feiler, kurt woodham (l-3 communications-titan group), david p. gluch.

This report describes how AADL support an instantiation of a reference architecture, address architectural themes, and provide a foundation for the analysis of performance elements and system assurance concerns.

Identifying Anomalous Port-Specific Network Behavior

In this report, Rhiannon Weaver describes a method for identifying network behavior that may be a sign of coming internet-wide attacks.

CERT Resilience Management Model, Version 1.0

By david w. white, julia h. allen, richard a. caralli, lisa r. young, pamela d. curtis.

In this report, the authors present CERT-RMM, an approach to managing operational resilience in complex, risk-evolving environments.

Java Concurrency Guidelines

By fred long, dhruv mohindra, david svoboda, robert c. seacord.

In this report, the authors describe the CERT Oracle Secure Coding Standard for Java, which provides guidelines for secure coding in Java.

Specifications for Managed Strings, Second Edition

By robert c. seacord, david svoboda, fred long, raunak rungta, hal burch.

In this report, the authors describe a managed string library for the C programming language.

Considerations for Using Agile in DoD Acquisition

April 1, 2010 • technical note, by mary ann lapham, alfred schenker, daniel burton, charles (bud) hammons, ray c. williams.

This 2010 report explores the questions: Can Agile be used in the DoD environment? If so, how?

As-If Infinitely Ranged Integer Model, Second Edition

By timothy wilson, thomas plum (plum hall, inc.), roger dannenberg (school of computer science, carnegie mellon university), alex volkovitsky, robert c. seacord, will dormann, david keaton, david svoboda.

In this report, the authors present the as-if infinitely ranged (AIR) integer model, a mechanism for eliminating integral exceptional conditions.

Data Rights for Proprietary Software Used in DoD Programs

By julie b. cohen, bonnie troup (the aerospace corporation), henry ouyang (the aerospace corporation).

This report examines how data rights issues were addressed in the TSAT program. It also reviews concerns posed by the use of commercial software in the TSAT program's Space Segment, …

Characterizing Technical Software Performance Within System of Systems Acquisitions: A Step-Wise Methodology

April 1, 2010 • technical report, by bryce l. meyer, james wessel.

This report focuses on both qualitative and quantitative ways of determining the current state of SWP (software performance) in terms of both test coverage and confidence for SOA-based SoS environments.

Measuring Software Security

March 1, 2010 • white paper.

This paper, extracted from the 2009 CERT Research Report, describes planned research tasks in the field of software security.

Cyber Assurance

By robert j. ellison, carol woody, christopher j. alberts.

This paper, extracted from the 2009 CERT Research Report, describes planned research tasks in the field of cyber assurance.

Evaluating Software's Impact on System and System and System of Systems Reliability

In this paper, the authors discuss how system engineers are uncertain about how to determine the impact of software on overall system.

A Research Agenda for Service-Oriented Architecture (SOA): Maintenance and Evolution of Service-Oriented Systems

March 1, 2010 • technical note.

This 2010 report describes the agenda of an SEI-led group that was formed to explore the business, engineering, and operations aspects of service-oriented architecture.

Extending Team Software Process (TSP) to Systems Engineering: A NAVAIR Experience Report

March 1, 2010 • technical report, by timothy a. chick, anita carleton, jeff schwalb, delwyn kellogg, james w. over.

This 2010 report communicates status, progress, lessons learned, and results on a joint collaboration between the SEI and NAVAIR.

Testing in Service-Oriented Environments

By david j. carney, sriram balasubramaniam, john morley, patrick r. place, soumya simanta, william anderson, edwin j. morris.

This report makes 65 recommendations for improving testing in service-oriented environments. It covers testing functionality and testing for interoperability, security, performance, and reliability qualities.

Reports from the Field on System of Systems Interoperability Challenges and Promising Approaches

By carol sledge.

In this report, Carol Sledge identifies challenges and successful approaches to achieving system of systems (SoS) interoperability.

Adapting the SQUARE Method for Security Requirements Engineering to Acquisition

February 22, 2010 • white paper.

In this paper, Nancy Mead adapts the SQUARE process for security requirements engineering to different acquisition situations.

0-knowledge fuzzing

February 9, 2010 • white paper, by vincenzo iozzo (zynamics).

In this paper, Vincenzo Iozzo describes how to effectively fuzz with no knowledge of the user-input and the binary.

MITRE, CWE, and CERT Secure Coding Standards

February 8, 2010 • white paper, by robert c. seacord, robert a. martin.

In this paper, the authors summarize the Common Weakness Enumeration (CWE) and CERT Secure Coding Standards and the relationship between the two.

A Probabilistic Population Study of the Conficker-C Botnet

February 1, 2010 • white paper.

In this paper, Rhiannon Weaver estimates the number of active machines per hour infected with the Conficker-C worm using a probability model.

Instrumented Fuzz Testing Using AIR Integers (Whitepaper)

By will dormann, robert c. seacord, david keaton, roger dannenberg (school of computer science, carnegie mellon university), thomas plum (plum hall, inc.), timothy wilson.

In this paper, the authors present the as-if infinitely ranged (AIR) integer model, which provides a mechanism for eliminating integral exceptional conditions.

Spotlight On: Insider Threat from Trusted Business Partners

By randall f. trzeciak, andrew p. moore, derrick spooner, dawn cappelli, robert weiland (carnegie mellon university).

In this report, the authors focus on cases in which the insider was employed by a trusted business partner of the victim organization.

Profiling Systems Using the Defining Characteristics of Systems of Systems (SoS)

February 1, 2010 • technical note, by donald firesmith.

This technical note identifies and describes the characteristics that have been used in various definitions of the term system of systems.

Proceedings of the 3rd International Workshop on a Research Agenda for Maintenance and Evolution of Service-Oriented Systems (MESOA 2009)

February 1, 2010 • special report.

This report contains selected papers from the 3rd International Workshop on a Research Agenda for Maintenance and Evolution of Service-Oriented Systems (MESOA 2009).

Acquisition Archetype: Shooting the Messenger

January 20, 2010 • white paper.

When problems are detected in programs, everyone needs to listen and work together towards a solution. Shooting the messenger only delays the process, and hurts program morale.

Industry Standard Notation for Architecture-Centric Model-Based Engineering

In this paper, Peter Feiler describes the AADL, an industry standard for modeling and analyzing the architecture of software-reliant systems.

Approaches to Process Performance Modeling: A Summary from the SEI Series of Workshops on CMMI High Maturity Measurement and Analysis

January 1, 2010 • technical report.

This report summarizes the results from the second and third high maturity measurement and analysis workshops.

Evaluating the Software Design of a Complex System of Systems

By steven crosson (u.s. army), barry boehm (university of california, los angeles), stephen blanchette, jr..

The report examines the application of the life-cycle architecture milestone to the software and computing elements of the former Future Combat Systems program.

Secure Coding Governance and Guidance

December 4, 2009 • white paper.

In this paper, the authors propose the use of secure coding standards in the development of software for surface combatants and submarines.

Secure Coding Plan

This plan is a government-provided customizable document that is part of the acquisition's government reference library.

Generalized Criteria and Evaluation Method for Center of Excellence: A Preliminary Report

December 1, 2009 • technical note, by william craig (amrdec sed), clay kaylor (amrdec sed), john porter (amrdec sed), scott reed, matt fisher, suzanne miller.

Criteria and standards to certify an organization as a COE are presented in this Carnegie Mellon Software Engineering Institute preliminary report.

A Structured Approach for Reviewing Architecture Documentation

By rich hilliard, david emery, robert nord, paul c. clements.

This technical note proposes a structured approach for reviewing architecture documentation that is centered on the documentation's stakeholders and engages them in a guided manner so as to ensure that …

Measurement and Analysis Infrastructure Diagnostic (MAID) Evaluation Criteria, Version 1.0

December 1, 2009 • technical report, by software engineering measurement and analysis (sema) group.

This 2009 report presents the criteria used during a MAID evaluation that serve as a checklist to rate the quality of an organization's measurement and analysis practices and the quality …

Results of SEI Independent Research and Development Projects (FY 2009)

By jörgen hansson (university of skovde), len bass, lutz wrage, cal waits, matthew geiger, karthik lakshmanan, ragunathan rajkumar, peter h. feiler, kurt c. wallnau, dionisio de niz, paul c. clements, mark h. klein, james ivers, daniel plakosh, kristopher rush, jeffrey hansen, gabriel moreno, scott hissam.

In this report, the authors describe the SEI independent research and development (IRAD) projects conducted during fiscal year 2009.

An Everyday Example of Architecture Documentation: Subway Maps

November 30, 2009 • white paper, by paul c. clements.

This white paper explores the idea that subway maps provide a good, common example of architecture documentation and that they might be instructive about good software architecture documentation.

System of Systems Software Assurance

November 2, 2009 • white paper, by john b. goodenough.

This white paper describes SEI investigation into ways to provide justified confidence that a system of systems will behave as needed in its actual and evolving usage environments.

Proceedings of the Workshop on Software Engineering Foundations for End-User Programming (SEEUP 2009)

November 1, 2009 • special report, by brad myers, len bass, dennis b. smith, grace lewis.

This report presents the papers that were given at SEEUP 2009, held at the 31st ICSE in Vancouver, British Columbia on May 23, 2009.

The Watts New Collection: Columns by the SEI’s Watts Humphrey

By watts s. humphrey.

news@sei columns written by the SEI's Watts Humphrey between June 1998 and August 2008

Evaluating Artifact Quality from an Appraisal Perspective

November 1, 2009 • technical note, by emanuel r. baker, matt fisher, charlene gross.

This report explores the lack of agreement among SCAMPI Lead Appraisers about what “artifact quality” means in the SCAMPI process context.

Evaluating Process Quality from an Appraisal Perspective

By matt fisher, emanuel r. baker.

This report explores the lack of agreement among SCAMPI Lead Appraisers about what “process quality” means in the SCAMPI process context.

A Method for Assessing Technical Progress and Quality Throughout the System Life Cycle

By robert ferguson, rita c. creel, summer c. fowler.

This 2009 paper provides a framework for evaluating a system from several perspectives for a comprehensive picture of progress and quality.

Integrating CMMI and TSP/PSP: Using TSP Data to Create Process Performance Models

By shurei tamura.

This report describes the fundamental concepts of process performance models (PPMs) and describes how they can be created using data generated by projects following the TSP.

System Architecture Virtual Integration: An Industrial Case Study

November 1, 2009 • technical report, by lutz wrage, jörgen hansson (university of skovde), peter h. feiler, dionisio de niz.

This report introduces key concepts of the SAVI paradigm and discusses the series of development scenarios used in a POC demonstration to illustrate the feasibility of improving the quality of …

The Software Quality Profile

October 29, 2009 • white paper.

The software community has been slow to use data to measure software quality. This paper discusses the reasons for this problem and describes a way to use process measurements to …

Acquisition Archetypes: Happy Path Testing

October 15, 2009 • white paper, by linda levine, william e. novak.

When time and budget are tight, it's tempting to follow the "happy path" in testing. But be careful: it may be a path that brings your program great unhappiness.

Acquisition Archetypes: Brooks' Law

This April 2009 whitepaper focuses on the problems of underspending, which can result in funds being shifted from one acquisition program to another.

The Economics of CMMI

This paper provides practical guidance for CMMI adopters in the effective use of CMMI, based upon established NDIA principles.

Insights on Program Success

October 1, 2009 • special report, by systems and software consortium, inc., the software engineering institute.

This 2009 report examines the reasons why some programs fail and studies the factors that lead to program success.

A Bibliography of the Personal Software Process (PSP) and the Team Software Process (TSP)

By marlene macdonald, rachel callison.

This 2009 special report provides a bibliography of books, articles, and other literature concerning the PSP and TSP methodologies.

Towards an Assurance Case Practice for Medical Devices

October 1, 2009 • technical note, by charles weinstock, john b. goodenough.

In this report, the authors explore how to enable manufacturers and federal regulators gain confidence in software-dominated medical devices.

Data Model as an Architectural View

By paulo merson.

This 2009 report describes the data model as an architectural style in an effort to help architects apply this style to create data model architectural views.

Secure Design Patterns

October 1, 2009 • technical report, by chad dougherty, david svoboda, robert c. seacord, kirk sayre, kazuya togashi (jpcert/cc).

In this report, the authors describe a set of general solutions to software security problems that can be applied in many different situations.

CMMI and Medical Device Engineering

September 29, 2009 • white paper, by david w. walker.

This paper summarizes the comparison performed between the CMMI and the regulations and standards that drive software intensive medical device product development.

Lessons Learned from a Large, Multi-Segment, Software-Intensive System

September 1, 2009 • technical note, by mary ann lapham, john t. foreman.

This 2009 report contains a series of observations and their associated lessons learned from a large, multi-segment, software-intensive system.

Effectiveness of the Vulnerability Response Decision Assistance (VRDA) Framework

August 1, 2009 • white paper, by fumihiko kousaka (jpcert/cc), shawn mccaffrey (carnegie mellon university), masanori yamaguchi (iij technology inc.), robert weiland (carnegie mellon university), joseph b. kadane, kazuya togashi (jpcert/cc), christopher king, art manion.

In this paper, the authors describe the Vulnerability Response Decision Assistance (VRDA) framework, a decision support and expert system.

The Personal Software Process (PSP) Body of Knowledge, Version 2.0

August 1, 2009 • special report, by robert cannon, marsha pomeroy-huff, julia l. mullaney, timothy a. chick, bill nichols.

The Personal Software Process (PSP) body of knowledge (BOK) provides guidance to software professionals who are interested in using proven-effective, disciplined methods to improve their personal software development process.

Formulation of a Production Strategy for a Software Product Line

August 1, 2009 • technical note, by john mcgregor, gary chastek, patrick donohoe.

This 2009 report describes a technique for formulating the production strategy of a production system.

Realizing and Refining Architectural Tactics: Availability

August 1, 2009 • technical report, by james scott, rick kazman.

Tactics are fundamental elements of software architecture that an architect employs to meet a system's quality requirements. This report describes an updated set of tactics that enable the architect to …

Team Software Process (TSP) Coach Mentoring Program Guidebook

German language translation of cmmi for development, v1.2, july 31, 2009 • white paper.

The German language translation of CMMI for Development, V1.2.

Insider Theft of Intellectual Property for Business Advantage: A Preliminary Model

July 20, 2009 • white paper, by randall f. trzeciak, andrew p. moore, dawn cappelli, thomas c. caron (john heinz iii college, school of information systems management, carnegie mellon university), eric d. shaw.

In this paper, the authors describe general observations about a preliminary system dynamics model of insider crime based on our empirical data.

Why Don't They Practice What We Preach?

July 17, 2009 • white paper.

One of the most intractable problems in software is getting engineers to consistently use effective methods. The Software Engineering Institute has worked on this problem for a number of years …

Resiliency Management Model: Communications

July 1, 2009 • white paper.

In this paper, the authors describe the purpose of Communications: to develop, deploy, and manage communications to support resiliency activities and processes.

Privacy Risk Assessment Case Studies in Support of SQUARE

July 1, 2009 • special report, by nancy r. mead, varokas panusuwan, prashanth batlagundu.

In this report, the authors describe enhancements to the SQUARE method for addressing privacy requirements.

A Proactive Means for Incorporating a Software Architecture Evaluation in a DoD System Acquisition

July 1, 2009 • technical note, by john k. bergey.

This technical note provides guidance on how to contractually incorporate architecture evaluations in an acquisition.

Building Process Improvement Business Cases Using Bayesian Belief Networks and Monte Carlo Simulation

By ben linders.

This report describes a collaboration between the SEI and Ericsson Research and Development to build a business case using high maturity measurement approaches that require limited measurement effort.

As-if Infinitely Ranged Integer Model

By thomas plum (plum hall, inc.), alex volkovitsky, timothy wilson, robert c. seacord, david svoboda, david keaton.

In this report, the authors present the as-if infinitely ranged (AIR) integer model, which eliminates integer overflow and integer truncation in C and C++ code.

People Capability Maturity Model (P-CMM), Version 2.0, Second Edition

July 1, 2009 • technical report, by bill curtis (cast research labs), william e. hefley, sarah miller.

This report documents an update to the People CMM, Version 2, which updates informative material within the People CMM and its subpractices and provides new information learned from the continuing …

Revealing Cost Drivers for Systems Integration and Interoperability Through Q Methodology

June 10, 2009 • white paper, by william anderson, maureen brown (university of north carolina).

The findings suggest that Q Methodology may prove helpful in isolating many of the non-technical latent cost factors associated with system integration and interoperability.

Spanish language translation of CMMI for Development, V1.2

June 5, 2009 • white paper.

The Spanish language translation of CMMI for Development, V1.2 was performed by Cátedra de Mejora de Procesos de Software en el Espacio, Iberoamericano de la Universidad Politécnica de Madrid and …

Spotlight On: Insider Theft of Intellectual Property Inside the United States Involving Foreign Governments or Organizations (2009)

June 1, 2009 • white paper, by dawn cappelli, derrick spooner, randall f. trzeciak, andrew p. moore.

In this report, the authors focus on employees, contractors, and business partners who stole intellectual property to benefit a foreign entity.

Computational Evaluation of Software Security Attributes

By richard c. linger (oak ridge national laboratory), gwendolyn h. walton, thomas a. longstaff.

This paper provides an introduction to the CSA approach, provides behavioral requirements for security attributes, and discusses possible application of the CSA approach.

Measurement for Improvement: Successful Measurement Practices Used in Army Software Acquisition

June 1, 2009 • technical note, by james wessel, robert ferguson.

This report summarizes the findings of a study conducted for the Army to find and describe software measurement practices that are being used successfully.

A Scenario-Based Technique for Developing SOA Technical Governance

By grace lewis, edwin j. morris, soumya simanta, dennis b. smith, sriram balasubramaniam.

Organizations can make the available SOA governance frameworks more effective in their organizations using the scenario-based tailoring technique introduced in this technical note.

Incremental Development in Large-Scale Systems: Finding the Programmatic IEDs

By charles (bud) hammons.

This paper explores how continued use of the acquisition roadmaps opens up the potential for running into program pitfalls (programmatic IEDs) that aren‰t acknowledged on the map at hand.

Integrating Quality-attribute Reasoning Frameworks in the ArchE Design Assistant

May 5, 2009 • white paper, by felix bachmann, philip bianco, len bass, hyunwoo kim, andres diaz-pace.

Bachmann et al present their work on a design assistant called ArchE that provides third-party researchers with an infrastructure to integrate their own quality-attribute models.

Incorporating Software Requirements into the System RFP: Survey of RFP Language for Software by Topic, v. 2.0

May 1, 2009 • special report.

The 2009 report defines and communicates software engineering and management events necessary to support the successful acquisition of software-intensive systems.

Evaluating Hazard Mitigations with Dependability Cases

April 21, 2009 • white paper, by matthew r. barry (software intensive systems, inc.), john b. goodenough.

In this 2009 paper, the authors present an example to show the value a dependability case adds to a traditional hazard analysis.

Risk Detection and Mitigation Metrics and Design Check Lists for Real Time and Embedded Systems

April 19, 2009 • white paper, by doug locke, lui r. sha.

A whitepaper by Lui Sha of the University of Illinois and C. Douglass Locke of LC System Services Inc. The paper discusses risk detection and mitigation metrics and design check …

Assurance Cases for Design Analysis of Complex System of Systems Software

April 1, 2009 • white paper, by stephen blanchette, jr..

This paper discusses the application of assurance cases as a means of building confidence that the software design of a complex system of systems will actually meet the operational objectives …

Acquisition Archetypes: Longer Begets Bigger

Planning for a long development period doesn't always solve acquisition scheduling problems. Sometimes it makes them worse.

Acquisition Archetypes: Robbing Peter to Pay Paul

This April 2009 whitepaper is one in a short series of acquisition failures. This paper focuses on the problems of underspending, which can result in funds being shifted from one …

April 1, 2009 • Special Report

By antonio drommi, john harrison, jeff ingalsbe (university of detroit mercy), art conklin, james rainey, dan shoemaker (university of detroit mercy), nancy r. mead, julia h. allen.

In this report, the authors provide advice for those making a business case for building software assurance into software products during software development.

Impact of Army Architecture Evaluations

By stephen blanchette, jr., john k. bergey, robert nord, mark h. klein.

This 2009 report describes the results of a study of the impact that the ATAM evaluations and QAWs had on Army programs.

A Workshop on Architecture Competence

April 1, 2009 • technical note, by jeannine siviy, len bass, rick kazman, paul c. clements, mark h. klein, john klein.

This report summarizes a June 2008 architecture competence workshop where practitioners discussed key issues in assessing architecture competence in organizations.

A Framework for Categorizing Key Drivers of Risk

April 1, 2009 • technical report.

This 2009 report features a systemic approach for managing risk that takes into account the complex nature of distributed environments.

Software Product Lines: Report of the 2009 U.S. Army Software Product Line Workshop

By sholom g. cohen, lawrence g. jones, john k. bergey, matt fisher, patrick donohoe.

This report is a synthesis of the presentations and discussions that took place during the 2009 U.S. Army Software Product Line Workshop.

Acquisition Archetypes: Everything for Everybody

March 1, 2009 • white paper.

When projects attempt to please too many customers, complexity mounts, schedules slip, costs expand, and no one is happy.

Spotlight On: Malicious Insiders with Ties to the Internet Underground Community

By michael hanley, dawn cappelli, andrew p. moore, randall f. trzeciak.

In this report, the authors focus on insider threat cases in which the insider had relationships with the internet underground community.

An Initial Comparative Analysis of the CMMI Version 1.2 Development Constellation and the ISO 9000 Family

March 1, 2009 • special report, by david kitson, robert vickroy, john walz, dave wynn.

A preliminary, high-level comparison of the CMMI Development constellation and the ISO 9001:2000 family of process improvement standards.

U.S. Army Workshop on Exploring Enterprise, System of Systems, System, and Software Architectures

March 1, 2009 • technical report, by stephen blanchette, jr., john k. bergey, john klein, michael j. gagliardi, william wood, robert wojcik, paul c. clements.

This report confirms that various architectural genres enjoy more commonalities than differences. Each one has its own important knowledge base, and openness among the various architectural tasks within an organization …

Deploying TSP on a National Scale: An Experience Report from Pilot Projects in Mexico

By rafael salazar, bill nichols.

This report communicates status, progress, lessons learned, and next steps for the Mexican TSP Initiative.

CMMI for Services V1.2 (Traditional Chinese)

February 1, 2009 • white paper.

The Traditional Chinese translation of CMMI for Services V.1.2.

Multi-View Decision Making (MVDM) Workshop

February 1, 2009 • special report, by christopher j. alberts, carol woody, james smith.

In this report, the authors describe the value of multi-view decision making, a set of practices that reflect the realities of complex development efforts.

Overview of the Lambda-* Performance Reasoning Frameworks

February 1, 2009 • technical report, by jeffrey hansen, gabriel moreno.

This report provides an overview of the Lambda-* performance reasoning frameworks, their current capabilities, and ongoing research.

Use and Organizational Effects of Measurement and Analysis in High Maturity Organizations: Results from the 2008 SEI State of Measurement and Analysis Practice Surveys

By dennis goldenson, robert w. stoddard, james mccurley.

This report contains results from a survey of high maturity organizations conducted by the Software Engineering Institute (SEI) in 2008. The questions center on the use of process performance modeling …

CMMI for Services, Version 1.2

A model of best practices to improve the processes of service providers.

The Arcade Game Maker Pedagogical Product Line

January 5, 2009 • white paper.

The Arcade Game Maker product line is an example product line created to support learning about and experimenting with software product lines in the classroom.

Common Sense Guide to Prevention and Detection of Insider Threats 3rd Edition – Version 3.1

January 1, 2009 • white paper, by andrew p. moore, randall f. trzeciak, timothy j. shimeall, dawn cappelli.

In this paper, the authors present findings from examining insider crimes in a new way and add new practices that were not present in the second edition.

Developing An Acquisition Strategy

An acquisition strategy is of great importance to those organizations that primarily acquire rather than develop.

High-Fidelity E-Learning: The SEI's Virtual Training Environment (VTE)

January 1, 2009 • technical report, by david w. white, julia h. allen, jim wrubel.

In this 2008 report, the authors compare various approaches and tools used to capture and analyze evidence from computer memory.

Statistical-Based WCET Estimation and Validation

December 31, 2008 • white paper, by gabriel moreno, jeffrey hansen, scott hissam.

This paper presents a measurement-based approach that produces both a WCET (Worst Case Execution Time) estimate and a prediction of the probability that a future execution time will exceed a …

Spotlight On: Programming Techniques Used as an Insider Attack Tool

December 1, 2008 • white paper, by andrew p. moore, randall f. trzeciak, dawn cappelli, thomas c. caron (john heinz iii college, school of information systems management, carnegie mellon university).

In this report, the authors focus on persons who use programming techniques to commit malicious acts against their organizations.

A Survey of Systems Engineering Effectiveness - Initial Results

December 1, 2008 • special report, by the ndia se effectiveness committee, khaled el emam, nichole donitelli, angelica neisa, joseph p. elm, dennis goldenson.

This survey quantifies the relationship between the application of Systems Engineering (SE) best practices to projects and programs, and the performance of those projects and programs.

Results of SEI Independent Research and Development Projects (FY 2008)

December 1, 2008 • technical report, by lui r. sha, lutz wrage, jörgen hansson (university of skovde), sherman eagles, paul jones, insup lee, ragunathan rajkumar, robert w. stoddard, robert nord, kurt c. wallnau, peter h. feiler, dionisio de niz, rick kazman, ipek ozkaya, gabriel moreno, john b. goodenough, charles weinstock, daniel plakosh, mark h. klein.

This report describes the independent research and development (IRAD) projects that were conducted during fiscal year 2008 (October 2007 through September 2008).

CMMI or Agile: Why Not Embrace Both!

November 1, 2008 • technical note, by hillel glazer - entinex inc., jeff dalton (broadsword solutions corporation), david anderson (david j. anderson & associates inc.), michael d. konrad, sandra shrum.

This report describes how CMMI and Agile methods can be used together successfully.

CMMI Roadmaps

By ben linders, jan j. cannegieter, andre heijstek, rini van solingen.

The report guides organizations that are starting a CMMI for development implementation and deciding to use the continuous representation. The report offers guidance for how to decide what process areas …

CMMI High Maturity Measurement and Analysis Workshop Report: March 2008

By robert w. stoddard, david zubrow, dennis goldenson, erin harper.

This report outlines a 2008 workshop, in which leaders discussed high maturity practices and how to sustain momentum for improvement.

Can You Trust Your Data? Establishing the Need for a Measurement and Analysis Infrastructure Diagnostic

By david zubrow, mark kasunic, james mccurley.

This report describes common errors in measurement and analysis and the need for a criterion-based assessment method that will allow organizations to evaluate key characteristics of their measurement programs.

Software Engineering Bibliography

September 29, 2008 • white paper.

In this paper, the authors provide a bibliography of sources related to software engineering.

Application Firewalls and Proxies - Introduction and Concept of Operations

September 27, 2008 • white paper, by howard f. lipson, ken van wyk (no affiliation).

In this paper, the authors describe one of the many potential topic areas involving the integration of business applications into a supporting IT security infrastructure.

Existence Plots: A Low-Resolution Time Series for Port Behavior Analysis

September 15, 2008 • white paper, by jeff janies.

In this paper, Jeff Janies introduces the existence plot as a visualization and discuss its use in gaining insight into a host's behavior.

Acquisition Archetypes: Underbidding the Contract

September 1, 2008 • white paper.

From the Acquisition Support Program, one in a series of short papers on acquisition patterns of failure.

Acquisition Archetypes: Staff Burnout and Turnover

Applying more pressure on staff can temporarily increase productivity, but burnout soon sets in.

T-Check in Technologies for Interoperability: Business Process Management in a Web Services Context

September 1, 2008 • technical note, by lutz wrage, fabian hueppi, grace lewis.

This technical note presents an investigation of the Business Process Execution Language, a popular BPM technology used to describe, analyze, execute, and monitor business processes.

Service Level Agreements in Service-Oriented Architecture Environments

By philip bianco, paulo merson, grace lewis.

This 2008 report surveys the state of practice in service level agreement specification and offers guidelines on how to assure that services are provided with high availability, security, performance, and …

Requirements and Their Impact Downstream: Improving Causal Analysis Processes Through Measurement and Analysis of Textual Information

September 1, 2008 • technical report, by dennis goldenson, lawrence t. osiecki, ira monarch.

Requirements documents, test procedures, and problem and change reports from a U. S. Army Software Engineering Center (SEC) were analyzed to identify, clarify, and begin categorizing recurring patterns of issues …

Computer Forensics: Results of Live Response Inquiry vs. Memory Image Analysis

August 1, 2008 • technical note, by joseph a. akinyele, richard nolan, cal waits, larry rogers.

The authors compare various approaches and tools used to capture and analyze evidence from computer memory.

Introducing Function Extraction into Software Testing (July 2008)

July 14, 2008 • white paper, by richard c. linger (oak ridge national laboratory), alan r. hevner (university of south florida), mark pleszkoch.

This paper describes the emerging technology of function extraction (FX).

Preview of the Mission Assurance Analysis Protocol (MAAP): Assessing Risk and Opportunity in Complex Environments

July 1, 2008 • technical note, by lisa marino, audrey j. dorofee, christopher j. alberts.

In this 2008 document, the authors preview a core set of activities and outputs that define a MAAP assessment.

A Data Specification for Software Project Performance Measures: Results of a Collaboration on Performance Measurement

July 1, 2008 • technical report.

This 2008 document contains defined software project performance measures and influence factors that can be used by software development projects so that valid comparisons can be made between completed projects.

Results of SEI Independent Research and Development Projects (FY 2007)

By stefan schuster, marin litoiu (ibm canada ltd.), lui r. sha, len bass, jörgen hansson (university of skovde), dennis b. smith, kostas kontogiannis, john j. hudak, mark h. klein, peter h. feiler, kurt c. wallnau, dionisio de niz, donald firesmith, grace lewis.

This report describes the independent research and development (IRAD) projects that were conducted during fiscal year 2007 (October 2006 through September 2007).

Proceedings of the International Workshop on the Foundations of Service-Oriented Architecture (FSOA 2007)

June 1, 2008 • special report, by dennis b. smith, grace lewis.

This report presents the results of the Foundations of Software-Oriented Architecture (FSOA) workshop held at the Third International Conference on Interoperability for Enterprise Software and Applications (I-ESA 2007).

SQUARE-Lite: Case Study on VADSoft Project

By nancy r. mead, ashwin gayash, venkatesh viswanathan, deepa padmanabhan.

In this 2008 report, the authors describe SQUARE and SQUARE-Lite, and using SQUARE-Lite to develop security requirements for a financial application.

SoS Navigator 2.0: A Context-Based Approach to System-of-Systems Challenges

June 1, 2008 • technical note, by william anderson, suzanne miller, lisa brownsword, john morley, philip j. boxer, dennis b. smith, david j. carney, patrick kirwan.

This report introduces the fundamental concepts, processes, and techniques of the SoS Navigator approach. It also presents case studies that show the use of SoS Navigator in healthcare, military, and …

SMART: Analyzing the Reuse Potential of Legacy Components in a Service-Oriented Architecture Environment

By dennis b. smith, grace lewis, edwin j. morris, soumya simanta.

Is legacy system migration feasible for your organization as a means of SOA adoption? The Service Migration and Reuse Technique (SMART) assists an organization in determining what to migrate, the …

Evaluation of CERT Secure Coding Rules through Integration with Source Code Analysis Tools

June 1, 2008 • technical report, by chad dougherty, robert c. seacord, david svoboda, david keaton, chris taschner, stephen dewhurst, yurie ito, dan saks, kazuya togashi (jpcert/cc).

In this report, the authors describe a study to evaluate CERT Secure Coding Standards and source code analysis tools in commercial software projects.

Strategic Technology Selection and Classification in Multimodel Environments

May 8, 2008 • white paper, by john morley, lisa marino, patrick kirwan, jeannine siviy.

This white paper is the second in a five-part series dedicated to examining problems organizations encounter when operating in multimodel environments and the current process improvement approaches such organizations need …

Leadership and Management in Software Architecture

May 1, 2008 • white paper, by brian berenbach, len bass.

The workshop on Leadership and Management in Software Architecture that took place at ICSE 2008 was focused on understanding these non-technical duties and the type of support an architect should …

Management and Education of the Risk of Insider Threat (MERIT): System Dynamics Modeling of Computer System

By andrew p. moore, timothy j. shimeall, dawn cappelli, bradford j. willke, akash g. desai (information networking institute, carnegie mellon university), elise a. weaver (worcester polytechnic institute).

In this paper, the authors describe the MERIT insider threat model and simulation results.

Implementation Challenges in a Multimodel Environment

By john morley, jeannine siviy, patrick kirwan, lisa marino.

This white paper is the fifth in a five-part series dedicated to examining problems organizations encounter when operating in multimodel environments.

Using Model-Based Engineering and Architectural Models to Build Secure Systems

By peter h. feiler, john morley, jörgen hansson (university of skovde).

In this paper, the authors present analytical techniques to model and validate security protocols for enforcing confidentiality and integrity.

Building Secure Systems Using Model-Based Engineering and Architectural Models

By jörgen hansson (university of skovde), john morley, peter h. feiler.

A system designer faces several challenges when specifying security for distributed computing environments or migrating systems to a new execution platform.

Proceedings of the First Workshop on Service-Oriented Architectures and Product Lines

May 1, 2008 • special report.

This 2008 report includes an overview of the First Workshop on Service-Oriented Architectures and Product Lines, four invited presentations, details of the workshops outcomes, and the workshop position papers.

Incorporating Security Quality Requirements Engineering (SQUARE) into Standard Life-Cycle Models

May 1, 2008 • technical note, by nancy r. mead, anusha raveendran, venkatesh viswanathan, deepa padmanabhan.

In this 2008 report, the authors describe how SQUARE can be incorporated into standard lifecycle models for security-critical projects.

Survivability Assurance for System of Systems

May 1, 2008 • technical report, by carol woody, robert j. ellison, charles weinstock, john b. goodenough.

In this report, the authors describe the Survivability Analysis Framework, a structured view of people, process, and technology.

The "Big Picture" of Insider IT Sabotage Across U.S. Critical Infrastructures

By randall f. trzeciak, andrew p. moore, dawn cappelli.

In this report, the authors describe seven observations about insider IT sabotage based on their empirical data and study findings.

CMMI for Acquisition (CMMI-ACQ) Primer, Version 1.2

By karen richter.

This primer can be used by projects that acquire products or services in government and non-government organizations to improve acquisition processes.

The State of Information Security Law A Focus on the Key Legal Trends

March 30, 2008 • white paper, by tom smedinghoff (wildman harrold).

This paper will examine new developments as they relate to three trends that are posing significant new challenges for most businesses..

The Value of Harmonizing Multiple Improvement Technologies: A Process Improvement Professional's View

March 1, 2008 • white paper, by jeannine siviy, patrick kirwan, lisa marino, john morley.

This white paper is the first in a five-part series dedicated to examining problems organizations encounter when operating in multimodel environments and the current process improvement approaches such organizations need …

Process Architecture in a Multimodel Environment

By lisa marino, patrick kirwan, jeannine siviy, john morley.

This white paper is the fourth in a five-part series that examines problems organizations encounter when operating in multimodel environments.

Improvement Technology Classification and Composition in Multimodel Environments

This paper is the third in a five-part series dedicated to examining problems organizations encounter when operating in multimodel environments and the current process improvement approaches such organizations need to …

Acquisition Archetypes: Feeding the Sacred Cow

Some programs take on a life of their own—privileged, and woven into an organization's existence. But when "sacred cow" projects begin to go wrong, that privilege and protection makes fixing …

Acquisition Archetypes: PMO versus Contractor Hostility

Everyone intends the best in project-driven marriages of PMOs and contractors, but good intentions can't overcome the hostility generated by loss of trust and squabbles in poorly developed relationships.

Acquisition Archetypes: Firefighting

All hands on deck helps put out the immediate blazes threatening projects, but falling into a routine of constant firefighting is not the way to guide a project across the …

Maximizing your Process Improvement ROI through Harmonization

By john morley, patrick kirwan, jeannine siviy, lisa marino.

This white paper is an executive overview of the business value in harmonizing process improvement efforts when multiple improvement technologies, models and standards are in use.

Lessons Learned Applying the Mission Diagnostic

March 1, 2008 • technical note, by christopher j. alberts, lisa marino, audrey j. dorofee.

This technical note describes the adaptation of the Mission Diagnostic (MD) necessary for a customer and the lessons we learned from its use.

Mission Diagnostic Protocol, Version 1.0: A Risk-Based Approach for Assessing the Potential for Success

March 1, 2008 • technical report.

This 2008 document describes the core set of activities and outputs that defines mission diagnostic protocol (MDP).

Models for Evaluating and Improving Architecture Competence

By len bass, mark h. klein, rick kazman, paul c. clements.

This report outlines the concepts of software architecture competence and describes four models for explaining, measuring, and improving the architecture competence of an individual or a software-producing organization.

Incident Management Mission Diagnostic Method, Version 1.0

By mark zajicek, robin ruefle, audrey j. dorofee, georgia killcrece.

This report is superseded by the Mission Risk Diagnostic for Incident Management Capabilities, CMU/SEI-2014-TN-004.

ASSIP Study of Real-Time Safety-Critical Embedded Software-Intensive System Engineering Practices

February 1, 2008 • special report, by peter h. feiler, dionisio de niz.

This report presents findings of a study of RTSCE software-intensive systems issues and develop recommendations for effectively dealing with those issues.

On the Anonymization and Deanonymization of NetFlow Traffic

January 8, 2008 • white paper, by michalis foukarakis (institute of computer science), demetres antoniades (institute of computer science), evangelos p. markatos (institute of computer science).

In this paper, the authors describe anontool, which allows per-field anonymization up to the NetFlow layer and offers a wide range of primitives to choose from.

Assessing Disclosure Risk in Anonymized Datasets

January 7, 2008 • white paper, by alexi kounine (epfl), michele bezzi (atl).

In this paper, the authors propose a framework for estimating disclosure risk using conditional entropy between the original and the anonymized datasets.

Vulnerability Detection in ActiveX Controls through Automated Fuzz Testing

January 1, 2008 • white paper, by will dormann, daniel plakosh.

In this 2008 paper, the authors explore results of a test of a large number of Active X controls, which provides insight into the current state of ActiveX security.

Insider Threat Study: Illicit Cyber Activity in the Information Technology and Telecommunications Sector Executive Summary

In this paper, the authors present the findings of research examining reported insider incidents in information technology and telecommunications sectors.

Insider Threat Study: Illicit Cyber Activity in the Government Sector Executive Summary

In this paper, the authors present the findings of a research effort to examine reported insider incidents within the government sector.

Insider Threat Study: Illicit Cyber Activity in the Information Technology and Telecommunications Sector

By eileen kowalski (united states secret service), dawn cappelli, andrew p. moore.

In this paper, the authors present the findings of research examining reported insider incidents in the information technology and telecommunications sectors.

Insider Threat Study: Illicit Cyber Activity in the Government Sector

By andrew p. moore, dawn cappelli, bradford j. willke, eileen kowalski (united states secret service).

In this paper, the authors present the findings of a research effort to examine reported insider incidents in the government sector.

T-Check in Technologies for Interoperability: Web Services and Security—Single Sign-On

January 1, 2008 • technical note, by grace lewis, soumya simanta, lutz wrage, saul jaspan.

This technical note presents the results of applying the T-Check approach in an initial investigation of two Web services standards, WS-Security and SAML, to create an SSO solution that works …

Moving Up the CMMI Capability and Maturity Levels Using Simulation

January 1, 2008 • technical report, by david m. raffo (portland state university), wayne wakeland (portland state university).

This report shows examples of how PSIM has been implemented within industry and government organizations to improve process consistency and results.

Using the Vickrey-Clarke-Groves Auction Mechanism for Enhanced Bandwidth Allocation in Tactical Data Networks

By daniel plakosh, mark h. klein, kurt c. wallnau.

This report describes an investigation of the potential for using computational mechanisms to improve the quality of a combat group's common operating picture, in a setting where network bandwidth is …

Handling Interdependent Values in an Auction Mechanism for Bandwidth Allocation in Tactical Data Networks

December 31, 2007 • white paper, by kurt c. wallnau, gabriel moreno, daniel plakosh, mark h. klein.

This paper introduces a mechanism that achieves efficient bandwidth allocation and provides incentive compatibility by conditioning payments on the realized value for data shared between agents.

The State of Information Security Law: A Focus on the Key Legal Trends

December 18, 2007 • white paper.

This paper provides information about the expanding duty to provide security and the emergency of a legal obligation for compliance.

Diagrams and Languages for Model-Based Software Engineering of Embedded Systems: UML and AADL

December 1, 2007 • white paper, by dionisio de niz.

The tools compared in this discussion, the Unified Modeling Language (UML) and the Architecture Analysis and Design Language (AADL), facilitate the modeling of software architecture and provide elements to understand …

Basic Principles and Concepts for Achieving Quality Parent

December 1, 2007 • technical note.

This report extends the quality concepts first articulated in "A Software Quality Framework (SQF)" developed in the early 1980s for the DoD by Baker and colleagues.

Flow Latency Analysis with the Architecture Analysis and Design Language (AADL)

By jörgen hansson (university of skovde), peter h. feiler.

This 2007 report describes the ability of AADL to determine a lower bound for the worst-case end-to-end latency in a system.

Software-Intensive Systems Producibility: A Vision and Roadmap (v 0.1)

This 2007 document is a draft in progress of a technology vision and roadmap to improve the ability of the DoD and industry to deliver needed SiS capability in a …

Programmatic Interoperability

By b. craig meyers, james smith.

This report introduces the concept of programmatic interoperability, which is the application of principles of interoperability to the acquisition management of systems. The report also discusses the orchestration of decisions …

Common Misconceptions About Service-Oriented Architecture

November 1, 2007 • white paper, by grace lewis, edwin j. morris, soumya simanta, dennis b. smith, lutz wrage.

This 2007 article from Crosstalk magazine suggests ways to more effectively address critical SOA issues that potential users, developers, and acquisition officers may have.

Traditional Chinese language translation of CMMI for Acquisition, V1.2

The Traditional Chinese language translation of CMMI for Acquisition (CMMI-ACQ), V1.2.

Classifying Architectural Elements as a Foundation for Mechanism Matching

By rick kazman, paul c. clements, len bass.

This paper presents a set of well-known but informally described software architectural elements used in system composition and taxonomizes them under a basic set of characteristic features.

A-Specification for the CMMI Product Suite, Version 1.7

The A-Specification for the CMMI Product Suite (A-SPEC) defines the scope and requirements the CMMI Product Suite must meet to be considered acceptable.

A Survey of Systems Engineering Effectiveness: Initial Results

November 1, 2007 • special report, by khaled el emam, nichole donitelli, angelica neisa, the ndia se effectiveness committee, dennis goldenson, joseph p. elm.

This survey quantifies the relationship between the application of systems engineering best practices to projects and the performance of those projects.

CMMI for Acquisition, Version 1.2

November 1, 2007 • technical report.

The CMMI-ACQ model provides guidance for the application of CMMI best practices by the acquirer.

Fishing for Phishes: Applying Capture-Recapture Methods to Estimate Phishing Populations

October 4, 2007 • white paper, by rhiannon weaver, m. p. collins (redjack).

In this paper, the authors describe addressing phishing problems by estimating population in terms of netblocks and by clustering phishing attempts into scams.

Acquisition Archetypes: The Bow Wave Effect

October 1, 2007 • white paper, cots and reusable software management planning: a template for life-cycle management, october 1, 2007 • technical report, by dennis b. smith, mary c. ward, edwin j. morris, william anderson.

This 2007 report presents a COTS and Reusable Software Management Plan that can serve as a guide for how to manage multiple COTS and other reusable software components in complex …

SCAMPI Lead Appraiser Body of Knowledge (SLA BOK)

By judah mogilensky, steve masters, sandra behrens, charlie ryan.

The SLA BOK identifies the competencies needed to carry out the method requirements and guidelines detailed in the MDD (Method Definition Document).

Hit-List Worm Detection and Bot Identification in Large Networks Using Protocol Graphs

September 5, 2007 • white paper, by m. p. collins (redjack), michael k. reiter.

In this paper, the authors present a novel method for detecting hit-list worms using protocol graphs.

Four Pillars of Service-Oriented Architecture

September 1, 2007 • white paper, by grace lewis, dennis b. smith.

This 2007 SEI whitepaper by Grace Lewis and Dennis B. Smith outlines four pillars to Service-Oriented Architecture (SOA) success.

Using ArchE in the Classroom: One Experience

September 1, 2007 • technical note, by len bass, mark h. klein, john mcgregor, philip bianco, felix bachmann.

The ArchE (Architecture Expert) tool serves as a software architecture design assistant. This report describes the use of a pre-alpha release of ArchE in a graduate-level software architecture class at …

Using Aspect-Oriented Programming to Enforce Architecture

This report illustrates how to use AOP (aspect-oriented programming) to ensure conformance to architectural design, proper use of design patterns and programming best practices, conformance to coding policies and naming …

Process Improvement Should Link to Security: SEPG 2007 Security Track Recap

In this document, Carol Woody summarizes the content shared at the 2007 SEPG conference and steps underway toward ties between security and process improvement.

Ranged Integers for the C Programming Language

By robert c. seacord, jeff gennari, fred long, shaun hedrick, justin pincar.

In this 2007 report, the authors describe an extension to the C programming language to introduce the notion of ranged integers.

Certified Binaries for Software Components

September 1, 2007 • technical report, by sagar chaki, kurt c. wallnau, james ivers, peter lee, noam zeilberger.

In this report, the authors present an approach to certify binary code against expressive policies to achieve the benefits of PCC and CMC.

Modifiability Tactics

By len bass, robert nord, felix bachmann.

This report describes how architectural tactics are based on the parameters of quality attribute models.

Evaluating a Service-Oriented Architecture

By philip bianco, paulo merson, rick kotermanski.

This report contains technical information about SOA design considerations and tradeoffs that can help the architecture evaluator to identify and mitigate risks in a timely and effective manner.

Business Rules for CMMI Focus Topics

August 22, 2007 • white paper.

This paper provides guidelines for organizations seeking to publish material related to any CMMI focus topics.

Governing for Enterprise Security (GES) Implementation Guide

August 1, 2007 • technical note, by julia h. allen, jody r. westby.

In this 2007 report, the authors provide prescriptive guidance for creating and sustaining an enterprise security governance program.

How To Compare the Security Quality Requirements Engineering (SQUARE) Method with Other Methods

In this 2007 report, Nancy Mead describes SQUARE, and outlines other methods used for identifying security requirements.

System Strategies References

July 17, 2007 • white paper.

In this paper, the authors provide references related to system strategies.

Governing for Enterprise Security Implementation Guide: Sample Artifact Roles and Responsibilities for an Enterprise Security Program

July 11, 2007 • White Paper

The use of malware analysis in support of law enforcement, by ross kinder.

In this paper, Ross Kinder discusses how malware analysis supports the efforts of those pursuing adversaries employing malicious code in their tradecraft.

Introduction to the Architecture of the CMMI Framework

July 1, 2007 • technical note, by the cmmi architecture team.

This 2007 document is an introduction to the CMMI Framework architecture, which guides how CMMI products are developed and integrated.

Dependability Modeling with the Architecture Analysis & Design Language (AADL)

By ana rugina, peter h. feiler.

This 2007 report explains the capabilities of the Error Model Annex and provides guidance on the use of the AADL and the error model in modeling dependability aspects of embedded …

Modeling of System Families

This report discusses how AADL can be used to model system families and configurations of system and component variants.

Results of SEI Independent Research and Development Projects (FY 2006)

July 1, 2007 • technical report, by christopher j. alberts, carol sledge, aaron greenhouse, thomas a. longstaff, james ivers, sagar chaki, mike phillips, lisa brownsword, suzanne miller, kurt c. wallnau, peter h. feiler, william anderson, pratyusa k. manadhata, j. wing, matt bass, peter lee, noam zeilberger, gwendolyn h. walton, philip j. boxer, james smith, len bass, eileen c. forrester, jörgen hansson (university of skovde), richard c. linger (oak ridge national laboratory), david fisher, b. craig meyers, james d. herbsleb.

This report describes the IRAD projects that were conducted during fiscal year 2006 (October 2005 through September 2006).

Developing AADL Models for Control Systems: A Practitioner's Guide

By peter h. feiler, john j. hudak.

This 2007 document helps practitioners use AADL and describes an approach for and the mechanics of constructing an architectural model that can be analyzed based on the AADL.

Progress Toward an Organic Software Architecture Capability in the U.S. Army

June 1, 2007 • technical report, by stephen blanchette, jr., john k. bergey.

This 2007 report describes the Software Architecture Initiative of the Army Strategic Software Improvement Program.

Case Study: Accelerating Process Improvement by Integrating the TSP and CMMI

By daniel s. wall, marsha pomeroy-huff, jim mchale.

This report describes how two NAVAIR organizations integrated the use of the TSP methodology and the CMM framework to progress from maturity level 1 to maturity level 4 in 30 …

SAAM: A Method for Analyzing the Properties of Software Architectures

May 1, 2007 • white paper, by len bass, mike webb (texas instruments), gregory abowd, rick kazman.

This paper describes three perspectives by which we can understand the description of a software architecture and proposes a five-step method for analyzing software architectures called SAAM (Software Architecture Analysis …

Quality-Attribute-Based Economic Valuation of Architectural Patterns

May 1, 2007 • technical report, by ipek ozkaya, rick kazman, mark h. klein.

This report shows how an analysis of the options embodied within architectural patterns allows a software and system architect or manager to make reasoned choices about the future value of …

Introducing the CERT® Resiliency Engineering Framework: Improving the Security and Sustainability Processes

By lisa r. young, james f. stevens, richard a. caralli, david w. white, william r. wilson, charles m. wallen.

In this 2007 report, the authors explore the transformation of security and business continuity into processes to support and sustain operational resiliency.

Introducing OCTAVE Allegro: Improving the Information Security Risk Assessment Process

By william r. wilson, james f. stevens, richard a. caralli, lisa r. young.

In this 2007 report, the authors highlight the design considerations and requirements for OCTAVE Allegro based on field experience.

Function Extraction: Automated Behavior Computation for Aerospace Software Verification and Certification

April 29, 2007 • white paper, by tim daly, richard c. linger (oak ridge national laboratory), stacy j. prowell.

This paper describes verification and certification challenges for avionics software.

Copper Manual, Tutorial, and Specification Grammar

April 1, 2007 • white paper, by sagar chaki.

Copper is a software model checker for concurrent message-passing C programs.

T-Check for Technologies for Interoperability: Open Grid Services Architecture (OGSA)—Part 1 Parent SEI Program

April 1, 2007 • technical note, by soumya simanta, grace lewis, lutz wrage.

This report investigates Open Grid Services Architecture (OGSA), one of the many technologies for accomplishing interoperability, using the T-Check technique.

Governing for Enterprise Security (GES) Implementation Guide Article 3: Enterprise Security Governance Activities

March 5, 2007 • White Paper

Governing for Enterprise Security (GES) Implementation Guide Article 2: Defining an Effective Enterprise Security Program (ESP)

March 1, 2007 • White Paper

Global information grid survivability: four studies, march 1, 2007 • special report, by richard c. ciampa, dawn day, jennifer r. franks, christopher t. tsuboi.

Four studies from 2006 that explore an issue relevant to the survivability of networks which are systems of systems.

Modeling and Analysis of Information Technology Change and Access Controls in the Business Context

March 1, 2007 • technical note, by andrew p. moore, rohit s. antao.

In this report, the authors describe progress in developing a system dynamics model of typical use of change and access controls to support IT operations.

Management and Education of the Risk of Insider Threat (MERIT): Mitigating the Risk of Sabotage to Employers Information, Systems, or Networks

By akash g. desai (information networking institute, carnegie mellon university), elise a. weaver (worcester polytechnic institute), bradford j. willke, dawn cappelli, andrew p. moore, timothy j. shimeall.

In this 2006 report, the authors describe MERIT insider threat model and simulation results.

+SAFE, V1.2: A Safety Extension to CMMI-DEV, V1.2

By defence materiel organisation, australian department of defence.

This technical report describes how to use +SAFE to appraise an organization's capability in developing, sustaining, maintaining, and managing safety-critical products.

Executive Overview of SEI MOSAIC: Managing for Success Using a Risk-Based Approach

This 2007 report provides an overview of the concepts and foundations of MOSAIC, a suite of advanced, risk-based analysis methods for assessing complex, distributed programs, processes, and information-technology systems.

Understanding and Leveraging a Supplier's CMMI Efforts: A Guidebook for Acquirers

March 1, 2007 • technical report, by the cmmi guidebook for acquirers team.

This guidebook is designed to help acquisition organizations benefit from their suppliers' use of CMMI for Development, a collection of best practices that addresses product development and maintenance activities throughout …

Governing for Enterprise Security (GES) Implementation Guide Article 1: Characteristics of Effective Security Governance1

February 5, 2007 • White Paper

A practical example of applying attribute-driven design (add), version 2.0, february 1, 2007 • technical report, by william wood.

This 2007 report describes an example application of the ADD method, an approach to defining a software architecture in which the design process is based on the quality attribute requirements …

Defining Computer Security Incident Response Teams

January 24, 2007 • white paper.

In this paper, Robin Ruefle describes the purpose and goals of a computer security incident response team (CSIRT).

Penetration Testing Tools

January 18, 2007 • white paper, cert® resiliency engineering framework, january 1, 2007 • white paper.

In this paper, the authors answer commonly asked questions about the CERT Resiliency Engineering Framework project.

Instructional Case of Insider Threat in the SDLC: The Case of InsureACure, Inc.

In this paper, the authors provide an instructional case of insider threat in the systems development lifecycle.

A Proposed Taxonomy for Software Development Risks for High-Performance Computing (HPC) Scientific/Engineering Applications

January 1, 2007 • technical note, by jeffrey carver, dale b. henderson, richard kendall, david fisher, douglass post (dod high performance computing modernization program).

In this report, the authors classify the sources of software development risk for scientific/engineering applications.

Case Study of the NENE Code Project

By douglass post (dod high performance computing modernization program), richard kendall, andrew mark (dod high performance computing modernization program).

This report outlines the case studies of high-performance code development projects. This is the fifth case study in this series.

Conditions for Achieving Network-Centric Operations in Systems of Systems

By david fisher, b. craig meyers, patrick r. place.

This 2007 report lists conditions that must prevail to achieve effective acquisition, development, and use of systems of systems.

Interpreting Capability Maturity Model Integration (CMMI) for Business Development Organizations in the Government and Industrial Business Sectors

By donald r. beynon.

This 2007 interpretation of CMMI best practices is for business development activities applicable to contractors doing business within the government (Department of Defense) and industrial business sectors.

The State of Software Measurement Practice: Results of 2006 Survey

December 1, 2006 • technical report.

This paper reports the results of a February 2006 study to gauge the state of the practice in software measurement.

Technology Foundations for Computational Evaluation of Software Security Attributes

In this 2006 report, the authors describe foundations for computational security attributes technology.

Comparing Insider IT Sabotage and Espionage: A Model-Based Analysis

By dawn cappelli, eric d. shaw, stephen r. band (counterintelligence field activity - behavioral science directorate), lynn f. fischer, andrew p. moore, randall f. trzeciak.

In this report, the authors examine the psychological, technical, organizational, and contextual factors that contribute to espionage and insider sabotage.

Action List for Developing a Computer Security Incident Response Team (CSIRT)

November 2, 2006 • white paper.

In this paper, the authors summarize actions to take and topics to address when planning and implementing a Computer Security Incident Response Team (CSIRT).

Army ASSIP System-of-Systems Test Metrics Task

November 1, 2006 • special report.

This report contains presents the results of an effort to improve the acquisition of software-intensive systems by focusing on acquisition programs, people, and production/sustainment and by institutionalizing continuous improvement.

Schedule Considerations for Interoperable Acquisition

November 1, 2006 • technical note, by carol sledge, b. craig meyers.

This 2006 report examines the issue of schedule considerations for interoperable acquisition.

Attribute-Driven Design (ADD), Version 2.0

November 1, 2006 • technical report, by william wood, robert wojcik, paul c. clements, paulo merson, felix bachmann, robert nord, len bass.

This report revises the steps of the Attribute-Driven Design (ADD) method and offers practical guidelines for carrying out each step.

A Traffic Analysis of a Small Private Network Compromised by an Online Gaming Host (White Paper)

October 10, 2006 • white paper, by ron mcleod (corporate development telecom applications research alliance).

In this paper, Ron McLeod describes a network traffic capture and analysis used to investigate network performance issues of a small private network.

System Requirements for Flow Processing

By raj srinivasan (bivio networks).

In this paper, Raj Srinivasan proposes an architecture that meets security requirements and is flexible enough to support future application needs.

Scalable Flow Analysis (White Paper)

By abhishek kumar (university of maryland), sapan bhatia (princeton).

In this paper, the authors present a new approach for summarization and analysis of flow records.

RAVE: The Retrospective Analysis and Visualization Engine

By john prevost, phil groce.

In this paper, the authors present RAVE as an analysis service provider.

The Effect of Packet Sampling on Anomaly Detection

By daniela brauckhoff (swiss federal institute of technology (eth)), bernhard tellenbach (swiss federal institute of technology (eth)), martin may (swiss federal institute of technology (eth)), anukool lakhina (boston university), arno wagner (communication systems laboratory swiss federal institute of technology zurich (eth zurich)).

In this paper, the authors empirically evaluate the impact of sampling on anomaly detection.

Attribution and Aggregation of Network Flows for Security Analysis (White Paper)

By vincent berk (dartmouth college), annarita giani (uc berkeley), iangregoriode souza (dartmouth college), george cybenko (dartmouth college).

In this paper, the authors describe a network flow analyzer capable of attribution and aggregation of different flows to identify suspicious behaviors.

IPFIX/PSAMP: What Future Standards Can Offer to Network Security (White Paper)

By thomas hirsch (fraunhofer fokus), tanja zseby (fraunhofer fokus), mark lutz (fraunhofer fokus), elisa boschi (hitachi).

In this paper, the authors show how IPFIX and PSAMP can be used to support network security.

Identifying Anomalous Network Traffic Through the Use of Client Port Distribution

By josh goldfarb (us-cert).

In this paper, Josh Goldfarb introduces an approach to IP flow analysis that examines server ports and client ports that exchange flows with them.

Anomaly Detection Through Blind Flow Analysis Inside a Local Network (White Paper)

October 2, 2006 • white paper, by ron mcleod (corporate development telecom applications research alliance), vagishwari nagaonkar (wipro technologies).

In this paper, the authors describe how hosts may be clustered into user workstations, servers, printers, and hosts compromised by worms.

An Examination of a Structural Modeling Risk Probe Technique

October 1, 2006 • special report, by william anderson, lisa brownsword, philip j. boxer.

This report examines a structural dynamic analysis modeling technique called Projective ANalysis (PAN) that was used on an interoperability technical probe of a NATO modernization program.

System-of-Systems Governance: New Patterns of Thought

October 1, 2006 • technical note, by dennis b. smith, patrick r. place, edwin j. morris.

This 2006 technical note examines the ways in which six key characteristics of good IT governance are affected by the autonomy of individual systems in a system of systems.

Topics in Interoperability: Structural Programmatics in a System of Systems

By james smith.

This technical note presents a case study on how choices of structural programmatics (e.g., hierarchical or peer-to-peer organization, centralized or decentralized execution) affect programmatic interoperability in complex systems of systems.

Next-Generation Software Engineering: Function Extraction for Computation of Software Behavior

September 9, 2006 • white paper, by gwendolyn h. walton, alan r. hevner (university of south florida), richard c. linger (oak ridge national laboratory).

This white paper discusses function extraction FX technology.

Finding Peer-To-Peer File-Sharing Using Coarse Network Behaviors?

September 4, 2006 • white paper, by michael collins, michael k. reiter.

In this paper, the authors propose a set of tests for identifying masqueraded peer-to-peer file-sharing based on traffic summaries (flows).

Quantitative Methods for Software Selection and Evaluation

September 1, 2006 • technical note, by michael s. bandor.

This 2006 report describes methods for selecting candidate commercial off-the-shelf packages for further evaluation, possible methods for evaluation, and other factors besides requirements to be considered.

Evolutionary Systems Design: Recognizing Changes in Security and Survivability Risks

In this 2006 report, Howard Lipson uses an example to illustrate the critical importance of evolutionary design changes in secure and survivable systems.

Assume-Guarantee Reasoning for Deadlock

By nishant sinha, sagar chaki.

This report shows how L^F can be used for compositional regular failure language containment and deadlock detection, using non-circular and circular assume-guarantee rules.

Certifying the Absence of Buffer Overflows

By sagar chaki, scott hissam.

In this report, the authors present a technique for certifying the safety of buffer manipulations in C programs.

Interoperable Acquisition for Systems of Systems: The Challenges

By james smith, mike phillips.

This 2006 report explores how systems-of-systems realities necessitate changes in the processes used to acquire, develop, field, and sustain operational capability.

Risk Themes Discovered Through Architecture Evaluations

September 1, 2006 • technical report, by len bass, robert nord, david zubrow, william wood.

This 2006 report analyzes the output of 18 evaluations conducted using the Architecture Tradeoff Analysis (ATAM). The goal of the analysis was to find patterns in the risk themes identified …

French language translation of CMMI for Development, V1.2

August 1, 2006 • white paper.

The French language translation of CMMI for Development (CMMI-DEV), V1.2.

Traditional Chinese language translation of CMMI for Development, V1.2

The Traditional Chinese language translation of CMMI for Development (CMMI-DEV), V1.2.

Security in the Software Lifecycle

Security in the Software Lifecycle: Making Software Development Processes--and Software Produced by Them--More Secure (Draft).

Portuguese language translation of CMMI for Development, V1.2

This is the Portuguese language translation of CMMI for Development, V1.2.

Workshop on Model-Driven Architecture and Program Generation

August 1, 2006 • technical note, by grace lewis, kurt c. wallnau, b. craig meyers.

This report summarizes the results of a June 2006 workshop, held to explore business and technical aspects of program generation in the context of the Object Management Group's model-driven architecture …

Risk Management Considerations for Interoperable Acquisition

By b. craig meyers.

In this report, Craig Meyers addresses interoperable risk management, the interoperability of organizations that engage in risk management.

Techniques for Developing an Acquisition Strategy by Profiling Software Risks

August 1, 2006 • technical report, by mary c. ward, joseph p. elm, susan kushner.

This report introduces a taxonomy of strategy drivers and strategy elements and provides a method for performing a comparative analysis of the strategy drivers and the resulting strategic choices for …

Performance Results of CMMI-Based Process Improvement

By keith kost, diane gibson, dennis goldenson.

This technical report summarizes much of the publicly available empirical evidence about the performance results that can occur as a consequence of CMMI-based process improvement.

CMMI for Development, Version 1.2

This report is an upgrade of CMMI-SE/SW/IPPD/SS, Version 1.1 and represents the model portion of the CMMI Product Suite.

Proceedings of the Second Software Architecture Technology User Network (SATURN) Workshop

By robert nord.

This report describes the second SATURN workshop format, discussion, and results, as well as plans for future SATURN workshops.

Appraisal Requirements for CMMI, Version 1.2 (ARC, V1.2)

The report defines the Appraisal Requirements for CMMI (ARC) V1.2 requirements that are considered to be essential to appraisal methods intended for use with Capability Maturity Model Integration (CMMI) models.

A Comparison of Requirements Specification Methods from a Software Architecture Perspective

By paulo merson, paul c. clements, ipek ozkaya, len bass, raghvinder sangwan, john k. bergey.

In this report, five methods for the elicitation and expression of requirements are evaluated with respect to their ability to capture architecturally significant requirements.

Systems of Systems: Scaling Up the Development Process

This report reviews the fundamental process and project-management problems of large-scale SoS-like programs and outlines steps to address these problems.

A Model for Opportunistic Network Exploits: The Case of P2P Worms

July 13, 2006 • white paper, by michael collins, carrie gates.

In this paper, the authors present VisFlowConnect-IP, a network flow visualization tool that detects and investigates anomalous network traffic.

Adapting CMMI for Acquisition Organizations: A Preliminary Report

June 1, 2006 • special report, by gowri s. ramani (hewlett packard), kathryn m. dodson (eds), hubert f. hofmann (general motors), deborah k. yedlin (general motors).

This 2006 document presents the initial draft CMMI-ACQ, which adapts CMMI for acquisition organizations.

Information Assurance: Building Educational Capacity

This 2006 report describes SEI and CERT Program efforts to increase the capacity of institutions of higher education to offer IA and IS courses.

Model Problems in Technologies for Interoperability: Web Services

June 1, 2006 • technical note, by grace lewis, lutz wrage.

This 2006 report presents the results of applying the model problem approach in an initial investigation of the potential of Web services to enable interoperability.

Specifying Initial Design Review (IDR) and Final Design Review (FDR) Criteria

By mary ann lapham.

This 2006 report presents definitions of IDR and FDR, their context in the acquisition life cycle, a comparison of engineering emphasis during IDR and FDR, IDR and FDR pre- and …

Joint Capabilities and System-of-Systems Solutions: A Case for Crossing Solution Domains

By william anderson, robert m. flowe, mary m. brown.

This 2006 report presents a case for the investigation and adaptation of structural and dynamic modeling techniques to the engineering of systems of systems.

Security Quality Requirements Engineering (SQUARE): Case Study Phase III

May 1, 2006 • special report, by lydia chung, frank hung, eric hough, don ojoko-adams, nancy r. mead.

In this report, the authors present their results of using SQUARE when working with three clients over the course of a semester.

Sustaining Software-Intensive Systems

May 1, 2006 • technical note, by carol woody, mary ann lapham.

This 2006 report discusses questions about sustaining new and legacy systems; the report presents definitions, related issues, future considerations, and recommendations for sustaining software-intensive systems.

Applying OCTAVE: Practitioners Report

By lisa r. young, johnathan coleman (no affiliation), michael fancher (no affiliation), carol myers (no affiliation), carol woody.

In this report, the authors describe how OCTAVE has been used and tailored to fit a wide range of organizational risk assessment needs.

PROxy Based Estimation (PROBE) for Structured Query Language (SQL)

By rob schoedel.

This 2006 report outlines a method for applying the PROxy Based Estimation (PROBE) technique to Structured Query Language (SQL).

Specifications for Managed Strings

May 1, 2006 • technical report, by fred long, hal burch, robert c. seacord.

This report has been superseded by Specifications for Managed Strings, Second Edition (CMU/SEI-2010-TR-018).

Autonomic Computing

April 1, 2006 • technical note, by william wood, mark h. klein, william o'brien, hausi a. muller (university of victoria).

This report examines selected aspects of autonomic computing and explores some of the strengths and weaknesses of that technology.

Sustaining Operational Resiliency: A Process Improvement Approach to Security Management

By richard a. caralli.

In this 2006 report, Richard Caralli describes the fundamental elements and benefits of a process approach to security and operational resiliency.

Common Elements of Risk

This technical note begins to define a foundation for effective risk management by identifying the basic elements of risk and exploring how these elements can affect the potential for mission …

Model Problems in Technologies for Interoperability: OWL Web Ontology Language for Services (OWL-S)

By chris metcalf c., grace lewis.

This 2006 report presents the results of applying the model problem approach to examine the feasibility of using OWL-S to allow applications to automatically discover, compose, and invoke services in …

System-of-Systems Navigator: An Approach for Managing System-of-Systems Interoperability

By edwin j. morris, lisa brownsword, james smith, david fisher, patrick kirwan.

This technical note introduces the System-of-Systems Navigator (SoS Navigator), the collection and codification of essential practices for building large-scale systems of systems.

Detecting Scans at the ISP Level

April 1, 2006 • technical report, by marc i. kellner, joseph b. kadane, josh mcnutt, carrie gates.

In this 2006 report, the authors present an approach to detecting scans against, or passing through, very large networks.

R2PL 2005 Proceedings of the First International Workshop on Reengineering Towards Product Lines

March 1, 2006 • special report, by barbara graaf, rafael capilla.

This 2006 report contains the proceedings from the First International Workshop on Reengineering Towards Product Lines (R2PL) 2005, which was held in November 2005.

On System Scalability

March 1, 2006 • technical note, by john b. goodenough, charles weinstock.

This 2006 report presents an analysis of what is meant by scalability and a description of factors to be considered when assessing the potential for system scalability.

Toward Measures for Software Architectures

By robert ferguson, gary chastek.

In this 2006 report, the authors describe the results of a preliminary investigation into measures for software architecture.

Requirements Management in a System-of-Systems Context: A Workshop

By peter capell, patrick r. place, b. craig meyers, james smith.

This 2006 report summarizes the results of a workshop focused on requirements management in a system of systems.

Product Line Acquisition in a DoD Organizational Guidance for Decision Makers

By john k. bergey, sholom g. cohen.

This 2006 report chronicles the decisions a program manager might face in considering the adoption of a product line approach.

An Emergent Perspective on Interoperation in Systems of Systems

March 1, 2006 • technical report.

This 2006 report facilitates discussion and reasoning about interoperation within systems of systems by showing some of the interdependencies among systems, emergence, and interoperation.

The Influence of System Properties on Software Assurance and Project Management

February 6, 2006 • white paper.

In this paper, Robert Ellison discusses characteristics of software and how they influence how software assurance should be managed.

The Architecture Analysis & Design Language (AADL): An Introduction

February 1, 2006 • technical note, by john j. hudak, peter h. feiler, david p. gluch.

This 2006 report provides an introduction to the AADL, a modeling language that supports early and repeated analyses of a system's architecture with respect to performance-critical properties.

Acquiring Evolving Technologies: Web Services Standards

By liam o'brien, harry l. levinson.

This technical note discusses some of the challenges of using Web services standards and presents the results generated by an assessment tool used to track the appropriateness of using this …

SAT-Based Software Certification

This 2006 report presents a technique that uses proofs to certify software.

2006 Tech Tip: UNIX Configuration Guidelines

January 1, 2006 • white paper.

This tech tip contains information about UNIX configuration guidelines.

Proceedings of the First International Research Workshop for Process Improvement in Small Settings, 2005

January 1, 2006 • special report, by keith kost, caroline graettinger, suzanne miller.

This 2006 report includes papers from the Proceedings of the First International Research Workshop for Process Improvement in Small Settings workshop, and presents conclusions and next steps for process improvement …

Incident Management

December 1, 2005 • white paper, by georgia killcrece.

In this paper, the author describes incident management capability and what it implies for controlling security events and incidents.

Botnets as a Vehicle for Online Crime

By aaron hackworth, nicholas ianelli.

In this paper, the authors describe the capabilities present in bot malware and the motivations for operating botnets.

Precise Buffer Overflow Detection via Model Checking

In this paper, the authors present an automated overflow detection technique based on model checking and iterative refinement.

December 1, 2005 • Special Report

By jim mchale, daniel s. wall, marsha pomeroy-huff.

This report describes how two NAVAIR organizations integrated the use of the Team Software Process methodology and the CMM framework to progress from Maturity Level 1 to Maturity Level 4 …

Relationships Between CMMI and Six Sigma

December 1, 2005 • technical note, by mary lynn penn, jeannine siviy, erin harper.

This 2005 report focuses on the joint use of two popular improvement initiatives: Capability Maturity Model Integration (CMMI) and Six Sigma.

Secure Software Development Life Cycle Processes: A Technology Scouting Report

The purpose of this 2005 technical note is to present overview information about existing processes, standards, life cycle models, frameworks, and methodologies that support or could support secure software development.

The CERT Function Extraction Experiment: Quantifying FX Impact on Software Comprehension and Verification

By richard c. linger (oak ridge national laboratory), rosann w. collins, gwendolyn h. walton, alan r. hevner (university of south florida).

In this report, the authors describe an experiment comparing traditional methods of comprehension with automated behavior computation using an FX prototype.

Verification of Evolving Software via Component Substitutability Analysis

December 1, 2005 • technical report, by sagar chaki, edmund clarke, natasha sharygina, nishant sinha.

This 2005 report describes the application of the SEI Architecture Tradeoff Analysis Method (ATAM) to the U.S. Army's Warfighter Information Network-Tactical (WIN-T) system.

Results of SEI Independent Research and Development Projects and Report on Emerging Technologies and Technology Trends (FY2005)

By stacy j. prowell, richard c. linger (oak ridge national laboratory), lutz wrage, jörgen hansson (university of skovde), natasha sharygina, rosann w. collins, angel jordan, gwendolyn h. walton, mark pleszkoch, alan r. hevner (university of south florida), sagar chaki, rick kazman, kurt c. wallnau, peter h. feiler, john j. hudak, john b. goodenough, charles weinstock, aaron greenhouse.

This report describes the IR&D projects that were conducted during fiscal year 2005 (October 2004 through September 2005). In addition, this report provides information on what the SEI has learned …

Categorizing Business Goals for Software Architectures

By rick kazman, len bass.

This report provides a categorization of possible business goals for software-intensive systems, so that individuals have some guidance in the elicitation, expression, and documentation of business goals.

Survivability and Information Assurance Curriculum Lab Overview

November 28, 2005 • white paper.

The overview provides information about the hardware and the software required for the lab in general and for each specific course. Other topics include configuration management, user identity and privileges, …

Survivability and Information Assurance Curriculum Overview

The Survivability and Information Assurance (SIA) Curriculum is designed to teach experienced system administrators about security and ways to integrate security into their routine tasks.

Foundations of the Survivability and Information Assurance Curriculum

This document highlights the foundations of the SIA Curriculum.

Safety-Critical Systems and the TSP

November 1, 2005 • technical note.

This 2005 report provides a brief overview of recent work in software safety, discusses the problems and implications of using the TSP for developing safety-critical systems, and presents some conclusions.

Topics in Interoperability: Infrastructure Replacement in a System of Systems

By james smith, david j. carney, patrick r. place.

This technical note examines the Common Operations System (COS), a large aggregation of independently developed systems, and the risks posed to it by an infrastructure upgrade.

Topics in Interoperability: Concepts of Ownership and Their Significance in Systems of Systems

By david j. carney, william anderson, patrick r. place.

This technical note is a brief examination of the concept of ownership and the ways in which it might apply to systems of systems.

Security Quality Requirements Engineering Technical Report

November 1, 2005 • technical report, by eric hough, ted stehney ii, nancy r. mead.

In this 2005 report, the authors present the SQUARE Methodology for eliciting and prioritizing security requirements in software development projects.

Software Product Lines: Experience from the Eighth DoD Software Product Line Workshop

By patrick donohoe, john k. bergey, lawrence g. jones, sholom g. cohen.

This 2005 report summarizes the discussions from a 2005 PLP workshop in which participants shared DoD product line practices, experiences, and issues and discussed ways in which specific product line …

Software Vulnerabilities in Java

October 1, 2005 • technical note, by fred long.

In this report, Fred Long briefly describes potential software vulnerabilities in Java version 5.

U.S. Army Acquisition: The Program Office Perspective

October 1, 2005 • special report.

This report documents the results of the interviews conducted during BFI engagements. These results are of interest to Program Executive Office staffs, Program Management Office staffs, and Department of Army …

VisFlowConnet-IP: An Animated Link Analysis Tool for Visualizing Netflows

September 20, 2005 • white paper, by xiaoxin yin (national center for supercomputing applications (ncsa) at university of illinois at urbana-champaign), adam slagell (national center for supercomputing applications (ncsa) at university of illinois at urbana-champaign), william yurcik (national center for supercomputing applications (ncsa) at university of illinois at urbana-champaign).

In this paper, the authors present VisFlowConnect-IP, a network flow visualization tool that allows operators to detect and investigate network traffic.

VisFlowConnect-IP: An Animated Link Analysis Tool For Visualizing Netflows (White Paper)

Identifying p2p heavy-hitters from network-flow data, by arno wagner (communication systems laboratory swiss federal institute of technology zurich (eth zurich)), thomas dubendorfer (communication systems laboratory swiss federal institute of technology zurich (eth zurich)), lukas hammerle (communication systems laboratory swiss federal institute of technology zurich (eth zurich)), bernhard plattner (communication systems laboratory swiss federal institute of technology zurich (eth zurich)).

In this September 2005 paper, the authors present measurements done on a medium-sized internet backbone and discuss accuracy issues.

Flow-Data Compressibility Changes During Internet Worm Outbreaks

By arno wagner (communication systems laboratory swiss federal institute of technology zurich (eth zurich)).

In this paper, Arno Wagner presents measurements and analysis done on a Swiss internet backbone during the Blaster and Witty internet worm outbreak.

A Proposed Translation Data Model for Flow Format Interoperability

By brian trammell.

In this paper, Brian Trammell presents a proposed solution to the problem of mutual unintelligibility of raw flow and intermediate analysis data.

R: A Proposed Analysis and Visualization Environment for Network Security Data (White Paper)

By josh mcnutt.

In this paper, Josh McNutt discusses the R statistical language as an analysis and visualization interface to SiLK flow analysis tools.

Correlations Between Quiescent Ports in Network Flows (White Paper)

By josh mcnutt, markus deshon.

In this paper, the authors introduce a method for detecting the onset of anomalous port-specific activity by recognizing deviation from correlated activity.

CANINE: A NetFlows Converter/Anonymizer Tool for Format Interoperability and Secure Sharing (White Paper)

By katherine luo (national center for supercomputing applications (ncsa) university of illinois at urbana-champaign), adam slagell (national center for supercomputing applications (ncsa) at university of illinois at urbana-champaign), william yurcik (national center for supercomputing applications (ncsa) at university of illinois at urbana-champaign), yifan li (national center for supercomputing applications (ncsa) university of illinois at urbana-champaign).

In this paper, the authors introduce a tool to address two problems with using Net-Flow logs for security analysis.

Detecting Distributed Attacks using Network-Wide Flow Traffic

By anukool lakhina (boston university), mark crovella (boston university), chrisophe diot (intel).

In this paper, the authors present their methods for detecting distributed attacks in backbone networks using sampled flow traffic data.

IP Flow Information Export (IPFIX): Applicability and Future Suggestions for Network Security

By elisa boschi (hitachi), tanja zseby (fraunhofer fokus), mark lutz (fraunhofer fokus), thomas hirsch (fraunhofer fokus).

In this paper, the authors present the IPFIX protocol and discuss its applicability with a special focus on network security.

NVisionIP: An Animated State Analysis Tool for Visualizing NetFlows (White Paper)

In this paper, the authors describe NVisionIP, a NetFlow visualization tool.

Using the OPEN Process Framework to Produce a Situation-Specific Requirements Engineering Method

September 1, 2005 • white paper, by donald firesmith, b. henderson-sellers, d. zowghi.

The OPEN Process Framework (or OPF) is an appropriate focused requirements engineering method (REM) that facilitates the search for a mechanism that will support the flexible creation of a number …

Covert Channel Detection Using Process Query Systems (White Paper)

By vincent berk (dartmouth college).

In this FloCon 2005 presentation, the author uses traffic analysis to investigate a stealthy form of data exfiltration.

Building Information Assurance Educational Capacity: Pilot Efforts to Date

September 1, 2005 • special report.

In this report, Carol Sledge describes work to increase the capacity of educational institutions to offer and expand IA and IS topics and courses.

Quality Attributes and Service-Oriented Architectures

September 1, 2005 • technical note, by len bass, liam o'brien, paulo merson.

This report examines the relationship between service-oriented architectures (SOAs) and quality attributes.

Using the SEI Architecture Tradeoff Analysis Method to Evaluate WIN-T: A Case Study

By paul c. clements, john k. bergey, dave mason.

This report describes the application of the SEI ATAM (Architecture Tradeoff Analysis Method) to the U.S. Army's Warfighter Information Network-Tactical (WIN-T) system.

SMART: The Service-Oriented Migration and Reuse Technique

By grace lewis, edwin j. morris, liam o'brien, lutz wrage, dennis b. smith.

This document has been superseded by CMU/SEI-2008-TN-008, SMART: Analyzing the Reuse Potential of Legacy Components in a Service-Oriented Architecture Environment.

Elements of a Usability Reasoning Framework

By len bass, jinhee lee.

This note describes an ARL implementation of two usability scenarios: displaying progress feedback and allowing cancel.

Mission Assurance Analysis Protocol (MAAP): Assessing Risk in Complex Environments

In this 2005 report, the authors present concepts and theories underlying the Mission Assurance Analysis Protocol.

Integrated Diagnostics: Operational Missions, Diagnostic Types, Characteristics, and Capability Gaps

By theodore f. marz.

This 2005 report attempts to fill in these gaps in knowledge and experience by presenting an overview of the operational diagnostic life cycle of a system.

A Taxonomy of Operational Risks

By rita c. creel, ray c. williams, brian p. gallagher, susan kushner, pamela j. case.

This report presents a taxonomy-based method for identifying and classifying risks to operational aspects of an enterprise.

Proceedings of the First Software Architecture Technology User Network (SATURN) Workshop

By james e. tomayko, len bass, linda m. northrop, paul c. clements, robert nord.

This report describes the format, discussion, and results of the first SATURN workshop, and outlines the plans for future SATURN workshops.

Lessons Learned Model Checking an Industrial Communications Library

By james ivers.

This 2005 report describes the application of a reasoning framework to the design of an industrial communications library and the problems that were found.

Experience Using the Web-Based Tool Wiki for Architecture Documentation

By paulo merson, felix bachmann.

This 2005 report discusses the benefits and challenges of using a wiki-based collaborative environment to create software architecture documentation.

Exploring Programmatic Interoperability: Army Future Force Workshop

This report documents the proceedings of the Future Force Workshop held at the SEI in 2004.

SAT-Based Predicate Abstraction of Programs

September 1, 2005 • technical report, by natasha sharygina, edmund clarke, daniel kroening, karen yorav (ibm).

This note presents technical details of a SAT-based predicate abstraction technique used in ComFoRT (component formal reasoning technology).

Variability in Software Product Lines

By paul c. clements, felix bachmann.

This 2005 report describes the concepts needed when creating core assets with included variability. These concepts provide guidelines to core asset creators on how to model the variability explicitly, so …

QuARS: A Tool for Analyzing Requirement

By giuseppe lami.

This 2005 report describes a disciplined method and a related automated tool that can be used for the analysis of natural language requirements documents.

Preparing for Automated Derivation of Products in a Software Product Line

By john mcgregor.

This 2005 report provides an end-to-end view of the activities that are needed to support the automatic derivation of products within a software product line.

The U.S. Army's Common Avionics Architecture System (CAAS) Product Line: A Case Study

By paul c. clements, john k. bergey.

This report offers a case study of organizations that have adopted a software product line approach for developing a family of software-intensive systems.

Limits to Effectiveness in Computer Security Incident Response Teams

August 22, 2005 • white paper, by johannes wiik (agder university college norway), jose j. gonzalez (agder university college norway).

In this paper, the authors present an attempt to gain a better understanding of how a CSIRT can handle a growing work load with limited resources.

Information Technology: Programming Languages, Their Environments and System Software Interfaces: Specification for Managed Strings

August 19, 2005 • white paper, by robert c. seacord, fred long.

In this paper, the authors present a standard specification for managed strings.

Simulating Insider Cyber-Threat Risks: A Model-Based Case and a Case-Based Model

August 11, 2005 • white paper, by elise a. weaver (worcester polytechnic institute), howard f. lipson, eliot rich (university at albany state university of new york), jose m. sarriegui (university of navarra spain), agata sawicka (agder university college norway), thomas r. stewart (university at albany state university of new york), jose m. torres (university of navarra spain), johannes wiik (agder university college norway), ignacio j. martinez-moyano (university at albany state university of new york), paul conrad, david f. andersen (university at albany state university of new york), dawn cappelli, andrew p. moore, dave mundie, timothy j. shimeall, robert j. ellison, jose j. gonzalez (agder university college norway).

In this paper, the authors identify actions that may inadvertently lead to increased vulnerability to threats from employees, contractors, and clients.

Obtaining the Benefits of Predictable Assembly from Certifiable Components (PACC)

August 1, 2005 • white paper, by kurt c. wallnau.

PACC combines the complementary features of software architecture technology and software component technology to improve both engineering productivity and product quality in the design and implementation of quality-critical software systems.

The Personal Software Process (PSP) Body of Knowledge, Version 1.0

August 1, 2005 • special report, by watts s. humphrey, marsha pomeroy-huff, julia l. mullaney, robert cannon, mark sebern.

The body of knowledge contained in this report is designed to complement the IEEE Computer Society's Software Engineering Body of Knowledge (SWEBOK) by delineating the key skills and concepts that …

Self-Assessment and the CMMI-AM—A Guide for Government Program Managers

August 1, 2005 • technical note, by stephen blanchette, jr., kristi keeler.

This 2005 report provides program managers with general information about the CMMI-AM, details about the self-assessment technique, and the questions used in a self-assessment.

Some Current Approaches to Interoperability

By david j. carney, david fisher, patrick r. place, edwin j. morris.

This 2005 report examines some of the complexities of interoperability and some recent research approaches to achieving it.

Using Containers to Enforce Smart Constraints for Performance in Industrial Systems

By gabriel moreno, scott hissam, kurt c. wallnau.

This technical note shows how smart constraints can be embedded in software infrastructure, so that systems conforming to those constraints are predictable by construction.

The ComFoRT Reasoning Framework

July 1, 2005 • white paper, by kurt c. wallnau, sagar chaki, james ivers, natasha sharygina.

Model checking is a promising technology for verifying critical behavior of software. However, software model checking is hamstrung by scalability issues and is difficult for software engineers to use directly. …

Comparing the SEI's Views and Beyond Approach for Documenting Software Architectures with ANSI-IEEE 1471-2000

July 1, 2005 • technical note.

This report summarizes the V&B and 1471 approaches to architecture description, and shows how a software architecture document prepared using V&B can be made compliant with 1471.

Product Line Adoption in a CMMI Environment

By lawrence g. jones, linda m. northrop.

This 2005 technical note addresses product line adoption in the context of an organization that is using the CMMI models to guide its process improvement effort.

Reasoning Frameworks

July 1, 2005 • technical report, by len bass, paulo merson, james ivers, mark h. klein.

This report describes a vehicle for encapsulating the quality attribute knowledge needed to understand a system's quality behavior as a reasoning framework that can be used by nonexperts.

The Impact of Function Extraction Technology on Next-Generation Software Engineering

By alan r. hevner (university of south florida), mark pleszkoch, gwendolyn h. walton, rosann w. collins, stacy j. prowell, richard c. linger (oak ridge national laboratory).

In this 2005 report, the authors summarize FX research and development and investigates the impact of FX on software engineering.

Designing for Reuse of Configurable Logic

This 2005 report provides an overview of a generic FPGA firmware design process and identifies the resulting work products that may be suitable for reuse in future development efforts.

Word Level Predicate Abstraction and Refinement for Verifying RTL Verilog

June 1, 2005 • white paper, by natasha sharygina, edmund clarke, daniel kroening.

This paper proposes to use predicate abstraction for verifying RTL Verilog, a technique successfully used for software verification.

Advanced Security Reporting Systems for Large Network Situational Awareness

By greg virgin (redjack), michael collins.

In this paper, the authors describe the technologies that support an asset inventory system and enable a flexible, ad-hoc intrusion detection capability.

The CENTAUR System: Helping to Protect the NIPRNet

By jeffrey jaime (applied technology unit, joint task force - global network operations, united states strategic command), marc i. kellner.

In this paper, the authors describe the CENTAUR system, which was developed to help DoD security analysts better understand and defend the NIPRNet.

By Aaron Hackworth

In this 2005 paper, the authors give an overview of spyware, provide examples of common threats, and describe how to defend against spyware.

Report on Annual Regional Information Assurance Symposia

June 1, 2005 • special report.

In this report, Carol Sledge explains why the annual Regional Information Assurance Symposia is a key transition component of Regional Collaborative Clusters.

Using Earned Value Management (EVM) in Spiral Development

June 1, 2005 • technical note, by james smith, lisa brownsword.

This report explores the fundamental challenges in using Earned Value Management (EVM) with spiral development processes and proposes adaptations to some EVM principles to render it more suitable for today's …

Information Asset Profiling

By bradford j. willke, james f. stevens, richard a. caralli.

In this 2005 report, the authors describe IAP, a documented and repeatable process for developing consistent asset profiles.

Governing for Enterprise Security

In this 2005 report, Julia Allen examines governance thinking, principles, and approaches and applies them to the subject of enterprise security.

A Process for Context-Based Technology Evaluation

By lutz wrage, grace lewis.

This report describes a process called context-based evaluation that determines the fitness of a technology within a specific context.

Insider Threat Study: Illicit Cyber Activity in the Banking and Finance Sector

June 1, 2005 • technical report, by andrew p. moore, dawn cappelli, marissa r. randazzo (united states secret service), michelle keeney (united states secret service), eileen kowalski (united states secret service).

In this 2005 report, the authors outline the ITS, a study of insider incidents identified by public reporting or in fraud cases from the Secret Service.

Secret Service and CERT Release Report Analyzing Acts of Insider Sabotage via Computer Systems in Critical Infrastructure Sectors

May 16, 2005 • white paper.

This press release is the second in a series of reports focusing on insider threats to information systems and data in critical infrastructure sectors.

2005 E-Crime Watch Survey Findings

May 3, 2005 • white paper.

In this 2005 report, the authors summarize the results of the 2005 E-Crime Watch Survey, conducted to unearth electronic crime fighting trends and techniques.

A Taxonomy of Security-Related Requirements

May 1, 2005 • white paper.

This paper addresses the problems associated with a lack of a clear security taxonomy by identifying four different types of security-related requirements, providing them with clear definitions, and placing them …

Reflections on Software Agility and Agile Methods: Challenges, Dilemmas, and the Way Ahead

By linda levine.

This 2005 whitepaper argues for that the shift toward agile models and methods signals a larger transformation in the workplace toward the organization of the 21st century. The transition is …

Method Engineering and COTS Evaluation

By b. henderson-sellers, c. gonzalez-perez, m. k. serour, donald firesmith.

This position paper argues that a successful COTS evaluation process should be based on the principles of method engineering (ME).

2005 E-Crime Watch Survey Results

This paper summarizes the results of a survey conducted to gauge the current state of cybercrime.

Technical Trends in Phishing Attacks

By jason milletary.

In this paper, Jason Milletary identifies technical capabilities used to conduct phishing scams, reviews trends, and discusses countermeasures.

System Quality Requirements Engineering (SQUARE): Case Study on Asset Management System, Phase II

May 1, 2005 • special report, by dan gordon, neha wattas, eugene yu, ted stehney ii, nancy r. mead.

In this report, the authors describe the second phase of an application of the SQUARE Methodology on an asset management system.

Model Problems in Technologies for Interoperability: Model-Driven Architecture

May 1, 2005 • technical note.

This 2005 report looks at Model-Driven Architecture (MDA) as one of many technologies for accomplishing interoperability.

CMMI Acquisition Module (CMMI-AM), Version 1.1

May 1, 2005 • technical report, by hal wilson, roger bate, thomas bernard, brian p. gallagher.

This report documents acquisition practices that should be performed by government acquisition projects acquiring systems or services.

Industry Best Practices in Achieving Service Oriented Architecture

April 22, 2005 • white paper.

This document represents the first iteration of a conversation and is neither a complete nor exhaustive coverage of the evolving subject of SOA.

Pin Component Technology (V1.0) and Its C Interface

April 1, 2005 • technical note, by kurt c. wallnau, daniel plakosh, james ivers, scott hissam.

This 2005 report describes the main concepts of Pin and documents the C-language interface to Pin V1.0.

Robustness Testing of Software-Intensive Systems: Explanation and Guide

By julie b. cohen, daniel plakosh, kristi keeler.

This 2005 technical note provides guidance and procedures for performing robustness testing as part of DoD or federal acquisition programs that have a software component.

Mapping TSP to CMMI

April 1, 2005 • technical report, by daniel s. wall, watts s. humphrey, jim mchale, michael d. konrad.

This 2005 report provides an essential element to facilitate the adoption of the TSP in organizations using CMMI, namely, a mapping of ideal TSP practices into the specific and generic …

U.S. Army Acquisition: The Program Executive Officer Perspective

March 1, 2005 • special report.

The U.S. Army Strategic Software Improvement Program (ASSIP) is a multi-year effort to improve the way the Army acquires software-intensive systems. As part of the ASSIP, the Carnegie Mellon Software …

Topics in Interoperability: System-of-Systems Evolution

March 1, 2005 • technical note, by patrick r. place, david fisher, david j. carney.

This report - the first in a series of reports on interoperability - examines how interoperable systems of systems evolve.

Eliciting and Analyzing Quality Requirements: Management Influences on Software Quality Requirements

In this 2005 report, Carol Woody documents how environments for system development can support or reject improved quality requirements elicitation mechanisms.

Software Product Lines: Experiences from the Seventh DoD Software Product Line Workshop

March 1, 2005 • technical report, by lawrence g. jones, patrick donohoe, john k. bergey, sholom g. cohen.

This 2005 report summarizes discussions and presentations from the Seventh Department of Defense (DoD) Product Line Practice Workshop.

Software Process Improvement Journey: IBM Australia Application Management Services

By robyn nichols, colin connaughton.

This report describes the work of the 2004 recipient of the IEEE Computer Society Software Process Achievement Award, jointly established by the SEI and IEEE to recognize outstanding achievements in …

Including Interoperability in the Acquisition Process

By ira monarch, james smith, b. craig meyers, linda levine.

This 2005 report explores achieving interoperability in the acquisition process.

Interpreting SCAMPI for a People CMM Appraisal at Tata Consultancy Services

February 1, 2005 • special report, by sarah miller, will hayes, ron radice, gian wemyss, jack r. ferguson, william e. hefley, bill curtis (cast research labs).

This 2005 report includes the draft interpretation guide used for four mini-appraisal pilots and the final enterprise-wide Class A appraisal at Tata Consultancy Services (TCS).

Software Architecture in DoD Acquisition: An Approach and Language for a Software Development Plan

February 1, 2005 • technical note.

This report discusses the Software Development Plan (SDP), providing an example approach and corresponding SDP language that enable software architecture to play a central role in the technical and organizational …

Software Architecture in DoD Acquisition: A Reference Standard for a Software Architecture Document

By john k. bergey, paul c. clements.

This report provides a reference standard for a Software Architecture Document (SAD). Acquisition organizations can use this to acquire documentation needed for communicating the architecture design and conducting software architecture …

The Structured Intuitive Model for Product Line Economics (SIMPLE)

February 1, 2005 • technical report, by sholom g. cohen, john mcgregor, paul c. clements.

This 2005 report presents SIMPLE, a general-purpose business model that supports the estimation of the costs and benefits in a product line development organization.

Preliminary System Dynamics Maps of the Insider Cyber-Threat Problem

January 1, 2005 • white paper, by andrew p. moore, timothy j. shimeall, dawn cappelli, david f. andersen (university at albany state university of new york), jose j. gonzalez (agder university college norway), eliot rich (university at albany state university of new york), jose m. sarriegui (university of navarra spain), elise a. weaver (worcester polytechnic institute), aldo zagonel (university at albany, rockefeller college of public affairs and policy), mohammad mojtahedzadeh (attune group, inc.), jeffrey m. stanton (syracuse university, school of information studies).

This paper discusses the preliminary system dynamic maps of the insider cyber-threat and describes the main ideas behind the research proposal.

MAAP Information Sheet

Information sheet on MAAP, a technique for assuring completion of defined missions by identifying and analyzing operational risks affecting mission-critical processes.

A Structured Approach to Classifying Security Vulnerabilities

January 1, 2005 • technical note, by robert c. seacord, allen d. householder.

In this 2005 report, the authors propose a classification scheme that uses attribute-value pairs to provide a multidimensional view of vulnerabilities.

2004 CERT Incident Notes

December 31, 2004 • white paper.

This document contains the CERT incident notes from 2004.

2004 CERT Advisories

This document contains the CERT advisories from 2004.

CMMI-Based Professional Certifications: The Competency Lifecycle Framework

December 1, 2004 • special report, by sandra behrens, steve masters, judah mogilensky.

This report describes how a competency life-cycle framework can be used as the basis for the CMMI-based professional certifications.

Systems Quality Requirements Engineering (SQUARE) Methodology: Case Study on Asset Management System

By marjon dean, don ojoko-adams, peter chen, hassan osman, lilian lopez, nick xie, nancy r. mead.

In this 2004 report, the authors describe the first case study that applied the SQUARE methodology to an organization.

Promising Technologies for Future Systems

December 1, 2004 • technical note, by lutz wrage, grace lewis, edwin j. morris.

This 2004 report presents of a few of the many programs, technologies, and research efforts that are addressing the challenges faced by future systems.

Managing for Enterprise Security

By william r. wilson, julia h. allen, richard a. caralli, james f. stevens, bradford j. willke.

In this 2004 report, the authors itemize characteristics of common approaches to security that limit effectiveness and success.

Discovering Architectures from Running Systems: Lessons Learned

December 1, 2004 • technical report, by jonathan aldrich, bradley schmerl, david garlan, hong yan, rick kazman.

This report describes a technique that uses automatically generated runtime observations of an executing system to construct an architectural view of the system.

Approaches to Constructive Interoperability

This report outlines several approaches to constructing systems of systems that have interoperability requirements, with respect to syntactic and semantic interoperability.

Rapid Integration Tools for Rapid Application Development A Case Study on Legacy Integration

By patrick r. place, amit midha, ravindra singh, lakshimi p. hari.

This 2004 report investigates the rapid integration tools available in the current market. The report presents a generic evaluation framework for identifying and evaluating rapid integration tools and an evaluation …

The Incident Object Description Exchange Format (IODEF) Implementation Guide

November 9, 2004 • white paper, by roman danyliw.

This document provides implementation guidelines for CSIRTs adopting the IODEF.

SQUARE Project: Cost/Benefit Analysis Framework for Information Security Improvement Projects in Small Companies

November 1, 2004 • technical note, by lilian lopez, nick xie, peter chen, marjon dean, don ojoko-adams, hassan osman, nancy r. mead.

In this 2004 report, the authors describe a cost/benefit analysis for estimations in small companies' information security improvement projects.

Predicate Abstraction with Minimum Predicates

October 1, 2004 • white paper, by sagar chaki, edmund clarke.

Predicate abstraction is a popular abstraction technique employed in formal software verification. Experiments show that predicate minimization can result in a significant reduction of both verification time and memory usage …

Results of SEI Independent Research and Development Projects and Report on Emerging Technologies and Technology Trends (FY 2004)

October 1, 2004 • technical report, by william o'brien, angel jordan, dennis b. smith, eileen c. forrester, sven dietrich, jeannine siviy, john k. bergey, howard f. lipson, charles weinstock, nancy r. mead, carol woody, edwin j. morris, rick kazman, donald firesmith, grace lewis.

This report describes the IR&D projects that were conducted during fiscal year 2004 (October 2003 through September 2004).

CMMI Interpretive Guidance Project: What We Learned

October 1, 2004 • special report, by mary beth chrissis, sandra shrum, michael d. konrad, gian wemyss, kenneth smith.

This report summarizes the results of the Capability Maturity Model Integration (CMMI) Interpretive Guidance Project, and summarizes and analyzes 7500 comments received regarding CMMI adoption that were reported by CMMI …

Illuminating Patterns of Perception: An Overview of Q Methodology

October 1, 2004 • technical note, by mary m. brown.

This 2004 technical note describes ways for applying Q methodology, a research method with a proven history for illuminating agreement and differences among individual and group perceptions, to assist software …

Defining Incident Management Processes for CSIRTs: A Work in Progress

By christopher j. alberts, robin ruefle, mark zajicek, audrey j. dorofee, georgia killcrece.

In this report, the authors present a prototype best practice model for performing incident management processes and functions.

Measurement and Analysis: What Can and Does Go Wrong?

September 1, 2004 • white paper, by dennis goldenson, maureen brown (university of north carolina).

Analyses of more than 1350 findings drawn from 663 Software CMM appraisals suggest several areas where both managers and engineers would benefit from better guidance about the proper use of …

A Taxonomy of Safety-Related Requirements

This paper describes a taxonomy of different kinds of safety-related requirements, and clearly and briefly defines and describes each of them.

A Roadmap of Risk Diagnostic Methods: Developing an Integrated View of Risk Identification and Analysis Techniques

September 1, 2004 • technical note, by kate ambrose, laura bentrem, ray c. williams.

This technical note describes the characteristics that determine whether a risk diagnostic method qualifies for the roadmap. It also describes the characteristics of diagnostic methods that do not qualify for …

Code of Professional Conduct for SEI Services, Version 1.0

September 1, 2004 • special report, by richard cox.

This report provides a set of expectations and practices for those operating under license or other applicable agreement with Carnegie Mellon University, acting through its Software Engineering Institute.

Benefits of Improvement Efforts

By peter capell.

This special report surveys the process improvement efforts undertaken by programs and projects that incorporate software-intensive systems.

Risk Based Diagnostics

By ray c. williams, laura bentrem, tom merendino, kate ambrose.

The SEI has constructed a tentative "roadmap" for personnel involved in the systems and software acquisition community. This report describes the characteristics that determine whether a risk diagnostic method qualifies …

Security and Survivability Reasoning Frameworks and Architectural Design Tactics

By andrew p. moore, felix bachmann, robert j. ellison, mark h. klein, len bass.

In this report, the authors describe an approach to disciplined software architecture design for the related quality attributes of security and survivability.

Applications of the Indicator Template for Measurement and Analysis

By wolfhart b. goethert, jeannine siviy.

This report presents guidance for adapting and completing an indicator template--an SEI-developed tool to describe an indicator's construction, interpretation, and how it can be best utilized.

Software Component Certification: 10 Useful Distinctions

This 2004 report discusses 10 useful distinctions that can help in understanding different aspects of certification in the context of software components.

Integrating Software-Architecture-Centric Methods into Extreme Programming (XP)

By robert nord, james e. tomayko, robert wojcik.

The report presents a summary of XP (Extreme Programming) and examines the potential uses of the SEI's architecture-centric methods.

Creating and Using Software Architecture Documentation Using Web-Based Tool Support

By judith a. stafford.

This report describes a design prototype that demonstrates a web-based approach to creating, communicating, and using software architecture throughout the life of the system.

Software Process Improvement and Product Line Practice: Building on Your Process Improvement Infrastructure

By lawrence g. jones.

This 2004 report describes how a process improvement infrastructure can provide a foundation for product line adoption.

Performance Property Theories for Predictable Assembly from Certifiable Components (PACC)

September 1, 2004 • technical report, by paulo merson, kurt c. wallnau, scott hissam, gabriel moreno, mark h. klein, john lehoczky (carnegie mellon university).

This report develops a queueing-theoretic solution to predict, for a real-time system, the average-case latency of aperiodic tasks managed by a sporadic server.

Software Product Line Adoption Roadmap

By linda m. northrop.

This 2004 report introduces a variant of the Factory Pattern called the Adoption Factory pattern that provides a generic roadmap to guide a manageable, phased product line adoption strategy.

Steps for Creating National CSIRTs

August 2, 2004 • white paper.

In this paper, Georgia Killcrece provides a high-level description of a National Computer Security Incident Response Team (NatCSIRT), its problems, and challenges.

Assumptions Management in Software Development

August 1, 2004 • technical note, by teeraphong mahatham, lutz wrage, grace lewis.

This technical note explores assumptions management as a method for improving software quality.

Statistical Methods for Flow Data

July 22, 2004 • white paper, by joseph b. kadane.

In this presentation, Joseph B. Kadane discusses how Bayesian methods help make the logistic regression approach to scan data stable and operationally feasible.

Integrating the Quality Attribute Workshop (QAW) and the Attribute-Driven Design (ADD) Method

July 1, 2004 • technical note, by william wood, robert nord, paul c. clements.

This technical note reports on a proposal to integrate the SEI Quality Attribute Workshop (QAW) and the SEI Attribute-Driven Design (ADD) method.

A Model Problem for an Open Robotics Controller

By scott hissam, mark h. klein.

This report describes the model problem created to support the continued enhancement and development of the PECT reasoning frameworks for an industrial trial in the domain of industrial robotics.

A Process for COTS Software Product Evaluation

July 1, 2004 • technical report, by santiago comella-dorda, john dean, tricia oberndorf, erin harper, grace lewis, edwin j. morris.

This 2004 report focuses on COTS product evaluations conducted for the purpose of selecting products to meet a known need in a system.

The Critical Success Factor Method: Establishing a Foundation for Enterprise Security Management

By william r. wilson, bradford j. willke, richard a. caralli, james f. stevens.

In this report, the authors describe the critical success factor method and present theories and experience in applying it to enterprise security management.

Integrating Software-Architecture-Centric Methods into the Rational Unified Process

By james e. tomayko, philippe kruchten, robert nord, rick kazman.

This report presents a summary of the RUP (Rational Unified Process) and examines the potential uses of the SEI's architecture-centric methods.

Acquisition Overview: The Challenges

June 1, 2004 • white paper.

In this paper, the authors raise issues involving how systems are integrated to provide required capabilities.

Embedded Systems Architecture Analysis Using SAE AADL

June 1, 2004 • technical note, by john j. hudak, peter h. feiler, david p. gluch, bruce lewis (u.s. army amcom).

This 2004 report discusses the role and benefits of using the AADL in the process of analyzing an existing avionics system.

2004 E-Crime Watch Survey Findings

May 25, 2004 • white paper.

In this report, the authors summarize the results of the 2004 E-Crime Watch Survey, conducted to unearth e-crime fighting trends and techniques.

An Empirical Analysis of Target-Resident DoS Filters

May 9, 2004 • white paper, by michael k. reiter, michael collins.

In this paper, the authors provide an empirical analysis of proposed techniques for filtering network traffic.

Software Patents: Innovation or Litigation?

May 1, 2004 • white paper, by linda levine, kurt m. saunders.

This paper summarizes the scope of patent protection in the European Union, the United States, and Japan. In doing so, it examines the patentability of computer software as inventions allowed …

Networked Technologies: The Role of Networks in the Diffusion and Adoption of Software Process Improvement (SPI) Approaches

By karlheinz kautz, jorn johansen, peter a. nielsen, linda levine, william e. hefley.

Social networks play a key role in the adoption and diffusion of software process improvement as a networked technology. This panel addressed actual examples of SPI networks and identified key …

Selecting Advanced Software Technology in Two Small Manufacturing Enterprises

May 1, 2004 • technical note, by william anderson, charles buhman, len estrin.

This 2004 report documents two small manufacturing enterprises' (SMEs') efforts to select advanced software technologies for their business operations.

Survivable Functional Units: Balancing an Enterprise's Mission and Technology

By larry rogers.

In this 2004 report, Larry Rogers describes enterprise networks in a way that helps system administrators see how technology supports the enterprise's mission.

Dependability Cases

By john j. hudak, john b. goodenough, charles weinstock.

In this 2004 report, the authors explain how to create a dependability case for a system that helps identify and keep track of details of large systems.

Case Study: A Measurement Program for Product Lines

By ed dunn (naval undersea warfare center), sholom g. cohen, david zubrow.

This report documents NUWC's approach for measurement by describing the Goal-Driven Software Measurement approach and providing early results of the measurement program.

Advanced Engineering Environments for Small Manufacturing Enterprises: Volume II

May 1, 2004 • technical report, by steven j. fenves (national institute of standards and technology), ram d. sriram (national institute of standards and technology), young choi (chung-ang university), joseph p. elm, john e. robert.

This report documents the Self-Assessment Tool for Engineering Environments (SAT-EE) and the Self-Assessment Tool for Engineering Tool Capabilities (SAT-ETC).

Standard Systems Group (SSG) Technology Adoption Planning Workshop

April 1, 2004 • special report, by lorraine nemeth-adams, jan vargas, suzanne miller.

This 2004 report presents the results of the SSG Technology Adoption Planning Workshop, which was held in October 2003 in Alabama.

Measuring Systems Interoperability: Challenges and Opportunities

April 1, 2004 • technical note, by william anderson, mark kasunic.

This 2004 report presents best practices for measuring systems interoperability and assisting military planners in the acquisition, development, and implementation of interoperable C4I systems.

Overview of ComFoRT: A Model Checking Reasoning Framework

By james ivers, natasha sharygina.

This 2004 report describes ComFoRT, a reasoning framework that packages the effectiveness of state-of-the-art model checking in a form that enables users to apply the analysis technique without being experts …

Systems of Systems Interoperability

April 1, 2004 • technical report, by edwin j. morris, patrick r. place, daniel plakosh, linda levine, b. craig meyers.

This technical report documents the findings of an internal research and development effort on system of systems interoperability (SOSI).

Documenting Component and Connector Views with UML 2.0

By james ivers, robert nord, paul c. clements, oviedo silva (carnegie mellon school of computer science), bradley schmerl, david garlan.

This 2004 report explores how changes in UML 2.0 affect UML's suitability for documenting component and connector views.

An Alternative to Technology Readiness Levels for Non-Developmental Item (NDI) Software

This report explores the difficulties in using TRLs as they apply to NDI software technology and products, and explores an alternative set of readiness criteria.

COTS Usage Risk Evaluation Participant’s Overview

March 29, 2004 • white paper.

This document provides an overview of the three steps of the COTS Usage Risk Evaluation (CURE) that involve participation by the program's team members. For each step, both the activity …

Sets, Bags, and Rock and Roll? Analyzing Large Data Sets of Network Data

March 24, 2004 • white paper, by john mchugh.

In this paper, John McHugh describes problems with monitoring and analyzing traffic on high-speed networks.

Conflict Patterns: Toward Identifying Suitable Middleware

March 1, 2004 • white paper.

This whitepaper describes patterns of interoperability conflicts along with their typical resolution in an effort to present reusable solutions for the design of integration architectures.

Software Product Lines: Experiences from the Sixth DoD Software Product Line Workshop

March 1, 2004 • technical note, by john k. bergey, dennis b. smith, lawrence g. jones, sholom g. cohen.

This 2004 report summarizes the presentations and discussions from the Sixth Department of Defense (DoD) Product Line Practice Workshop in September 2003.

A Study of Product Production in Software Product Lines

By john mcgregor, patrick donohoe, gary chastek.

This 2004 report presents the results of a study that focused on how product line organizations create products.

Case Study: IRS Business System Modernization Process Improvement

March 1, 2004 • technical report, by lloyd anderson, jon gross, matt fisher.

This report provides an overview of applying the SA-CMM to the IRS modernization effort to establish and implement more effective acquisition management processes and practices.

Army Strategic Software Improvement Program (ASSIP) Survey of Army Acquisition Managers

This report analyzes a survey that covered four areas of the acquisition system: the acquirer's environment, the developer's environment, communication between the acquirer and developer, and external factors that could …

An Integrated Approach to Software Process Improvement at Wipro Technologies: veloci-Q

By deb sambuddha (wipro technologies), priya krishnaswamy (wipro technologies), rituparna ghosh (wipro technologies).

This report describes the work of the 2002 recipient of the IEEE Computer Society Software Process Achievement Award, jointly established by the SEI and IEEE to recognize outstanding achievements in …

Current Perspectives on Interoperability

By b. craig meyers, david j. carney, lutz wrage, david fisher, james smith, grace lewis, edwin j. morris, lisa brownsword, patrick r. place.

This 2004 report describes current research within the software engineering community on the topic of interoperability between software systems.

A-Specification for the CMMI Product Suite, Version 1.6

February 6, 2004 • white paper.

The A-Specification for the CMMI Product Suite defines the scope, lists applicable documents, and defines the requirements the CMMI Product Suite must meet to be considered acceptable.

Upgrading from SW-CMM to CMMI

February 1, 2004 • white paper.

This whitepaper shows how organizations can promptly move from a maturity level of the SW-CMM to the corresponding maturity level of CMMI.

CMMI Acquisition Module (CMMI-AM) Version 1.0

February 1, 2004 • technical report, by thomas bernard, roger bate, brian p. gallagher, hal wilson.

This report contains the acquisition practices that should be performed by government acquisition organizations acquiring systems and/or services.

Working with Small Manufacturing Enterprises: An Analysis of TIDE

By john t. foreman, len estrin, john e. robert, alfred schenker, william anderson, suzanne miller, joseph p. elm.

This 2004 paper documents some of the challenges and risks facing programs or organizations trying to help small manufacturing enterprises (SMEs).

Financial Institution CSIRT Case Study

January 22, 2004 • white paper.

This case study describes the experiences of a financial institution CSIRT in getting its organization up and running.

Eight Architecture Lessons from History

January 1, 2004 • white paper.

This 2004 whitepaper offers eight lessons from history for the software architecture field, drawn from peer fields i.e. Military, Civil, Finance, Mathematics, Astronomy, Social and Medical.

COTS Acquisition Evaluation Process: Preacher's Practice

January 1, 2004 • technical note, by vijay sai.

This paper outlines a successful effort to apply COTS-based engineering principles to a software acquisition by various groups at the SEI.

2003 CERT Incident Notes

December 31, 2003 • white paper.

This document contains the CERT incident notes from 2003.

CERT Research 2003 Annual Report

This report provides brief abstracts for major research projects, followed by more detailed descriptions of these projects, for all CERT research conducted in the year 2003.

2003 CERT Advisories

This document contains the CERT advisories from 2003.

Common Concepts Underlying Safety, Security, and Survivability Engineering

December 1, 2003 • technical note.

In this report, Donald Firesmith presents information models that identify and define concepts underlying safety, security, and survivability engineering.

Integrating the Architecture Tradeoff Analysis Method (ATAM) with the Cost Benefit Analysis Method (CBAM)

By rick kazman, paul c. clements, robert nord, mark h. klein, james e. tomayko, liam o'brien, mario r. barbacci.

This technical note reports on a proposal to integrate the SEI ATAM (Architecture Tradeoff Analysis Method) and the CBAM (Cost Benefit Analysis Method).

SACAM: The Software Architecture Comparison Analysis Method

December 1, 2003 • technical report, by felix bachmann, christoph stoermer, chris verhoef.

The report describes SACAM, a method that provides rationale for an architecture selection process by comparing the fitness of architecture candidates for required systems.

Advanced Engineering Environments for Small Manufacturing Enterprises: Volume I

By joseph p. elm, john e. robert, steven j. fenves (national institute of standards and technology), ram d. sriram (national institute of standards and technology), young choi (chung-ang university).

This report provides an overview of AEE technologies, their benefits for subject matter experts, and the technical considerations for AEE adoption.

Real-Time Application Development with OSEK: A Review of the OSEK Standards

November 1, 2003 • technical note.

This 2003 report examines the OSEK OS, OSEK COM, and OSEK OIL specifications from the perspective of a real-time application developer.

Interpreting Capability Maturity Model Integration (CMMI) for Service Organizations' Systems Engineering and Integration Services Example

By mike phillips, julie a. walker, mary anne herndon (saic), robert moore (saic), laura west (saic).

This 2003 technical note presents one organization's interpretation of CMMI best practices for organizations that primarily provide services.

Architecture Reconstruction of J2EE Applications: Generating Views from the Module Viewtype

By liam o'brien, vorachat tamarree.

This report outlines the application of architecture reconstruction techniques to the Sun Microsystems' Duke's Bank system- Java2 Platform, Enterprise Edition/Enterprise JavaBeans (J2EE/EJB) application implemented mainly in Java.

Architecture Reconstruction Guidelines, Third Edition

November 1, 2003 • technical report, by liam o'brien, chris verhoef, rick kazman.

This report describes the process of architecture reconstruction using the Dali architecture reconstruction workbench.

Developing a Communication Strategy for a Research Institute

October 1, 2003 • white paper, by bill pollak, mike petock, anne humphreys.

This 2004 white paper presents a communication strategy that defines products and internal processes for optimizing communication with the Software Engineering Institute's (SEI) most important stakeholders.

CMMI Interpretive Guidance Project: Preliminary Report

October 1, 2003 • special report, by gian wemyss, kenneth smith, agapi svolou, michael d. konrad, mary beth chrissis, dennis goldenson.

The SEI collected data to learn more about how CMMI is being accepted by various organizations. This report describes those activities and includes summaries of the data collected.

Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results

By dennis goldenson, diane gibson.

This 2003 report demonstrates credible quantitative evidence that CMMI-based process improvement can result in better project performance and higher quality products.

Deriving Enterprise-Based Measures Using the Balanced Scorecard and Goal-Driven Measurement Techniques

October 1, 2003 • technical note, by matt fisher, wolfhart b. goethert.

This 2003 report describes the application of the balanced scorecard and goal-driven measurement methodologies to ways to measure an organization's health and performance.

A Template for Documenting Prediction-Enabled Component Technologies

This report proposes a template for documenting a PECT, and provides guidelines and a few examples to help PECT developers consolidate the broad range of information produced into the PECT …

Measures for Software Product Lines

By david zubrow, gary chastek.

This 2003 report characterizes the status of measurement associated with the operation of a software product line, suggests a small set of measures to support its management, and provides guidance …

State of the Practice of Computer Security Incident Response Teams (CSIRTs)

October 1, 2003 • technical report, by klaus-peter kossakowski, georgia killcrece, mark zajicek, robin ruefle.

In this 2003 report, the authors provide a study of the state of the practice of incident response, based on how CSIRTs around the world are operating.

Quality Attribute Workshops (QAWs), Third Edition

By robert j. ellison, charles weinstock, william wood, anthony j. lattanze, judith a. stafford, mario r. barbacci.

This report describes the newly revised QAW (Quality Attribute Workshop) and describes potential uses of the refined scenarios generated during it.

Analyzing and Specifying Reusable Security Requirements

September 1, 2003 • white paper.

A system cannot have high assurance if it has poor security, and thus, requirements for high assurance systems will logically include security requirement as well as availability, reliability, and robustness …

Requirements Engineering for Survivable Systems

September 1, 2003 • technical note.

In this 2003 report, Nancy Mead describes the state of requirements engineering for survivable systems.

A Life-Cycle View of Architecture Analysis and Design Methods

By robert nord, rick kazman, mark h. klein.

This report examines the architecture-centric analysis and design methods that were created at the SEI between 1993 and 2003.

DoD Experience with the C4ISR Architecture Framework

By sholom g. cohen, william wood.

This report discusses the context for using the C4ISRAF, the observations made during the interviews about its use, and the strengths and challenges of using it.

Predictable Assembly of Substation Automation Systems: An Experiment Report, Second Edition

September 1, 2003 • technical report, by judith a. stafford, magnus larsson, william wood, scott hissam, mark h. klein, james ivers, john j. hudak, linda m. northrop, gabriel moreno, daniel plakosh, kurt c. wallnau.

This 2003 report describes the results of an exploratory PECT prototype for substation automation, an application area in the domain of power generation, transmission, and management.

Product Line Analysis for Practitioners

By gary chastek, patrick donohoe.

This 2003 technical report describes the addition of development requirements to product line analysis.

The Team Software Process (TSP) in Practice: A Summary of Recent Results

By julia l. mullaney, noopur davis.

This 2003 report provides results and implementation data from projects and individuals that have adopted the TSP.

SEI Independent Research and Development Projects (FY 2003)

By edwin j. morris, suzanne miller, robert c. seacord, peter h. feiler, felix bachmann, mark h. klein, daniel plakosh, patrick r. place, anthony j. lattanze, david j. carney, sven dietrich, b. craig meyers, len bass, john mchugh.

This report describes the IR&D projects that were conducted during fiscal year 2003 (October 2002 through September 2003).

A Model Problem Approach to Measurement-to-Track Association

By b. craig meyers, grace lewis.

This report illustrates the use of model problems in the design of a system.

Preliminary Design of ArchE: A Software Architecture Design Assistant

By felix bachmann, mark h. klein, len bass.

This 2003 report presents a procedure for moving from a set of quality attribute scenarios to an architecture design that satisfies those scenarios.

Interpreting Capability Maturity Model Integration (CMMI) for COTS-Based Systems

By barbara tyson, cecilia albert, lisa brownsword.

This 2003 report shows that developing and maintaining COTS-based systems is more than selecting products and managing vendor relationships.

Identifying Commercial Off-the-Shelf (COTS) Product Risks: The COTS Usage Risk Evaluation

By edwin j. morris, patrick r. place, david j. carney.

This 2003 report describes the development of an approach to reduce the number of program failures attributable to COTS software: the COTS Usage Risk Evaluation (CURE).

Locality: A New Paradigm for Thinking About Normal Behavior and Outsider Threat

August 18, 2003 • white paper, by carrie gates, john mchugh.

In this paper, the authors describe how locality appears in many dimensions and applies to diverse mechanisms.

Building Relationships between Small Manufacturing Enterprises and Vendors: Findings from the TIDE Program

August 1, 2003 • technical note, by len estrin, john t. foreman.

This report presents findings to help vendors, VARs, and SMEs develop mutually beneficial and successful relationships.

Preserving Real Concurrency

July 1, 2003 • white paper, by kurt c. wallnau, james ivers.

In this 2003 whitepaper, the authors make use of information provided by components and extracted from static assembly topologies to faithfully model real concurrency. The result is more effective analysis.

Measurement and Analysis in Capability Maturity Model Integration Models and Software Process Improvement

By dennis goldenson, joe jarzombek (osd-nii), terry rout (griffith university).

This article reviews the content and rationale behind the new process area and describes how the ideas introduced there are further elaborated and evolved throughout capability maturity model integration models.

Organizational Interoperability Maturity Model for C2

A model of organizational interoperability is proposed in this paper, which extends the LISI model into the more abstract layers of C2 Support, that is, the C2 Frameworks, C2 Processes, …

Gobus Toolkit 3 Core - A Grid Service Container Framework

By derek gabbard.

The core infrastructure of Globus Toolkit 3 (GT3 Core) is based on the OGSI primitives and protocols. The main design goal has been to make the OGSI technology easy to …

International Liability Issues for Software Quality

July 1, 2003 • special report.

In this 2003 report, Nancy Mead focuses on international liability as it relates to information security for critical infrastructure applications.

Using the Architecture Tradeoff Analysis Method (ATAM) to Evaluate the Software Architecture for a Product Line of Avionics Systems: A Case Study

July 1, 2003 • technical note, by anthony j. lattanze, mario r. barbacci, paul c. clements, linda m. northrop, william wood.

This 2003 technical note describes an ATAM evaluation of the software architecture for an avionics system developed for the Technology Applications Program Office (TAPO) of the U.S. Army Special Operations …

CMM-Based Process Improvement and Schedule Deviation in Software Maintenance

By dennis goldenson, ho-won jung.

This study evaluates the predictive validity of the Capability Maturity Model (CMM) for Software (SW-CMM) as applied to software maintenance.

Predicting When Product Line Investment Pays

By sholom g. cohen.

This 2003 report defines key factors to consider in taking an incremental approach to fielding a product line.

What About Ada? The State of the Technology in 2003

This 2003 report documents a recent investigation which characterized the technical and programmatic risks in reusing significant quantities of legacy Ada code in a new system.

Documenting Software Architectures in an Agile World

By judith a. stafford, robert nord, paul c. clements, james ivers, reed little.

This report compares the Software Engineering Institute's Views and Beyond approach for documenting software architectures with the documentation philosophy embodied in agile software-development methods.

Third International Workshop on Adoption-Centric Software Engineering

June 1, 2003 • special report, by jens-holger jahnke (university of victoria), marin litoiu (ibm canada ltd.), hausi a. muller (university of victoria), margaret-anne storey (university of victoria), scott r. tilley (florida institute of technology), kenny wong (university of alberta), anke weber (university of victoria), robert balzer (teknowledge corporation), dennis b. smith.

This report contains a set of papers that focus on overcoming barriers to adopting research tools. The papers were presented at the Third International Workshop on Adoption-centric Software Engineering (ACSE).

Proceedings of the System of Systems Interoperability Workshop (February 2003)

June 1, 2003 • technical note, by linda levine, b. craig meyers, edwin j. morris, patrick r. place, daniel plakosh.

This report documents the model of interoperability presented and the findings from the System of Systems Interoperability Workshop, held in February 2003.

Snapshot of CCL: A Language for Predictable Assembly

By james ivers, kurt c. wallnau.

This 2003 report presents a snapshot of the construction and composition language (CCL) by examining a small example CCL specification.

The Software Engineering Institute's Second Workshop on Predictable Assembly: Landscape of Compositional Predictability

By judith a. stafford, scott hissam.

To further its work in predictable assembly focusing on compositional reasoning techniques, the Software Engineering Institute (SEI) held its second Predictable Assembly from Certifiable Components (PACC) Workshop on January 10-11, …

Interactions Among Techniques Addressing Quality Attributes

June 1, 2003 • technical report, by mario r. barbacci, hernan r. eguiluz.

This report provides software architects a chart for determining the relationships among techniques that promote different architectural qualities.

The Evolution of Product Line Assets

The focus of this 2003 technical report is how evolutionary changes affect the various types of assets in a software product line.

Fifth DoD Product Line Practice Workshop Report

By lawrence g. jones, william o'brien, matt fisher, john k. bergey, sholom g. cohen, linda m. northrop.

This 2003 document summarizes the presentations and discussions from the Fifth Department of Defense (DoD) Product Line Practice Workshop, held in August 2002.

Overcoming Barriers to Technology Adoption in Small Manufacturing Enterprises (SMEs)

By len estrin, john t. foreman, suzanne miller.

This 2003 report summarizes technology demonstrations, workforce development activities, and technology development efforts of the SEI's TIDE Program.

Integration of Computer-Aided Design and Finite Element Analysis Tools in a Small Manufacturing Enterprise

By john e. robert, joseph p. elm.

This 2003 report summarizes two case studies of tool integration activities at one small manufacturer.

Architecture, Design, Implementation

May 1, 2003 • white paper, by rick kazman.

Architecture, design, and implementation are used informally in partitioning software specifications into three coarse strata of abstraction. These strata are not well-defined in either research or practice, causing miscommunication and …

The SAE Avionics Architecture Description Language (AADL) Standard: A Basis for Model-Based Architecture-Driven Embedded Systems Engineering

By peter h. feiler, steve vestal (honeywell technology center).

The AADL standard will include a UML profile useful for avionics, space, automotive, robotics and other real-time concurrent processing domains including safety critical applications.

A Basis for an Assembly Process for COTS-Based Systems (APCS)

May 1, 2003 • technical report, by patrick r. place, david j. carney, tricia oberndorf.

This paper describes a generic process framework for developing software systems based on commercial off-the-shelf (COTS) products.

Case Study: Computer Supplier Evaluation Practices of the Parenteral Drug Association

By tricia oberndorf, david j. carney, harvey greenawalt, grigonis grigonis.

This case study describes the development of a method for evaluating computer and software suppliers for the pharmaceutical industry.

Architecture Reconstruction Case Study

April 1, 2003 • technical note.

This report outlines an architecture reconstruction carried out at the SEI on a software system called VANISH, which was developed for prototyping visualizations.

Volume III: A Technology for Predictable Assembly from Certifiable Components

April 1, 2003 • technical report.

This 2003 report, the final in a three-volume series on CBSE, identifies the key technical concepts of PACC, with an emphasis on the theory of prediction-enabled component technology (PECT).

DoD Architecture Framework and Software Architecture Workshop Report

March 1, 2003 • technical note, by lyn uzzle, john weiler, art krummenoehl, huei-wan ang, loring bernhardt, fatma dandashi, david emery, mario r. barbacci, steve palmquist, paul c. clements, william wood, sarah sheard.

This report summarizes the activities of the Workshop on the Department of the 2003 Defense Architecture Framework and Software Architecture workshop.

A Federation Object Model (FOM) Flexible Federate Framework

By reed little, regis dumond.

This 2003 report describes an approach to designing a domain framework that encapsulates expertise in developing an HLA federate by hiding RTI internal operations from the developer.

Application of Options Analysis for Reengineering in a Lead System Integrator Environment

This note describes the use of OAR to guide decision making on mining assets within an LSI (lead system integrator) context.

Relating the Team Software Process (TSP) to the Capability Maturity Model for Software (SW-CMM)

March 1, 2003 • technical report, by noopur davis, watts s. humphrey, jim mchale.

This 2003 report helps process professionals, process managers, project leaders, and organizational managers establish process improvement strategies and plans.

Deriving Architectural Tactics: A Step Toward Methodical Architectural Design

By len bass, felix bachmann, mark h. klein.

This 2003 technical report provides the status on the work being done by the SEI to understand the relationship between quality requirements and architectural design.

On the Suitability of Tcl/Tk for SYS

February 1, 2003 • technical note, by fred hansen.

This 2003 report reviews various websites and considers other factors that should influence the choice of Tcl/Tk as a tool for further development of SYS.

Rendering Tcl/Tk Windows as HTML

Tcl is a programming language having a Toolkit library that provides a standard set of GUI widgets. Since these are aimed at direct presentation via a window manager, Tcl/Tk applications …

Applying FSQ Engineering Foundations to Automated Calculation of Program Behavior

By richard c. linger (oak ridge national laboratory).

In this report, Richard Linger describes the application of function-theoretic mathematical foundations to the problem of program behavior calculation.

Quantifying the Value of Architecture Design Decisions: Lessons from the Field

January 1, 2003 • white paper, by rick kazman, mark h. klein, mike moore (nasa goddard space flight center), jai asundi.

This paper outlines experiences with using economic criteria to make architecture design decisions.

Simple Network Management Protocol (SNMP) Vulnerabilities Frequently Asked Questions (FAQ)

This tech tip provides advice about the Simple Network Management Protocol (SNMP).

2003 Tech Tip: W32/Blaster Recovery Tips

This tech tip contains information about recovery from W32/Blaster.

2002 CERT Incident Notes

December 31, 2002 • white paper.

This document contains the CERT incident notes from 2002.

2002 CERT Advisories

This document contains the CERT advisories from 2002.

Supporting the CANCEL Command Through Software Architecture

December 1, 2002 • technical note, by bonnie e. john, len bass.

This report, published in 2002, details the responsibilities that a system must implement to support command cancellation.

PECT Infrastructure: A Rough Sketch

By james ivers, scott hissam.

This 2002 paper investigates the nature of PECT infrastructures, summarizes the activities that a PECT infrastructure should support, and proposes a design for the tools that make up a PECT …

Rules of Thumb for the Use of COTS Products

December 1, 2002 • technical report, by michele motsko, ellen-jane pairo, james smith, tricia oberndorf.

This 2002 report provides information to help guide decisions about when COTS products are an appropriate solution,and when they are not.

The Internal Consistency of Key Process Areas in the Capability Maturity Model (CMM) for Software (SW-CMM)

This report examines the dimensions underlying the maturity construct in the Capability Maturity Model (CMM) for Software (SW-CMM) and then estimates the internal consistency (reliability) of each dimension.

Network Survivability Analysis Using Easel

By alan m. christie.

In this 2002 report, Alan Christie describes the results of exploring the use of simulation in examining internet survivability.

CSIRT Services

November 25, 2002 • white paper.

In this paper, the authors define computer security incident response team (CSIRT) services.

Tracking and Tracing Cyber-Attacks: Technical Challenges and Global Policy Issues

November 1, 2002 • special report.

In this report, Howard Lipson describes the technical challenges and global policy issues related to cyber attacks.

Evolutionary Process for Integrating COTS-Based Systems (EPIC) Building, Fielding, and Supporting Commercial-off-the-Shelf (COTS) Based Solutions

November 1, 2002 • technical report, by david bentley (u.s. air force), thomas bono (mitre), deborah pruitt (mitre), edwin j. morris, lisa brownsword, cecilia albert.

This 2002 document is the first release of a full description of the EPIC framework along with its activities and artifacts.

Salion, Inc.: A Software Product Line Case Study

By paul c. clements, linda m. northrop.

This 2002 case study outlines the efforts of Salion, Inc., an enterprise software company providing Revenue Acquisition Management solutions tailored to the unique needs of automotive suppliers.

2002 Tech Tip: Problems With The FTP PORT Command or Why You Don't Want Just Any Port in a Storm

October 16, 2002 • white paper.

This tech tip contains discussion about problems with the FTP PORT command.

SEI Architecture Analysis Techniques and When to Use Them

October 1, 2002 • technical note, by mario r. barbacci.

When analyzing system and software architectures, the Quality Attribute Workshop (QAW) and the Architecture Tradeoff Analysis Method (ATAM) can be used in combination to obtain early and continuous benefits.

Model-Based Verification: Abstraction Guidelines

By santiago comella-dorda, david p. gluch, charles weinstock, john j. hudak, grace lewis.

This 2002 report presents abstraction techniques that can be used to build essential models of system behavior in the context of MBV and details a methodology for creating state machine …

Using CMMI to Improve Earned Value Management

By paul solomon.

For organizations using Earned Value Management (EVM) or that plan to implement EVM during Capability Maturity Model Integration (CMMI) implementation, this technical note provides guidance for cost-effective process improvement and …

SEI Independent Research and Development Projects

October 1, 2002 • technical report, by linda levine, dennis b. smith, richard c. linger (oak ridge national laboratory), eileen c. forrester, steve cross, ira monarch, thomas a. longstaff, scott hissam, kurt c. wallnau, rick kazman.

This report describes the IR&D projects that were conducted during fiscal year 2002 (October 2001 through September 2002).

Life-Cycle Models for Survivable Systems

By carol sledge, nancy r. mead, richard c. linger (oak ridge national laboratory), howard f. lipson, john mchugh.

In this 2002 report, the authors describe a software development life-cycle model for survivability and illustrate techniques to support survivability goals.

Trustworthy Refinement Through Intrusion-Aware Design

By robert j. ellison, andrew p. moore.

This document has been superseded by CMU/SEI-2003-TR-002.

Trustworthy Refinement Through Intrusion-Aware Design (TRIAD)

In this report, the authors demonstrate the application of TRIAD to refining a survivability strategy for a business that sells products on the internet.

Using the Technology Readiness Levels Scale to Support Technology Management in the DoD's ATD/STO Environments (A Findings and Recommendations Report Conducted for Army CECOM)

September 1, 2002 • special report, by caroline graettinger, jeannine siviy, peter j. van syckle, robert j. schenk, suzanne miller.

This report describes the results of the SEI study of the feasibility of (a) using TRLs in STO technology screening, (b) developing or acquiring a TRL tool, and (c) implementing …

Product Line State of the Practice Report

September 1, 2002 • technical note.

This 2002 report outlines the state of software product line practice in industry. The report blends a case study with the results of a product line questionnaire that was sent …

Successful Product Line Development and Sustainment: A DoD Case Study

By sholom g. cohen, ed dunn (naval undersea warfare center), albert soule.

This case study describes the Naval Undersea Warfare Center's (NWUC) efforts to sustain and support the evolution of RangeWare, a software product line asset base used to test range operations.

A Basis for Composition Language CL

By nishant sinha, james ivers, kurt c. wallnau.

This report describes the composition language CL and its rudimentary graphical syntax, and defines and illustrates the compositional semantics for CL using Hoare's CSP.

An Application of an Iterative Approach to DoD Software Migration Planning

By liam o'brien, dennis b. smith, john k. bergey.

This 2002 report outlines the early results of an approach to support software migration planning that focused on deriving actionable plans for focus areas that were identified in an initial …

Product Line Production Planning for the Home Integration System Example

By gary chastek, patrick donohoe, john mcgregor.

This 2002 technical note examines the significant characteristics of the production plans of three hypothetical organizations that create product lines of home integration systems.

Discovery Colloquium: Quality Software Development @ Internet Speed

September 1, 2002 • technical report, by richard baskerville, jan pries-heje, balasubramaniam ramesh, sandra slaughter, linda levine.

This report presents the data collected during a 2001 colloquium to explore issues associated with developing quality software at Internet speed.

Making Architecture Design Decisions: An Economic Approach

By rick kazman, mark h. klein, jai asundi.

This report describes the improvements to the CBAM (Cost Benefit Analysis Method) and provides a pilot case study conducted with NASA.

2002 Tech Tip: Securing an Internet Name Server

August 1, 2002 • white paper, by brian king, allen d. householder.

This document discusses name server security and focuses on BIND, which is the most commonly used software for DNS servers.

A Report on the May 2002 CMMI Workshop

August 1, 2002 • special report, by mark c. paulk, bill curtis (cast research labs), michael d. konrad, mary beth chrissis.

This report summarizes the results of the CMMI Workshop held on May 7-8, 2002.

PAMD: Developing a Plug-In Architecture for Palm OS-Powered Devices Using Software Engineering

August 1, 2002 • technical note, by hernan r. eguiluz, venkat govi, you jung kim, adrian sia.

This 2002 technical note describes a plug-in architecture for Palm Operating System devices developed by the authors, a team of graduate students from the CMU Master of Software Engineering program.

Plug-In Architecture for Mobile Devices

By madhu keshavamurthy, jung soo kim, mona li, vichaya sagetong.

This 2002 report describes plug-in architecture for mobile devices (PAMD), an architectural specification that extends the function of applications in mobile devices.

Model-Based Verification: An Engineering Practice

August 1, 2002 • technical report, by julie a. walker, david p. gluch, santiago comella-dorda, charles weinstock, john j. hudak, grace lewis, david zubrow.

This 2002 report summarizes MBV and outlines the responsibilities of engineers engaged in Model-Based Verification.

Software Architecture Reconstruction: Practice Needs and Current Approaches

By christoph stoermer, chris verhoef, liam o'brien.

This report presents the concept of practice scenarios for architecture reconstruction.

Illuminating the Fundamental Contributors to Software Architecture Quality

This 2002 report presents the basic concepts of analysis models for two quality attributes-modifiability and performance, identifies a collection of tactics that can be used to control responses within those …

CMMI for Software Engineering, Version 1.1, Continuous Representation (CMMI-SW, V1.1, Continuous)

This CMMI model is designed to help organizations improve their product and service development, acquisition, and maintenance processes.

CMMI for Software Engineering, Version 1.1, Staged Representation (CMMI-SW, V1.1, Staged)

Home computer security, july 8, 2002 • white paper.

This 2002 document provides tips for securing your home computer.

Reeducation to Expand the Software Engineering Workforce: Successful Industry/University Collaborations

July 1, 2002 • special report, by nancy r. mead, heidi j. ellis, stephen b. seidman, ana m. moreno (universidad politecnica de madrid).

In this 2002 report, the authors describe a study of reeducating non-software professionals and practitioners to become software engineers.

Replaceable Components and the Service Provider Interface

July 1, 2002 • technical note, by lutz wrage, robert c. seacord.

This 2002 report considers the motivation for using replaceable components and defines the requirements of replaceable component models.

Software Process Improvement and Product Line Practice: CMMI and the Framework for Software Product Line Practice

By lawrence g. jones, albert soule.

This 2002 report explores the relationship between software product line practice, as defined by the Framework for Software Product Line Practice, and software engineering process discipline, as defined by the …

Evolutionary Process for Integrating COTS-Based Systems (EPIC): An Overview

July 1, 2002 • technical report, by lisa brownsword, edwin j. morris, cecilia albert, deborah pruitt (mitre), david bentley (u.s. air force), thomas bono (mitre).

This document is the first release of an overview of the EPIC framework along with its activities and artifacts.

Distributed Software: From Component Model to Software Architecture

June 1, 2002 • white paper.

This 2002 whitepaper presents a component model for redeveloping software.

A Software Product Line Vision for Defense Acquisition

June 1, 2002 • technical note.

This report presents a vision for software product lines as an acquisition focus and suggests extensions to current Department of Defense policy and practices to increase the awareness of and …

Use of the Architecture Tradeoff Analysis Method (ATAM) in Source Selection of Software-Intensive Systems

By matt fisher, john k. bergey, lawrence g. jones.

This report explains the role of software architecture evaluation in a source selection and describes the contractual elements that are needed to support its use.

Use of Quality Attribute Workshops (QAWs) in Source Selection for a DoD System Acquisition: A Case Study

By john k. bergey, william wood.

This case study outlines how a DoD organization used architecture analysis and evaluation in a major system acquisition to reduce program risk.

Documenting Software Architecture: Documenting Interfaces

By reed little, james ivers, paul c. clements, felix bachmann, robert nord, judith a. stafford, david garlan, len bass.

This report provides guidance for documenting the interfaces to software elements.

Flow-Service-Quality (FSQ) Engineering: Foundations for Network System Analysis and Development

By mark pleszkoch, gwendolyn h. walton, alan r. hevner (university of south florida), richard c. linger (oak ridge national laboratory).

In this 2002 report, the authors describe Flow-Service-Quality engineering, an emerging technology for management, acquisition, and more.

Guidelines for Developing a Product Line Production Plan

June 1, 2002 • technical report, by john mcgregor, gary chastek.

This 2002 technical report provides guidance for creating, using, and evaluating a production plan, which is a description of how core assets are to be used to develop a product …

Quality Attribute Workshops, 2nd Edition

By judith a. stafford, anthony j. lattanze, mario r. barbacci, william wood, robert j. ellison, charles weinstock.

This report clarifies the context in which a QAW (Quality Attribute Workshop) is applicable, provides a rationale for developing the process and describes it in detail, and concludes with a …

Using EVMS with COTS-Based Systems

By carol sledge, mary jo staley, tricia oberndorf.

This 2002 report focuses is on the use of Earned Value in the context of a COTS-Based System (CBS).

Packaging and Deploying Predictable Assembly

May 31, 2002 • white paper, by scott hissam, gabriel moreno, kurt c. wallnau, judith a. stafford.

This paper describes prediction-enabled component technology (PECT), which integrates component technology with analysis models.

Foundations for Survivable Systems Engineering

May 20, 2002 • white paper, by andrew p. moore, nancy r. mead, robert j. ellison, richard c. linger (oak ridge national laboratory).

In this paper, the authors describe their efforts to perform risk assessment and analyze and design robust survivable systems.

Issues in Predicting the Reliability of Components

May 1, 2002 • white paper, by john mcgregor, judith a. stafford.

This whitepaper presents the design of an experiment that forms the basis of a reliability prediction-enabled component technology (PECT). It also discusses aspects of models that need to be adapted …

Statistical Models for Empirical Component Properties and Assembly-Level Property Predictions: Toward Standard Labeling

April 30, 2002 • white paper, by kurt c. wallnau, gabriel moreno, scott hissam.

This paper identifies statistical models that could form a basis for standard industry labels for component properties and prediction theories.

Is Third Party Certification Necessary?

April 1, 2002 • white paper, by kurt c. wallnau, judith a. stafford.

This paper describes a model for the component marketplace, along with two possible forms that the model may take in order to establish trust among participants in component-based design.

The Potential for Synergy Between Certification and Insurance

By mary shaw, p. luo li, kevin stolarick, kurt c. wallnau.

Because of their affordability and availability, reusable software components have long been a tantalizing IT investment, but they are not without their risks. Certification and insurance are potential approaches to …

Interpreting Capability Maturity Model Integration (CMMI) for Operational Organizations

April 1, 2002 • technical note, by brian p. gallagher.

This 2002 report details how operational organizations that perform a variety of missions can benefit from the concepts in CMMI to improve the processes and effectiveness of mission operations.

MAP and OAR Methods: Techniques for Developing Core Assets for Software Product Lines from Existing Assets

By dennis b. smith, liam o'brien.

This 2002 report describes the MAP and OAR methods, the activities that each involves, and examples of applying them.

SCAMPI V1.1 Use in Supplier Selection and Contract Process Monitoring

By rick barbour, thomas bernard.

A newer document covers this topic in more detail. If you want to see the newer document, see Standard CMMI Appraisal Method for Process Improvement (SCAMPI) A, Version 1.3: Method …

Experiences in Architecture Reconstruction at Nokia

March 1, 2002 • technical note, by liam o'brien.

This 2002 report outlines details of past and current architecture reconstruction work on several systems at Nokia.

Software Acquisition Capability Maturity Model (SA-CMM) Version 1.03

March 1, 2002 • technical report, by jack cooper, matt fisher.

This 2002 version of the SA-CMM incorporates change requests that have been received, as well as the results of lessons learned from conducting appraisals and from the use of Version …

CMMI for Systems Engineering/Software Engineering/Integrated Product and Process Development/Supplier Sourcing, Version 1.1, Continuous Representation (CMMI-SE/SW/IPPD/SS, V1.1, Continuous)

Cmmi for systems engineering/software engineering/integrated product and process development/supplier sourcing, version 1.1, staged representation (cmmi-se/sw/ippd/ss, v1.1, staged), the road to cmmi: results of the first technology transition workshop, february 1, 2002 • technical report, by caroline graettinger, lynn carter, shelly zasadni, gian wemyss, mac patrick.

This 2002 paper reports the findings of the First Technology Transition Workshop, held in November 2001.

2002 Tech Tip: A Brief Tour of the Simple Network Management Protocol

January 1, 2002 • white paper.

In this 2002 tech tip, the authors provide a brief overview of the Simple Network Management Protocol (SNMP).

2002 Tech Tip: Email Bombing and Spamming

This CERT Division tech tip describes email bombing and spamming.

2002 Tech Tip: Spoofed/Forged Email

This tech tip contains information about spoofed and forged email.

2002 Tech Tip: Securing Your Web Browser

This tech tip contains ways to secure your web browser.

The 2001 High Maturity Workshop

January 1, 2002 • special report, by mary beth chrissis, mark c. paulk.

This report contains overviews of more than 30 high maturity organizations and the various working group reports from the workshop.

Documenting Software Architecture: Documenting Behavior

January 1, 2002 • technical note, by reed little, james ivers, paul c. clements, robert nord, felix bachmann, judith a. stafford, len bass, david garlan.

This report describes ways to document the behavior of systems, subsystems, and components of software architecture.

Model-Based Verification: Guidelines for Generating Expected Properties

By santiago comella-dorda, david p. gluch, grace lewis, john j. hudak, charles weinstock.

This report presents a basic set of guidelines to facilitate the generation of expected properties in the context of Model-Based Verification.

2001 CERT Incident Notes

December 31, 2001 • white paper.

This document contains the CERT incident notes from 2001.

2001 CERT Advisories

This document contains the CERT advisories from 2001.

Using the Architecture Tradeoff Analysis Method to Evaluate a Wargame Simulation System: A Case Study

December 1, 2001 • technical note, by lawrence g. jones, anthony j. lattanze.

This report describes the application of the ATAM (Architecture Tradeoff Analysis Method) to a major wargaming simulation system.

Model-Based Verification: Analysis Guidelines

By john j. hudak, charles weinstock, grace lewis, santiago comella-dorda, david p. gluch.

This technical note provides guidance for the analysis activity that occurs during the interpretation of results produced by model-checking tools.

Can We Ever Build Survivable Systems from COTS Components?

By howard f. lipson, andrew p. moore, nancy r. mead.

In this 2001 report, the authors describe a risk-mitigation framework for deciding when and how COTS components can be used to build survivable systems.

A Framework for the Specification of Acquisition Models

December 1, 2001 • technical report, by b. craig meyers, tricia oberndorf.

This special report provides a bibliography of books, articles, and other literature concerning the PSP and TSP methodologies.

OCTAVE Criteria, Version 2.0

This 2001 report defines a general approach for evaluating and managing information security risks.

Testing a Software Product Line

This report expands on the testing practice area described by Clements and Northrop. Test-related activities that can be used to form the test process for a product line organization are …

Appraisal Requirements for CMMI, Version 1.1 (ARC, V1.1)

This report defines the ARC V1.1 requirements that are considered to be essential to appraisal methods intended for use with CMMI models ARC, V1.1.

Using Economic Considerations to Choose Among Architecture Design Alternatives

By jai asundi, mark h. klein, rick kazman.

The SEI developed the CBAM (Cost Benefit Analysis Method), which incorporates the costs and benefits of architectural design decisions and provides an effective means of making such decisions. This paper …

CMMI for Systems Engineering/Software Engineering, Version 1.1, Continuous Representation (CMMI-SE/SW, V1.1, Continuous)

Cmmi for systems engineering/software engineering, version 1.1, staged representation (cmmi-se/sw, v1.1, staged), cmmi for systems engineering/software engineering/integrated product and process development, version 1.1, continuous representation (cmmi-se/sw/ippd, v1.1, continuous), cmmi for systems engineering/software engineering/integrated product and process development, version 1.1, staged representation (cmmi-se/sw/ippd, v1.1, staged), army workshop on lessons learned from software upgrade programs, november 1, 2001 • special report, by john k. bergey, matt fisher, caroline graettinger, fred hansen, halbert stevens, ray obenza, dennis b. smith, william anderson.

This report summarizes the results of the SEI-sponsored Software Upgrade Workshop for Legacy Systems at the Redstone Arsenal on June 5-7, 2001.

Experiences in Implementing Measurement Programs

November 1, 2001 • technical note, by wolfhart b. goethert, will hayes.

This 2001 report describes lessons learned at several organizations that have implemented measurement programs using the Goal-Driven Software Measurement methodology.

Perspectives on Open Source Software

November 1, 2001 • technical report, by jai asundi, scott hissam, daniel plakosh, charles weinstock.

This 2001 report summarizes the results of a study of the benefits of pitfalls of using open source software.

Analysis of CMM-Based Appraisal for Internal Process Improvement (CBA IPI) Assessment Feedback

By marie baker, donna k. dunaway, michele falce.

This report updates the analysis of feedback from users of the CMM-Based Appraisal for Internal Process Improvement (CBA IPI) method.

Packaging Predictable Assembly with Prediction-Enabled Component Technology

By gabriel moreno, scott hissam, kurt c. wallnau, judith a. stafford.

This report describes the major structures of a PECT. It then discusses the means of validating the predictive powers of a PECT so that consumers may obtain measurably bounded trust …

CMM-Based Appraisal for Internal Process Improvement (CBA IPI) Version 1.2 Method Description

By donna k. dunaway, steve masters.

This report provides a high-level overview of the CBA IPI V1.2 assessment method and is an update to the CBA IPI V1.1 .

2001 Tech Tip: Trends in Denial of Service Attack Technology

October 1, 2001 • white paper, by kevin houle, george weaver.

In this 2001 paper, the authors highlight trends in the deployment, use, and impact of DoS attack technology based on intruder activity and attack tools.

2001 Tech Tip: Managing the Threat of Denial-of-Service Attacks

By allen d. householder, art manion, linda pesante.

In this 2001 paper, the authors describe the then-current situation regarding denial-of-service (DOS) attacks and ways of addressing the problem.

Quality Attribute Design Primitives and the Attribute Driven Design Method

This paper discusses the understanding of quality attributes and their application to the design of a software architecture.

Framework Document: Model-Based Verification Pilot Study

October 1, 2001 • special report, by julie a. walker, robert janousek, david p. gluch, john j. hudak, charles weinstock, david zubrow.

This 2001 document describes the processes, activities, artifacts, and deliverables associated with an Engineering Practice Investigation of MBV.

Architectural Refinement for the Design of Survivable Systems

October 1, 2001 • technical note.

This paper describes a process for systematically refining an enterprise system architecture to resist, recognize, and recover from deliberate, malicious attacks by applying reusable design primitives that help ensure the …

Model-Based Verification: Claim Creation Guidelines

By grace lewis, john j. hudak, charles weinstock, santiago comella-dorda, david p. gluch.

This 2001 report describes a pattern-based approach to facilitate claim generation.

Model-Based Verification: Scope, Formalism, and Perspective Guidelines

By julie a. walker, david p. gluch, santiago comella-dorda, charles weinstock, john j. hudak, grace lewis.

This report provides guidance for defining the scope, formalism, and perspective for applying MBV, a systematic approach to finding defects in software requirements, designs, or code.

Analyzing Enterprise JavaBeans Systems Using Quality Attribute Design Primitives

By mark h. klein, anna liu, len bass.

This report introduces the notion of quality attribute design primitives, which are architectural building blocks that target the achievement of one or sometimes several quality attribute requirements.

Applicability of General Scenarios to the Architecture Tradeoff Analysis Method

October 1, 2001 • technical report, by len bass, gabriel moreno, mark h. klein.

In this report, we compare the scenarios elicited from five ATAM (Architecture Tradeoff Analysis Method) evaluations with the scenarios used to characterize the quality attributes.

Fourth DoD Product Line Practice Workshop Report

By dennis b. smith, matt fisher, john k. bergey, lawrence g. jones, grady campbell, albert soule, william o'brien, robert w. krut, jr., sholom g. cohen, linda m. northrop.

The report summarizes the workshop presentations and discussions from the Fourth Department of Defense (DoD) Software Product Line Practice Workshop, held in March 2001.

An Enterprise Information System Data Architecture Guide

By daniel plakosh, patrick r. place, grace lewis, robert c. seacord, santiago comella-dorda.

This report describes a sample data architecture in terms of a collection of generic architectural patterns that define and constrain how data is managed in a system that uses the …

OCTAVE Catalog of Practices, Version 2.0

By christopher j. alberts, julia h. allen, audrey j. dorofee.

In this report, the authors describe OCTAVE practices, which enable organizations to identify risks and mitigate them.

Foundations for Survivable System Development: Service Traces, Intrusion Traces, and Evaluation Models

By andrew p. moore, richard c. linger (oak ridge national laboratory).

This 2001 paper describes initial work in the foundations stage for survivability specification and intrusion specification, as well as survivability evaluation models that draw upon both of these areas.

Control Channel Toolkit: A Software Product Line Case Study

By patrick donohoe, paul c. clements, sholom g. cohen, linda m. northrop.

This 2001 report is a case study of the Control Channel Toolkit (CCT), a software asset base for a software product line of ground-based spacecraft command and control systems built …

Use of the ATAM in the Acquisition of Software-Intensive Systems

September 1, 2001 • technical note, by john k. bergey, matt fisher.

This report discusses the role of software architecture evaluations in a system acquisition and describes the contractual elements that are needed to accommodate architecture evaluations in an acquisition. The report …

Fifth Product Line Practice Workshop Report

September 1, 2001 • technical report, by kyo c. kang (pohang university of science and technology), patrick donohoe, paul c. clements, linda m. northrop, john mcgregor.

This report synthesizes the workshop presentations and discussions from the Fifth Software Engineering Institute Product Line Practice Workshop, held in December 2000.

Proceedings of the Real-Time Systems Engineering Workshop

August 1, 2001 • special report, by peter h. feiler, theodore f. marz, b. craig meyers.

This report presents the results of a workshop on real-time systems engineering. The workshop was held as part of the SEI Symposium in Washington, DC, during September 2000.

Documenting Software Architectures: Organization of Documentation Package

August 1, 2001 • technical note, by len bass, david garlan, judith a. stafford, paul c. clements, felix bachmann, robert nord, james ivers, reed little.

This comprehensive handbook outlines how to produce high-quality documentation for software architectures.

DoD Software Migration Planning

By dennis b. smith, john k. bergey, liam o'brien.

This 2001 report describes migration planning, identifies influencing factors, outlines a set of migration planning activities, and offers a set of guidelines for the migration planning process.

Beyond the Black Box: A Case Study in C to Java Conversion and Product Extensibility

By pisey huy, ming-hsun liu, grace lewis.

This case study describes the experience of converting and enhancing NDBS 1.0, a programmatic library to extract private keys and digital certificates from a Netscape database written in C and …

Maintaining Transactional Context: A Model Problem

August 1, 2001 • technical report, by patrick r. place, daniel plakosh, robert c. seacord, grace lewis, santiago comella-dorda.

This 2001 report outlines a model problem constructed to verify the feasibility of building a mechanism to modernize a legacy system.

Architecture Reconstruction Guidelines

By rick kazman, liam o'brien, chris verhoef, incremental modernization for legacy systems, july 1, 2001 • technical note, by daniel plakosh, robert c. seacord, patrick r. place, grace lewis, santiago comella-dorda.

This 2001 report shows an objective technique for developing an incremental code-migration strategy for large legacy Common Business-Oriented Language (COBOL) systems.

Architecture Reconstruction to Support a Product Line Effort: Case Study

This report describes the architecture reconstruction process that was followed when the SEI performed architecture reconstructions on three small automotive motor systems.

Legacy System Modernization Strategies

July 1, 2001 • technical report, by santiago comella-dorda, robert c. seacord, grace lewis, patrick r. place, daniel plakosh.

This 2001 report discusses alternative development approaches for incrementally modernizing legacy systems.

Real-Time Systems Engineering: Lessons Learned from Independent Technical Assessments

June 1, 2001 • technical note, by theodore f. marz, daniel plakosh.

This 2001 paper contains observations, recurring themes, trends, and lessons learned about systems development as derived from real-time/mission-critical programs that have been reviewed over the last three years.

Options Analysis for Reengineering (OAR): A Method for Mining Legacy Assets

By liam o'brien, john k. bergey, dennis b. smith.

OAR is a systematic, architecture-centric, decision-making method for mining existing components for a product line or new software architecture.

Managing Variability in Software Architectures

May 1, 2001 • white paper, by felix bachmann, len bass.

This paper presents experience with explicitly managing variability within a software architecture.

Quality Attribute Workshops

May 1, 2001 • technical report, by mario r. barbacci, judith a. stafford, charles weinstock, robert j. ellison, william wood.

This report describes the QAW (Quality Attribute Workshop) approach, which is a method for evaluating a software-intensive system architecture during the acquisition phase of major programs.

Spiral Development and Evolutionary Acquisition

May 1, 2001 • special report.

DoD Instruction 5000.2 introduced innovations throughout the acquisition cycle. To address this, a workshop was held September 2000. This 2001 report summarizes the workshop and presents its recommendations.

SEI Workshop on Software Architecture Representation, 16-17 January 2001

This report summarizes the discussions from the 2001 Architecture Representation Workshop, where five leading software architects and practitioners were invited to discuss aspects of the architecture representation with senior members …

Case Study: Building and Communicating a Business Case for a DoD Product Line

April 1, 2001 • technical note.

This case study describes a DoD weapon system development effort and compares the current way of developing software systems to the product line approach.

Developing a Product Line Acquisition Strategy for a DoD Organization: A Case Study

By john k. bergey, wolfhart b. goethert.

This 2001 report describes the approach a DoD organization used to develop alternative acquisition strategies and analyzes the pros and cons of each.

Product Line Analysis: A Practical Introduction

April 1, 2001 • technical report, by kyo c. kang (pohang university of science and technology), steffen thiel (robert bosch gmbh), gary chastek, patrick donohoe.

This 2001 report provides a practical introduction to product line requirements modeling. The report describes product line analysis in the context of product line development and shows how a requirements …

Guidance on Commercial-Based and Open Systems for Program Managers

April 1, 2001 • special report.

This 2001 document discusses various risks and provides guidance that may be used to mitigate those risks.

Attack Modeling for Information Security and Survivability

March 1, 2001 • technical note, by richard c. linger (oak ridge national laboratory), robert j. ellison, andrew p. moore.

This technical note describes and illustrates an approach for documenting attack information in a structured and reusable form.

Achieving Usability Through Software Architecture

March 1, 2001 • technical report, by len bass, jesse kates, bonnie e. john.

This paper outlines an approach to improving the usability of software systems by means of software architectural decisions.

K-BACEE: A Knowledge-Based Automated Component Ensemble Evaluation Tool

February 1, 2001 • technical note, by somjai boonsiri, robert c. seacord, dave mundie.

This 2001 report describes an automated approach to evaluating ensembles of components within the context of a system requirements specification.

2001 Tech Tip: Using PGP to Verify Digital Signatures

January 1, 2001 • white paper, by shawn hernan, linda pesante.

This white paper discusses how to use Pretty Good Privacy (PGP) to verify digital signatures.

2001 Tech Tip: Cross-Site Scripting Vulnerabilities

By jason rafail.

In this paper, Jason Rafail discusses cross-site scripting vulnerabilities.

Before You Connect a New Computer to the Internet

This tech tip provides advice about connecting a new computer to the Internet.

Topic modeling in software engineering research

Open access
Published: 06 September 2021
Volume 26 , article number 120 , ( 2021 )

Cite this article

You have full access to this open access article

Camila Costa Silva ORCID: orcid.org/0000-0002-3690-1711 1 ,
Matthias Galster ORCID: orcid.org/0000-0003-3491-1833 1 &
Fabian Gilson ORCID: orcid.org/0000-0002-1465-3315 1

6930 Accesses

31 Citations

1 Altmetric

Explore all metrics

Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic “topics” (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been used to analyze textual data in empirical studies (e.g., to find out what developers talk about online), but also to build new techniques to support software engineering tasks (e.g., to support source code comprehension). Topic modeling needs to be applied carefully (e.g., depending on the type of textual data analyzed and modeling parameters). Our study aims at describing how topic modeling has been applied in software engineering research with a focus on four aspects: (1) which topic models and modeling techniques have been applied, (2) which textual inputs have been used for topic modeling, (3) how textual data was “prepared” (i.e., pre-processed) for topic modeling, and (4) how generated topics (i.e., word clusters) were named to give them a human-understandable meaning. We analyzed topic modeling as applied in 111 papers from ten highly-ranked software engineering venues (five journals and five conferences) published between 2009 and 2020. We found that (1) LDA and LDA-based techniques are the most frequent topic modeling techniques, (2) developer communication and bug reports have been modelled most, (3) data pre-processing and modeling parameters vary quite a bit and are often vaguely reported, and (4) manual topic naming (such as deducting names based on frequent words in a topic) is common.

A survey on the use of topic models when mining software repositories

Tse-Hsun Chen, Stephen W. Thomas & Ahmed E. Hassan

Latent Dirichlet Allocation (LDA) Based on Automated Bug Severity Prediction Model

Avoid common mistakes on your manuscript.

1 Introduction

Text mining is about searching, extracting and processing text to provide meaningful insights from the text based on a certain goal. Techniques for text mining include natural language processing (NLP) to process, search and understand the structure of text (e.g., part-of-speech tagging), web mining to discover information resources on the web (e.g., web crawling), and information extraction to extract structured information from unstructured text and relationships between pieces of information (e.g., co-reference, entity extraction) (Miner et al. 2012 ). Text mining has been widely used in software engineering research (Bi et al. 2018 ), for example, to uncover architectural design decisions in developer communication (Soliman et al. 2016 ) or to link software artifacts to source code (Asuncion et al. 2010 ).

Topic modeling is a text mining and concept extraction method that extracts topics (i.e., coherent word clusters) from large corpora of textual documents to discovery hidden semantic structures in text (Miner et al. 2012 ). An advantage of topic modeling over other techniques is that it helps analyzing long texts (Treude and Wagner 2019 ; Miner et al. 2012 ), creates clusters as “topics” (rather than individual words) and is unsupervised (Miner et al. 2012 ).

Topic modeling has become popular in software engineering research (Sun et al. 2016 ; Chen et al. 2016 ). For example, Sun et al. ( 2016 ) found that topic modeling had been used to support source code comprehension, feature location and defect prediction. Additionally, Chen et al. ( 2016 ) found that many repository mining studies apply topic modeling to textual data such as source code and log messages to recommend code refactoring (Bavota et al. 2014b ) or to localize bugs (Lukins et al. 2010 ).

Probabilistic topic models such as Latent Semantic Indexing (LSI) (Deerwester et al. 1990 ) and Latent Dirichlet Allocation (LDA) (Blei et al. 2003b ) discover topics in a corpus of textual documents, using the statistical properties of word frequencies and co-occurrences (Lin et al. 2014 ). However, Agrawal et al. ( 2018 ) warn about systematic errors in the analysis of LDA topic models that limit the validity of topics. Lin et al. ( 2014 ) also advise that classical topic models usually generate sub-optimal topics when applied “as is” to small amounts or short text documents.

Considering the limitations of topic modeling techniques and topic models on the one hand and their potential usefulness in software engineering on the other hand, our goal is to describe how topic modeling has been applied in software engineering research. In detail, we explore the following research questions:

RQ1. Which topic modeling techniques have been used and for what purpose? There are different topic modeling techniques (see Section 2 ), each with their own limitations and constraints (Chen et al. 2016 ). This RQ aims at understanding which topic modeling techniques have been used (e.g., LDA, LSI) and for what purpose studies applied such techniques (e.g., to support software maintenance tasks). Furthermore, we analyze the types of contributions in studies that used topic modeling (e.g., a new approach as a solution proposal, or an exploratory study).

RQ2. What are the inputs into topic modeling? Topic modeling techniques accept different types of textual documents and require the configuration of parameters (see Section 2.1 ). Carefully choosing parameters (such as the number of topics to be generated) is essential for obtaining valuable and reliable topics (Agrawal et al. 2018 ; Treude and Wagner 2019 ). This RQ aims at analysing types of textual data (e.g., source code), actual documents (e.g., a Java class or an individual Java method) and configured parameters used for topic modeling to address software engineering problems.

RQ3: How are data pre-processed for topic modeling? Topic modeling requires that the analyzed text is pre-processed (e.g., by removing stop words) to improve the quality of the produced output (Aggarwal and Zhai 2012 ; Bi et al. 2018 ). This RQ aims at analysing how previous studies pre-processed textual data for topic modeling, including the steps for cleaning and transforming text. This will help us understand if there are specific pre-processing steps for a certain topic modeling technique or types of textual data.

RQ4. How are generated topics named? This RQ aims at analyzing if and how topics (word clusters) were named in studies. Giving meaningful names to topics may be difficult but may be required to help humans comprehend topics. For example, naming topics can provide a high-level view on topics discussed by developers in Stack Overflow (a Q&A website) (Barua et al. 2014 ) or by end mobile app users in tweets (Mezouar et al. 2018 ). Analysts (e.g., developers interested in what topics are discussed on Stack Overflow or app reviews) can then look at the name of the topic (i.e., its “label”) rather than the cluster of words. These labels or names must capture the overarching meaning of all words in a topic. We describe different approaches to naming topics generated by a topic model, such as manual or automated labeling of clusters with names based on the most frequent words of a topic (Hindle et al. 2013 ).

In this paper, we provide an overview of the use of topic modeling in 111 papers published between 2009 and 2020 in highly ranked venues of software engineering (five journals and five conferences). We identify characteristics and limitations in the use of topic models and discuss (a) the appropriateness of topic modeling techniques, (b) the importance of pre-processing, (c) challenges related to defining meaningful topics, and (d) the importance of context when manually naming topics.

The rest of the paper is organized as follows. In Section 2 we provide an overview of topic modeling. In Section 3 we describe other literature reviews on the topic as well as “meta-studies” that discuss topic modeling more generally. We describe the research method in Section 4 and present the results in Section 5 . In Section 6 , we summarize our findings and discuss implications and threats to validity. Finally, in Section 7 we present concluding remarks and future work.

2 Topic Modeling

Topic modeling aims at automatically finding topics, typically represented as clusters of words, in a given textual document (Bi et al. 2018 ). Unlike (supervised) machine learning-based techniques that solve classification problems, topic modeling does not use tags, training data or predefined taxonomies of concepts (Bi et al. 2018 ). Based on the frequencies of words and frequencies of co-occurrence of words within one or more documents, topic modeling clusters words that are often used together (Barua et al. 2014 ; Treude and Wagner 2019 ). Figure 1 illustrates the general process of topic modeling, from a raw corpus of documents (“Data input”) to topics generated for these documents (“Output”). Below we briefly introduce the basic concepts and terminology of topic modeling (based on Chen et al. ( 2016 )):

Word w : a string of one or more alphanumeric characters (e.g., “software” or “management”);

Document d : a set of n words (e.g., a text snippet with five words: w 1 to w 5 );

Corpus C : a set of t documents (e.g., nine text snippets: d 1 to d 9 );

Vocabulary V : a set of m unique words that appear in a corpus (e.g., m = 80 unique words across nine documents);

Term-document matrix A : an m by t matrix whose A i , j entry is the weight (according to some weighting function, such as term-frequency) of word w i in document d j . For example, given a matrix A with three words and three documents as

A 1,1 = 5 indicates that “code” appears five times in d 1 , etc.;

Topic z : a collection of terms that co-occur frequently in the documents of a corpus. Considering probabilistic topic models (e.g., LDA), z refers to an m -length vector of probabilities over the vocabulary of a corpus. For example, in a vector z 1 = ( c o d e : 0.35; t e s t : 0.17; b u g : 0.08),

0.35 indicates that when a word is picked from a topic z 1 , there is a 35% chance of drawing the word “code”, etc.;

Topic-term matrix ϕ (or T ): a k by m matrix with k as the number of topics and ϕ i , j the probability of word w j in topic z i . Row i of ϕ corresponds to z i . For example, given a matrix ϕ as

0.05 in the first column indicates that the word “code” appears with a probability of 0.5% in topic z 3 , etc.;

Topic membership vector 𝜃 d : for document d i , a k -length vector of probabilities of the k topics. For example, given a vector \(\theta _{d_{i}} = (z_{1}: 0.25; z_{2}: 0.10; z_{3}: 0.08)\) ,

0.25 indicates that there is a 25% chance of selecting topic z 1 in d i ;

Document-topic matrix 𝜃 (or D ): an n by k matrix with 𝜃 i , j as the probability of topic z j in document d i . Row i of 𝜃 corresponds to \(\theta _{d_{i}}\) . For example, given a matrix 𝜃 as

0.10 in the first column indicates that document d 2 contains topic z 1 with probability of 10%, etc.

General topic modeling process

2.1 Data Input

Data used as input into topic modeling can take many forms. This requires decisions on what exactly are documents and what the scope of individual documents is (Miner et al. 2012 ). Therefore, we need to determine which unit of text shall be analyzed (e.g., subject lines of e-mails from a mailing list or the body of e-mails).

To model topics from raw text in a corpus C (see Fig. 1 ), the data needs to be converted into a structured vector-space model, such as the term-document matrix A . This typically also requires some pre-processing. Although each text mining approach (including topic modeling) may require specific pre-processing steps, there are some common steps, such as tokenization, stemming and removing stop words (Miner et al. 2012 ). We discuss pre-processing for topic modeling in more detail when presenting the results for RQ3 in Section 5.4 .

2.2 Modeling

Different models can be used for topic modeling. Models typically differ in how they model topics and underlying assumptions. For example, besides LDA and LSI mentioned before, other examples of topic modeling techniques include Probabilistic Latent Semantic Indexing (pLSI) (Hofmann 1999 ). LSI and pLSI reduce the dimensionality of A using Singular Value Decomposition (SVD) (Hofmann 1999 ). Furthermore, variants of LDA have been proposed, such as Relational Topic Models (RTM) (Chang and Blei 2010 ) and Hierarchical Topic Models (HLDA) (Blei et al. 2003a ). RTM finds relationships between documents based on the generated topics (e.g., if document d 1 contains the topic “microservices”, document d 2 contains the topic “containers” and document d n contains the topic “user interface”, RTM will find a link between documents d 1 and d 2 (Chang and Blei 2010 )). HLDA discovers a hierarchy of topics within a corpus, where each lower level in the hierarchy is more specific than the previous one (e.g., a higher topic “web development” may have subtopics such as “front-end” and “back-end”).

Topic modeling techniques need to be configured for a specific problem, objectives and characteristics of the analyzed text (Treude and Wagner 2019 ; Agrawal et al. 2018 ). For example, Treude and Wagner ( 2019 ) studied parameters, characteristics of text corpora and how the characteristics of a corpus impact the development of a topic modeling technique using LDA. Treude and Wagner ( 2019 ) found that textual data from Stack Overflow (e.g., threads of questions and answers) and GitHub (e.g., README files) require different configurations for the number of generated topics ( k ). Similarly, Barua et al. ( 2014 ) argued that the number of topics depends on the characteristics of the analyzed corpora. Furthermore, the values of modeling parameters (e.g., LDA’s hyperparameters α and β which control an initial topic distribution) can also be adjusted depending on the corpus to improve the quality of topics (Agrawal et al. 2018 ).

By finding words that are often used together in documents in a corpus, a topic modeling technique creates clusters of words or topics z k . Words in such a cluster are usually related in some way, therefore giving the topic a meaning. For example, we can use a topic modeling technique to extract five topics from unstructured document such as a combination of Stack Overflow posts. One of the clusters generated could include the co-occurring words “error”, “debug” and “warn”. We can then manually inspect this cluster and by inference suggest the label “Exceptions” to name this topic (Barua et al. 2014 ).

3 Related Work

3.1 previous literature reviews.

Sun et al. ( 2016 ) and Chen et al. ( 2016 ), similar to our study, surveyed software engineering papers that applied topic modeling. Table 1 shows a comparison between our study and prior reviews. As shown in the table, Sun et al. ( 2016 ) focused on finding which software engineering tasks have been supported by topic models (e.g., support source code comprehension, feature location, traceability link recovery, refactoring, software testing, developer recommendations, software defects prediction and software history comprehension), and Chen et al. ( 2016 ) focused on characterizing how studies used topic modeling to mine software repositories.

Furthermore, as shown in Table 1 , in comparison to Sun et al. ( 2016 ) and Chen et al. ( 2016 ), our study surveys the literature considering other aspects of topic modeling such as data inputs (RQ2), data pre-processing (RQ3), and topic naming (RQ4). Additionally, we searched for papers that applied topic models to any type of data (e.g., Q&A websites) rather than to data in software repositories. We also applied a different search process to identify relevant papers.

Although some of the search venues of these two previous studies and our study overlap, our search focused on specific venues. We also searched papers published between 2009 and 2020, a period which only partially overlaps with the searches presented by Sun et al. ( 2016 ) and Chen et al. ( 2016 ).

Regarding the data analysed in previous studies, Chen et al. ( 2016 ) analyzed two aspects not covered in our study: (a) tools to implement topic models in papers, and (b) how papers evaluated topic models (note that even though we did not cover this aspect explicitly, we checked whether papers compared different topic models, and if so, what metrics they used to compare topic models). However, different to Chen et al. ( 2016 ) we analyzed (a) the types of contribution of papers (e.g., a new approach); (b) details about the types of data and documents used in topic modeling techniques, and (c) whether and how topics were named. Additionally, we extend the survey of Chen et al. ( 2016 ) by investigating hyperparameters (see Section 2.1 ) of topic models and data pre-processing in more detail. We provide more details and a justification of our research method in Section 4 .

3.2 Meta-studies on Topic Modeling

In addition to literature surveys, there are “meta-studies” on topic modeling that address and reflect on different aspects of topic modeling more generally (and are not considered primary studies for the purpose of our review, see our inclusion and exclusion criteria in Section 4 ). In the following paragraphs we organized their discussion into three parts: (1) studies about parameters for topic modeling, (2) studies on topic models based on the type of analyzed data, and (3) studies about metrics and procedures to evaluate the performance of topic models. We refer to these studies throughout this manuscript when reflecting on the findings of our study.

Regarding parameters used for topic modeling, Treude and Wagner ( 2019 ) performed a broad study on LDA parameters to find optimal settings when analyzing GitHub and Stack Overflow text corpora. The authors found that popular rules of thumb for topic modeling parameter configuration were not applicable to their corpora, which required different configurations to achieve good model fit. They also found that it is possible to predict good configurations for unseen corpora reliably. Agrawal et al. ( 2018 ) also performed experiments on LDA parameter configurations and proposed LDADE, a tool to tune the LDA parameters. The authors found that due to LDA topic model instability, using standard LDA with “off-the-shelf” settings is not advisable. We also discuss parameters for topic modeling in Section 2.2 .

For studies on topic models based on the analyzed data, researchers have investigated topic modeling involving short texts (e.g., a tweet) and how to improve the performance of topic models that work well with longer text (e.g., a book chapter) (Lin et al. 2014 ). For example, the study of Jipeng et al. ( 2020 ) compared short-text topic modeling techniques and developed an open-source library of the short-text models. Another example is the work of Mahmoud and Bradshaw ( 2017 ) who discussed topic modeling techniques specific for source code.

Finally, regarding metrics and procedures to evaluate the performance of topic models, some works have explored how semantically meaningful topics are for humans (Chang et al. 2009 ). For example, Poursabzi-Sangdeh et al. ( 2021 ) discuss the importance of interpretability of models in general (also considering other text mining techniques). Another example is the work of Chang et al. ( 2009 ) who presented a method for measuring the interpretability of a topic model based on how well words within topics are related and how different topics are between each other. On the other hand, as an effort to quantify the interpretability of topics without human evaluation, some studies developed topic coherence metrics . These metrics score the probability of a pair of words from topics being found together in (a) external data sources (e.g., Wikipedia pages) or (b) in the documents used by the model that generated those topics (Röder et al. 2015 ). Röder et al. ( 2015 ) combined different implementations of coherence metrics in a framework. Perplexity is another measure of performance for statistical models in natural language processing, which indicates the uncertainty in predicting a single word (Blei et al. 2003b ). This metric is often applied to compare the configurations of a topic modeling technique (e.g., Zhao et al. ( 2020 )). Other studies use perplexity as an indicator of model quality (such as Chen et al. 2019 and Yan et al. 2016b ).

4 Research Method

We conducted a literature survey to describe how topic modeling has been applied in software engineering research. To answer the research questions introduced in Section 1 , we followed general guidelines for systematic literature review (Kitchenham 2004 ) and mapping study methods (Petersen et al. 2015 ). This was to systematically identify relevant works, and to ensure traceability of our findings as well as the repeatability of our study. However, we do not claim to present a fully-fledged systematic literature review (e.g., we did not assess the quality of primary studies) or a mapping study (e.g., we only analyzed papers from carefully selected venues). Furthermore, we used parts of the procedures from other literature surveys on similar topics (Bi et al. 2018 ; Chen et al. 2016 ; Sun et al. 2016 ) as discussed throughout this section.

4.1 Search Procedure

To identify relevant research, we selected high-quality software engineering publication venues. This was to ensure that our literature survey includes studies of high quality and described at sufficient level of detail. We identified venues rated as A and A ∗ for Computer Science and Information Systems research in the Excellence Research for Australia (CORE) ranking (ARC 2012 ). Only one journal was rated B (IST), but we included it due to its relevance for software engineering research. These venues are a subset of venues also searched by related previous literature surveys (Chen et al. 2016 ; Sun et al. 2016 ), see Section 3 . The list of searched venues includes five journals: (1) Empirical Software Engineering (EMSE); (2) Information and Software Technology (IST); (3) Journal of Systems and Software (JSS); (4) ACM Transactions on Software Engineering & Methodology (TOSEM); (5) IEEE Transaction on Software Engineering (TSE). Furthermore, we included five conferences: (1) International Conference on Automated Software Engineering (ASE); (2) ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM); (3) International Symposium on the Foundations of Software Engineering / European Software Engineering Conference (ESEC/FSE); (4) International Conference on Software Engineering (ICSE); (5) International Workshop/Working Conference on Mining Software Repositories (MSR).

We performed a generic search on SpringerLink (EMSE), Science Direct (IST, JSS), ACM DL (TOSEM, ESEC/FSE, ASE, ESEM, ICSE, MSR) and IEEE Xplore (TSE, ASE, ESEM, ICSE, MSR) using the venue (journal or conference) as a high-level filtering criterion. Considering that the proceedings of ASE, ESEM, ICSE and, MSR are published by ACM and IEEE, we searched these venues on ACM DL and IEEE Xplore to avoid missing relevant papers. We used a generic search string (“topic model[l]ing” and “topic model”). Furthermore, in order to find studies that apply specific topic models but do not mention the term “topic model”, we used a second search string with topic model names (“lsi” or “lda” or “plsi” or “latent dirichlet allocation” or “latent semantic”). This second string was based on the search string used by Chen et al. ( 2016 ), who also present a review and analysis of topic modeling techniques in software engineering (see Section 3 ). We applied both strings to the full text and metadata of papers. We considered works published between 2009 and 2020. The search was performed in March 2021. Limiting the search to the last twelve years allowed us to focus on more mature and recent works.

4.2 Study Selection Criteria

We only considered full research papers since full papers typically report (a) mature and complete research, and (b) more details about how topic modeling was applied. Furthermore, to be included, a paper should either apply, experiment with, or propose a topic modeling technique (e.g., develop a topic modeling technique that analyzes source code to recommend refactorings (Bavota et al. 2014b )), and meet none of the exclusion criteria: (a) the paper does not apply topic models (e.g., it applies other text mining techniques and only cites topic modeling in related or future work, such as the paper by Lian et al. ( 2020 ); (b) the paper focuses on theoretical foundation and configurations for topic models (e.g., it discusses how to tune and stabilize topic models, such as Agrawal et al. ( 2018 ) and other meta-studies listed in Section 3.2 ); and (c) the paper is a secondary study (e.g., a literature review like the studies discussed in Section 3.1 ). We evaluated inclusion and exclusion criteria by first reading the abstracts and then reading full texts.

The search with the first search string (see Section 4.1 ) resulted in 215 papers and the search with the second search string resulted in an additional 324 papers. Applying the filtering outlined above resulted in 114 papers. Furthermore, we excluded three papers from the final set of papers: (a) Hindle et al. ( 2011 ), (b) Chen et al. ( 2012 ), and (c) Alipour et al. ( 2013 ). These papers were earlier and shorter versions of follow-up publications; we considered only the latest publications of these papers (Hindle et al. 2013 ; Chen et al. 2017 ; Hindle et al. 2016 ). This resulted in a total of 111 papers for analysis.

4.3 Data Extraction and Synthesis

We defined data items to answer the research questions and characterize the selected papers (see Table 2 ). The extracted data was recorded in a spreadsheet for analysis (raw data are available online Footnote 1 ). One of the authors extracted the data and the other authors reviewed it. In case of ambiguous data, all authors discussed to reach agreement. To synthesize the data, we applied descriptive statistics and qualitatively analyzed the data as follows:

RQ1: Regarding the data item “Technique”, we identified the topic modeling techniques applied in papers. For the data item “Supported tasks”, we assigned to each paper one software engineering task. Tasks emerged during the analysis of papers (see more details in Section 5.2.2 ). We also identified the general study outcome in relation to its goal (data item “Type of contribution”). When analyzing the type of contribution, we also checked whether papers included a comparison of topic modeling techniques (e.g., to select the best technique to be included in a newly proposed approach). Based on these data items we checked which techniques were the most popular, whether techniques were based on other techniques or used together, and for what purpose topic modeling was used.

RQ2: We identified types of data (data item “Type of data”) in selected papers as listed in Section 5.3.1 . Considering that some papers addressed one, two or three different types of data, we counted the frequency of types of data and related them with the document. Regarding “Document”, we identified the textual document and (if reported in the paper) its length. For the data item “Parameters”, we identified whether papers described modeling parameters and if so, which values were assigned to them.

RQ3: Considering that some papers may have not mentioned any pre-processing, we first checked which papers described data pre-processing. Then, we listed all pre-processing steps found and counted their frequencies.

RQ4: Considering the papers that described topic naming, we analyzed how generated topics were named (see Section 5.5 ). We used three types of approaches to describe how topics were named: (a) Manual - manually analysis and labeling of topics; (b) Automated - use automated approaches to label names to topics; and (c) Manual & Automated - mix of both manual and automated approaches to analyse and name topics. We also described the procedures performed to name topics.

5.1 Overview

As mentioned in Section 4.1 , we analyzed 111 papers published between 2009 and 2020 (see Appendix A.1 - Papers Reviewed). Most papers were published after 2013. Furthermore, most papers were published in journals (68 papers in total, 32 in EMSE alone), while the remaining 43 papers appeared in conferences (mostly MSR with sixteen papers). Table 3 shows the number of papers by venue and year.

5.2 RQ1: Topic Models Used

In this Section we first discuss which topic modeling techniques are used (Section 5.2.1 ). Then, we explore why or for what purpose these techniques were used (Section 5.2.2 ). Finally, we describe the general contributions of papers in relation to their goals (Section 5.2.3 ).

5.2.1 Topic Modeling Techniques

The majority of the papers used LDA (80 out of 111), or a LDA-based technique (30 out of 111), such as Twitter-LDA (Zhao et al. 2011 ). The other topic modeling technique used is LSI. Figure 2 shows the number of papers per topic modeling technique. The total number (125) exceeds the number of papers reviewed (111), because ten papers experimented with more than one technique: Thomas et al. ( 2013 ), De Lucia et al. ( 2014 ), Binkley et al. ( 2015 ), Tantithamthavorn et al. ( 2018 ), Abdellatif et al. ( 2019 ) and Liu et al. ( 2020 ) experimented with LDA and LSI; Chen et al. ( 2014 ) experimented with LDA and Aspect and Sentiment Unification Model (ASUM); Chen et al. ( 2019 ) experimented with Labeled Latent Dirichlet Allocation (LLDA) and Label-to-Hierarchy Model (L2H); Rao and Kak ( 2011 ) experimented with LDA and MLE-LDA; and Hindle et al. ( 2016 ) experimented with LDA and LLDA. ASUM, LLDA, MLE-LDA and L2H are techniques based on LDA.

Number of papers per topic modeling technique

The popularity of LDA in software engineering has also been discussed by others, e.g., Treude and Wagner ( 2019 ). LDA is a three-level hierarchical Bayesian model (Blei et al. 2003b ). LDA defines several hyperparameters, such as α (probability of topic z i in document d i ), β (probability of word w i in topic z i ) and k (number of topics to be generated) (Agrawal et al. 2018 ).

Thirty-seven (out of 75) papers applied LDA with Gibbs Sampling (GS). Gibbs sampling is a Markov Chain Monte Carlo algorithm that samples from conditional distributions of a target distribution. Used with LDA, it is an approximate stochastic process for computing α and β (Griffiths and Steyvers 2004 ). According to experiments conducted by Layman et al. ( 2016 ), Gibbs sampling in LDA parameter estimation ( α and β ) resulted in lower perplexity than the Variational Expectation-Maximization (VEM) estimations. Perplexity is a standard measure of performance for statistical models of natural language, which indicates the uncertainty in predicting a single word. Therefore, lower values of perplexity mean better model performance (Griffiths and Steyvers 2004 ).

Thirty papers applied modified or extended versions of LDA (“LDA-based” in Fig. 2 ). Table 4 shows a comparison between these LDA-based techniques. Eleven papers proposed a new extension of LDA to adapt LDA to software engineering problems (hence the same reference in the third and fourth column of Table 4 ). For example, the Multi-feature Topic Model (MTM) technique by Xia et al. ( 2017b ), which implements a supervised version of LDA to create a bug triaging approach. The other 19 papers applied existing modifications of LDA proposed by others (third column in Table 4 ). For example, Hu and Wong ( 2013 ) used the Citation Influence Topic Model (CITM), developed by Dietz et al. ( 2007 ), which models the influence of citations in a collection of publications.

The other topic modeling technique, LSI (Deerwester et al. 1990 ), was published in 1990, before LDA which was published in 2003. LSI is an information extraction technique that reduces the dimensionality of a term-document matrix using a reduction factor k (number of topics) (Deerwester et al. 1990 ). Compared to LDA, LDA follows a generative process that is statistically more rigorous than LSI (Blei et al. 2003b ; Griffiths and Steyvers 2004 ). From the 16 papers that used LSI, seven papers compared this technique to others:

One paper (Rosenberg and Moonen 2018 ) compared LSI with other two dimensionality reduction techniques: Principal Component Analysis (PCA) (Wold et al. 1987 ) and Non-Negative Matrix Factorization (NMF) (Lee and Seung 1999 ). The authors applied these models to automatically group log messages of continuous deployment runs that failed for the same reasons.

Four papers applied LDA and LSI at the same time to compare the performance of these models to Vector Space Model (VSM) (Salton et al. 1975 ), an algebraic model for information extraction. These studies supported documentation (De Lucia et al. 2014 ); bug handling (Thomas et al. 2013 ; Tantithamthavorn et al. 2018 ); and maintenance tasks (Abdellatif et al. 2019 )).

Regarding the other two papers, Binkley et al. ( 2015 ) compared LSI to Query likelihood LDA (QL-LDA) and other information extraction techniques to check the best model for locating features in source code; and Liu et al. ( 2020 ) compared LSI and LDA to Generative Vector Space Model (GVSM), a deep learning technique, to select the best performer model for documentation traceability to source code in multilingual projects.

5.2.2 Supported Tasks

As mentioned before, we aimed to understand why topic modeling was used in papers, e.g., if topic modeling was used to develop techniques to support specific software engineering tasks, or if it was used as a data analysis technique in exploratory studies to understand the content of large amounts of textual data. We found that the majority of papers aimed at supporting a particular task, but 21 papers (see Table 5 ) used topic modeling in empirical exploratory and descriptive studies as a data analysis technique.

We extracted the software engineering tasks described in each study (e.g., bug localization, bug assignment, bug triaging) and then grouped them into eight more generic tasks (e.g., bug handling) considering typical software development activities such as requirements, documentation and maintenance (Leach 2016 ). The specific tasks collected from papers are available online 1 . Note that we kept “Bug handling” and “Refactoring” separate rather than merging them into maintenance because of the number of papers (bug handling) and the cross-cutting nature (refactoring) in these categories. Each paper was related to one of these tasks:

Architecting: tasks related to architecture decision making, such as selection of cloud or mash-up services (e.g., Belle et al. ( 2016 ));

Bug handling: bug-related tasks, such as assigning bugs to developers, prediction of defects, finding duplicate bugs, or characterizing bugs (e.g., Naguib et al. ( 2013 ));

Coding: tasks related to coding, e.g., detection of similar functionalities in code, reuse of code artifacts, prediction of developer behaviour (e.g., Damevski et al. ( 2018 ));

Documentation: support software documentation, e.g., by localizing features in documentation, automatic documentation generation (e.g., Souza et al. ( 2019 ));

Maintenance: software maintenance-related activities, such as checking consistency of versions of a software, investigate changes or use of a system (e.g., Silva et al. ( 2019 ));

Refactoring: support refactoring, such as identifying refactoring opportunities and removing bad smell from source code (e.g., Bavota et al. ( 2014b ));

Requirements: related to software requirements evolution or recommendation of new features (e.g., Galvis Carreno and Winbladh ( 2012 ));

Testing: related to identification or prioritization of test cases (e.g., Thomas et al. ( 2014 )).

Table 5 groups papers based on the topic modeling technique and the purpose. Few papers applied topic modeling to support Testing (three papers) and Refactoring (three papers). Bug handling is the most frequent supported task (33 papers). From the 21 exploratory studies, 13 modeled topics from developer communication to identify developers’ information needs: 12 analyzed posts on Stack Overflow, a Q&A website for developers (Chatterjee et al. 2019 ; Bajaj et al. 2014 ; Ye et al. 2017 ; Bagherzadeh and Khatchadourian 2019 ; Ahmed and Bagherzadeh 2018 ; Barua et al. 2014 ; Rosen and Shihab 2016 ; Zou et al. 2017 ; Chen et al. 2019 ; Han et al. 2020 ; Abdellatif et al. 2020 ; Haque and Ali Babar 2020 ) and one paper analyzed blog posts (Pagano and Maalej 2013 ). Regarding the other eight exploratory studies, three papers investigated web search queries to also identify developers’ information needs (Xia et al. 2017a ; Bajracharya and Lopes 2009 ; 2012 ); four papers investigated end user documentation to analyse users’ feedback on mobile apps (Tiarks and Maalej 2014 ; El Zarif et al. 2020 ; Noei et al. 2018 ; Hu et al. 2018 ); and one paper investigated historical “bug” reports of NASA systems to extract trends in testing and operational failures (Layman et al. 2016 ).

5.2.3 Types of Contribution

For each study, we identified what type of contribution it presents based on the study goal. We used three types of contributions (“Approach”, “Exploration” and “Comparison”, as described below) by analyzing the research questions and main results of each study. A study could contribute either an “Approach” or an “Exploration”, while “Comparison” is orthogonal, i.e., a study that presents a new approach could present a comparison of topic models as part of this contribution. Similarly, a comparison of topic models can also be part of an exploratory study.

Approach: a study develops an approach (e.g., technique, tool, or framework) to support software engineering activities based on or with the support of topic models. For example, Murali et al. ( 2017 ) developed a framework that applies LDA to Android API methods to discover types of API usage errors, while Le et al. ( 2017 ) developed a technique (APRILE+) for bug localization which combines LDA with a classifier and an artificial neural network.

Exploration: a study applies topic modeling as the technique to analyze textual data collected in an empirical study (in contrast to for example open coding). Studies that contributed an exploration did not propose an approach as described in the previous item, but focused on getting insights from data. For example, Barua et al. ( 2014 ) applied LDA to Stack Overflow posts to discover what software engineering topics were frequently discussed by developers; Noei et al. ( 2018 ) explored the evolution of mobile applications by applying LDA to app descriptions, release notes, and user reviews.

Comparison: the study (that can also contribute with an “Approach” or an “Exploration”) compares topic models to other approaches. For example, Xia et al. ( 2017b ) compared their bug triaging approach (based on the so called Multi-feature Topic Model - MTM) with similar approaches that apply machine learning (Bugzie (Tamrawi et al. 2011 )) and SVM-LDA (combining a classifier with LDA (Somasundaram and Murphy 2012 )). On the other hand, De Lucia et al. ( 2014 ) compared LDA and LSI to define guidelines on how to build effective automatic text labeling techniques for program comprehension.

From the papers that contributed an approach , twenty-two combined a topic modeling technique with one or more other techniques applied for text mining:

Information extraction (e.g., VSM) (Nguyen et al. 2012 ; Zhang et al. 2018 ; Chen et al. 2020 ; Thomas et al. 2013 ; Fowkes et al. 2016 );

Classification (e.g., Support Vector Machine - SVM) (Hindle et al. 2013 ; Le et al. 2017 ; Liu et al. 2017 ; Demissie et al. 2020 ; Zhao et al. 2020 ; Shimagaki et al. 2018 ; Gopalakrishnan et al. 2017 ; Thomas et al. 2013 );

Clustering (e.g., K-means) (Jiang et al. 2019 ; Cao et al. 2017 ; Liu et al. 2017 ; Zhang et al. 2016 ; Altarawy et al. 2018 ; Demissie et al. 2020 ; Gorla et al. 2014 );

Structured prediction (e.g., Conditional Random Field - CRF) (Ahasanuzzaman et al. 2019 );

Artificial neural networks (e.g., Recurrent Neural Network - RNN) (Murali et al. 2017 ; Le et al. 2017 );

Evolutionary algorithms (e.g., Multi-Objective Evolutionary Algorithm - MOEA) (Blasco et al. 2020 ; Pérez et al. 2018 );

Web crawling (Nabli et al. 2018 ).

Pagano and Maalej ( 2013 ) was the only study that contributed an exploration that combined LDA with another text mining technique. To analyze how developer communities use blogs to share information, the authors applied LDA to extract keywords from blog posts and then analyzed related “streams of events” (commit messages and releases by time in relation to blog posts), which were created with Sequential pattern mining.

Regarding comparisons we found that (1) 13 out of the 63 papers that contribute an approach also include some form of comparison, and (2) ten out of the 48 papers contribute an exploration also include some form of comparison. We discuss comparisons in more detail below in Section 6.1.2

5.3 RQ2: Topic Model Inputs

In this section we first discuss the type of data (Section 5.3.1 ). Then we discuss the actual textual documents used for topic modeling (Section 5.3.2 ). Finally, we describe which model parameters were used (Section 5.3.3 ) to configure models.

5.3.1 Types of Data

Types of data help us describe the textual software engineering content that has been analyzed with topic modeling. We identified 12 types of data in selected papers as shown in Table 6 . In some papers we identified two or three of these types of data; for example, the study of Tantithamthavorn et al. ( 2018 ) dealt with issue reports, log information and source code.

Source code (37 occurrences), issue/bug reports (22 occurrences) and developer communication (20 occurrences) were the most frequent types of data used. Seventeen papers used two to four types of data in their topic modeling technique; twelve of these papers used a combination of source code with another type of data. For example, Sun et al. ( 2015 ) generated topics from source code and developer communication to support software maintenance tasks, and in another study, Sun et al. ( 2017 ) used topics found in source code and commit messages to assign bug-fixing tasks to developers.

5.3.2 Documents

A document refers to a piece of textual data that can be longer or shorter, such as a requirements document or a single e-mail subject. Documents are concrete instances of the types of data discussed above. Figure 3 shows documents (per type of data) and how often we found them in papers. The most frequent documents are bug reports (12 occurrences), methods from source code (9 occurrences), Q&A posts (9 occurrences) and user reviews (8 occurrences).

Documents (leaves in the figure) by type of data (nodes in the figure)

We also analyzed document length and found the following:

In general, papers described the length of documents in number of words, see Table 7 . Footnote 2 On the other hand, two papers (Moslehi et al. 2016 , 2020 ) described their documents’ length in minutes of screencast transcriptions (videos with one to ten minutes, no information about the size of transcripts). Sixteen papers mentioned the actual length of the documents, see Table 7 . Ten papers that described the actual document length did that when describing the data used for topic modeling; four papers discussed document length while describing results; and one mentioned document length as a metric for comparing different data sources;

Most papers (80 out of 111) did not mention document length and also do not acknowledge any limitations or the impact of document length on topics.

Fifteen papers did not mention the actual document length, but at some point acknowledge the influence of document length on topic modeling. For example, Abdellatif et al. ( 2019 ) mentioned that the documents in their data set were “not long”. Similarly, Yan et al. ( 2016b ) did not mention the length of the bug reports used but discussed the impact of the vocabulary size of their corpus on results. Moslehi et al. ( 2018 ) mentioned document length as a limitation and acknowledge that using LDA on short documents was a threat to construct validity. According to these authors, using techniques specific for short documents could have improved the outcomes of their topic modeling.

5.3.3 Model Parameters

Topic models can be configured with parameters that impact how topics are generated. For example, LDA has typically been used with symmetric Dirichlet priors over 𝜃 (document-topic distributions) and ϕ (topic-word distributions) with fixed values for α and β (Wallach et al. 2009 ). Wallach et al. ( 2009 ) explored the robustness of a topic model with asymmetric priors over 𝜃 (i.e., varying values for α ) and a symmetric prior (fixed value for β ) over ϕ . Their study found that such topic model can capture more distinct and semantically-related topics, i.e., the words in clusters are more distinct. Therefore, we checked which parameters and values were used in papers. Overall, we found the following:

Eighteen of the 111 papers do not mention parameters (e.g., number of topics k , hyperparameters α and β ). Thirteen of these papers use LDA or an LDA-based technique, four papers use LSI, while (Liu et al. 2020 ) use LDA and LSI.

The remaining 93 papers mention at least one parameter. The most frequent parameters discussed were k , α and β :

Fifty-eight papers mentioned actual values for k , α and β ;

Two papers mentioned actual values for α and β , but no values for k ;

Twenty-nine papers included actual values for k but not for α and β ;

Thirty-two (out of 58) papers mentioned other parameters in addition to k , α and β . For example, Chen et al. ( 2019 ) applied L2H (in comparison to LLDA), which uses the hyperparameters γ 1 and γ 2 ;

One paper (Rosenberg and Moonen 2018 ) that applied LSI, mentioned the parameter “similarity threshold” rather than k , α and β .

We then had a closer look at the 60 papers that mentioned actual values for hyperparameters α and β :

α based on k : The most frequent setting (29 papers) was α = 50/ k and β = 0.01 (i.e., α was depending on the number of topics, a strategy suggested by Steyvers and Griffiths ( 2010 ) and Wallach et al. ( 2009 )). These values are a default setting in Gibbs Sampling implementations for LDA such as Mallet. Footnote 3

Fixed α and β : Five papers fixed 0.01 for both hyperparameters, as suggested by Hoffman et al. ( 2010 ). Another eight papers fixed 0.1 for both hyperparameters, a default setting in Stanford Topic Modeling Toolbox (TMT); Footnote 4 and three other papers fixed α = 0.1 and β = 1 (these three studies applied RTM).

Varying α or β : Four papers tested different values for α , where two of these papers also tested different values for β ; and one paper varied β but fixed a value for α .

Optimized parameters : Four papers obtained optimized values for hyperparameters (Sun et al. 2015 ; Catolino et al. 2019 ; Yang et al. 2017 ; Zhang et al. 2018 ). These papers applied LDA-GA (as proposed by Panichella et al. ( 2013 )) which, based on genetic algorithms; finds the best values for LDA hyperparameters. In regards to the actual values chosen for optimized hyperparameters, Catolino et al. ( 2019 ) did not mention the values for hyperparameters; Sun et al. ( 2015 ) and Yang et al. ( 2017 ) mentioned only the values used for k ; and Zhang et al. ( 2018 ) described the values for k , α and β .

Regarding the values for k we observed the following:

The 90 papers that mentioned values for k modeled three (Cao et al. 2017 ) to 500 (Li et al. 2018 ; Lukins et al. 2010 ; Chen et al. 2017 ) topics;

Twenty-four (out of 90) papers mentioned that a range of values for k was tested in order to check the performance of the technique (e.g., Xia et al. ( 2017b )) or as a strategy to select the best number of topics (e.g., Layman et al. ( 2016 ));

Although the remaining 66 (out of 90) papers mentioned a single value used for k , most of them acknowledged that had tried several number of topics or used the number of topics suggested by other studies.

As can be seen in Table 7 , there is no common trend of what values for hyperparameter or k depending on the document or document length.

5.4 RQ3: Pre-processing Steps

Thirteen of the papers did not mention what pre-processing steps were applied to the data before topic modeling. Seven papers only described how the data analyzed were selected, but not how they were pre-processed. Table 8 shows the pre-processing steps found in the remaining 91 papers. Each of these papers mentioned at least one of these steps.

Removing noisy content (76 occurrences), Stemming terms (61 occurrences) and Splitting terms (33 occurrences) were the most used pre-processing steps. The least frequent pre-processing step (Resolving negations) was found only in the studies of Noei et al. ( 2019 ) and Noei et al. ( 2018 ). Resolving synonyms and Expanding contractions were also less frequent, with three occurrences each.

Table 9 shows the types of noise removal in papers and their frequency. Most of the papers that described pre-processing steps removed stop words (76 occurrences). Stop words are the most common words in a language, such as “a/an” and “the” in English. Removing stop words allows topic modeling techniques to focus on more meaningful words in the corpus (Miner et al. 2012 ). Eight papers mentioned the stop words list used: Layman et al. ( 2016 ) and Pettinato et al. ( 2019 ) used the SMART stop words list; Footnote 5 Martin et al. ( 2015 ) and Hindle et al. ( 2013 ) used the Natural Language Toolkit English stop words list; Footnote 6 Bagherzadeh and Khatchadourian ( 2019 ), Ahmed and Bagherzadeh ( 2018 ) and Yan et al. ( 2016b ) used the Mallet stop words list; Footnote 7 and Mezouar et al. ( 2018 ) used the Moby stop words list. Footnote 8

As can be seen in Table 9 , some papers removed words based on the frequency of their occurrence (most or least frequent terms) or length (words shorter than four, three or two letters or long terms). Other papers removed long paragraphs. For example, Henß et al. ( 2012 ) removed paragraphs longer than 800 characters because most paragraphs in their data set were shorter than that. We also found two papers that removed short documents: Gorla et al. ( 2014 ) removed documents with fewer than ten words, and Palomba et al. ( 2017 ) removed documents with fewer than three words. The concept of non-informative content depends on the context of each paper. In general, it refers to any data considered not relevant for the objective of the study. For example, Choetkiertikul et al. ( 2017 ), which aimed at predicting bugs in issue reports, removed issues that took too much time to be resolved. Noei et al. ( 2019 ) and Fu et al. ( 2015 ) removed content (end user reviews and commit messages) that did not describe feedback or cause of change.

5.5 RQ4: Topic Naming

Topic naming is about assigning labels (names) to topics (word clusters) to give the clusters a human-understandable meaning. Seventy-five papers (out of 111) did not mention whether or how topics were named. These papers only used the word clusters for analysis, but did not require a name. For example, Xia et al. ( 2017a ) and Canfora et al. ( 2014 ) did not name topics, but mapped the word clusters to the documents (search queries and source code comments) used as input for topic modeling. These papers used the probability of a document to belong to a topic ( 𝜃 ) to associate a document to the topic with the highest probability.

From the 36 papers (out of 111) that mentioned topic naming (see Table 10 ), we identified three ways of how they named topics:

Automated: Assigning names to word clusters without human intervention;

Manual: Manually checking the meaning and the combination of words in cluster to “deduct” a name, sometimes validated with expert judgment;

Manual & Automated: Mix of manual and automated; e.g., topics are manually labeled for one set of clusters to then train a classifier for naming another set of clusters.

Most of the papers (30 out of 36) assigned one name to one topic. However, we identified six papers that used one name for multiple topics (Hindle et al. 2013 ; Pagano and Maalej 2013 ; Bajracharya and Lopes 2012 ; Rosen and Shihab 2016 ) or labeled a topic with multiple names (Zou et al. 2017 ; Gao et al. 2018 ). Two of the papers (Hindle et al. 2013 ; Bajracharya and Lopes 2012 ) that assigned one name to multiple topics used predefined labels, and in the other two papers (Pagano and Maalej 2013 ; Rosen and Shihab 2016 ) authors interpreted words in the clusters to deduct names.

Regarding the papers that assigned multiple names to a topic, Zou et al. ( 2017 ) assigned no, one or more names, depending on how many words in the predefined word list matched words in clusters. Gao et al. ( 2018 ) used an automated approach to label topics with the three most relevant phrases and sentences from the end user reviews inputted to their topic model. The relevance of phrases and sentences were obtained with the metrics Semantic and Sentiment scores proposed by these authors.

6 Discussion

6.1 rq1: topic modeling techniques, 6.1.1 summary of findings.

LDA is the most frequently used topic model. Almost all papers (95 out of 111) applied LDA or a LDA-based technique, while nine papers applied LSI to identify topics and seven papers used LDA and LSI. Regarding the papers that used LDA-based techniques, eleven (out of 30) proposed their own LDA-based technique (Fu et al. 2015 ; Nguyen et al. 2011 ; Liu et al. 2017 ; Cao et al. 2017 ; Panichella et al. 2013 ; Yan et al. 2016a ; Xia et al. 2017b ; Nguyen et al. 2012 ; Damevski et al. 2018 ; Gao et al. 2018 ; Rao and Kak 2011 ). This may indicate that the LDA default implementation may not be adequate to support specific software engineering tasks or extract meaningful topics from all types of data. We discuss more about topic modeling techniques and their inputs in Section 6.2.2 . Furthermore, we found that topic modeling is used to develop tools and methods to support software engineers and concrete tasks (the most frequently supported task we found was bug handling), but also as a data analysis technique for textual data to explore empirical questions (see for example the “oldest” paper in our sample published in 2009 (Bajracharya and Lopes 2009 )).

One aspect that we did not specifically address in this review, but which impacts the applicability of topics models is their computational overhead. Computational overhead refers to processing time and computational resources (e.g., memory, CPU) required for topic modeling. As discussed by others, topic modeling can be computational intensive (Hoffman et al. 2010 ; Treude and Wagner 2019 ; Agrawal et al. 2018 ). However, we found that only few papers (seven out of 111) mentioned computational overhead at all. From these seven papers, five mentioned processing time (Bavota et al. 2014b ; Zhao et al. 2020 ; Luo et al. 2016 ; Moslehi et al. 2016 ; Chen et al. 2020 ), one paper mentioned computational requirements and some processing times (e.g., processor, data pre-processing time, LDA processing time and clustering processing time), and one paper only mention that their technique was processed in “few seconds” (Murali et al. 2017 ). Hence, based on the reviewed studies we cannot provide broader insights into the practical applicability and potential constraints of topic modeling based on the computational overhead.

6.1.2 Comparative Studies

As mentioned in Sections 5.2.1 and 5.2.3 , we identified studies that used more than one topic modeling technique and compared their performance. In detail, we found studies that (1) compared topic modeling techniques to information extraction techniques, such as Vector Space Model (VSM), an algebraic model (Salton et al. 1975 ) (see Table 11 ), (2) proposed an approach that uses a topic modeling technique and compared it to other approaches (which may or may not use topic models) with similar goals (see Table 12 ), and (3) compared the performance of different settings for a topic modeling technique or a newly proposed approach that utilizes topic models (see Table 13 ). In column “Metric” of Tables 11 , 12 and 13 the metrics show the metrics used in the comparisons to decide which techniques performed “better” (based on the metrics’ interpretation). Metrics in bold were proposed for or adapted to a specific context (e.g., SCORE and Effort reduction), while the other metrics are standard NLP metrics (e.g., Precision, Recall and Perplexity). Details about the metrics used to compare the techniques are provided in Appendix A.2 - Metrics Used in Comparative Studies.

As shown in Table 11 , ten papers compared topic modeling techniques to information extraction techniques. For example, Rosenberg and Moonen ( 2018 ) compared LSI with two other dimensionality reduction techniques (PCA and NMF) to group log messages of failing continuous deployment runs. Nine out of these ten papers presented explorations, i.e., studies experimented with different models to discuss their application to specific software engineering tasks, such as bug handling, software documentation and maintenance. Thomas et al. ( 2013 ) on the other hand experimented with multiple models to propose a framework for bug localization in source code that applies the best performing model.

Four papers in Table 11 (De Lucia et al. 2014 ; Tantithamthavorn et al. 2018 ; Abdellatif et al. 2019 ; Thomas et al. 2013 ) compared the performance of LDA, LSI and VSM with source code and issue/bug reports. Except for De Lucia et al. ( 2014 ), these studies applied Top-k accuracy (see Appendix A.2 - Metrics Used in Comparative Studies) to measure the performance of models, and the best performing model was VSM. Tantithamthavorn et al. ( 2018 ) found that VSM achieves both the best Top-k performance and the least required effort for method-level bug localization. Additionally, according to De Lucia et al. ( 2014 ), VSM possibly performed better than LSI and LDA due to the nature of the corpus used in their study: LDA and LSI are ideal for heterogeneous collections of documents (e.g., user manuals from different systems), but in De Lucia et al. ( 2014 ) study each corpus was a collection of code classes from a single software system.

Ten studies proposed an approach that uses a topic modeling technique and compared it to similar approaches (shown in Table 12 ). In column “Approaches compared” of Table 12 , the approach in bold is the one proposed by the study (e.g., Cao et al. 2017 ) or the topic modeling technique used in their approach (e.g., Thomas et al. 2014 ). All newly proposed approaches were the best performing ones according to the metrics used.

In addition to the papers mentioned in Tables 11 and 12 , four papers compared the performance of different settings for a topic modeling technique or tested which topic modeling technique works best in their newly proposed approach (see Table 13 ). Biggers et al. ( 2014 ) offered specific recommendations for configuring LDA when localizing features in Java source code, and observed that certain configurations outperform others. For example, they found that commonly used heuristics for selecting LDA hyperparameter values ( beta = 0.01 or beta = 0.1) in source code topic modeling are not optimal (similar to what has been found by others, see Section 3.2 ). The other three papers (Chen et al. 2014 ; Fowkes et al. 2016 ; Poshyvanyk et al. 2012 ) developed approaches which were tested with different settings (e.g., the approach applying LDA or ASUM (Chen et al. 2014 )).

Regarding the datasets used by comparative studies, only Rao and Kak ( 2011 ) used a benchmarking dataset (iBUGS). Most of the comparative studies (13 out of 24) used source code or issue/bug reports from open source software, which are subject to evolution. The advantage of using benchmarking datasets rather than “living” datasets (e.g., an open source Java system) is that its data will be static and the same across studies. Additionally, data in benchmarking datasets are usually curated. This means that the results of replicating studies can be compared to the original study when both used the same benchmarking dataset.

Finally, we highlight that each of the above mentioned comparisons has a specific context. This means that, for example, the type of data analyzed (e.g., Java classes), the parameter setting (e.g., k = 50), the goal of the comparison (e.g., to select the best model for bug localization or for tracing documentation in source code) and pre-processing (e.g., stemming and stop word removal) were different. Therefore, it is not possible to “synthesize” the results from the comparisons across studies by aggregating the different comparisons in different papers, even for studies that appear to have similar goals or use the same topic modeling techniques, such as comparing the same models with similar types of data (such as Tantithamthavorn et al. 2018 and Abdellatif et al. 2019 ).

6.2 RQ2: Inputs to Topic Models

6.2.1 summary of findings.

Source code, developer communication and issue/bug reports were the most frequent types of data used for topic modeling in the reviewed papers. Consequently, most of the documents referred to individual or groups of functions or methods, individual Q&A posts, or individual bug reports; another frequent document was an individual user review (more discussions are in Section 6.2.3 ). We also found that few papers (16 out of 111) mentioned the actual length of documents used for topic modeling (we discuss this more in Section 6.2.2 ).

Regarding modeling parameters, most of the papers (93 out of 111) explicitly mentioned the configuration of at least one parameter, e.g., k , α or β for LDA. We observed that the setting α = 50/ k and β = 0.01 (asymmetric α and symmetric β ) as suggested by Steyvers and Griffiths ( 2010 ) and Wallach et al. ( 2009 ) was frequently used (28 out of 93 papers). Additionally, papers that applied LDA mostly used the default parameters of the tools used to implement LDA (e.g., Mallet 3 with α = 50/ k and β = 0.01 as default). This finding is similar to what has been reported by others, e.g., according to another review by Agrawal et al. ( 2018 ), LDA is frequently applied “as is out-of-the-box” or with little tuning. This means that studies may rely on the default settings of the tools used with their topic modeling technique, such as Mallet and TMT, rather than try to optimize parameters.

6.2.2 Documents and Parameters for Topic Models

Short texts : According to Lin et al. ( 2014 ), topic models such as LDA have been widely adopted and successfully used with traditional media like edited magazine articles. However, applying LDA to informal communication text such as tweets, comments on blog posts, instant messaging, Q&A posts, may be less successful. Their user-generated content is characterized by very short document length, a large vocabulary and a potentially broad range of topics. As a consequence, there are not enough words in a document to create meaningful clusters, compromising the performance of the topic modeling. This means that probabilistic topic models such as LDA perform sub-optimally when applied “as is” with short documents even when hyperparameters ( α and β in LDA) are optimized (Lin et al. 2014 ). In our sample there were only two papers that mentioned the use of a LDA-based technique specifically for short documents (Hu et al. 2019 ; Hu et al. 2018 ). Hu et al. ( 2019 ) and Hu et al. ( 2018 ) applied Twitter-LDA with end user reviews. Furthermore, Moslehi et al. ( 2018 ) used a weighting algorithm in documents to generate topics with more relevant words, they also acknowledge that the use of a short text technique could have improved their topic model.

As shown in Table 7 , few papers mentioned the actual length of documents. Considering a single document from a corpus, we observed that most papers potentially used short texts (all documents found in papers are shown in Fig. 3 ). For example, papers used an individual search query (Xia et al. 2017a ), an individual Q&A post (Barua et al. 2014 ), an individual user review (Nayebi et al. 2018 ), or an individual commit message (Canfora et al. 2014 ) as a document. Among the papers that mentioned document length, the shortest documents were an individual commit message (9 to 20 words) (Canfora et al. 2014 ) and an individual method (14 words) (Tantithamthavorn et al. 2018 ). Both studies applied LDA.

Two approaches to improve the performance of LDA when analyzing short documents are pooling and contextualization (Lin et al. 2014 ). Pooling refers to aggregating similar (e.g., semantically or temporally) documents into a single document (Mehrotra et al. 2013 ). For example, among the papers analysed, Pettinato et al. ( 2019 ) used temporal pooling and combined short log messages into a single document based on a temporal order. Contextualization refers to creating subsets of documents according to a type of context; considering tweets as documents, the type of context can refer to time, user and hashtags associated with tweets (Tang et al. 2013 ). For example, Weng et al. ( 2010 ) combined all the individual tweets of an author into one pseudo-document (rather than treating each tweet as a document). Therefore, with the contextualization approach, the topic model uses word co-occurrences at a context level instead of at the document level to discover topics.

Hyperparameters Table 14 shows the hyperparameter settings and types of data of the papers that mentioned the value of at least one model parameter. In Table 14 we also highlight the topic modeling techniques used. Note that some topic modeling techniques (e.g., RTM) can receive more parameters that the ones mentioned in Table 14 (e.g., number of documents, similarity thresholds); all parameters mentioned in papers are available online in the raw data of our study 1 . When comparing hyperparameter settings, topic modeling techniques and types of data, we observed the following:

Papers that used LDA-GA, an LDA-based technique that optimizes hyperparameters with Genetic algorithms, applied it to data from developer documentation or source code;

LDA was used with all three types of hyperparameter settings across studies. The most common setting was α based on k for developer communication and source code;

Most of the LDA-based techniques applied fixed values for α and β .

Most of the papers that applied only LSI as the topic modeling technique did not mention hyperparameters. As LSI is a model simpler than LDA, it generally requires the number of topics k . For example, a paper that applied LSI to source code mentioned α and k (Poshyvanyk et al. 2012 ).

Number of topics By relating the type of data to the number of topics, we aimed at finding whether the choice of the number of topics is related to the data used in the topic modeling techniques (see also Table 7 ). However, the number of topics used and data in the studies are rather diverse. Therefore, synthesizing practices and offering insights from previous studies on how to choose the number topics is rather limited.

From the 90 papers that mentioned number of topics ( k ), we found that 66 papers selected a specific number of topics (e.g., based on previous works with similar data or addressing the same task), while 24 papers used several numbers of topics (e.g., Yan et al. ( 2016b ) used 10 to 120 topics in steps of 10). To provide an example of how the number of topics differed even when the same type of data was analyzed with the same topic modeling technique, we looked at studies that applied LDA in textual data from developer communication (mostly Q&A posts) to propose an approach to support documentation. For these papers we found one paper that did not mention k (Henß et al. 2012 ), one paper that modeled different numbers of topics ( k = 10,20,30) (Asuncion et al. 2010 ), one paper that modeled k = 15 (Souza et al. 2019 ) and another paper that modeled k = 40 (Wang et al. 2015 ). This illustrates that there is no common or recommended practice that can be derived from the papers.

Some papers mentioned that they tested several numbers of topics before selecting the most appropriate value for k (in regards to studies’ goals) but did not mention the range of values tested. In regards to papers that mentioned such range, we identified four studies (Nayebi et al. 2018 ; Chen et al. 2014 ; Layman et al. 2016 ; Nabli et al. 2018 ) that tested several values for k and used perplexity (see details in Appendix A.2 - Metrics Used in Comparative Studies) of models to evaluate which value of k generated the best performing model; three studies (Zhao et al. 2020 ; Han et al. 2020 ; El Zarif et al. 2020 ) also selected the number of topics after testing several values for k ; however they used topic coherence (Röder et al. 2015 ) to evaluate models. One paper (Haque and Ali Babar 2020 ) used both perplexity and topic coherence to select a value for k . Metrics of topic coherence score the probability of a pair of words from the resulted word clusters being found together in (a) external data sources (e.g., Wikipedia pages) or (b) in the documents used by the topic model that generated those word clusters (Röder et al. 2015 ).

6.2.3 Supported Tasks, Types of Data and Types of Contribution

We looked into the relationship between the tasks supported by papers, the type of data used and the types of contributions (see Table 15 ). We observed the following:

Source code was a frequent type of data in papers; consequently it appeared for almost all supported tasks, except for exploratory studies;

Considering exploratory studies, most papers used developer communication (13 out of 21), followed by search queries and end user communication (three papers each);

Papers that supported bug handling mostly used issue/bug reports, source code and end user communication;

Log information was used by papers that supported maintenance, bug handling, and coding;

Considering the papers that supported documentation, three used transcript texts from speech;

From the four papers related to the type of data developer documentation, two supported architecting tasks and the other two, documentation tasks.

Regarding the type of data, URLs and transcripts were only used in studies that contributed an approach.

We found that most of the exploratory studies used data that is less structured. For example, developer communication, such as Q&A posts and conversation threads generally do not follow a standardized template. On the other hand, issue reports are typically submitted through forms which enforces a certain structure.

6.3 RQ3: Data Pre-processing

6.3.1 summary of findings.

Most of the papers (91 out of 111) pre-processed the textual data before topic modeling. Removing noisy content was the most frequent pre-processing step (as typical for natural language processing), followed by stemming and splitting words. Miner et al. ( 2012 ) consider tokenizing as one of the basic data pre-processing steps in text mining. However, in comparison to other basic pre-processing steps such as stemming, splitting words and removing noise, tokenizing was not frequently found in papers (it was at least not mentioned in papers).

Eight papers (Henß et al. 2012 ; Xia et al. 2017b ; Ahasanuzzaman et al. 2019 ; Abdellatif et al. 2019 ; Lukins et al. 2010 ; Tantithamthavorn et al. 2018 ; Poshyvanyk et al. 2012 ; Binkley et al. 2015 ) tested how pre-processing steps affected the performance of topic modeling or topic model-based approaches. For example, Henß et al. ( 2012 ) tested several pre-processing steps (e.g., removing stop words, long paragraphs and punctuation) in e-mail conversations analyzed with LDA. They found that removing such content increased LDA’s capability to grasp the actual semantics of software mailing lists. Ahasanuzzaman et al. ( 2019 ) proposed an approach which applies LDA and Conditional Random Field (CRF) to localize concerns in Stack Overflow posts. The authors did not incorporate stemming and stop words removal in their approach because in preliminary tests these pre-processing steps decreased the performance of the approach.

6.3.2 Pre-processing Different Types of Data

Table 16 shows how different types of data were pre-processed. We observed that stemming, removing noise, lowercasing, and splitting words were commonly used for all types of data. Regarding the differences, we observed the following:

For developer communication there were specific types of noisy content that was removed: URLs, HTML tags and code snippets. This might have happened because most of the papers used Q&A posts as documents, which frequently contain hyperlinks and code examples;

Removing non-informative content was frequently applied to end user communication and end user documentation;

Expanding contracted terms (e.g., “didn’t” to “did not”) were applied to end user communication and issue/bug reports;

Removing empty documents and eliminating extra white spaces were applied only in end user communication. Empty documents occurred in this type of data because after the removal of stop words no content was left (Chen et al. 2014 );

For source code there was a specific noise to be removed: program language specific keywords (e.g., “public”, “class”, “extends”, “if”, and “while”).

Table 16 shows that splitting words, stop words removal and stemming were frequently applied to source code and most of these studies (15) applied these three steps at the same time. Studies that performed these pre-processing steps to source code mostly used methods, classes, or comments in classes/methods as documents. For example, Silva et al. ( 2016 ) who applied LDA, performed these three pre-processing steps in classes from two open source systems using TopicXP (Savage et al. 2010 ). TopicXP is a Eclipse plug-in that extracts source code, pre-process it and executes LDA. This plug-in implements splitting words, stop words removal and stemming.

Splitting words was the most frequent pre-processing step in source code. Studies used this step to separate Camel Cases in methods and classes (e.g., the class constructor InvalidRequestTest produces the terms “invalid”, “request” and “test”). For example, Tantithamthavorn et al. ( 2018 ) compared LDA, LSI and VSM testing different combinations of pre-processing steps to the methods’ identifiers inputted to these techniques. The best performing approach was VSM with splitting words, stop words removal and stemming.

Removing stop words in source code refer to the exclusion of the most common words in a language (e.g., “a/an” and “the” in English), as in studies that used other types of data. Removing stop words in source code is also different from removing programming language keywords and studies mentioned these as separate steps. Lukins et al. ( 2010 ), for example, tested how removing stop words from their documents (comments and identifiers of methods) affected the topics generated by their LDA-based approach. They found that this step did not improve the results substantially.

As mentioned in Section 5.4 , stemming is the process of normalizing words into their single forms by identifying and removing prefixes, suffixes and pluralisation (e.g., “development”, “developer”, “developing” become “develop”). Regarding stemming in source code, papers normalized identifiers of classes and methods, comments related to classes and methods, test cases or a source code file. Three papers tested the effect of this pre-processing step in the performance of their techniques (Tantithamthavorn et al. 2018 ; Poshyvanyk et al. 2012 ; Binkley et al. 2015 ), and one of these papers also tested removing stop words and splitting words (Tantithamthavorn et al. 2018 ). Poshyvanyk et al. ( 2012 ) tested the effect of stemming classes in the performance of their LSI-based approach. The authors concluded that stemming can positively impact features localization by producing topics (“concept lattices” in their study) that effectively organize the results of searches in source code. Binkley et al. ( 2015 ) compared the performance of LSI, QL-LDA and other techniques. They also tested the effects of stemming (with two different stemmers: Porter Footnote 9 and Krovetz Footnote 10 ) and non-stemming methods from five open source systems. These authors found that they obtained better performances in terms of models’ Mean Reciprocal Rank (MRR, details in Appendix A.2 - Metrics Used in Comparative Studies) with non-stemming.

Additionally, we found that even though some papers used the same type of data, they pre-processed data differently since they had different goals and applied different techniques. For example, Ye et al. ( 2017 ), Barua et al. ( 2014 ) and Chen et al. ( 2019 ) used developer communication (Q&A posts as documents). Ye et al. ( 2017 ) and Barua et al. ( 2014 ) removed stop words, code snippets and HTML tags, while Barua et al. ( 2014 ) also stemmed words. On the other hand, Chen et al. ( 2019 ) removed stop words and the least and the most frequent words, and identified bi-grams. Some studies considered the advice on data pre-processing from previous studies (e.g., Chen et al. 2017 ; Li et al. 2018 ), while others adopted steps that are commonly used in NLP, such as noise removal and stemming (Miner et al. 2012 ) (e.g., Demissie et al. 2020 ). This means that the choice of pre-processing steps do not only depend on the characteristics of the type of data inputted to topic modeling techniques.

6.4 RQ4: Assigning Names to Topics

Most papers did not mention if or how they named topics. The majority of papers that explicitly assigned names to topics (27 out of 36) used a manual approach and relied on human judgment (researchers’ interpretation) of words in clusters. One paper (Rosen and Shihab 2016 ) justified their use of a manual approach by arguing that there was no tool that could give human readable topics based on word clusters. Thus, authors checked every word cluster generated and the documents used (an individual question of a Q&A website) to make sure they would label topics appropriately.

Table 17 shows how topics were named and the type of data analyzed. Table 18 shows how topics were named and the type of contributions they make. We observed the following:

Studies that modeled topics from developer documentation, transcripts and URLs did not mention topic naming. Studies that contributed with both exploration and comparison also did not mention topic naming;

Topics were mostly named in studies that used data from developer communication (ten occurrences) and in exploratory studies (22 occurrences).

From studies that compared topic models or topic modeling-based approaches (see Section 6.1.2 ), only one study (Yan et al. 2016b ) named topics (automatically with predefined labels).

Fourteen papers acknowledged limitations of manual topic naming:

Twelve papers (Bagherzadeh and Khatchadourian 2019 ; Ahmed and Bagherzadeh 2018 ; Martin et al. 2015 ; Hindle et al. 2013 ; Pagano and Maalej 2013 ; Zou et al. 2017 ; Pettinato et al. 2019 ; Layman et al. 2016 ; Ray et al. 2014 ; Tiarks and Maalej 2014 ; Mezouar et al. 2018 ; Abdellatif et al. 2020 ) acknowledged that how topics were named could be a threat to validity. For example, Layman et al. ( 2016 ) mentioned that they did not evaluate the accuracy of the manual topic naming, which was based on their expertise.

Three papers (Hindle et al. 2015 ; Bajracharya and Lopes 2012 ; Li et al. 2018 ) mentioned difficulties to assign names to topics. Hindle et al. ( 2015 ), for example, explained that labeling topics was difficult due to many project specific and unclear terms in clusters.

One paper (Pettinato et al. 2019 ) acknowledged that there is another topic naming approach that could be applied to their data: authors acknowledged that an automated extraction of topic names could replace manual labeling.

Hindle et al. ( 2015 ) provided some recommendations on topic analysis in software engineering based on their experiences. Below are some of their recommendations related to topic naming:

Some of the generated topics will not be relevant (e.g., clusters filled with common terms may not address any particular subject) and topics may be duplicated. This means that not all topics have to be named and used for analysis;

Domain experts can label topics better than non-experts, because they are more familiar to domain-specific keywords that may appear in word clusters;

It is important to rely on the relationship between topics generated and the original data. Hindle et al. ( 2015 ) argued that “the content of the topic can be interpreted in many different ways and LDA does not look for the same patterns that people do”.

6.5 Implications

The goal of this study was to describe how topic modeling is applied in software engineering research. We found studies that experimented, explored data, or proposed solutions to support different software engineering tasks with topic models. Our findings help researchers and practitioners as follows:

Understand which topic modeling techniques to use for what purpose . Researchers and practitioners that are going to select and apply a topic modeling technique, for example, to refactor legacy systems; may consider the experiences of other studies with similar objectives.

Pre-processing based on the type of data to be modeled . Pre-processing steps depend on the type of data analyzed (e.g., removing HTML tags in developer communication, mainly Q&A posts). Researchers and practitioners who, for example, intend to model topics from source code; may consider the same pre-processing steps that other studies applied to source code.

Understand how to name topics . Researchers and practitioners may check how other studies named topics to get insights on how to give meaning to their own topics.

We present some additional insights:

Appropriateness of topic modeling . Although we found that most of papers applied LDA “as is”, it may not be the best approach for other studies or for practical application. LDA is popular because it is an unsupervised model, i.e., it does not require previous knowledge about the data (e.g., pre-defined classes for model training), it is statistically more rigorous than other techniques (e.g., LSI), and it discovers latent relationships (i.e., topics) between documents in a large textual corpus (Griffiths and Steyvers 2004 ). However, LDA is an unstable and non-deterministic model. This means that generated topics cannot be replicated by others, even if the same model inputs (data pre-processing and configuration of parameters) are used. Furthermore, LDA performs poorly with short documents (Lin et al. 2014 ).

Meaningful topics . Topic models should discover semantically meaningful topics. Chang et al. ( 2009 ) argue about the importance of the interpretability of topics generated by probabilistic topic modeling techniques such as LDA. To create meaningful and replicable topics with LDA, Mantyla et al. ( 2018 ) highlight the importance of stabilizing the topic model (e.g., through tuning (Agrawal et al. 2018 )) and advocate the use of stability metrics (e.g., rank-biased overlap - RBO (Mantyla et al. 2018 )).

Research opportunities . Researchers interested in investigating topic modeling in software engineering may consider developing guidelines for researchers on how to use topic modeling, depending on the type of data, goals, etc. Further studies may also explore issues related to approaches for naming topics (e.g., based on domain experts), on the evaluation of the semantic accuracy of topics generated (e.g., how meaningful the topics are and if the context of document have to be considered), and on metrics to measure the performance of topic models supporting different software engineering tasks.

6.6 Threats to Validity

We analysed the validity threats to our study considering four types of threats to validity in systematic literature mapping studies (Petersen et al. 2015 ):

Theoretical validity This threat to validity refers to concerns related to capturing the data as intended, i.e., bias and limitations in the data selection and extraction. As we focused on the practice of topic modeling in software engineering, we restricted the search to highly ranked software engineering venues, which generally publish more mature studies. We used “topic model”, “topic model[l]ing”, “lsi”, “lda”, “plsi”, “latent dirichlet allocation”, “latent semantic” as search keywords to find all papers related to topic modeling. To select papers to the survey, we established inclusion and exclusion criteria. One author selected the papers and the others checked whether the selection criteria were applied appropriately. Furthermore, to minimize this threat in relation to data extraction, we first defined the data items (details are in Table 2 ) to be extracted from papers and the relevance of the data for each research question. Then, one author extracted the data and the others reviewed the results. Controversial data results were discussed to reach agreement.

Descriptive validity In the context of a literature survey, descriptive validity refers to bias and limitations in data synthesis and the accurate and objective description of the data. To mitigate this threat, we described in detail how the data was synthesized (see Section 4.3 ); furthermore, one of the authors synthesized the data and the others reviewed the results. Still, data and results depend on what is reported in papers which was sometimes incomplete, inconsistent or inaccurate (see for example information about document length).

Interpretive validity This threat to validity refers to bias and limitations in the results of the data analysis. We frequently reviewed the synthesized data during the data analysis and the authors with more experience in this type of study checked the occurrence of inconsistencies in results. Still, we recognize that interpretation bias may not have been removed completely.

Repeatability This threat to validity concerns whether the study and its results can be replicated. To reduce this threat, we described our search procedures in detail (Section 4 ), and the processes of data selection, extraction and synthesis in detail. We also followed general guidelines for systematic literature review as suggested by Kitchenham ( 2004 ) and mapping study method as suggested by Petersen et al. ( 2015 ). Furthermore, raw data of our study are available online 1 .

7 Conclusions

We analyzed 111 papers that applied topic modeling. These papers were published in the last twelve years (2009-2020) in ten highly ranked software engineering venues (five conferences and five journals). Below we summarize our findings:

LDA and LDA-based techniques are the most frequently used topic modeling techniques;

Topic modeling was mostly used to develop techniques for handling bugs (e.g., to predict defects). Exploratory studies that use topic modeling as a data analysis technique were also frequent;

Most papers modeled topics from source code (using methods as documents);

Most papers used LDA “as is” and without adapting values of hyperparameters ( α and β );

Most papers describe pre-processing. Some pre-processing steps depend on the type of textual data used (e.g., removal of URL and HTML tags), while others are commonly used in NLP techniques (e.g., stop words removal or stemming);

Only 36 (out of 111) papers named the topics. When naming topics, papers mostly adopted manual topic naming approaches such as deducting names (or labeling pre-defined names) based on the meaning of frequent words in that topic.

By analysing topic modeling techniques, data inputs, data pre-processing, and how topics were named, we identified characteristics and limitations in the use of topic models. Our study can provide insights and references to researchers and practitioners to make the best use of topic modeling, considering the experiences from previous studies.

Our study did not investigate all potential characteristics of topic modeling in software engineering or compared topic models to other text mining techniques. To answer our research questions, we analyzed data items shown in Table 2 . Future studies may investigate other characteristics of the use of topic modeling in software engineering, for example, topic modeling tools or libraries (e.g., Mallet) used; the context of a specific supported software engineering task; or compare topic modeling techniques to other text mining techniques, such as clustering and summarization (e.g., sentence or document embeddings). Furthermore, future work can reflect on other fields or uses of topic modeling to contrast how topic modeling is applied in software engineering. Further studies may also investigate how papers evaluate the performance of their topic modeling techniques, how papers evaluate the the quality of the generated topics, and how exactly word clusters were used when topics were not named.

https://doi.org/10.5281/zenodo.5280890

This table also shows hyperparameters and the number of topics which are discussed in the following subsection.

http://mallet.cs.umass.edu/topics.php

https://nlp.stanford.edu/software/tmt/tmt-0.4/

http://www.ai.mit.edu/projects/jmlr/papers/volume5/lewis04a/a11-smart-stop-list/english.stop

https://gist.github.com/sebleier/554280

https://github.com/mengjunxie/ae-lda/blob/master/misc/mallet-stopwords-en.txt

http://icon.shef.ac.uk/Moby/mwords.html

https://tartarus.org/martin/PorterStemmer/

https://pypi.org/project/krovetz/

Abdellatif A, Costa D, Badran K, Abdalkareem R, Shihab E (2020) Challenges in Chatbot Development: A Study of Stack Overflow Posts. In: Proceedings of the 17th international conference on mining software repositories. https://doi.org/10.1145/3379597.3387472 , vol 12. IEEE/ACM, Seoul, pp 174–185

Abdellatif TM, Capretz LF, Ho D (2019) Automatic recall of software lessons learned for software project managers. Inf Softw Technol 115:44–57. https://doi.org/10.1016/j.infsof.2019.07.006

Article Google Scholar

Aggarwal CC, Zhai C (2012) Mining text data. Springer, New York. https://doi.org/10.1007/978-1-4614-3223-4

Book Google Scholar

Agrawal A, Fu W, Menzies T (2018) What is wrong with topic modeling? And how to fix it using search-based software engineering. Inf Softw Technol 98(January 2017):74–88. https://doi.org/10.1016/j.infsof.2018.02.005

Ahasanuzzaman M, Asaduzzaman M, Roy CK, Schneider KA (2019) CAPS: a supervised technique for classifying Stack Overflow posts concerning API issues. Empir Softw Eng 25:1493–1532. https://doi.org/10.1007/s10664-019-09743-4

Ahmed S, Bagherzadeh M (2018) What do concurrency developers ask about?: A large-scale study using Stack Overflow. In: Proceedings of the international symposium on empirical software engineering and measurement. https://doi.org/10.1145/3239235.3239524 . ACM, Oulu, pp 1–10

Ali N, Sharafi Z, Guéhéneuc Y G, Antoniol G (2015) An empirical study on the importance of source code entities for requirements traceability. Empir Softw Eng 20(2):442–478. https://doi.org/10.1007/s10664-014-9315-y

Alipour A, Hindle A, Stroulia E (2013) A contextual approach towards more accurate duplicate bug report detection. In: IEEE international working conference on mining software repositories. pp 183–192. https://doi.org/10.1109/MSR.2013.662402

Altarawy D, Shahin H, Mohammed A, Meng N (2018) LASCAD: Language-agnostic software categorization and similar application detection. J Syst Softw 142:21–34. https://doi.org/10.1016/j.jss.2018.04.018

ARC ARC (2012) Excellence in research for australia (ERA). https://www.arc.gov.au/excellence-research-australia http://www.arc.gov.au/pdf/era12/ERAFactsheet_Jan2012_1.pdf

Asuncion HU, Asuncion AU, Taylor RN (2010) Software traceability with topic modeling. In: Proceedings of the international conference on software engineering. IEEE/ACM, Cape Town, pp 95–104

Bagherzadeh M, Khatchadourian R (2019) Going big: a large-scale study on what big data developers ask. In: Proceedings of the 27th joint european software engineering conference and symposium on the foundations of software engineering. https://doi.org/10.1145/3338906.3338939 . ACM, Tallinn, pp 432–442

Bajaj K, Pattabiraman K, Mesbah A (2014) Mining questions asked by web developers. In: Proceedings of the 11th working conference on mining software repositories. https://doi.org/10.1145/2597073.2597083 . ACM, Hyderabad, pp 112–121

Bajracharya S, Lopes C (2009) Mining search topics from a code search engine usage log. In: Proceedings of the 6th international working conference on mining software repositories. https://doi.org/10.1109/MSR.2009.5069489 . IEEE, Vancouver, pp 111–120

Bajracharya SK, Lopes CV (2012) Analyzing and mining a code search engine usage log. Empir Softw Eng 17:424–466. https://doi.org/10.1007/s10664-010-9144-6

Barua A, Thomas SW, Hassan AE (2014) What are developers talking about? An analysis of topics and trends in Stack Overflow. Empir Softw Eng 19 (3):619–654. https://doi.org/10.1007/s10664-012-9231-y

Bavota G, Gethers M, Oliveto R, Poshyvanyk D, Lucia ADE (2014a) Improving software modularization via automated analysis of latent. ACM Trans Softw Eng Methodol 23(1):1–33. https://doi.org/10.1145/2559935

Bavota G, Oliveto R, Gethers M, Poshyvanyk D, De Lucia A (2014b) Methodbook: Recommending move method refactorings via relational topic models. IEEE Trans Softw Eng 40(7):671–694. https://doi.org/10.1109/TSE.2013.60

Beitzel SM, Jensen EC, Frieder O (2009) MAP. In: Encyclopedia of database systems. https://doi.org/10.1007/978-0-387-39940-9_492 . Springer US, Boston, pp 1691–1692

Belle AB, Boussaidi GE, Kpodjedo S (2016) Combining lexical and structural information to reconstruct software layers. Inf Softw Technol 74:1–16. https://doi.org/10.1016/j.infsof.2016.01.008

Bi T, Liang P, Tang A, Yang C (2018) A systematic mapping study on text analysis techniques in software architecture. J Syst Softw 144:533–558. https://doi.org/10.1016/j.jss.2018.07.055

Biggers LR, Bocovich C, Capshaw R, Eddy BP, Etzkorn LH, Kraft NA (2014) Configuring latent Dirichlet allocation based feature location. Empir Softw Eng 19(3):465–500. https://doi.org/10.1007/s10664-012-9224-x

Binkley D, Lawrie D, Uehlinger C, Heinz D (2015) Enabling improved IR-based feature location. J Syst Softw 101:30–42. https://doi.org/10.1016/j.jss.2014.11.013

Blasco D, Cetina C, Pastor O (2020) A fine-grained requirement traceability evolutionary algorithm: Kromaia, a commercial video game case study. Inf Softw Technol 119:1–12. https://doi.org/10.1016/j.infsof.2019.106235

Blei DM, Jordan MI, Griffiths TL, Tenenbaum JB (2003a) Hierarchical topic models and the nested chinese restaurant process. In: Proceedings of the 16th international conference on neural information processing systems. Neural Information Processing Systems Foundation, Vancouver, pp 17–24

Blei DM, Ng AY, Jordan MI (2003b) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022. https://doi.org/10.1162/jmlr.2003.3.4-5.993

MATH Google Scholar

Brank J, Mladenić D, Grobelnik M, Liu H, Mladenić D, Flach PA, Garriga GC, Toivonen H, Toivonen H (2011) F 1-measure. In: Encyclopedia of machine learning. https://doi.org/10.1007/978-0-387-30164-8_298 . Springer US, pp 397–397

Canfora G, Cerulo L, Cimitile M, Di Penta M (2014) How changes affect software entropy: An empirical study. Empir Softw Eng 19:1–38. https://doi.org/10.1007/s10664-012-9214-z

Cao B, Frank Liu X, Liu J, Tang M (2017) Domain-aware Mashup service clustering based on LDA topic model from multiple data sources. Inf Softw Technol 90:40–54. https://doi.org/10.1016/j.infsof.2017.05.001

Capiluppi A, Ruscio DD, Rocco JD, Nguyen PT, Ajienka N (2020) Detecting Java software similarities by using different clustering techniques. Inf Softw Technol 122. https://doi.org/10.1016/j.infsof.2020.106279

Catolino G, Palomba F, Zaidman A, Ferrucci F (2019) Not all bugs are the same: Understanding, characterizing, and classifying bug types. J Syst Softw 152:165–181. https://doi.org/10.1016/j.jss.2019.03.002

Chang J, Blei DM (2009) Relational topic models for document networks. In: Proceedings of the 12th international conference on artificial intelligence and statistics. Society for Artificial Intelligence and Statistics, Clearwater Beach, pp 81–88

Chang J, Blei DM (2010) Hierarchical relational models for document networks. Ann Appl Stat 4(1):124–150. https://doi.org/10.1214/09-AOAS309

Article MathSciNet MATH Google Scholar

Chang J, Boyd-Graber J, Gerrish S, Wang C, Blei DM (2009) Reading tea leaves: How humans interpret topic models. In: Proceedings of the 2009 conference advances in neural information. Neural Information Processing Systems Foundation, Vancouver, pp 288–296

Chatterjee P, Damevski K, Pollock L (2019) Exploratory study of slack q&a chats as a mining source for software engineering tools. In: Proceedings of the 16th international conference on mining software repositories. IEEE, Montreal, pp 1–12

Chen H, Coogle J, Damevski K (2019) Modeling stack overflow tags and topics as a hierarchy of concepts. J Syst Softw 156:283–299. https://doi.org/10.1016/j.jss.2019.07.033

Chen L, Hassan F, Wang X, Zhang L (2020) Taming behavioral backward incompatibilities via cross-project testing and analysis. In: Proceedings of the 42nd international conference on software engineering. https://doi.org/10.1145/3377811.3380436 . IEEE/ACM, Seoul, pp 112–124

Chen N, Lin J, Hoi SC, Xiao X, Zhang B (2014) AR-miner: Mining informative reviews for developers from mobile app marketplace. In: Proceedings of the international conference on software engineering. https://doi.org/10.1145/2568225.2568263 , vol 1. IEEE/ACM, Hyderabad, pp 767–778

Chen TH, Thomas SW, Nagappan M, Hassan AE (2012) Explaining software defects using topic models. In: Proceedings of the international working conference on mining software repositories. https://doi.org/10.1109/MSR.2012.6224280 . IEEE, Zurich, pp 189–198

Chen TH, Thomas SW, Hassan AE (2016) A survey on the use of topic models when mining software repositories. Empir Softw Eng 21(5):1843–1919. https://doi.org/10.1007/s10664-015-9402-8

Chen TH, Shang W, Nagappan M, Hassan AE, Thomas SW (2017) Topic-based software defect explanation. J Syst Softw 129:79–106. https://doi.org/10.1016/j.jss.2016.05.015

Choetkiertikul M, Dam HK, Tran T, Ghose A (2017) Predicting the delay of issues with due dates in software projects. Empir Softw Eng 22:1223–1263. https://doi.org/10.1007/s10664-016-9496-7

Craswell N (2009) Mean reciprocal rank. In: Encyclopedia of database systems. https://doi.org/10.1007/978-0-387-39940-9_488 . Springer US, pp 1703–1703

Croft WB, Metzler D (2010) Search engines: Information retrieval in practice. Addison-Wesley, Reading

Google Scholar

Cui D, Liu T, Cai Y, Zheng Q, Feng Q, Jin W, Guo J, Qu Y (2019) Investigating the impact of multiple dependency structures on software defects, IEEE/ACM, Montreal. https://doi.org/10.1109/ICSE.2019.00069

Damevski K, Chen H, Shepherd DC, Kraft NA, Pollock L (2018) Predicting future developer behavior in the IDE using topic models. IEEE Trans Softw Eng 44(11):1100–1111. https://doi.org/10.1109/TSE.2017.2748134

De Lucia A, Di Penta M, Oliveto R, Panichella A, Panichella S (2014) Labeling source code with information retrieval methods: An empirical study. Empir Softw Eng 19(5):1383–1420. https://doi.org/10.1007/s10664-013-9285-5

Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6): 391-407 https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9

Demissie BF, Ceccato M, Shar LK (2020) Security analysis of permission re-delegation vulnerabilities in Android apps. Empir Softw Eng 25:5084–5136. https://doi.org/10.1007/s10664-020-09879-8

Dietz L, Bickel S, Scheffer T (2007) Unsupervised prediction of citation influences. In: Proceedings of the 24th international conference on machine learning. https://doi.org/10.1145/1273496.1273526 . ACM, Corvallis, pp 233–240

Dit B, Revelle M, Poshyvanyk D (2013) Integrating information retrieval, execution and link analysis algorithms to improve feature location in software. Empir Softw Eng 18(2):277–309. https://doi.org/10.1007/s10664-011-9194-4

El Zarif O, Da Costa DA, Hassan S, Zou Y (2020) On the relationship between user churn and software issues. In: Proceedings of the 17th international conference on mining software repositories. https://doi.org/10.1145/3379597.3387456 . ACM, New York, pp 339–349

Fowkes J, Chanthirasegaran P, Ranca R, Allamanis M, Lapata M, Sutton C (2016) Autofolding for source code summarization. Proc Int Conf Softw Eng 43(12):649–652. https://doi.org/10.1145/2889160.2889171

Fu Y, Yan M, Zhang X, Xu L, Yang D, Kymer JD (2015) Automated classification of software change messages by semi-supervised Latent Dirichlet Allocation. Inf Softw Technol 57:369–377. https://doi.org/10.1016/j.infsof.2014.05.017

Galvis Carreno LV, Winbladh K (2012) Analysis of user comments: an approach for software requirements evolution. In: Proceedings of the international conference on software engineering. IEEE/ACM, San Francisco, pp 582–591

Gao C, Zeng J, Lyu MR, King I (2018) Online app review analysis for identifying emerging issues. In: Proceedings of the 40th international conference on software engineering. https://doi.org/10.1145/3180155.3180218 . IEEE/ACM, Gothenburg, pp 48–58

Gopalakrishnan R, Sharma P, Mirakhorli M, Galster M (2017) Can latent topics in source code predict missing architectural tactics?. In: Proceedings of the 39th international conference on software engineering, IEEE/ACM, pp 15–26. https://doi.org/10.1109/ICSE.2017.10 . http://ghtorrent.org/

Gorla A, Tavecchia I, Gross F, Zeller A (2014) Checking app behavior against app descriptions. In: Proceedings of the international conference on software engineering. https://doi.org/10.1145/2568225.2568276 . IEEE/ACM, Hyderabad, pp 1025–1035

Griffiths TL, Steyvers M (2004) Finding scientific topics. In: Proceedings of the national academy of sciences. https://doi.org/10.1073/pnas.0307752101 , vol 101. Neural Information Processing Systems Foundation, Irvine, pp 5228–5235

Haghighi A, Vanderwende L (2009) Exploring content models for multi-document summarization. In: Proceedings of the conference on human language technologies: the 2009 annual conference of the north american chapter of the association for computational linguistics. https://doi.org/10.3115/1620754.1620807 , http://www-nlpir.nist.gov/projects/duc/data.html . Association for Computational Linguistics, Boulder, pp 362–370

Han J, Shihab E, Wan Z, Deng S, Xia X (2020) What do programmers discuss about deep learning frameworks. Empir Softw Eng 25:2694–2747. https://doi.org/10.1007/s10664-020-09819-6

Haque MU, Ali Babar M (2020) Challenges in docker development: a large-scale study using stack overflow. In: Proceedings of the 14th international symposium on empirical software engineering and measurement. https://doi.org/10.1145/3382494.3410693 . IEEE/ACM, Bari, pp 1–11

Hariri N, Castro-Herrera C, Mirakhorli M, Cleland-Huang J, Mobasher B (2013) Supporting domain analysis through mining and recommending features from online product listings. IEEE Trans Softw Eng 39(12):1736–1752. https://doi.org/10.1109/TSE.2013.39

Henß S, Monperrus M, Mezini M (2012) Semi-automatically extracting FAQs to improve accessibility of software development knowledge. In: Proceedings of the international conference on software engineering. https://doi.org/10.1109/ICSE.2012.6227139 . IEEE/ACM, Zurich, pp 793–803

Hindle A, Godfrey MW, Ernst NA, Mylopoulos J (2011) Automated topic naming to support cross-project analysis of software maintenance activities. In: Proceedings of the 33rd international conference on software engineering. ACM, Waikiki, pp 163–172

Hindle A, Ernst NA, Godfrey MW, Mylopoulos J (2013) Automated topic naming: Supporting cross-project analysis of software maintenance activities. Empir Softw Eng 18(6):1125–1155. https://doi.org/10.1007/s10664-012-9209-9

Hindle A, Bird C, Zimmermann T, Nagappan N (2015) Do topics make sense to managers and developers? Empir Softw Eng 20:479–515. https://doi.org/10.1007/s10664-014-9312-1

Hindle A, Alipour A, Stroulia E (2016) A contextual approach towards more accurate duplicate bug report detection and ranking. Empir Softw Eng 21 (2):368–410. https://doi.org/10.1007/s10664-015-9387-3

Hoffman M, Blei D, Bach F (2010) Online learning for latent dirichlet allocation. In: Proceedings of the neural information processing systems conference. https://doi.org/10.1.1.187.1883. Neural Information Processing Systems Foundation, Vancouver, pp 1–9

Hofmann T (1999) Probabilistic latent semantic indexing. In: Proceedings of the 22nd annual international conference on research and development in information retrieval. ACM, Berkeley, pp 50–57

Hu H, Bezemer CP, Hassan AE (2018) Studying the consistency of star ratings and the complaints in 1 & 2-star user reviews for top free cross-platform Android and iOS apps. Empir Softw Eng 23(6):3442–3475. https://doi.org/10.1007/s10664-018-9604-y

Hu H, Wang S, Bezemer CP, Hassan AE (2019) Studying the consistency of star ratings and reviews of popular free hybrid Android and iOS apps. Empir Softw Eng 24:7–32. https://doi.org/10.1007/s10664-018-9617-6

Hu W, Wong K (2013) Using citation influence to predict software defects. In: Proceedings of the international working conference on mining software repositories. https://doi.org/10.1109/MSR.2013.6624058 . IEEE, San Francisco, pp 419–428

Jiang H, Zhang J, Ren Z, Zhang T (2017) An unsupervised approach for discovering relevant tutorial fragments for APIs. In: Proceedings of the 39th international conference on software engineering. https://doi.org/10.1109/ICSE.2017.12 . IEEE/ACM, Buenos Aires, pp 38–48

Jiang HE, Zhang J, Li X, Ren Z, Lo D, Wu X, Luo Z (2019) Recommending new features from mobile app descriptions. ACM Trans Softw Eng Methodol 28(4):1–29. https://doi.org/10.1145/3344158

Jipeng Q, Zhenyu Q, Yun L, Yunhao Y, Xindong W (2020) Short text topic modeling techniques, applications, and performance: a survey. https://doi.org/10.1109/TKDE.2020.2992485

Jo Y, Oh A (2011) Aspect and sentiment unification model for online review analysis. In: Proceedings of the fourth ACM international conference on Web search and data mining. https://doi.org/10.1145/1935826 . ACM, New York, pp 815–824

Jones JA, Harrold MJ (2005) Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th international conference on automated software engineering. https://doi.org/10.1145/1101908.1101949 , http://portal.acm.org/citation.cfm?doid=1101908.1101949 . IEEE/ACM, New York, pp 273–282

Kakas AC, Cohn D, Dasgupta S, Barto AG, Carpenter GA, Grossberg S, Webb GI, Dorigo M, Birattari M, Toivonen H, Timmis J, Branke J, Toivonen H, Strehl AL, Drummond C, Coates A, Abbeel P, Ng AY, Zheng F, Webb GI, Tadepalli P (2011) Area under curve. In: Encyclopedia of machine learning. https://doi.org/10.1007/978-0-387-30164-8_28 . Springer US, pp 40–40

Kitchenham BA (2004) Procedures for performing systematic reviews. Keele, UK, Keele University 33(TR/SE-0401):28. https://doi.org/10.1.1.122.3308

Layman L, Nikora AP, Meek J, Menzies T (2016) Topic modeling of NASA space system problem reports research in practice. In: Proceedings of the 13th working conference on mining software repositories. https://doi.org/10.1145/2901739.2901760 . ACM, Austin, pp 303–314

Le TDB, Thung F, Lo D (2017) Will this localization tool be effective for this bug? Mitigating the impact of unreliability of information retrieval based bug localization tools. Empir Softw Eng 22:2237–2279. https://doi.org/10.1007/s10664-016-9484-y

Leach RJ (2016) Introduction to software engineering, 2nd edn. CRC Press LLC, Boca Raton. https://ebookcentral.proquest.com/lib/canterbury/detail.action?docID=4711469&query=Software+Engineering

Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

Article MATH Google Scholar

Li H, Chen THP, Shang W, Hassan AE (2018) Studying software logging using topic models. Empir Softw Eng 23:2655–2694. https://doi.org/10.1007/s10664-018-9595-8

Lian X, Liu W, Zhang L (2020) Assisting engineers extracting requirements on components from domain documents. Inf Softw Technol 118(September 2019):106196. https://doi.org/10.1016/j.infsof.2019.106196

Lin T, Tian W, Mei Q, Cheng H (2014) The dual-sparse topic model: Mining focused topics and focused terms in short text. In: Proceedings of the 23rd international conference on world wide web. https://doi.org/10.1145/2566486.2567980 . ACM, Seoul, pp 539–549

Liu Y, Liu L, Liu H, Wang X, Yang H (2017) Mining domain knowledge from app descriptions. J Syst Softw 133:126–144. https://doi.org/10.1016/j.jss.2017.08.024

Liu Y, Lin J, Cleland-Huang J (2020) Traceability support for multi-lingual software projects. In: Proceedings of the 17th international conference on mining software repositories. https://doi.org/10.1145/3379597.3387440 . ACM, Seoul, pp 443–454

Lukins SK, Kraft NA, Etzkorn LH (2010) Bug localization using latent Dirichlet allocation. Inf Softw Technol 52:972–990. https://doi.org/10.1016/j.infsof.2010.04.002

Luo Q, Moran K, Poshyvanyk D (2016) A large-scale empirical comparison of static and dynamic test case prioritization techniques. In: Proceedings of the 24th international symposium on foundations of software engineering. https://doi.org/10.1145/2950290.2950344 . ACM, Seattle, pp 559–570

Mahmoud A, Bradshaw G (2017) Semantic topic models for source code analysis. Empir Softw Eng 22(4):1965–2000. https://doi.org/10.1007/s10664-016-9473-1

Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60. https://doi.org/10.1214/aoms/1177730491 , http://projecteuclid.org/euclid.aoms/1177730491

Manning CD, Raghavan P, Schütze H (2008) Evaluation of Clustering. In: Introduction to information retrieval. chap 16, https://doi.org/10.33899/csmj.2008.163987 . https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html , http://nlp.stanford.edu/IR?book/html/htmledition/evaluation?of?clustering?1.htmlwhereisthesetofclustersan . Cambridge University Press

Mantyla MV, Claes M, Farooq U (2018) Measuring LDA topic stability from clusters of replicated runs, ACM, Oulu. https://doi.org/10.1145/3239235.3267435

Martin W, Harman M, Jia Y, Sarro F, Zhang Y (2015) The app sampling problem for app store mining. In: Proceedings of the 12th international working conference on mining software repositories. https://doi.org/10.1109/MSR.2015.19 . IEEE, Florence, pp 123–133

Martin W, Sarro F, Harman M (2016) Causal impact analysis for app releases in google play. In: Proceedings of the 24th international symposium on foundations of software engineering. https://doi.org/10.1145/2950290.2950320 . ACM, Seattle, pp 435–446

McIlroy S, Ali N, Khalid H, E Hassan A (2016) Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empir Softw Eng 21:1067–1106. https://doi.org/10.1007/s10664-015-9375-7

Mehrotra R, Sanner S, Buntine W, Xie L (2013) Improving LDA Topic Models for Microblogs via Tweet Pooling and Automatic Labeling. In: Proceedings of the 36th International Conference on Research and Development in Information Retrieval. ACM, Dublin, pp 889–892

Mezouar ME, Zhang F, Zou Y (2018) Are tweets useful in the bug fixing process? An empirical study on Firefox and Chrome. Empir Softw Eng 23 (3):1704–1742. https://doi.org/10.1007/s10664-017-9559-4

Miner G, Elder J, Fast A, Hill T, Nisbet R, Delen D (2012) Practical text mining and statistical analysis for non-structured text data applications. Elsevier Science & Technology, Waltham . https://doi.org/10.1016/C2010-0-66188-8

Moslehi P, Adams B, Rilling J (2016) On mining crowd-based speech documentation. In: Proceedings of the 13th working conference on mining software repositories. https://doi.org/10.1145/2901739.2901771 . ACM, Austin, pp 259–268

Moslehi P, Adams B, Rilling J (2018) Feature location using crowd-based screencasts. In: Proceedings of the 15th international conference on mining software repositories. https://doi.org/10.1145/3196398.3196439 . ACM, New York, pp 192–202

Moslehi P, Adams B, Rilling J (2020) A feature location approach for mapping application features extracted from crowd-based screencasts to source code. Empir Softw Eng 25:4873–4926. https://doi.org/10.1007/s10664-020-09874-z

Murali V, Chaudhuri S, Jermaine C (2017) Bayesian specification learning for finding API usage errors. In: Proceedings of the Joint european software engineering conference and symposium on the foundations of software engineering. https://doi.org/10.1145/3106237.3106284 . ACM, Paderborn, pp 151–162

Nabli H, Ben Djemaa R, Ben Amor IA (2018) Efficient cloud service discovery approach based on LDA topic modeling. J Syst Softw 146:233–248. https://doi.org/10.1016/j.jss.2018.09.069

Naguib H, Narayan N, Brügge B, Helal D (2013) Bug report assignee recommendation using activity profiles. In: Proceedings of the international working conference on mining software repositories. https://doi.org/10.1109/MSR.2013.6623999 . IEEE, San Francisco, pp 22–30

Nayebi M, Cho H, Ruhe G (2018) App store mining is not enough for app improvement. Empir Softw Eng 23:2764–2794. https://doi.org/10.1007/s10664-018-9601-1

Nguyen AT, Nguyen TT, Al-Kofahi J, Nguyen HV, Nguyen TN (2011) A topic-based approach for narrowing the search space of buggy files from a bug report. In: Proceedings of the 26th international conference on automated software engineering. https://doi.org/10.1109/ASE.2011.6100062 . IEEE/ACM, Lawrence, pp 263–272

Nguyen AT, Nguyen TT, Nguyen TN, Lo D, Sun C (2012) Duplicate bug report detection with a combination of information retrieval and topic modeling. In: Proceedings of the 27th international conference on automated software engineering. https://doi.org/10.1145/2351676.2351687 . IEEE/ACM, Essen, pp 70–79

Nguyen VA, Boyd-Graber J, Resnik P, Chang J, Graber JB (2014) Learning a concept hierarchy from multi-labeled documents. In: Proceedings of the neural information processing systems conference. Neural Information Processing Systems Foundation, Montreal, pp 1–9

Noei E, Heydarnoori A (2016) EXAF: A search engine for sample applications of object-oriented framework-provided concepts. Inf Softw Technol 75:135–147. https://doi.org/10.1016/j.infsof.2016.03.007

Noei E, Da Costa DA, Zou Y (2018) Winning the app production rally. In: Proceedings of the 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering. https://doi.org/10.1145/3236024.3236044 . ACM, Lake Buena Vista, pp 283–294

Noei E, Zhang F, Wang S, Zou Y (2019) Towards prioritizing user-related issue reports of mobile applications. Empir Softw Eng 24:1964–1996. https://doi.org/10.1007/s10664-019-09684-y

Pagano D, Maalej W (2013) How do open source communities blog? Empir Softw Eng 18(6):1090–1124. https://doi.org/10.1007/s10664-012-9211-2

Palomba F, Salza P, Ciurumelea A, Panichella S, Gall H, Ferrucci F, De Lucia A (2017) Recommending and localizing change requests for mobile apps based on user reviews. In: Proceedings of the 39th international conference on software engineering. https://doi.org/10.1109/ICSE.2017.18 . IEEE/ACM, Buenos Aires, pp 106–117

Panichella A, Dit B, Oliveto R, Di Penta M, Poshynanyk D, De Lucia A (2013) How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms. In: Proceedings of the international conference on software engineering. https://doi.org/10.1109/ICSE.2013.6606598 . IEEE/ACM, San Francisco, pp 522–531

Pérez F, Lapeṅa R, Font J, Cetina C (2018) Fragment retrieval on models for model maintenance: Applying a multi-objective perspective to an industrial case study. Inf Softw Technol 103:188–201. https://doi.org/10.1016/j.infsof.2018.06.017

Petersen K, Vakkalanka S, Kuzniarz L (2015) Guidelines for conducting systematic mapping studies in software engineering: An update. Inf Softw Technol 64(1):1–18. https://doi.org/10.1016/j.infsof.2015.03.007

Pettinato M, Gil JP, Galeas P, Russo B (2019) Log mining to re-construct system behavior: An exploratory study on a large telescope system. Inf Softw Technol 114:121–136. https://doi.org/10.1016/j.infsof.2019.06.011

Poshyvanyk D, Gueheneuc YG, Marcus A, Antoniol G, Rajlich V (2007) Feature location using probabilistic ranking of methods based on execution scenarios and information retrieval. https://doi.org/10.1109/TSE.2007.1016 . https://www.researchgate.net/publication/3189749 , vol 33, pp 420–431

Poshyvanyk D, Marcus A, Ferenc R, Gyimóthy T (2009) Using information retrieval based coupling measures for impact analysis. Empir Softw Eng 14(1):5–32. https://doi.org/10.1007/s10664-008-9088-2 , http://www.mozilla.org/

Poshyvanyk D, Gethers M, Marcus A (2012) Concept location using formal concept analysis and information retrieval. ACM Trans Softw Eng Methodol 21(4):1–34. https://doi.org/10.1145/2377656.2377660

Poursabzi-Sangdeh F, Goldstein DG, Hofman JM, Vaughan JW, Wallach H (2021) Manipulating and measuring model interpretability. In: Proceedings of the conference on human factors in computing systems. https://doi.org/10.1145/3411764.3445315 . ACM, Yokohama

Ramage D, Hall D, Nallapati R, Manning CD (2009) Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the conference on empirical methods in natural language processing. https://doi.org/10.5555/1699510.1699543 . ACL/AFNLP, Singapore, pp 248–256

Rao S, Kak A (2011) Retrieval from software libraries for bug localization: A comparative study of generic and composite text models. In: Proceedings of the international conference on software engineering. https://doi.org/10.1145/1985441.1985451 . IEEE/ACM, Waikiki, pp 43–52

Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in GitHub. In: Proceedings of the symposium on the foundations of software engineering, pp 155–165. https://doi.org/10.1145/2635868.2635922

Revelle M, Gethers M, Poshyvanyk D (2011) Using structural and textual information to capture feature coupling in object-oriented software. Empir Softw Eng 16(6):773–811. https://doi.org/10.1007/s10664-011-9159-7

Röder M, Both A, Hinneburg A (2015) Exploring the space of topic coherence measures. In: Proceedings of the eighth ACM international conference on web search and data mining - WSDM ’15. https://doi.org/10.1145/2684822.2685324 . ACM, Shanghai, pp 399–408

Rosen C, Shihab E (2016) What are mobile developers asking about? A large scale study using Stack Overflow. Empir Softw Eng 21:1192–1223. https://doi.org/10.1007/s10664-015-9379-3

Rosenberg CM, Moonen L (2018) Improving problem identification via automated log clustering using dimensionality reduction. In: Proceedings of the international symposium on empirical software engineering and measurement. https://doi.org/10.1145/3239235.3239248 . ACM, Oulu, pp 1–10

Rothermel G, Untcn RH, Chu C, Harrold MJ (2001) Prioritizing test cases for regression testing. IEEE Trans Softw Eng 27(10):929–948. https://doi.org/10.1109/32.962562

Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11):613–620. https://doi.org/10.1145/361219.361220

Savage T, Dit B, Gethers M, Poshyvanyk D (2010) TopicXP: exploring topics in source code using latent Dirichlet allocation. IEEE, Timisoara. https://doi.org/10.1109/ICSM.2010.5609654

Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

Shimagaki J, Kamei Y, Ubayashi N, Hindle A (2018) Automatic topic classification of test cases using text mining at an android smartphone vendor. In: Proceedings of the 12th international symposium on empirical software engineering and measurement. https://doi.org/10.1145/3239235.3268927 . IEEE/ACM, Oulu, pp 1–10

Silva B, Sant’anna C, Rocha N, Chavez C (2016) The effect of automatic concern mapping strategies on conceptual cohesion measurement. Inf Softw Technol 75:56–70. https://doi.org/10.1016/j.infsof.2016.03.006

Silva LL, Valente MT, Maia MA (2019) Co-change patterns: A large scale empirical study. J Syst Softw 152:196–214. https://doi.org/10.1016/j.jss.2019.03.014

Soliman M, Galster M, Salama AR, Riebisch M (2016) Architectural knowledge for technology decisions in developer communities: An exploratory study with Stack Overflow. In: Proceedings of the 13th working conference on software architecture. https://doi.org/10.1109/WICSA.2016.13 . IEEE, Venice, pp 128–133

Somasundaram K, Murphy GC (2012) Automatic categorization of bug reports using latent Dirichlet allocation. In: Proceedings of the 5th India software engineering conference. https://doi.org/10.1145/2134254.2134276 , vol 12. ACM, pp 125–130

Souza LB, Campos EC, Madeiral F, Paixão K, Rocha AM, Maia M d A (2019) Bootstrapping cookbooks for APIs from crowd knowledge on Stack Overflow. Inf Softw Technol 111(March 2018):37–49. https://doi.org/10.1016/j.infsof.2019.03.009

Steyvers M, Griffiths T (2010) Probalistic Topic Models. In: Landauer T, McNamara D, Dennis S, Kintsch W (eds) Latent semantic analysis: a road to meaning. https://doi.org/10.1016/s0364-0213(01)00040-4 . University of California, Irvine, pp 993–1022

Sun X, Li B, Leung H, Li B, Li Y (2015) MSR4SM: Using topic models to effectively mining software repositories for software maintenance tasks. Inf Softw Technol 66:1–12. https://doi.org/10.1016/j.infsof.2015.05.003

Sun X, Liu X, Li B, Duan Y, Yang H, Hu J (2016) Exploring topic models in software engineering data analysis: A survey, IEEE, Shangai. https://doi.org/10.1109/SNPD.2016.7515925

Sun X, Yang H, Xia X, Li B (2017) Enhancing developer recommendation with supplementary information via mining historical commits. J Syst Softw 134:355–368. https://doi.org/10.1016/j.jss.2017.09.021

Taba SES, Keivanloo I, Zou Y, Wang S (2017) An exploratory study on the usage of common interface elements in android applications. J Syst Softw 131:491–504. https://doi.org/10.1016/j.jss.2016.07.010

Tairas R, Gray J (2009) An information retrieval process to aid in the analysis of code clones. https://doi.org/10.1007/s10664-008-9089-1 , http://www.cis.uab.edu/tairasr/clones/literature , vol 14, pp 33–56

Tamrawi A, Nguyen TT, Al-Kofahi JM, Nguyen TN (2011) Fuzzy set and cache-based approach for bug triaging. In: Proceedings of the 19th ACM symposium on foundations of software engineering. https://doi.org/10.1145/2025113.202516 . ACM, pp 365–375

Tang J, Zhang M, Mei Q (2013) One theme in all views: modeling consensus topics in multiple contexts. In: Proceedings of the 19th international conference on knowledge discovery and data mining. ACM, New York, pp 5–13

Tantithamthavorn C, Lemma Abebe S, Hassan AE, Ihara A, Matsumoto K (2018) The impact of IR-based classifier configuration on the performance and the effort of method-level bug localization. Inf Softw Technol 102(June):160–174. https://doi.org/10.1016/j.infsof.2018.06.001

Teh YW, Jordan MI, Beal MJ, Blei DM (2006) Hierarchical Dirichlet processes. J Am Stat Assoc 101(476):1566–1581. https://doi.org/10.1198/016214506000000302

Thomas SW, Nagappan M, Blostein D, Hassan AE (2013) The impact of classifier configuration and classifier combination on bug localization. IEEE Trans Softw Eng 39(10):1427–1443. https://doi.org/10.1109/TSE.2013.27

Thomas SW, Hemmati H, Hassan AE, Blostein D (2014) Static test case prioritization using topic models. Empir Softw Eng 19:182–212. https://doi.org/10.1007/s10664-012-9219-7

Tiarks R, Maalej W (2014) How does a typical tutorial for mobile development look like?. In: Proceedings of the 11th international conference on mining software repositories. https://doi.org/10.1145/2597073.2597106 . IEEE/ACM, Hyderabad, pp 272–281

Treude C, Wagner M (2019) Predicting good configurations for GitHub and stack overflow topic models. In: Proceedings of the 16th international conference on mining software repositories. https://doi.org/10.1109/MSR.2019.00022 . IEEE, Montreal, pp 84–95

Vargha A, Delaney HD (2000) A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J Educ Behav Stat 25(2):101–132. https://doi.org/10.3102/10769986025002101

Wallach HM, Mimno D, McCallum A (2009) Rethinking LDA: Why priors matter. In: Proceedings of the conference on advances in neural information processing systems. Curran Associates Inc., Vancouver, pp 1973–1981. http://rexa.info/

Wang C, Blei DM (2011) Collaborative topic modeling for recommending scientific articles. In: Proceedings of the international conference on knowledge discovery and data mining. https://doi.org/10.1145/2020408.2020480 . ACM, New York, pp 448–456

Wang W, Malik H, Godfrey MW (2015) Recommending posts concerning API issues in developer Q&A sites. In: Proceedings of the international working conference on mining software repositories. https://doi.org/10.1109/MSR.2015.28 . http://stackoverflow.com/questions/5358219/ . IEEE/ACM, pp 224–234

Wei X, Croft WB (2006) LDA-based document models for ad-hoc retrieval. In: Proceedings of the 29th annual international conference on research and development in information retrieval. https://doi.org/10.1145/1148170.1148204 . ACM, Seattle, pp 178–185

Weng J, Lim EP, Jiang J, He Q (2010) TwitterRank: Finding topic-sensitive influential twitterers. In: Proceedings of the 3rd international conference on web search and data mining. https://doi.org/10.1145/1718487.1718520 . ACM, New York, pp 261–270

Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2:37–52. https://doi.org/10.1016/0169-7439(87)80084-9

Xia X, Bao L, Lo D, Kochhar PS, Hassan AE, Xing Z (2017a) What do developers search for on the web? Empir Softw Eng 22(6):3149–3185. https://doi.org/10.1007/s10664-017-9514-4

Xia X, Lo D, Ding Y, Al-Kofahi JM, Nguyen TN, Wang X (2017b) Improving automated bug triaging with specialized topic model. IEEE Trans Softw Eng 43(3):272–297. https://doi.org/10.1109/TSE.2016.2576454

Yan M, Fu Y, Zhang X, Yang D, Xu L, Kymer JD (2016a) Automatically classifying software changes via discriminative topic model: Supporting multi-category and cross-project. J Syst Softw 113:296–308. https://doi.org/10.1016/j.jss.2015.12.019

Yan M, Zhang X, Yang D, Xu L, Kymer JD (2016b) A component recommender for bug reports using Discriminative Probability Latent Semantic Analysis. Inf Softw Technol 73:37–51. https://doi.org/10.1016/j.infsof.2016.01.005

Yang X, Lo D, Li L, Xia X, Bissyandé T F, Klein J (2017) Characterizing malicious Android apps by mining topic-specific data flow signatures. Inf Softw Technol 90:27–39. https://doi.org/10.1016/j.infsof.2017.04.007

Ye D, Xing Z, Kapre N (2017) The structure and dynamics of knowledge network in domain-specific Q&A sites: a case study of stack overflow. Empir Softw Eng 22(1):375–406. https://doi.org/10.1007/s10664-016-9430-z

Zaman S, Adams B, Hassan AE (2011) Security versus performance bugs: A case study on firefox. In: Proceedings - international conference on software engineering. https://doi.org/10.1145/1985441.198545 , pp 93–102

Zeugmann T, Poupart P, Kennedy J, Jin X, Han J, Saitta L, Sebag M, Peters J, Bagnell JA, Daelemans W, Webb GI, Ting KM, Ting KM, Webb GI, Shirabad JS, Fürnkranz J, Hüllermeier E, Matwin S, Sakakibara Y, Flener P, Schmid U, Procopiuc CM, Lachiche N, Fürnkranz J (2011) Precision and recall. In: Encyclopedia of machine learning. https://doi.org/10.1007/978-0-387-30164-8_652 . Springer US, pp 781–781

Zhang E, Zhang Y (2009) Average precision. In: Encyclopedia of database systems. https://doi.org/10.1007/978-0-387-39940-9_482 . Springer US, pp 192–193

Zhang T, Chen J, Yang G, Lee B, Luo X (2016) Towards more accurate severity prediction and fixer recommendation of software bugs. J Syst Softw 117:166–184. https://doi.org/10.1016/j.jss.2016.02.034

Zhang Y, Lo D, Xia X, Scanniello G, Le TDB, Sun J (2018) Fusing multi-abstraction vector space models for concern localization. Empir Softw Eng 23:2279–2322. https://doi.org/10.1007/s10664-017-9585-2

Zhao N, Chen J, Wang Z, Peng X, Wang G, Wu Y, Zhou F, Feng Z, Nie X, Zhang W, Sui K, Pei D (2020) Real-time incident prediction for online service systems. In: Proceedings of the 28th ACM joint meeting european software engineering conference and symposium on the foundations of software engineering. https://doi.org/10.1145/3368089.3409672 , vol 20. ACM, pp 315–326

Zhao WX, Jiang J, Weng J, He J, Lim EP, Yan H, Li X (2011) Comparing twitter and traditional media using topic models. In: Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-642-20161-5-34 , vol 6611. Springer, Berlin, chap Advances i, pp 338–349

Zhao Y, Zhanq F, Shlhab E, Zou Y, Hassan AE (2016) How are discussions associated with bug reworking? an empirical study on open source projects. In: Proceedings of the 10th international symposium on empirical software engineering and measurement. https://doi.org/10.1145/2961111.296259 . IEEE/ACM, Ciudad Real, pp 1–10

Zou J, Xu L, Yang M, Zhang X, Yang D (2017) Towards comprehending the non-functional requirements through Developers’ eyes: An exploration of Stack Overflow using topic analysis. Inf Softw Technol 84(1):19–32. https://doi.org/10.1016/j.infsof.2016.12.003

Download references

Acknowledgements

We would like to thank the editor and the anonymous reviewers for their insightful and detailed feedback that helped us to significantly improve the manuscript.

Author information

Authors and affiliations.

University of Canterbury, Christchurch, New Zealand

Camila Costa Silva, Matthias Galster & Fabian Gilson

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Camila Costa Silva .

Ethics declarations

Conflict of interests.

The authors declare that they have no conflict of interest.

Additional information

Communicated by: Andrea De Lucia

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

1.1 A.1 Papers Reviewed

1.2 a.2 metrics used in comparative studies.

The column “Context-specific” indicates if the metric was proposed or adapted to a specific context (“Yes”) or is a standard NLP metric (“No”).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Silva, C.C., Galster, M. & Gilson, F. Topic modeling in software engineering research. Empir Software Eng 26 , 120 (2021). https://doi.org/10.1007/s10664-021-10026-0

Download citation

Accepted : 29 July 2021

Published : 06 September 2021

DOI : https://doi.org/10.1007/s10664-021-10026-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Topic modeling
Text mining
Natural language processing
Literature analysis
Find a journal
Publish with us
Track your research

Mon - Sat 9:00am - 12:00am

Get a quote

Latest Thesis and Research Topics in Software Engineering

Unique software engineering research topics for students.

more software engineers are needed as a result of the growing reliance on technology in both personal and professional spheres of life. Software engineering research topics are essential for solving complicated issues, increasing productivity, and fostering innovation. While software engineering is so important, it is equally difficult for students to get their degree in Software engineering.

Being said that many students struggle to keep up academically because software engineering is one of the most desired degrees. The final year thesis or dissertation is the most challenging assignment; many students are on the edge of losing their minds over it. While writing a thesis is one duty, coming up with an original and creative software engineering research topic is the first and most challenging step. Students with their assignments and activities don’t have enough time or energy to build a topic that is exactly right for them, finding a topic that is feasible and corresponds with your interests requires a lot of effort.

However this issue can be resolved as our PhD experts can provide you with well researched software engineering dissertation topics . We have plenty of topics for you to choose from mentioned below, and even if you don’t find anything according to your interests here you can simply contact us and request your topics according to your requirements and our experts will get you a tailored software engineering thesis topic.

Get an Immediate Response

Discuss your requirments with our writers

Get 3 Customize Research Topic within 24 Hours

Undergraduate Masters PhD Others

List of Free Software Engineering Research Topics

An analysis of the undertaking of good outcome factors and difficulties in software engineering projects:, how “the research guardian” can help you a lot.

Our top thesis writing experts are available 24/7 to assist you the right university projects. Whether its critical literature reviews to complete your PhD. or Master Levels thesis.

Automated software testing and quality control:

The study aims to improve programming testing and quality control through the execution of mechanized testing methods.

Objectives:

To efficiently detect software defeat and ensure complete test coverage, create an automated testing framework.
To determine which automated testing frameworks and tools are best suited to software development.
To analyze key metrics, and contrast them with the manual testing method to investigate the effects.

Impact of DevOps practices on software development:

The study aims to examine how DevOps practices affect software development productivity and efficiency.

To encourage cross-functional teams to collaborate, share information, and jointly advanced the development process.
To automate testing procedures like unit root tests, integration tests, and regression tests.
To change the activities for quality assurance and testing in the development process.

Get Help from Expert Thesis Writers!

TheresearchGuardian.com providing expert thesis assistance for university students at any sort of level. Our thesis writing service has been serving students since 2011.

Role of upgrading software security to enhance protection:

The aim of upgrading programming security through weakness identification and enhancing protection from possible breach

To find security flaws and weaknesses early on, employ, methods like vulnerability scanning, code reviews, and penetration testing.
To reduce the likelihood of being exploited, establish a procedure for resolving vulnerabilities as soon as possible.
To provide extensive security awareness and training programs, an organization can foster a security-conscious culture.

Adoption and effectiveness of continuous development:

The study aims to identify how effectively software engineering can be used for continuous development along with integration practices

To determine the benefit of implementing continuous deployment practices in numbers.
To evaluate the effect of computerizing the arrangement cycle, including code joining, testing, and delivery to the executive.
To analyze the impact of continuous integration practices on software development lifecycle enhancement.
To analyze how team communication and collaboration are affected by adopting software engineering practices and continuous development.

Looking For Customize Thesis Topics?

Take a review of different varieties of thesis topics and samples from our website TheResearchGuardian.com on multiple subjects for every educational level.

Planning and assess client-driven approaches in software programming:

The study aims to plan and assess client driven approaches to programing necessities and designing.

To identify the beneficial client-driven approaches necessary for programming and designing.
To ensure the successful implementation of these approaches in an organization.
To investigate the outcomes of these approaches in the success or failure of an organization.

Analyzing software metrics and their applications:

The study aims to analyze software metrics and their application to predictive software quality assurance.

To evaluate a comprehensive set of software metrics that can shed light on software product quality.
To create predictive models that make use of the software metrics that have been identified to predict potential risk and quality issues.
To compare the predictions made by the predictive models to actual software quality outcomes.

Applying Block chain Innovation:

The study aims to investigate how the distinctive characteristics of Block chain technology can be used to enhance software development and deployment process

To assess the potential use cases and advantages of coordinating block chain innovation into the product advancement lifecycle.
To investigate the application of block chain for transparent deployment histories, and decentralized package management.
To influence block chain’s straightforwardness to work with reviewing and consistence process in programming advancement.

Investigation of augmented and Virtual Reality into Software Engineering Methods and Tools:

The study aims to deeply analyse the integration of Augmented and Virtual Reality into Software Engineering Methods and tools to enhance the efficiency

To measure the impact of the integration of AR and VR technologies on software engineering
To examine the practical and technical obstacles to incorporate to incorporating augmented reality and virtual reality into existing software engineering techniques and tools.
To analyze existing frameworks and solution that make it possible to integrate AR and VR Software.

Complete Solution of All Your Hectic Thesis Papers

Our Expert online thesis writers are qualified and have expertise in almost all subject areas. This gives us an edge and we can help a lot of students who are struggling. Having a PhD expert in Software engineering gives us an advantage as we can help students looking for research topics in software engineering for masters, and then further help them with their research proposals and complete thesis.

Meet Our Professionals Ranging From Renowned Universities

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

Notifications

📚 A curated list of papers for Software Engineers

facundoolano/software-papers

Folders and files, repository files navigation, papers for software engineers.

A curated list of papers that may be of interest to Software Engineering students or professionals. See the sources and selection criteria below.

Von Neumann's First Computer Program. Knuth (1970) . Computer History; Early Programming

The Education of a Computer. Hopper (1952) .
Recursive Programming. Dijkstra (1960) .
Programming Considered as a Human Activity. Dijkstra (1965) .
Goto Statement Considered Harmful. Dijkstra (1968) .
Program development by stepwise refinement. Wirth (1971) .
The Humble Programmer. Dijkstra (1972) .
Computer Programming as an Art. Knuth (1974) .
The paradigms of programming. Floyd (1979) .
Literate Programming. Knuth (1984) .

Computing Machinery and Intelligence. Turing (1950) . Early Artificial Intelligence

Some Moral and Technical Consequences of Automation. Wiener (1960) .
Steps towards Artificial Intelligence. Minsky (1960) .
ELIZA—a computer program for the study of natural language communication between man and machine. Weizenbaum (1966) .
A Theory of the Learnable. Valiant (1984) .

A Method for the Construction of Minimum-Redundancy Codes. Huffman (1952) . Information Theory

A Universal Algorithm for Sequential Data Compression. Ziv, Lempel (1977) .
Fifty Years of Shannon Theory. Verdú (1998) .

Engineering a Sort Function. Bentley, McIlroy (1993) . Data Structures; Algorithms

On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem. Kruskal (1956) .
A Note on Two Problems in Connexion with Graphs. Dijkstra (1959) .
Quicksort. Hoare (1962) .
Space/Time Trade-offs in Hash Coding with Allowable Errors. Bloom (1970) .
The Ubiquitous B-Tree. Comer (1979) .
Programming pearls: Algorithm design techniques. Bentley (1984) .
Programming pearls: The back of the envelope. Bentley (1984) .
Making data structures persistent. Driscoll et al (1986) .

A Design Methodology for Reliable Software Systems. Liskov (1972) . Software Design

On the Criteria To Be Used in Decomposing Systems into Modules. Parnas (1971) .
Information Distribution Aspects of Design Methodology. Parnas (1972) .
Designing Software for Ease of Extension and Contraction. Parnas (1979) .
Programming as Theory Building. Naur (1985) .
Software Aging. Parnas (1994) .
Towards a Theory of Conceptual Design for Software. Jackson (2015) .

Programming with Abstract Data Types. Liskov, Zilles (1974) . Abstract Data Types; Object-Oriented Programming

The Smalltalk-76 Programming System Design and Implementation. Ingalls (1978) .
A Theory of Type Polymorphism in Programming. Milner (1978) .
On understanding types, data abstraction, and polymorphism. Cardelli, Wegner (1985) .
SELF: The Power of Simplicity. Ungar, Smith (1991) .

Why Functional Programming Matters. Hughes (1990) . Functional Programming

Recursive Functions of Symbolic Expressions and Their Computation by Machine. McCarthy (1960) .
The Semantics of Predicate Logic as a Programming Language. Van Emden, Kowalski (1976) .
Can Programming Be Liberated from the von Neumann Style? Backus (1978) .
The Semantic Elegance of Applicative Languages. Turner (1981) .
The essence of functional programming. Wadler (1992) .
QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. Claessen, Hughes (2000) .
Church's Thesis and Functional Programming. Turner (2006) .

An Incremental Approach to Compiler Construction. Ghuloum (2006) . Language Design; Compilers

The Next 700 Programming Languages. Landin (1966) .
Programming pearls: little languages. Bentley (1986) .
The Essence of Compiling with Continuations. Flanagan et al (1993) .
A Brief History of Just-In-Time. Aycock (2003) .
LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. Lattner, Adve (2004) .
A Unified Theory of Garbage Collection. Bacon, Cheng, Rajan (2004) .
A Nanopass Framework for Compiler Education. Sarkar, Waddell, Dybvig (2005) .
Bringing the Web up to Speed with WebAssembly. Haas (2017) .

No Silver Bullet: Essence and Accidents of Software Engineering. Brooks (1987) . Software Engineering; Project Management

How do committees invent? Conway (1968) .
Managing the Development of Large Software Systems. Royce (1970) .
The Mythical Man Month. Brooks (1975) .
On Building Systems That Will Fail. Corbató (1991) .
The Cathedral and the Bazaar. Raymond (1998) .
Out of the Tar Pit. Moseley, Marks (2006) .

Communicating sequential processes. Hoare (1978) . Concurrency

Solution Of a Problem in Concurrent Program Control. Dijkstra (1965) .
Monitors: An operating system structuring concept. Hoare (1974) .
On the Duality of Operating System Structures. Lauer, Needham (1978) .
Software Transactional Memory. Shavit, Touitou (1997) .

The UNIX Time- Sharing System. Ritchie, Thompson (1974) . Operating Systems

An Experimental Time-Sharing System. Corbató, Merwin Daggett, Daley (1962) .
The Structure of the "THE"-Multiprogramming System. Dijkstra (1968) .
The nucleus of a multiprogramming system. Hansen (1970) .
Reflections on Trusting Trust. Thompson (1984) .
The Design and Implementation of a Log-Structured File System. Rosenblum, Ousterhout (1991) .

A Relational Model of Data for Large Shared Data Banks. Codd (1970) . Databases

Granularity of Locks and Degrees of Consistency in a Shared Data Base. Gray et al (1975) .
Access Path Selection in a Relational Database Management System. Selinger et al (1979) .
The Transaction Concept: Virtues and Limitations. Gray (1981) .
The design of POSTGRES. Stonebraker, Rowe (1986) .
Rules of Thumb in Data Engineering. Gray, Shenay (1999) .

A Protocol for Packet Network Intercommunication. Cerf, Kahn (1974) . Networking

Ethernet: Distributed packet switching for local computer networks. Metcalfe, Boggs (1978) .
End-To-End Arguments in System Design. Saltzer, Reed, Clark (1984) .
An algorithm for distributed computation of a Spanning Tree in an Extended LAN. Perlman (1985) .
The Design Philosophy of the DARPA Internet Protocols. Clark (1988) .
TOR: The second generation onion router. Dingledine et al (2004) .
Why the Internet only just works. Handley (2006) .
The Network is Reliable. Bailis, Kingsbury (2014) .

New Directions in Cryptography. Diffie, Hellman (1976) . Cryptography

A Method for Obtaining Digital Signatures and Public-Key Cryptosystems. Rivest, Shamir, Adleman (1978) .
How To Share A Secret. Shamir (1979) .
A Digital Signature Based on a Conventional Encryption Function. Merkle (1987) .
The Salsa20 family of stream ciphers. Bernstein (2007) .

Time, Clocks, and the Ordering of Events in a Distributed System. Lamport (1978) . Distributed Systems

Self-stabilizing systems in spite of distributed control. Dijkstra (1974) .
The Byzantine Generals Problem. Lamport, Shostak, Pease (1982) .
Impossibility of Distributed Consensus With One Faulty Process. Fisher, Lynch, Patterson (1985) .
Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial. Schneider (1990) .
Practical Byzantine Fault Tolerance. Castro, Liskov (1999) .
Paxos made simple. Lamport (2001) .
Paxos made live - An Engineering Perspective. Chandra, Griesemer, Redstone (2007) .
In Search of an Understandable Consensus Algorithm. Ongaro, Ousterhout (2014) .

Designing for Usability: Key Principles and What Designers Think. Gould, Lewis (1985) . Human-Computer Interaction; User Interfaces

As We May Think. Bush (1945) .
Man-Computer symbiosis. Licklider (1958) .
Some Thoughts About the Social Implications of Accessible Computing. David, Fano (1965) .
Tutorials for the First-Time Computer User. Al-Awar, Chapanis, Ford (1981) .
The star user interface: an overview. Smith, Irby, Kimball (1982) .
Design Principles for Human-Computer Interfaces. Norman (1983) .
Human-Computer Interaction: Psychology as a Science of Design. Carroll (1997) .

The anatomy of a large-scale hypertextual Web search engine. Brin, Page (1998) . Information Retrieval; World-Wide Web

A Statistical Interpretation of Term Specificity in Retrieval. Spärck Jones (1972) .
World-Wide Web: Information Universe. Berners-Lee et al (1992) .
The PageRank Citation Ranking: Bringing Order to the Web. Page, Brin, Motwani (1998) .

Dynamo, Amazon’s Highly Available Key-value store. DeCandia et al (2007) . Internet Scale Data Systems

The Google File System. Ghemawat, Gobioff, Leung (2003) .
MapReduce: Simplified Data Processing on Large Clusters. Dean, Ghemawat (2004) .
Bigtable: A Distributed Storage System for Structured Data. Chang et al (2006) .
ZooKeeper: wait-free coordination for internet scale systems. Hunt et al (2010) .
The Hadoop Distributed File System. Shvachko et al (2010) .
Kafka: a Distributed Messaging System for Log Processing. Kreps, Narkhede, Rao (2011) .
CAP Twelve Years Later: How the "Rules" Have Changed. Brewer (2012) .
Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases. Verbitski et al (2017) .

On Designing and Deploying Internet Scale Services. Hamilton (2007) . Operations; Reliability; Fault-tolerance

Ironies of Automation. Bainbridge (1983) .
Why do computers stop and what can be done about it? Gray (1985) .
Recovery Oriented Computing (ROC): Motivation, Definition, Techniques, and Case Studies. Patterson et al (2002) .
Crash-Only Software. Candea, Fox (2003) .
Building on Quicksand. Helland, Campbell (2009) .

Thinking Methodically about Performance. Gregg (2012) . Performance

Performance Anti-Patterns. Smaalders (2006) .
Thinking Clearly about Performance. Millsap (2010) .

Bitcoin, A peer-to-peer electronic cash system. Nakamoto (2008) . Crytpocurrencies

Ethereum: A Next-Generation Smart Contract and Decentralized Application Platform. Buterin (2014) .

A Few Useful Things to Know About Machine Learning. Domingos (2012) . Machine Learning

Statistical Modeling: The Two Cultures. Breiman (2001) .
The Unreasonable Effectiveness of Data. Halevy, Norvig, Pereira (2009) .
ImageNet Classification with Deep Convolutional Neural Networks. Krizhevsky, Sutskever, Hinton (2012) .
Playing Atari with Deep Reinforcement Learning. Mnih et al (2013) .
Generative Adversarial Nets. Goodfellow et al (2014) .
Deep Learning. LeCun, Bengio, Hinton (2015) .
Attention Is All You Need. Vaswani et al (2017) .
Von Neumann's First Computer Program. Knuth (1970) .
Computing Machinery and Intelligence. Turing (1950) .
A Method for the Construction of Minimum-Redundancy Codes. Huffman (1952) .
Engineering a Sort Function. Bentley, McIlroy (1993) .
A Design Methodology for Reliable Software Systems. Liskov (1972) .
Programming with Abstract Data Types. Liskov, Zilles (1974) .
Why Functional Programming Matters. Hughes (1990) .
An Incremental Approach to Compiler Construction. Ghuloum (2006) .
No Silver Bullet: Essence and Accidents of Software Engineering. Brooks (1987) .
Communicating sequential processes. Hoare (1978) .
The UNIX Time- Sharing System. Ritchie, Thompson (1974) .
A Relational Model of Data for Large Shared Data Banks. Codd (1970) .
A Protocol for Packet Network Intercommunication. Cerf, Kahn (1974) .
New Directions in Cryptography. Diffie, Hellman (1976) .
Time, Clocks, and the Ordering of Events in a Distributed System. Lamport (1978) .
Designing for Usability: Key Principles and What Designers Think. Gould, Lewis (1985) .
The anatomy of a large-scale hypertextual Web search engine. Brin, Page (1998) .
Dynamo, Amazon’s Highly Available Key-value store. DeCandia et al (2007) .
On Designing and Deploying Internet Scale Services. Hamilton (2007) .
Thinking Methodically about Performance. Gregg (2012) .
Bitcoin, A peer-to-peer electronic cash system. Nakamoto (2008) .
A Few Useful Things to Know About Machine Learning. Domingos (2012) .

This list was inspired by (and draws from) several books and paper collections:

Papers We Love
Ideas That Created the Future
The Innovators
The morning paper
Distributed systems for fun and profit
Readings in Database Systems (the Red Book)
Fermat's Library
Classics in Human-Computer Interaction
Awesome Compilers
Distributed Consensus Reading List
The Decade of Deep Learning

A few interesting resources about reading papers from Papers We Love and elsewhere:

Should I read papers?
How to Read an Academic Article
How to Read a Paper. Keshav (2007) .
Efficient Reading of Papers in Science and Technology. Hanson (1999) .
On ICSE’s “Most Influential Papers”. Parnas (1995) .

Selection criteria

The idea is not to include every interesting paper that I come across but rather to keep a representative list that's possible to read from start to finish with a similar level of effort as reading a technical book from cover to cover.
I tried to include one paper per each major topic and author. Since in the process I found a lot of noteworthy alternatives, related or follow-up papers and I wanted to keep track of those as well, I included them as sublist items.
The papers shouldn't be too long. For the same reasons as the previous item, I try to avoid papers longer than 20 or 30 pages.
They should be self-contained and readable enough to be approachable by the casual technical reader.
They should be freely available online.
Examples of this are classic works by Von Neumann, Turing and Shannon.
That being said, where possible I preferred the original paper on each subject over modern updates or survey papers.
Similarly, I tended to skip more theoretical papers, those focusing on mathematical foundations for Computer Science, electronic aspects of hardware, etc.
I sorted the list by a mix of relatedness of topics and a vague chronological relevance, such that it makes sense to read it in the suggested order. For example, historical and seminal topics go first, contemporary internet-era developments last, networking precedes distributed systems, etc.

Sponsor this project

Contributors 4.

Python 100.0%

Research Topics in Software Engineering

This seminar is an opportunity to become familiar with current research in software engineering and more generally with the methods and challenges of scientific research.

Each student will be asked to study some papers from the recent software engineering literature and review them. This is an exercise in critical review and analysis. Active participation is required (a presentation of a paper as well as participation in discussions).

The aim of this seminar is to introduce students to recent research results in the area of programming languages and software engineering. To accomplish that, students will study and present research papers in the area as well as participate in paper discussions. The papers will span topics in both theory and practice, including papers on program verification, program analysis, testing, programming language design, and development tools.

Funding sources in top Software Engineering conference publications

Ieee account.

Change Username/Password
Update Address

Purchase Details

Payment Options
Order History
View Purchased Documents

Profile Information

Communications Preferences
Profession and Education
Technical Interests
US & Canada: +1 800 678 4333
Worldwide: +1 732 981 0060
Contact & Support
About IEEE Xplore
Accessibility
Terms of Use
Nondiscrimination Policy
Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

"We've shifted the responsibility of extracting relevant context for software engineering tasks from developers to the AI agents": Microsoft's AI-based framework turns developers to overnight 'mere supervisors'

What you need to know.

A research paper recently published by Microsoft details how its AI framework is turning software engineering into a fully automated task, rendering developers "mere supervisors."
NVIDIA's CEO had previously warned that coding is not a viable career option for the future generation as AI will eventually take over the profession.
Upskilling seems like a viable option, especially if you want to explore opportunities in coding.
Recruiters are actively seeking professionals with AI skills.

While safety and privacy are among the significant concerns among users with the prevalence and fast adoption of AI, the loss of jobs to AI is quickly rising in the ranks, too. Microsoft’s Bill Gates recently expressed fear of losing his career to AI but indicated the technology presents a 3-day workweek opportunity as it can handle mundane and recurring tasks.

NVIDIA’s CEO, Jensen Huang, shared the same sentiments and claimed that coding might be dead in the water as a career option for the next generation , given the rapid adoption of generative AI. As it turns out, Huang and Microsoft might be on the same train of thought regarding coding as a viable career option for the future generation.

Microsoft recently published a research paper that painted a clearer picture, highlighting the future of coding and developers as artificial intelligence becomes more widespread. The paper provides an in-depth analysis of AutoDev — an AI-powered framework designed to ‘assist’ developers with software development, ultimately redefining coding and automation.

The research paper further details instances where the framework was tested and performed well by providing repositories to tackle technical software engineering work. It’s worth noting that the technology also ships with AI-powered capabilities to validate its outcomes. AutoDev supports file editing, retrieval, build processes, execution, testing, and git operations.

As highlighted by the researchers in the paper:

“The developer’s role within the AutoDev framework transforms from manual actions and validation of AI suggestions to a supervisor overseeing multi-agent collaboration on tasks, with the option to provide feedback. Developers can monitor AutoDev’s progress toward goals by observing the ongoing conversation used for communication among agents and the repository.”

The report further outlines:

“We’ve shifted the responsibility of extracting relevant context for software engineering tasks and validating AI-generated code from users (mainly developers) to the AI agents themselves.”

With this in mind, It’s only a matter of time before the AI-based framework becomes self-sufficient and can run operations without human supervision or intervention. This shift ultimately means coding might not be a viable career option in the foreseeable future.

Upskilling seems like the only viable option to remain relevant

While commenting on the viability of coding as a career option for the next generation, NVIDIA's boss indicated that the youth are better off seeking opportunities in biology, education, manufacturing, or farming. He added that the only way around this challenge for people already invested in coding is to upskill (specifically in AI). This way, it'll be possible to maintain relevance and contribute to programming projects.

Coding isn't the only profession impacted by the fast adoption of AI. Architecture and graphic design jobs are also at risk , too. AI-powered tools like Image Creator from Designer (Bing Image Creator), ChatGPT, Midjourney, and more are already great at generating detailed and impressive structural designs within a moment's notice.

However, they aren't perfect either. Did you know AI struggles to create a simple, plain white image ? This limitation is on top of the heightened censorship of the tools, which has seemingly lobotomized their capabilities . There's been an alarming increase in reports flagging deepfakes and explicit images surfacing online. A study also revealed recruiters are seeking professionals with AI skills , so it might not be a bad idea to upskill in the area.

"We've shifted the responsibility of extracting relevant context for software engineering tasks from developers to the AI agents": Microsoft's AI-based framework turns developers to overnight 'mere supervisors'

If you have forgotten your password, we can send you a new one .

MyU : For Students, Faculty, and Staff

Fall 2024 CSCI Special Topics Courses

Cloud computing.

Meeting Time: 09:45 AM‑11:00 AM TTh Instructor: Ali Anwar Course Description: Cloud computing serves many large-scale applications ranging from search engines like Google to social networking websites like Facebook to online stores like Amazon. More recently, cloud computing has emerged as an essential technology to enable emerging fields such as Artificial Intelligence (AI), the Internet of Things (IoT), and Machine Learning. The exponential growth of data availability and demands for security and speed has made the cloud computing paradigm necessary for reliable, financially economical, and scalable computation. The dynamicity and flexibility of Cloud computing have opened up many new forms of deploying applications on infrastructure that cloud service providers offer, such as renting of computation resources and serverless computing. This course will cover the fundamentals of cloud services management and cloud software development, including but not limited to design patterns, application programming interfaces, and underlying middleware technologies. More specifically, we will cover the topics of cloud computing service models, data centers resource management, task scheduling, resource virtualization, SLAs, cloud security, software defined networks and storage, cloud storage, and programming models. We will also discuss data center design and management strategies, which enable the economic and technological benefits of cloud computing. Lastly, we will study cloud storage concepts like data distribution, durability, consistency, and redundancy. Registration Prerequisites: CS upper div, CompE upper div., EE upper div., EE grad, ITI upper div., Univ. honors student, or dept. permission; no cr for grads in CSci. Complete the following Google form to request a permission number from the instructor ( https://forms.gle/6BvbUwEkBK41tPJ17 ).

CSCI 5980/8980

Machine learning for healthcare: concepts and applications.

Meeting Time: 11:15 AM‑12:30 PM TTh Instructor: Yogatheesan Varatharajah Course Description: Machine Learning is transforming healthcare. This course will introduce students to a range of healthcare problems that can be tackled using machine learning, different health data modalities, relevant machine learning paradigms, and the unique challenges presented by healthcare applications. Applications we will cover include risk stratification, disease progression modeling, precision medicine, diagnosis, prognosis, subtype discovery, and improving clinical workflows. We will also cover research topics such as explainability, causality, trust, robustness, and fairness.

Registration Prerequisites: CSCI 5521 or equivalent. Complete the following Google form to request a permission number from the instructor ( https://forms.gle/z8X9pVZfCWMpQQ6o6 ).

Visualization with AI

Meeting Time: 04:00 PM‑05:15 PM TTh Instructor: Qianwen Wang Course Description: This course aims to investigate how visualization techniques and AI technologies work together to enhance understanding, insights, or outcomes.

This is a seminar style course consisting of lectures, paper presentation, and interactive discussion of the selected papers. Students will also work on a group project where they propose a research idea, survey related studies, and present initial results.

This course will cover the application of visualization to better understand AI models and data, and the use of AI to improve visualization processes. Readings for the course cover papers from the top venues of AI, Visualization, and HCI, topics including AI explainability, reliability, and Human-AI collaboration. This course is designed for PhD students, Masters students, and advanced undergraduates who want to dig into research.

Registration Prerequisites: Complete the following Google form to request a permission number from the instructor ( https://forms.gle/YTF5EZFUbQRJhHBYA ). Although the class is primarily intended for PhD students, motivated juniors/seniors and MS students who are interested in this topic are welcome to apply, ensuring they detail their qualifications for the course.

Visualizations for Intelligent AR Systems

Meeting Time: 04:00 PM‑05:15 PM MW Instructor: Zhu-Tian Chen Course Description: This course aims to explore the role of Data Visualization as a pivotal interface for enhancing human-data and human-AI interactions within Augmented Reality (AR) systems, thereby transforming a broad spectrum of activities in both professional and daily contexts. Structured as a seminar, the course consists of two main components: the theoretical and conceptual foundations delivered through lectures, paper readings, and discussions; and the hands-on experience gained through small assignments and group projects. This class is designed to be highly interactive, and AR devices will be provided to facilitate hands-on learning. Participants will have the opportunity to experience AR systems, develop cutting-edge AR interfaces, explore AI integration, and apply human-centric design principles. The course is designed to advance students' technical skills in AR and AI, as well as their understanding of how these technologies can be leveraged to enrich human experiences across various domains. Students will be encouraged to create innovative projects with the potential for submission to research conferences.

Registration Prerequisites: Complete the following Google form to request a permission number from the instructor ( https://forms.gle/Y81FGaJivoqMQYtq5 ). Students are expected to have a solid foundation in either data visualization, computer graphics, computer vision, or HCI. Having expertise in all would be perfect! However, a robust interest and eagerness to delve into these subjects can be equally valuable, even though it means you need to learn some basic concepts independently.

Sustainable Computing: A Systems View

Meeting Time: 09:45 AM‑11:00 AM Instructor: Abhishek Chandra Course Description: In recent years, there has been a dramatic increase in the pervasiveness, scale, and distribution of computing infrastructure: ranging from cloud, HPC systems, and data centers to edge computing and pervasive computing in the form of micro-data centers, mobile phones, sensors, and IoT devices embedded in the environment around us. The growing amount of computing, storage, and networking demand leads to increased energy usage, carbon emissions, and natural resource consumption. To reduce their environmental impact, there is a growing need to make computing systems sustainable. In this course, we will examine sustainable computing from a systems perspective. We will examine a number of questions: • How can we design and build sustainable computing systems? • How can we manage resources efficiently? • What system software and algorithms can reduce computational needs? Topics of interest would include: • Sustainable system design and architectures • Sustainability-aware systems software and management • Sustainability in large-scale distributed computing (clouds, data centers, HPC) • Sustainability in dispersed computing (edge, mobile computing, sensors/IoT)

Registration Prerequisites: This course is targeted towards students with a strong interest in computer systems (Operating Systems, Distributed Systems, Networking, Databases, etc.). Background in Operating Systems (Equivalent of CSCI 5103) and basic understanding of Computer Networking (Equivalent of CSCI 4211) is required.

Future undergraduate students
Future transfer students
Future graduate students
Future international students
Diversity and Inclusion Opportunities
Learn abroad
Living Learning Communities
Mentor programs
Programs for women
Student groups
Visit, Apply & Next Steps
Information for current students
Departments and majors overview
Departments
Undergraduate majors
Graduate programs
Integrated Degree Programs
Additional degree-granting programs
Online learning
Academic Advising overview
Academic Advising FAQ
Academic Advising Blog
Appointments and drop-ins
Academic support
Commencement
Four-year plans
Honors advising
Policies, procedures, and forms
Career Services overview
Resumes and cover letters
Jobs and internships
Interviews and job offers
CSE Career Fair
Major and career exploration
Graduate school
Collegiate Life overview
Scholarships
Diversity & Inclusivity Alliance
Anderson Student Innovation Labs
Information for alumni
Get engaged with CSE
Upcoming events
CSE Alumni Society Board
Alumni volunteer interest form
Golden Medallion Society Reunion
50-Year Reunion
Alumni honors and awards
Outstanding Achievement
Alumni Service
Distinguished Leadership
Honorary Doctorate Degrees
Nobel Laureates
Alumni resources
Alumni career resources
Alumni news outlets
CSE branded clothing
International alumni resources
Inventing Tomorrow magazine
Update your info
CSE giving overview
Why give to CSE?
College priorities
Give online now
External relations
Giving priorities
Donor stories
Impact of giving
Ways to give to CSE
Matching gifts
CSE directories
Invest in your company and the future
Recruit our students
Connect with researchers
K-12 initiatives
Diversity initiatives
Research news
Give to CSE
CSE priorities
Corporate relations
Information for faculty and staff
Administrative offices overview
Office of the Dean
Academic affairs
Finance and Operations
Communications
Human resources
Undergraduate programs and student services
CSE Committees
CSE policies overview
Academic policies
Faculty hiring and tenure policies
Finance policies and information
Graduate education policies
Human resources policies
Research policies
Research overview
Research centers and facilities
Research proposal submission process
Research safety
Award-winning CSE faculty
National academies
University awards
Honorary professorships
Collegiate awards
Other CSE honors and awards
Staff awards
Performance Management Process
Work. With Flexibility in CSE
K-12 outreach overview
Summer camps
Outreach events
Enrichment programs
Field trips and tours
CSE K-12 Virtual Classroom Resources
Educator development
Sponsor an event

COMMENTS

Top 10 Software Engineer Research Topics for 2024
Top Software Engineer Research Topics. 1. Artificial Intelligence and Software Engineering. Intersections between AI and SE. The creation of AI-powered software engineering tools is one potential research area at the intersection of artificial intelligence (AI) and software engineering. These technologies use AI techniques that include machine ...
Software Engineering's Top Topics, Trends, and Researchers
For this theme issue on the 50th anniversary of software engineering (SE), Redirections offers an overview of the twists, turns, and numerous redirections seen over the years in the SE research literature. Nearly a dozen topics have dominated the past few decades of SE research—and these have been redirected many times. Some are gaining popularity, whereas others are becoming increasingly ...
Proceedings of the ACM on Software Engineering
The Proceedings of the ACM on Software Engineering (PACMSE) is a premier Gold Open Access journal that publishes top-quality, original research on all aspects of software engineering, from requirements elicitation to quality assessment and from design to maintenance, evolution, and deployment. PACMSE covers a broad range of topics and methods that help conceive, create, and maintain better ...
software engineering Latest Research Papers
End To End . Predictive Software. The paper examines the principles of the Predictive Software Engineering (PSE) framework. The authors examine how PSE enables custom software development companies to offer transparent services and products while staying within the intended budget and a guaranteed budget.
Journal of Software Engineering Research and Development
They wanted to define values and basic principles for better software development. On top of being brought into focus, the ... Philipp Hohl, Jil Klünder, Arie van Bennekum, Ryan Lockard, James Gifford, Jürgen Münch, Michael Stupperich and Kurt Schneider. Journal of Software Engineering Research and Development 2018 6 :15.
Trending Topics in Software Engineering
In this new column Trending Topics in Software Engineering, we aim at providing insights, reports, and outlooks on how researchers and practitioners around the world are working (or planning to work) on those trends. We intend to collect the challenges they are facing or foresee, and explore them in future issues.
Software Engineering
Software Engineering. At Google, we pride ourselves on our ability to develop and launch new products and features at a very fast pace. This is made possible in part by our world-class engineers, but our approach to software development enables us to balance speed and quality, and is integral to our success. Our obsession for speed and scale is ...
Software Engineering and Applications
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications. ... This Topic on software ...
2022 Research Review
2022 Research Review. At the 2022 Research Review, our researchers detail how they are forging a new path for software engineering by executing the SEI's technical strategy to deliver tangible results. Researchers highlight methods, prototypes, and tools aimed at the most important problems facing the DoD, industry, and academia, including AI ...
Machine Learning for Software Engineering
There is therefore significant potential for utilizing ML approaches to address the problem of increasing software application complexity and scale. This Research Topic will concern the application of AI techniques such as machine learning to software engineering: the application of AI techniques to accelerate software development and to ...
[2204.03254] The General Index of Software Engineering Papers
We introduce the General Index of Software Engineering Papers, a dataset of fulltext-indexed papers from the most prominent scientific venues in the field of Software Engineering. The dataset includes both complete bibliographic information and indexed ngrams (sequence of contiguous words after removal of stopwords and non-words, for a total of 577 276 382 unique n-grams in this release) with ...
150 Best Research Paper Topics For Software Engineering
This paper reviews software tools to solve complicated tasks in the analysis of data. The paper compares NVivo, HyperRESEARCH, and Dedoose. Data Scientist and Software Development. Data scientists convert data into insights, giving elaborate guidance to those who use the data to make educated decisions and take action.
Software Engineer Research Paper Topics 2021: Top 5
Thus, to help you land on the best topic for your needs, we have listed the top 5 software engineer research paper topics in the next sections. Machine Learning. Machine learning is one of the most used research topics of software engineers. If you're not yet familiar with this, it's a field that revolves around producing programs that ...
Carnegie Mellon University, Software Engineering Insitute
Explainable Verification: Survey, Situations, and New Ideas April 16, 2024 • White Paper By Bjorn Andersson, Mark H. Klein, Dionisio de Niz This report focuses on potential changes in software development practice and research that would help tools used for formal methods explain their output, making software practitioners more likely to trust …
(PDF) Software Engineering Research Topics
5) Software Testing. 6) Software Measurement. 7) Software Product Lines. 8) Software Architecture. 9) software verification. 10) software business. 11) Software Refactoring. 12) software design ...
Research Topics in Software Engineering
Overview. This seminar is an opportunity to become familiar with current research in software engineering and more generally with the methods and challenges of scientific research. Each student will be asked to study some papers from the recent software engineering literature and review them. This is an exercise in critical review and analysis.
Topic modeling in software engineering research
Topic modeling using models such as Latent Dirichlet Allocation (LDA) is a text mining technique to extract human-readable semantic "topics" (i.e., word clusters) from a corpus of textual documents. In software engineering, topic modeling has been used to analyze textual data in empirical studies (e.g., to find out what developers talk about online), but also to build new techniques to ...
[1608.08100] Finding Trends in Software Research
This paper explores the structure of research papers in software engineering. Using text mining, we study 35,391 software engineering (SE) papers from 34 leading SE venues over the last 25 years. These venues were divided, nearly evenly, between conferences and journals. An important aspect of this analysis is that it is fully automated and repeatable. To achieve that automation, we used a ...
An Analysis of Research in Software Engineering:
This paper presents a software-aided method for assessment and trend analysis, which can be used in software engineering as well as other research fields in computer science (or other disciplines). The method proposed in this paper is modular and automated compared with the method in prior studies [7, 10-22, 2].
PDF Writing Good Software Engineering Research Papers
Minitutorial. Mary Shaw. Carnegie Mellon University [email protected]. Abstract. Software engineering researchers solve problems of several different kinds. To do so, they produce several different kinds of results, and they should develop appropriate evidence to validate these results. They often report their research in conference papers.
Unique List of Software Engineering Research Topics
Unique Software Engineering Research Topics for Students. more software engineers are needed as a result of the growing reliance on technology in both personal and professional spheres of life. Software engineering research topics are essential for solving complicated issues, increasing productivity, and fostering innovation.
Papers for Software Engineers
A curated list of papers that may be of interest to Software Engineering students or professionals. See the sources and selection criteria below. List of papers by topic. Von Neumann's First Computer Program. Knuth (1970). Computer History; Early Programming. The Education of a Computer. Hopper (1952). Recursive Programming.
Research Topics in Software Engineering
Overview. This seminar is an opportunity to become familiar with current research in software engineering and more generally with the methods and challenges of scientific research. Each student will be asked to study some papers from the recent software engineering literature and review them. This is an exercise in critical review and analysis.
Funding sources in top Software Engineering conference publications
Bibliometric studies analyze existing research pub-lications and assist in better understanding a research area. Concerning the area of Software Engineering, many works on analysis of research topics and citations exist but additional data have not been analyzed. In this paper, we present a preliminary work toward a funding analysis in top Software Engineering conferences. We have chosen ICSE ...
"We've shifted the responsibility of extracting relevant context for
A research paper recently published by Microsoft details how AI is turning software engineering into a fully automated task, rendering developers "mere supervisors." This reiterates the NVIDIA CEO ...
Program Analysis and Verification (Seminar)
Program analysis is the research area that studies the automatic analysis of computer programs. The methods that are developed in this research area e.g., help programmers to understand complex programs, allow compilers to optimize their code, and enable computers to check the correctness of programs. In this seminar each student will study a research paper and give a talk in which he/she ...
Fall 2024 CSCI Special Topics Courses
Visualization with AI. Meeting Time: 04:00 PM‑05:15 PM TTh. Instructor: Qianwen Wang. Course Description: This course aims to investigate how visualization techniques and AI technologies work together to enhance understanding, insights, or outcomes. This is a seminar style course consisting of lectures, paper presentation, and interactive ...
MizzouForward Keynote: Valorization of Sustainable Resources for Multi
Join us for "Valorization of Sustainable Resources for Multi-industry Decarbonization", a keynote presentation by Dr. Xianglan Bai, College of Engineering Mechanical and Aerospace, ChBME, MizzouForward faculty candidate. Dr. Bai will present on her research for approximately 40-minutes with a 20-minute question and answer session to follow. Dr. Xianglan Bai is an Associate Professor in the ...

Top 10 Software Engineer Research Topics for 2024