Continuous Experimentation Cookbook: How to do Lean Startup Experiments

Bildschirmfoto 2017-07-10 um 10.03.23

We have co-authored a cookbook on continuous experimentation based on our research in the N4S program:

Continuous experimentation Cookbook – An introduction to systematic experimentation for software-intensive businesses provides an introduction to continuous experimentation, which is a systematic way to continuously test your product or service value and whether your business strategy is working.

An increasing number of companies are involved in building software-intensive products and services – hence the popular slogan “every business is a software business”. Software allows companies to disrupt existing markets because of its flexibility. This creates highly dynamic and competitive environments, imposing high risks to businesses. One risk is that the product or service is of only little or no value to customers, meaning the effort to develop it is wasted. In order to reduce such risks, you can adopt an experiment- driven development approach where you validate your product ideas before spending resources on fully developing them. Experiments allow you to test assumptions about what customers really want and react if the assumptions are wrong.

This book provides an introduction to continuous experimentation, which is a systematic way to continuously test your product or service value and whether your business strategy is working. With real case examples from Ericsson, Solita, Vaadin, and Bittium, the book not only gives you the concepts needed to start performing continuous experimentation, but also shows you how others have been doing it.

Take a look at the book!

Traditional, Agile and Beyond: Book on “Managing Software Process Evolution”

Bildschirmfoto 2016-03-15 um 16.27.34

  • Collects and summarizes the state of the art in analysis, design, implementation, management and governance, improvement and enactment of software processes
  • Provides the foundations of current research on software process improvement and management and lays the basis for further problem-driven research
  • Addresses researchers and practitioners by providing recent research results as well as experiences and best practices

This book focuses on the design, development, management, governance and application of evolving software processes that are aligned with changing business objectives, such as expansion to new domains or shifting to global production. In the context of an evolving business world, it examines the complete software process lifecycle, from the initial definition of a product to its systematic improvement. In doing so, it addresses difficult problems, such as how to implement processes in highly regulated domains or where to find a suitable notation system for documenting processes, and provides essential insights and tips to help readers manage process evolutions. And last not least, it provides a wealth of examples and cases on how to deal with software evolution in practice.

Reflecting these topics, the book is divided into three parts. Part 1 focuses on software business transformation and addresses the questions of which process(es) to use and adapt, and how to organize process improvement programs. Subsequently, Part 2 mainly addresses process modeling. Lastly, Part 3 collects concrete approaches, experiences, and recommendations that can help to improve software processes, with a particular focus on specific lifecycle phases.

This book is aimed at anyone interested in understanding and optimizing software development tasks at their organization. While the experiences and ideas presented will be useful for both those readers who are unfamiliar with software process improvement and want to get an overview of the different aspects of the topic, and for those who are experts with many years of experience, it particularly targets the needs of researchers and Ph.D. students in the area of software and systems engineering or information systems who study advanced topics concerning the organization and management of (software development) projects and process improvements projects.


  • [PDF] [DOI] Marco Kuhrmann, Jürgen Münch, Ita Richardson, Andreas Rausch, Jason He Zhang, editor. Managing Software Process Evolution: Traditional, Agile and Beyond – How to handle process change?. Springer-Verlag, 2016.
    [Bibtex] [doi] [url] [pdf]
    editor = {Marco Kuhrmann, Jürgen Münch, Ita Richardson, Andreas Rausch, Jason He Zhang},
    title = {Managing Software Process Evolution: Traditional, Agile and Beyond - How to handle process change?},
    publisher = {Springer-Verlag},
    year  = {2016},
    url = {},
    doi = {10.1007/978-3-319-31545-4},

Springer Website of the Book


How to Onboard in Open Source Projects?

Onboarding is a process that helps newcomers become integrated members of their organisation. Successful onboarding programs can result in increased performance in conventional organisations, but there is little guidance on how to onboard new developers in Open Source Software (OSS) projects.

Bildschirmfoto 2015-12-27 um 15.12.34

[blue = mentored, red = non-mentored]

In several studies, we examined how mentoring and project characteristics influence the effectiveness and efficiency of the onboarding process.

Recommendations for project leaders and managers:

  • Identify core developers who can spend a limited time on intensive mentoring. Provide direct incentives for mentoring. For example, the opportunity to get help for pending tasks can be attractive for potential mentors. Clearly limiting the duration of mentoring reduces the negative effect on the mentor’s performance in other project tasks and can reduce some of the resistance to participate.
  • Organize or sponsor collocated events, such as Hackathons, and use them to kick off the mentoring period. Face-to-face events can help team members and mentors to focus on problems which are difficult to overcome in a distributed setting, and can further boost the success of onboarding new members into virtual teams. Many open source projects already arrange periodic collocated events and welcome participation by newcomers. Engaging with these provides direct access to the project community.
  • Expect considerable variation in performance increases over time. Assessing the cost and outcomes of mentoring requires understanding onboarding as a learning process which does not proceed linearly. Some onboarding activity will not be publicly visible. Engage directly with mentors and newcomers to gain insight of how onboarding is progressing.
  • Adapt the onboarding program to project characteristics and culture. Take the maturity of the target project and its existing onboarding practices into account. Low-maturity projects may require more support to instill a productive mentoring culture, while mature projects may already have an existing culture of integrating new developers and may be ready for tailoring towards more specific inclusion targets.

Keywords: Onboarding, Organisational Socialisation, Open Source Software, Case Study, Mentoring, Software Teams, Distributed Development


Read more about the general effect of onboarding support on newcomer activity and the moderating effect of project characteristics, such as age, number of contributors, and appeal, on the speed of the onboarding process:

  • [PDF] [DOI] Fabian Fagerholm, Alejandro S. Guinea, Jürgen Münch, Jay Borenstein. The Role of Mentoring and Project Characteristics for Onboarding in Open Source Software Projects. In Proceedings of the 8th ACM-IEEE International Symposium on Software Engineering and Measurement (ESEM 2014), Torino, Italy, September 2014.
    [Bibtex] [doi] [pdf]
    author = {Fabian Fagerholm, Alejandro S. Guinea, Jürgen Münch, Jay Borenstein}, 
    title = {The Role of Mentoring and Project Characteristics for Onboarding in Open Source Software Projects}, 
    booktitle = {Proceedings of the 8th ACM-IEEE International Symposium on Software Engineering and Measurement (ESEM 2014)},
    year = {2014},
    month = {September},
    address = {Torino, Italy}

Read more about the developer activity during onboarding, the potential cost of mentoring in terms of lost productivity, and find guidelines for using mentoring as an onboarding support mechanism.

  • [PDF] [DOI] Fabian Fagerholm, Alejandro Sanchez Guinea, Jay Borenstein, Jürgen Münch. Onboarding in Open Source Projects. IEEE Software, 31(6):54-61, 2014.
    [Bibtex] [doi] [url] [pdf]
    author={Fabian Fagerholm, Alejandro Sanchez Guinea, Jay Borenstein, Jürgen Münch}, 
    journal={IEEE Software}, 
    title={Onboarding in Open Source Projects}, 
    volume = {31},
    number = {6},
    publisher = {IEEE},
    URL = {},
    pages = {54-61},
    doi = {10.1109/MS.2014.107},
    keywords={open source software projects; virtual teams; mentoring; global software development; distributed software development; case study}}

Talk: “From Agile Development to Continuous Experimentation – How to Create Value with Software?“

I will give a talk on the topic
“From Agile Development to Continuous Experimentation – How to Create Value with Software“

After my talk, Fabian Fagerholm will give talk on the topic
“Software Factory – A Retrospective and Future Directions”

When: Thursday, December 10, 4:30pm
Where: University of Helsinki, Kumpula campus, Physicum building, auditorium D101 (Gustaf Hällströmin katu 2a)

Before the talk, cocktails will be served in front of the auditorium.
I am looking forward to seeing you.

Interview at Herman Hollerith Center

I was interviewed about my avenues for future research at the Herman Hollerith Center (HHZ) (in German).

The Herman Hollerith Center for digital business and innovative software in Böblingen is operated in partnership between Reutlingen University, University of Stuttgart, Esslingen University, and the five companies Daimler AG, Hewlett-Packard, IBM, Novatec, and Robert Bosch GmbH. It is supported by the city and the district of Böblingen.

The Herman Hollerith Center offers a PhD program and two master’s degree programs in Services Computing and Digital Business Management. The center is hosted by the Department of Computer Science of Reutlingen University.

The Herman Hollerith Center is located in Böblingen, the city with Europe’s largest pool of innovation. Böblingen is very close to Stuttgart with many companies around. Daimler and IBM are in Böblingen as well as Hewlett-Packard and Elektrobit. Porsche, Bosch, and many other companies are close by. Several start-up incubators are in Böblingen to facilitate radical innovations.

The research within the Herman Hollerith Center focuses on sound and effective methods for the development and evolution of software-enabled services and products and the digital transformation of organizations. The research addresses significant problems that reflect important needs of companies. The research areas include software engineering, service science, data management, smart data services, business intelligence, internet of things, enterprise architectures, business modeling and enterprise social networks.

See the interview here.

Talk: “The Wheels of Value Model: Driving Product Ideas to Their Fullest Strength”

The Wheels of Value Model is a tool for driving product ideas to their fullest strength by systematically unearthing critical product assumptions. Instead of identifying assumptions for each element of a business model it generates a closed value chain among the right actors and ensures that you do not miss important links. By doing this you can rapidly see what you need to validate. This talk explains the main elements.

Presenter: Prof. Dr. Jürgen Münch
When: 16 Sept. 2015, 10:15 am
Where: Pori, N4SQ3, Yyteri Hotel, Finland

wheels of value excerpt

Is Water-Scrum-Fall Reality? On the Use of Agile and Traditional Development Practices – Accepted at PROFES 2015

by Georgios Theocharis, Marco Kuhrmann, Jürgen Münch, Philipp Diebold

Abstract. For years, agile methods are considered the most promising route toward successful software development, and a considerable number of publications studies the (successful) use of agile methods and reports on the benefits companies have from adopting agile methods. Yet, since the world is not black or white, the question for what happened to the traditional models arises. Are traditional models replaced by agile methods? How is the transformation toward Agile managed, and, moreover, where did it start? With this paper we close a gap in literature by studying the general process use over time to investigate how traditional and agile methods are used. Is there coexistence or do agile methods accelerate the traditional processes’ extinction? The findings of our literature study comprise two major results: Studies and reliable numbers on the general process model use are rare, i.e., we lack quantitative data on the actual process use and, thus, we often lack the ability to ground process-related research in practically relevant issues. Second, despite the assumed dominance of agile methods, our results clearly show that companies enact context-specific hybrid solutions in which traditional and agile development approaches are used in combination.



  • [PDF] [DOI] Georgios Theocharis, Marco Kuhrmann, Jürgen Münch, Philipp Diebold. Is Water-Scrum-Fall Reality? On the Use of Agile and Traditional Development Practices. In Proceedings of the 16th International Conference on Product-Focused Software Process Improvement (PROFES 2015), volume 9459 of LNCS, pages 149-167. Springer-Verlag, 2015.
    [Bibtex] [doi] [url] [pdf]
    booktitle={Proceedings of the 16th International Conference on Product-Focused Software Process Improvement (PROFES 2015)},
    title={Is Water-Scrum-Fall Reality? On the Use of Agile and Traditional Development Practices},
    keywords={Development Practices, Agile Methods, Software Process, Systematic Literature Review, Comparative Study},
    author={Georgios Theocharis, Marco Kuhrmann, Jürgen Münch, Philipp Diebold},

Keynote at ICSSP 2015: Software Development as an Experiment System

“Software Development as an Experiment System” is the title of the keynote that Jürgen Münch, University of Helsinki, is going to present on 25th of August 1 pm at the International Conference on Software and System Process 2015.


Abstract: Most modern software development activities are focusing on domains of emergence where experts cannot know a priori what kind of software provides value to users and customers. This is fundamentally different to traditional software engineering for large systems where a priori analysis by experts is used to identify requirements. While the latter is gaining a niche software category, developing and establishing development practices for domains of emergence is becoming significantly important and urgent. A major challenge is to find the right scope for software development. There are many options on what to deliver. Software practices are needed that help in determining what customers want and creating the right capabilities for them. In this talk I introduce an approach for steering software development towards the right scope by continuously conducting experiments. This includes systematically observing users’ behavioral responses to stimuli such as features. Insights from experiments directly influence frequent iterative deliveries. Success cases from industry show that such an experimental approach helps companies to gain competitive advantage by reducing uncertainties and rapidly finding product roadmaps that work. This presentation is aimed at process engineers, researchers, product managers, startup founders, business people, software developers, and anyone who is interested in making an impact with their products. It shows the relevance of experimentation in software development and how it influences the software process. In addition, new methods and practices are presented that have been tested in different industry environments. The talk answers questions such as:

  • How do we rapidly and effectively create value for users and customers by integrating experimentation into software processes?
  • How do we identify the relevant experiments we need to conduct for making good product decisions?
  • What are the components of a good hypothesis?
  • How do we link the experimental findings with product decisions and dynamically change a product roadmap?
  • What are the key obstacles when introducing continuous experimentation in an organization and how can we address them?
  • What are future avenues for software process research?

Date: Tuesday, 25th of August 1 pm

Venue: Tallinn, Viro Hotel, Estonia

Jürgen Münch is a full professor in the Department of Computer Science at the University of Helsinki. He specializes in software and systems engineering, in particular data- and value-driven software engineering, software measurement, innovative software processes, software quality assurance, and global software development. Results are documented in five books and more than 140 refereed publications. He is co-inventor of GQM+, a method for aligning company’s software strategies with long-term business goals. Dr. Münch has been a principal investigator of numerous research and industrial development projects and regularly consults for companies on issues including quality improvement, IT business alignment, software measurement, process engineering, and software technology in general.

All keynotes at ICSSP 2015 at a glance.