What actually is agile?
Agile development … there is a certain lightness to it. In fact, agile development helps achieve better results more quickly. However, the process is also stricter than what you are probably living at present. Scrum is more strict towards management and requires a working integration and test process, especially in embedded systems.
The first agile value is “Individuals and interactions over processes and tools.” To value people and interactions more highly in everyday work, it is important that working processes and tools exist that are naturally used.
Agile sounds innovative, just like Scrum with its user stories and collaboration. The agile principle of welcoming change has customers and senior management in development organizations beaming with anticipation: It provides the option to change course mid-project without project managers looking annoyed. And all this has the blessing of the Agile Manifesto and uses a framework called Scrum. Seriously? If your organization wants to be truly successful with Scrum, it is necessary to take a deeper look. It is highly probable that the process is significantly stricter than the one you are living today. In addition, outside the team, Scrum is stricter towards management. Moreover, at the early stage of its introduction, Scrum is more taxing than your current integration and test process, especially for embedded systems.
Principles
Much has already been written about the agile mindset, for example, in the Agile Manifesto [1]. The principles behind Scrum are the same as the ones the Agile Manifesto describes. “We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.” [2]
We often find good approaches and motivation; however, equally often, the reception in teams only focuses on the above four core values. Furthermore, the introduction of Scrum may cause confusion, duplicated work, and insecurity, especially when the team is in the process of self-discovery or when collaborating with customers that do not yet use agile approaches, as can be read in [3].
Some requirements of embedded software engineering – strongly coupled to hardware, with very price-sensitive electronic components and long lead times – differ.
The problems start when teams overlook the added “while there is value in the items on the right.” The agile principles do not mean that processes, tools, and documentation no longer matter. The preposition is helpful: “Individuals and interactions over processes and tools.” Over.
When individuals and interactions are above processes and tools, the latter are below them and therefore form the basis. Individuals are more important to us; however, for this to have space, we need useful processes and tools.
If working software is more important to us, we need just enough documentation below so that everyone knows how the existing software and development work.
If collaboration with the customer is more important to us, we need meaningful contract negotiations to enable and shape this collaboration. Anyone who wants to meet all functional safety requirements for embedded software engineering of automotive systems must define where responsibilities lie.
Particularly if we want to respond to change quickly, a plan that specifies the original idea and the resulting objectives is a useful basis. The development of embedded systems has framework conditions that are best laid down in a plan.
Planning replaces coincidence with error. And the team can learn from error [1].
The Agile Manifesto is often used as a weapon against any classical method. But the Agile Manifesto is much more: It outlines a way, amid all the classical methods, to focus even more on succeeding – as a result of collaboration between people in a changing world.
Infrastructure 1 – DevOps
Scrum states that the cross-functional team should act quasi-autonomously and define its processes and methods in a way that is best for the project. And yes, with absolutely constant teams, this can actually work. In reality, however, companies are faced with turnover so that new members keep joining the team. Projects are often so big that multiple teams work on them, additionally increasing the probability of new team members.
If the team wants to provide working software at regular intervals, it will not be able to avoid frequent software functionality checks. This brings continuous integration and nightly builds into focus. In an appropriate infrastructure, they automatically get code and other artifacts from a version control system, build and comprehensively test the software.
According to Scrum, each team is allowed to self-organize and choose the infrastructure. However, it is essential to ensure that each team uses a working build infrastructure. An existing infrastructure should be available to teams that cannot or do not want to select their own infrastructure.
Creating a project and build infrastructure is a complex task for a developer and not easily achieved in an afternoon. All this results in the build infrastructure and everything that goes with it: Development Operations (DevOps) [4]. Without claiming to be complete, it needs:
- A version control system, e.g. Subversion [5], Git [6].
- An issue tracking system such as Jira [7], Trello, Leankit, Kanbanize.
- A build server, e.g. Jenkins [8].
- Deployment automation, e.g. GoCD [9].
- A static code analysis tool, e.g. Polyspace [10], Klocwork [11] run both on the build server and locally for the developers.
- A code coverage tool such as Bullseye [12].
- Test automation, the framework depends on the technology used.
- A branching strategy for risk-free development of new features [13]
- Fully-automated reporting. No one should have to manually compile statistics and states.
- A multi-stage integration and merge process on the build server to merge branches [14].
- A sandbox system such as Docker [15] so that everyone has the same environment in a reproducible fashion.
- Role definitions for Test Manager, Integration Manager, Release Manager, Build Manager.
Aside from working tools, it is important that the team member, with knowledge of the above systems and processes, stays on the project to be able to adequately adapt changes. This is where the Scrum requirement for cross-functional teams becomes important: The person concerned must be and remain part of the project; otherwise, there are repeated delays. For large-scale projects with multiple teams, a DevOps team can actually be the most useful solution. To counter turnover problems, the required knowledge is ensured by a whole team of “knowing individuals”. However, this DevOps team must be directly integrated into the development teams. Another thing is also clear: The infrastructure must run reliably; it is used as a production system.
Scrum states: “There is no test phase. There is only a development phase that includes test activities.” [3, p. 228] The same applies to system software releases: “Releasing must become a standard process that is completely noiseless.” [3, p. 230]
Infrastructure 2 – embedded hardware
Another challenge in creating the infrastructure is illustrated by the question of how we get to the target platform as quickly as possible. In the context of embedded systems, the Agile Manifesto’s “working software” must actually mean a “working system.”
Embedded hardware is a bit hard to handle, which is not due to the board’s material. Mechatronic hardware developers think in terms of lead times for board layouts, delivery times, and availability of current processors.
A working system does not necessarily have to mean an actually existing unit of software, hardware, and mechanical system. The working system can also be designed as a simulation that is used to verify the actual customer benefits at a very early stage [16]. Especially frameworks such as AUTOSAR help enable developers to look at, initially separately from hardware, functionalities within the scope of a simulation [17]. In this context, a cautious approach must be taken regarding ill-considered modeling and thoughtless simulation. Setting up simulations that do not provide necessary insight can consume significant time and effort [18].
The Processor-in-the-Loop (PiL) and Hardware-in-the-Loop (HiL) methods are then successively added to the pure simulation in software to include specific effects of process architecture, ECU hardware, and mechatronic environment in automated testing. All DevOps infrastructure requirements continue to apply: Only what can be tested in an automated manner enables the team to supply a version of a system at fixed intervals.
And the system architecture can do the rest. Sharply defined modules – either as microservices [19] or as AUTOSAR components – make it possible to separately develop, test, and supply them.
All these frameworks and methods seem to restrict the teams’ freedom. However, they actually enable more freedom in solving the actual task, provided that the framework conditions are known and remain relatively stable.
Managers and the introduction of Scrum
Scrum applies not only to the development team(s). It applies to the entire organization. Even if only parts of an organization actually work in accordance with Scrum, everyone who interacts with these teams should understand what Scrum means.
This also, and especially, applies to managers. Scrum explicitly states that work should be done in sprints, i.e., development periods that last a few weeks. At the end of these sprints, a runnable system is supplied to the customer.
The requirements for the next sprint are planned after each completed sprint. Important: During the sprint, the requirements remain stable. This means there is no manager walking through the door, suddenly assigning different tasks to the developers. If new requirements arise during a sprint, they will generally not be integrated into the current one but added to the product backlog. If there is an increase in the number of cases where requirements scheduled for the current sprint suddenly become obsolete, this is an implicit job for the Scrum Master and Product Owner: Think about better requirements and their coordination with the customer.
The Scrum Master’s primary task is now to protect the team from external organizational interrupts. The senior management of an organization should do its best to support these efforts by not intervening in current processes and using the sprint cycle to include changed situations and requirements.
As a result, team training alone is not enough to introduce Scrum. Training managers in adjacent organizational units is another explicit requirement.
Agile means that we must change the entire organization to a structure that enables the project outcome and is geared towards adding value. This does not mean that each department has to use agile methods; however, they should know their role in the agile organization [20].
The introduction of Scrum has a beginning, but, due to the nature of self-organization, it has no end. Changed framework conditions and project environments will result in changed implementations of Scrum. Your manager will have to get used to the fact that there is no static development process that is laid out once and stays that way forever.
Summary
Agile approaches in embedded software engineering are not a sure-fire success. Enabling teams, through a good infrastructure, to regularly deliver with minimal effort and providing them with methods to consider the concerns of embedded systems is a good basis. When, in addition, managers let teams work in peace, success is almost guaranteed.
At Elektrobit, we have successfully supported large parts of our own development organization and renowned embedded software engineering customer organizations in their transition to agile methods – and we still keep learning. Contact us.
Functional safety in combination with agile software development
Everything important at a glance
Download our Infographic on the topic how to combine agile & safety in automotive development, and have all relevant information available at a glance.
References
[1] J. Sutherland and J. Sutherland, Scrum: The Art of Doing Twice the Work in Half the Time, UK: Random House Business Books, 2014.
[2] W. Cunningham and others, “Manifest für Agile Softwareentwicklung”, 2001. [Online]. Available: http://agilemanifesto.org/iso/de/manifesto.html.
[3] B. Gloger, Scrum: Produkte zuverlässig und schnell entwickeln, 5. Auflage Hrsg., München: Carl Hanser Verlag, 2016.
[4] G. Kim, J. Humble, P. Debois, J. Willis and T. Demmig, Das DevOps-Handbuch, Heidelberg: O’Reilly / dpunkt.verlag GmbH, 2017.
[5] B. Collins-Sussman, B. W. Fitzpatrick and C. M. Pilato, “Version Control with Subversion”, 2017. [Online]. Available: http://svnbook.red-bean.com/index.de.html.
[6] S. Chacon and B. Straub, Pro Git book, US: Apress, 2009.
[7] M. Doar, Practical JIRA Administration, UK: O’Reilly, 2011.
[8] “Jenkins Handbook”, [Online]. Available: https://jenkins.io/doc/book/. [Accessed 11 10 2017].
[9] „GoCD: Open Source Continuous Delivery and Automation Server“, [Online]. Available: https://www.gocd.org/. [Access 11 10 2017].
[10] „Polyspace. Making Critical Code Safe and Secure“, MathWorks, [Online]. Available: https://de.mathworks.com/products/polyspace.html. [Access 11 10 2017].
[11] „Klocwork: Source Code Analysis Tools for Security & Reliability“, [Online]. Available: https://www.klocwork.com/. [Access 11 10 2017].
[12] „BullseyeCoverage Code Coverage Analyzer“, [Online]. Available: http://www.bullseye.com/. [Access 11 10 2017].
[13] P. Hodgson, „Feature Branching vs. Feature Flags: What’s the Right Tool for the Job?“, 23 05 2017. [Online]. Available: https://devops.com/feature-branching-vs-feature-flags-whats-right-tool-job/. [Access 11 10 2017].
[14] W. Buchwalter, „A Git Workflow for Continuous Delivery“, 21 06 2016. [Online]. Available: https://blogs.technet.microsoft.com/devops/2016/06/21/a-git-workflow-for-continuous-delivery/. [Access 11 10 2017].
[15] “Docker – Build, Ship, and Run Any App, Anywhere”, [Online]. Available: https://www.docker.com/. [Accessed 11 10 2017].
[16] J. Schlosser, „Frühe Verifikation von Regelungssystemen mit Model-Based Design“, in Embedded Software Engineering Konress, Sindelfingen, 2011.
[17] J. Schlosser und G. Sandmann, „Keine Angst vor Autosar – Leitfaden und nutzenbetrachtung für Funktionsentwickler“, ATZelektronik, Bd. 4, Nr. 2, pp. 66-72, 2009.
[18] J. Schlosser, Architektursimulation von verteilten Steuergerätesystemen, Berlin: Logos Verlag, 2006.
[19] T. Schneider, „Achieving Cloud Scalability with Microservices and DevOps in the Connected Car Domain“, in Software Engineering (Workshops), Wien, 2016.
[20] M. Hillbrand, „Agile Methoden skalieren, aber bitte nicht dogmatisch! Erfolgreich skalieren!“, OBJEKTspektrum, Nr. 2, 2017.
Author
Dr. Joachim Schlosser
Senior Manager
