Nine months ago my journey led me back into R&D in the life sciences industry, working with (amongst other things) software in a medical setting. The healthcare industry is clearly heavily regulated. Moving from releasing web products several times a week to software that’s released a few times a year (if you’re lucky) has been an eye opener.
Tasked to bring in a new process, it would have been easy to slip into the wrong mindset – long spells of developer work, followed by copious manual documentation and a throw over the fence to a testing team for a few months. A few post-deadline back-and-forths between testers and developers thrown in at the end for good measure, obviously. I’ve seen it happen with installed software in far less regulated industries, and frankly, there’s rarely an excuse for it. Seven months later, we gained our ISO 13485 certification using the agile process we created.
If you work where you don’t need to worry about your software telling someone they’re ill or not, and the thought of deploying anything less than daily horrifies you, this post might not be for you. It is interesting however that many of the fundamental principles and practices still (or should) apply even in a regulated industry. How did we steer clear of the waterfall trap? Here are the lessons we learned:
1. Get the right quality team
It would be easy for a quality team to force a process upon a new software team because that’s the way that they validate everything else – usually by using a hideously inefficient and burdensome quality software. Whilst that definitely exists, we were lucky enough to have a quality manager working with us that saw the need for agility, and how the systems that we implemented (in this case Azure DevOps) were a better match for compliance in an agile software development process. Having a quality team with this flexible mentality is crucial, and may require a lot of dialog upfront before anything new is implemented.
2. Get help from subject matter experts
Just to make sure we were on the right track, we consulted an external embedded software house that had experience implementing agile processes in the medical industry. There’s no way I was going to wave the Agile card if what we were proposing wouldn’t actually meet regulatory requirements. Luckily, there is a highly respected set of guidelines that offers some fantastic direction on the use of Agile practices in the medical industry (though the fact they felt the need to capitalise AGILE still drives me crazy).
3. Use DevOps practices
Turns out that if you follow the same good DevOps practices that high performing teams do, you’re most of the way to being compliant anyway. I was confronted with the unfortunate reality that most of our instrumentation products shared the same code repository, and relied on a gatekeeper to manually build the code in the correct configuration to produce the right product. The first task was to create a “master” build in Gradle that would take parameters based on which flavour software we wanted to build. Then, a separate build pipeline was built for each software flavour that built their respective installers and artefacts. We could then connect this to release pipelines for each flavour and have a record of all work items, pull requests and artefacts linked to a particular release was given to the validation team (and yes, there are rare occasions where a separate testing team makes sense – like when developers can’t use a pipette and people’s lives depend on it).
If you want to learn more about how good DevOps practices lead to high software delivery optimisation, read Accelerate or the latest State of DevOps report at the time of writing.
4. If you don’t store your documentation in your QMS, you’re not going to burn.
Living, accessible documentation is essential for software teams to be agile. It’s no good gathering cobwebs in a quality system that takes an age to find anything (or worse, no one has access to). If you want to change your branching strategy slightly, you don’t want to be spending weeks getting a change approved by a QA who knows nothing about branching strategies.
Here is a blog by the consultancy I mentioned previously on the topic of living documentation for software teams in regulated industries. The idea is that your wikis / repos / markdown files are absolutely fine, as long as they’re kept up to date and go through a robust, audited pull request procedure and you’re able to produce them in an auditor-friendly format. This can all be automated, and in my experience, most modern software lifecycle systems are more secure and auditable than many quality systems on the market anyway.
5. Definition of Release
In healthcare, you’re not going to be releasing a product to a customer multiple times a day. At least not one that’s locally installed in a public sector environment. You generally have little control over the appetite for change of the various governing bodies and indeed individual sites. So then how can we conceivably aim to be an elite performing team according to the widely accepted DORA metrics? You aim to use all the previous steps to create validated, tested, shippable increments anyway. I mean everything – review lifecycle, packaging, deployment to an immutable release pipeline and signed off – preferably with automated documentation and release notes. If there’s a regulatory blocker, or the customer is simply reluctant to update their procedures with a new software module (even if it has features in it they’ve requested and want!), you’re still safe in the knowledge that those releases have gone through a complete release process in isolation and are ready to go. For us, that’s “production”. Just don’t let it end up like this:
In summary
In the time that we’ve had I’m pretty proud of my small team taking a 12 year old codebase that had no automation to speak of into a cloud-first, DevOps driven compliant process. Not only that, but we’ve learned a ton going through this process that will help us build our next generation of digital products.
We’ve still got a long way to go. If anyone has any other insights in remaining agile in a medical setting, or you can see some hideous oversights in these lessons, please do let me know. Just don’t be mean.
It’s a cliché, but don’t accept the status quo or your environment as an excuse to abandon an agile mindset. Get the right people around you, make noise, and move the needle – one day at a time.
Oranjutanj House 