Testing in Agile: Security

security-265130_1280Barely a week goes by without a news report of hackers stealing customer data, taking a website or network offline or even remotely hacking and controlling cars.News of vulnerabilities are published online and exploited very quickly and this punishes any company which is too slow to act.

Using the example of a ‘traditional’ waterfall project, it might be common to see applications developed and an external company booked in to perform a penetration test (or IT Health Check) before release. Usually this is too late to address all vulnerabilities. The options are then to fix vulnerabilities at the cost of delaying launch, or to release with known vulnerabilities to be patched later.  In any case, it is expensive to fix defects at this late stage.

Testing in-sprint

So, when should we start testing security?  All testing can be completed in-sprint provided you have the right environments, one of which has to be a scaled, or scalable production-like environment (Failure to have this environment would mean not meeting Digital Service Standard number 10 after all!).

Appropriate acceptance criteria or user stories should be created to cover non-functional aspects, including security.  By having this focus in-sprint, we’re actually now thinking about building security in, because we know that the definition of done will depend on passing the acceptance tests.

What do we (and hackers) check for?

As most hackers start by scanning for things like open ports, operating system version and enabled protocols, this is clearly an area that we should be looking at in test.

Nmap is a powerful, open-source port-scanning tool used by hackers, but what are we looking for? What is there to be concerned about?  At the most simplistic level, we are concerned about ports that are open that don’t need to be because every open port is a possible open door.  From the information gained from a port-scan, it is small step to looking up publicly available vulnerabilities and worked exploits!

Note that use of Nmap must be authorised well in advance with your Security Manager and with written agreement of hosting companies and other third parties due to potential Computer Misuse Act implications, which your Security Manager can help you with.

And what about checking for insecure configurations and missing patches (which in my experience, is one of the most common issues found in external penetration tests)?  Microsoft’s Baseline Security Analyser (MBSA) is a useful, free tool which will highlight areas to look at.  Often changes required are a line or two in a configuration file... we don’t need expensive pen tests to tell us about these issues!

The Open Web Application Security Project (OWASP) is an online community dedicated to web application security and one of their projects is the OWASP Top 10 which details the most critical web application security flaws. This useful and free resource should therefore also be something we look to use in our testing.  I recommend that use of the open-source OWASPZAP tool (an OWASP project), which is a penetration testing tool for finding vulnerabilities in web applications. There are various plugins which mean that this testing can be automated and run as part of each Continuous Integration (CI) build.

A formal IT Health Check is still likely to be required for accreditation purposes before go-live, however by following these guidelines, a smaller number of application issues should be discovered at that late stage.

The SFA's Quality Management System (QMS) now incorporates the Agile Test Framework which includes the guidance above and additionally we have some usage guides available for the tools mentioned.  Please contact me on Yammer or Twitter for information if you would like to discuss anything related to testing.

You may also be interested in:


  1. Colin Heward

    Excellent blog Mark. Security may not the most alluring aspect of a project but it is a day one consideration. Building it in from the start will save a lot of pain, and expense!

    Link to this comment Reply

Leave a comment