Remember me

Register  |   Lost password?

The Trading Mesh

Roundtable Report - Software Testing in Financial Services - Is the Price of Success Paid in Advance?

Wed, 09 Aug 2017 11:05:10 GMT           

It is well documented that banking technology is both aging and under increasing pressure to perform. At the heart of this pressure is the need for faster and more reliable technology and software.  To drive the deployment of up to date technologies and applications under tight timescales but without compromising quality, the banks are increasingly aware of the need to adopt a ‘shift left’ approach in the assurance and testing process.


On 13 July 2017 a roundtable of testing experts in financial services, organised by The Realization Group and change management  and testing specialists Certeco, met to discuss the challenges this process poses. The opinion of the group was that although structural changes in the finance industry are driving earlier testing or 'shift left' in the development process with greater urgency, there are still significant hurdles to overcome to enable this. 

Understanding the mechanics behind making a ‘shift left’, and the associated software development models such as Agile and DevOps, are key to understanding the barriers it faces to adoption in finance. 


Defining shift left

When defining ‘shift left’ It is a mistake to think about it as simply a process issue that testers and developers can make happen. 

Firstly, underpinning it is the ability to deploy whole environments very quickly using platforms such as Terraform, Cloud Formation or AWS which can create reusable environments in order to be able to conduct testing.

Financial services firms must have this ability to create their test environments and to use the right static information sets if the tests they are to conduct in the environments are to be meaningful. The ability to put them together, tear them apart and put them back together again is fundamental to getting value from 'shift left'. Therefore while the skills of the team are a critical part of this, there is also very much a technological component that requires investment. 

Secondly there is a huge cultural element to adopting ‘shift left’. In effect, the ‘counter-cultural’ challenge in financial institutions is that the focus on cost reduction and rigid hierarchical organisations can often prevent more effective models of working from being employed. 

Banks must overcome these two issues and the operational challenge of moving to non-linear Agile or DevOps systems development models if they are to reap the rewards of shift left, which are the reduction of waste, both of time and resources, in a project through ongoing testing and engagement.

Without a successful deployment it was agreed that firms will be stuck with old systems and old methods of testing and that causes real problems. From the antiquated technologies decried by bank CEOs and regulators, to the deployment of live systems that cost banks dear, such as Knight Capital’s near bankruptcy when a trading system went awry in 2012, the industry has already seen the cracks appear.

Making it happen

There are a number of ways in which shift left may be integrated in the development process. 

Two common approaches applied by banks take very different routes. The first is to centralise the deployment of the shift left methodology and then try to spread it out from the core of the organisation. The second is that every team becomes a micro-kernel and works it out their own way. 

In the latter version the teams have to communicate with one another and somewhat organically become consistent in their approach. That was seen as an the ideal by several participants at the roundtable as it supports the teams becoming creative in the way that they work. However, often the centralised model is the one that is preferred by management.

Testing also involves engagement with the rest of the business. The testers must have access to the right data and the best testers of a system – namely the users themselves. Ideally test data should be close to live data or live itself.

If both the users and live data can be used during the development process wastage will decrease even further. 

Barriers to change

However, to get the ideal development models (and therefore early testing) embedded within the process of software development, certain hurdles need to be overcome.

For example, while the data should be close to live data or live itself, arranging for users and live data to be brought into the testing process is a major challenge. As one attendee at the event noted, that alone requires that the senior management of an organisation would have to be aligned with the need for increased testing. 

In many cases although firms have heard of ‘shift left’ the leadership team is not clear in how it wants to achieve it or what the objectives are in the move. The same is equally true of new development approaches like Agile, which are popularly supported by senior management but not necessarily understood.

One attendee observed that jargon chasing is a big problem, with senior management thinking that by using jargon they can achieve something bigger, cheaper or faster than before. In some cases it may be that a tester well-versed with how the business works is able to look at designs and understand where issues will develop or where data is missing. That alone can make can make the testing process far more effective.

Proving the value of ‘shift left’ to senior management is also challenging. The existing linear approach to testing is straightforward to measure. Running a cost-benefit analysis on earlier and more frequent testing then comparing it with linear testing is difficult as there is not a direct comparison of outcomes. 

Additionally, within larger organisations there are practical problems in matching up the developers, engineers and testers/users where the teams have unequal numbers or are geographically distributed. 

Making the change

Roundtable participants were agreed on one key issue; a cultural change is needed within financial services firms in order to adapt the testing process. The reporting of slippage and success, the monitoring and measuring of success against budget and milestones clashes with ‘shift left’ and Agile development methodologies.

Any reform of the testing process must be led by the technology teams and presented to the business with either a clear cost-benefit analysis or a clear explanation of the risks involved in not engaging in earlier testing. Without business buy-in the process will not succeed. 

However it is also important to understand what the use of shift left, Agile and DevOps cannot do. They are no substitute for strategic thought. Behavioural change lags behind technological change and many leaders in the IT space are unlikely to change their behaviour without good reason. Therefore an understanding of how these changes in development fit a more strategic view will be necessary to drive change.