Monthly Archives: February 2013

Agile and PEYour are not Agile

Integrating Software performance engineering with Agile software development methods.  This should be easy, right? These two methods are not naturally suited for each other;

Performance engineering has rigorous and defined methods for defining non-functional requirements in the development cycle, and performance testing requires production like test systems with a proper transaction workload mix, and a large database. Performance testing can introduce more time in a release schedule when the application was designed and developed without well defined performance and scalability requirements.  The value in performance engineering methods is to manage risk and to design and build highly scalable systems that support the busines growth.

Agile methods start with stories and themes, a defined system architecture, and a partial list of required features. The team leader defines a series of releases comprised of a series of scrums.  Each release will have partial features and functions, Each scrum may not finish all the features listed in the scrum and need to be added to feature backlog. The schedule dominates the decisions and value of Agile methods is to get new features into production qucikly.

The Challenge

Businesses are adopting Agile methods for larger and larger software development projects. The projects are moving from department focus to Enterprise focus. The teams are getting larger and distributed. The Non-functional requirements of systems are becoming more stringent.  The end user experience is critical.

How do we implement performance engineering and performance testing tasks and activities in the Agile methods? The introduction of PE/PT cannot compromise the original intent of Agile, faster and paprtial delivery of software. The goal of Agile methods is to produce frequent working copies of the system, with incomplete features.  Features are introduced for each release. Traditional performance testing occurs after all the features have been developed and the application is nearing production.

This is not about fitting Agile methods into performance engineering, it is about adapting performance engineering to Agile methods.  Business are using Agile to accomplish business goals, that goal is speed to market.

The performance team must approach the project that same way the Development team does. People over process, multi-disciplined, learning and adapting. The performance engineer and the performance tester/scripter must understand the performance test process extremely well. During an Agile Performance Sprint, the Performance team will have to adjust and adapt to meet the timeline.  They will have to communicate performance issues to the development team so they can get the fixes into the next Sprint backlog.

Amending performance terms to Agile terms

Define the Performance Theme for the project. This is a project wide message to let everyone know the project has a performance and scalability focus, and the risk associated with the project require PE tasks, activities, and people. This must be conveyed to the Architect and the lead technical person, that best practices, tips and techniques for performance and scalability must be used for this project.

Performance Stories – The performance tests for the project, the workload models, and the test types. How do you add performance needs to the Story?

Performance backlog – The performance tests that need to be executed. The performance issues uncovered. When a performance issue is found, it must be put in the scrum/release backlog.

Spike – How will the performance tests be executed in context of the Release schedule and Sprints? Will there be focused out of band set of performance tests?

Definition of Done – Clearly defined exit criteria for the Performance tests.

New role on the Agile team.  A performance engineer for architecture reviews, code reviews, database reviews, and test planning.

The development team

Since the project has been set to a performance theme, the development team must use best practices for performance when developing code/services. The skillsets and techniques required to design and build highly performing systems must be communicated to the team. A critical success factor requires the development team to build perform into the system from the beginnning.  This will  help eliminate rework or technical debt due to poorly performing code. Performance is designed into applications, it is not tested into applications.  If the development team builds for performance the performance testing process will be much easier.

Performance testing

How many environments are avaialble for the Releases and Sprints? Is there a dedicated performance testing environment ready to go for performance testing to start immedately? The enviroment must be ready to begin testing in order to keep the value of Agile going.

Will there be multiple rounds (sprints) for PT, one or more tests is executed with various degrees of partial feature completion. There could be three or more PT Sprints, plus a final PT once all the features have been completed.

A round of performance testing will be done at the end of either a release or a sprint.  The performance team must be fully aware of the features available for testing. Each feature will require the development of one or more test scripts, with the supporting data.

Performance test database: Tracking the data readiness.  During the course of the sprints and releases, the database structure may change. The performance team must be fully aware of the changes and understand the impact to the test schedule.  If the database changes are significant, you may have to reload the Performance database. This would need to be accounted for in the project schedule. The performance and developmen team should automate the data creation process and reload process as much as possible to adhere to the value of Agile methods.

The typical performance test project has a startup phase where the test suites are planned, the test cases identified and the test scripts created.  The scripts and test cases are vetted and prepped for formal test execution. In the Agile approach, the performance test cases and the performance test scripts are developed along the way.  The performance team will use an Agile approach as well.

Performance testing in Sprints.

The performance plan will follow the same Agile methodology, the upfront planning, testing lab build out (similar to the architecture phase), then a series of sprints leading up to a final release. The workload will be evolving at each sprint. The Sprint should be three weeks, with the rule being “Make no Changes” this will be a measurement only exercise. The first week is the recoding of new scirpts, updating the test User Profiles, and modify the Database as required. Then there will be two weeks of test execution, with a report listing the issues.

more to come..