Why do we need a Body of Knowledge?
We can take these industry trends as the catalyst for introducing a Body of Knowledge. Using the framework of a body of knowledge will allow the profession to formalize and define a standard for language and terms, critical job descriptions with career paths, and a consistent method to communicate performance engineering value to the business. This can start in your company.
SPE activities not visibly aligned to business value
You must be able to demonstrate how SPE activities provide value to the business. In some cases it is easier; designing, coding, testing and monitoring activities can be aligned to show an increase in shopping cart conversions on the web site, due to stability and performance. The SPE can directly and positively impact the user experience.
Converting data from one system to a new system faster will result in business value. SPE can make sure the non-functional requirements are defined early, in the case of a large data conversion, this can make all the difference in the business case. If the business can convert the data in five days instead of 15, then there is real measureable business value. Having a goal of five days upfront is critical. Often times many people back into the conversion schedule, and that appears only to be an estimate. Of course, if it goes longer then the business case gets worse. The design activities and testing activities will align under this as well. Does it add value to plan and execute a number of pre-conversion performance tests to measure the throughput of the conversion?
Providing accurate and timely capacity forecasts offers great value to the business.
No clear definition of the SPE profession and roles
In most companies there is no formal title or job description of a performance engineer. Many people arrive at this point in their career on their own. They attained deep technical knowledge in two or more subject areas, they have great problem solving skills, and they understand the business goals. Their formal title may be architect, database architect, lead tech, etc. Many of the performance engineering roles are split across the IT organization. Another critical role is the Capacity Planner; where they monitor the current system, forecast the future needs and track the costs associated with the system.
Often times the performance engineer and the capacity planner work in different groups, they share many similar competencies and techniques. They track the same information, however the key split has been the production focus of the CP and the pre-production focus of the PE.
The performance test developer role tends to align with the Quality Assurance group or even be part of Center of Excellence team (these CoE’s typically do not add much value, they know how to test, but do not know anything about your application), the capacity planner with production operations, performance architect with the development group, database performance in the database engineering team.
The most common cross role confusion is that of the performance engineer and the performance tester. There are Sr. testing people who are also Performance engineers. The performance engineer is a more Sr. technical architect who understands a large body of performance practices across the SDLC, performance testing, application performance management, troubleshooting, capacity planning, etc. They are also a leader and great communicator and they understand the business goals. They understand the connection between the SLA’s, for example, the login transaction must be completed in 1.5 seconds for the 95th percentile under month end peak load conditions. The batch process must achieve 34 TPS in order to meet the SLA of one hour.
The terminology varies by industry and company
There is a large vocabulary for performance engineering terms; from the business terminology to the design and coding terminology, performance testing, and monitoring and measuring. Performance goals start with business terms; one business is concerned with Shopping cart conversions, another is concerned with throughput of a message hub that communicates sales information to the distributors and wholesales in real time. An insurance company is concerned with the number of policy quotes, throughput, and the time to get the pricing (response time).
How is a critical business transaction defined in your company, who decides that? Does everyone in your PE group have a common understanding of them? Is this used as the starting point for the capacity planning process?
There are the core competency technical terms;
- Physical and Virtual CPU utilization, CPU entitlements, shared pools, CPU run queue,
- Database wait events, full table scan, left outer joins, buffer gets,
- number of IO operations per seconds,
- transaction arrival rates, system throughput, component throughput
- response time, service time & queue time,
- heap size and garbage collection,
- queuing, capacity planning and forecasting, closed system, open system, standard deviation
- page load time, event correlation, etc.
A body of knowledge can help you capture these terms and share them across your Enterprise.
Introduce Agile techniques to SPE activities
Everyone is tackling this on their own, wouldn’t it be nice to reach out to a peer group working on the SPE BoK to define and implement a standard approach for companies to start with? This can be applied to any new technology or methodology that rolls through the IT industry.
The performance team must approach the Agile project that same way the Development team does; People over process, multi-disciplined, learning and adapting. The performance engineer and the performance test developer must understand the performance test process extremely well. During an Agile Performance Sprint, the Performance team will have to adjust and adapt to meet the timeline. They will have to communicate performance issues to the development team so they can get the fixes into the next Sprint backlog.
Develop a Performance Theme for the project. This is a project wide message to let everyone know the project has a performance and scalability focus, and the risk associated with the project require PE tasks, activities, and people. This must convey to the Architect and the lead technical person, that best practices, tips and techniques for performance and scalability must be used for this project.
- Performance Stories – The performance tests for the project, the workload models, and the test types. How do you add performance needs to the Story?
- Performance backlog – The performance tests that need to be executed.
- Spike – How will the performance tests be executed in context of the Release schedule and Sprints? Will there be focused out of band set of performance tests?
- Definition of Done – Clearly defined exit criteria for the Performance tests.
- New role on the Agile team. A performance engineer for architecture reviews, code reviews, database reviews, and test planning.
Since the project has been set to a performance theme, the development team must use best practices for performance when developing code/services.
Addressing career path and organizational challenges
The goal of the BoK is to overcome the organizational challenges you and other companies have. The key task you must undertake is to understand the gaps in your organization for performance engineering, the plan out how to define the roadmap to implement the BoK.