Performance engineering tasks and activities in the software development lifecycle. Designing and building software that meets expectations for transactions response times, throughput, and scalability as the business grows. Also, not all applications need performance engineering.
There are many Software development methodologies out there. They have the same phases and they implement them as a Waterfall approach, Iterative approach, or Agile, and some methods are not really enforced. Does the SDLC your company uses have specfic activties and tasks within each phase of the lifecycle? Are there workproducts specific to performance that must be created?
Performance, stability and scalabity must be designed into the application. It is diffuclt and expensive to add performance late in the project. Also, you cannot “test in” performance. You design for performance and then performance testing should be a validation exercise, not a discovery process.
You need requirements regardless of the development methodology, waterfalls, iterative or Agile. You need to specify the non-functional requirements for each key business transaction, the performance goals of the application, the scalability goals, and workload expectations. Are there regulatory requirements for performance or penalties from business partners?
Key business transaction: Identify the response time requirements. Categorize the transaction types, online web based, business intelligence and reporting, messages and batch. For example, the key transaction must respond in 2.00 seconds for the 90th percentile, under a peak load of 100 Transactions per second. The invoice component must be able to process 25 invoices per second and be able to handle our peak time of back to school of 4 times that.
Workload patterns and key marketing events: Does the usage pattern of the web site change signigicantly?
Workload model and volumes: define the different user profiles and build a workload mode estimating the number of transactions per second the system must support. Define average, 75 and 90 percentiles, are there extreme peaking factors on the workload?
Software Performance Requirements
Software performance requirements is about setting the performance and scalability context for getting the design and development right for your web application, your web service, your messaging hub, your reporting system, your mobile device. I am creating a few checklists;
- Project risk profile: Is performance important for this project and what works against performance in the project?
- Business workflows: What is the duration of a workflow, how many types of workflow and what is the peak workflow?
- Application business volumes and growth: The application automates the workflow with transactions and how will the volumes grow?
- Non-user interaction processes (batch, messages): This is about component throughput, how many orders per second?
- Communication to down-stream SDLC processes and phases: Setting the stage for design and development.
Here is a checklist to help set the context for your team;
Project Risk profile: These are the overarching performance and capacity considerations that the entire team must be aware of. This would communicate to the team that the business just acquired 2,000 new stores and this new application must now process twice as many users and orders in the same time. Or a new government regulation will be in effect that requires every retail brokerage order to have additional review that must not slow down the order processing system.
- Performance risk: Has performance or scalability been defined as a risk? Have you asked the business this question, or the production operations team?
- Extreme response times: Are there key business transactions with extreme response time requirements? User response under one second, a component that must process 500,000 transactions per second. If you have one second to respond to a user request, you better make sure the developers know this.
- Batch windows: Is there a very strict batch processing window and is the current system neat the end of the window? Yes, there are still large batch systems, that process tremendous volumes of data, for instance Mutual fund calculations.
- Third party: Does the system depend on third party software to complete the workflow? This could be software you buy and install as part of your application, or you could be using a Web Service.
- Third party SLA: Do the third party’s provide enforceable Service Level Agreements? Are you using a SaaS vendor and do you specify the response time?
- Peak Workload: Have the key business transactions been evaluated to determine average and peak workloads? And is there process in place to review these?
- Calculation: What are the key calculations and how is their response time influenced by the type of calculation (some do more work than others). Are there key pricing calculations, or rate quoting engines, or preference calculations, inventory allocation, etc. I have seen a few pricing calculations get caught-up in the volume discount calculation, and go from 100 milliseconds to 1 second.
- System Peak: What are the attributes that drive the peak load of the application? Is it seasonal, advertising driven, back to school, weather related (insurance claims), and do you model the peak? How many developers are not aware of the peak?
- Regulatory requirements: Is there an auditing component, reporting timeline, are there large volumes of data required to capture and provide to an agency
Your projects must have a performance risk profile defined. There may be no risk, or there may be significant risk.
Business workflow, business process
The System response time must not impact the workflow. The transition from transaction to transaction must be seamless and the user must not notice the system. One might even describe the interaction between the person and the system as graceful and flowing, where the system responds before they can even finish a sip of their coffee, do your users cozy up to the system (too far??).
Understanding each workflow in the application is crucial to setting the proper response time goals of the application when defining the software performance requirements for the systems and for each transition that supports the workflow. The systems today are highly distributed with web servers, application servers and web services, and message hubs and multiple database, etc. In the requirements phase, once the workflows are defined with performance goals, it is critical to make everyone who makes a component in the workflow aware of those goals.
There are call center workflows, document management workflows, order placement workflow, business intelligence and analytical workflow, and of course Big Data workflow.
When in the requirements phase you might consider this checklist for the workflow;
1) Identify the workflows: Have the key workflows been defined that have performance requirements and are the response time goals defined?
2) Duration: have you defined the overall duration of the workflow?
3) Downstream processing: Have you defined when the data from the workflow must be available for the downstream workflows? For instance, after collecting a customers demographic information and vehicle information, when is it available for rating a policy? 30 seconds, 24 hours?
4) Business transactions: These support the workflow. Have performance critical business transaction been defined, with response time goals?
5) System Transactions: These support the business transactions. Have you defined the response time goals for critical systems transactions supporting the critical business transactions? This is where share system transaction can be found, have your requirements captured enough performance information to tell the developer how fast this system transaction must be and how many transactions per second it will support?
6) Performance budget: Now that you have business transaction response time goal, have you allocated the response among all the technical components supporting the business transaction? You should create a system interaction diagram to help with this
7) Database query requests: Have you categorized your database queries? Simple transactions to the complex? Is there a response time goal for each? Is there difference between the first request and subsequent request?
8) Report requests: Have you categorized the report request types? Simple reports are 2 seconds, complex multi table grouping, ordering reports take longer that cross fiscal quarters?
9) Discussion and negotiation with the end-user or business sponsor. All along you must be in discussion with business people who own the system. The role of the architect is to work with the business to tell them what is possible and how much it will cost. The business priorities are critical. The business might want to spend the extra money to have near real time reporting to gain advantage or they might be satisfied with a four hour reporting window.
How to handle the response time discussion
Categorize: You should look to categorize the response time into; satisfied, tolerating, frustrated and abandonment. Two seconds could keep the people satisfied, while eight seconds will make them frustrated for an online transaction. Another transaction at five seconds could keep people satisfied and 15 seconds make them frustrated.
Percentiles: You need to establish a goal for what percentage of the user population would be satisfied, 50th, 80th, 90th percentile? 90 percent of the people should have a satisfied experience.
Under what load: You need to discuss with the business people that there is a normal workload, a peak workload, an above peak workload and define target for each. This business might ok with a relaxed target where the people are tolerating or frustrated for a period of time during a peak load for a short duration.