Design decisions for software performance
Most software performance and scalability problems can be solved in the requirements and architecture phases of the project lifecycle. The application requirements define the goals of the system, specifically the non-functional requirements, affectionately called the NFR’s, that specify; the response time of key business transactions (2 seconds or less), peaking factors (Monday afternoons at 2X), web site user population (500,000 active) and concurrency (250 concurrent average), system throughput for messaging or batch processes (10 invoices per second), and reporting.
It is critical to the overall success of the application Release to define the response time of critical business transactions. This is best done by working with the business owners of the application to provide guidance, and to set their expectations on the options and the costs of those options. Applications continue to become more complex and offer end-users a richer experience on the web site or the mobile application. The distance between the web site user and his/her orders continues to increase.
Take the two business transactions of logging into a web site and viewing the order history. By sitting with the business you can walk through varies options early to assess the performance and scalability impact of each. What information should be displayed at the log in? In general, the login process should be fast and almost unnoticeable by the web site user, not impacting their workflow. I define fast as one second. Too many times I have seen the login process bring back too much information the user was not interested in. The business may have a very valid reason to display history or current order state. The Software Architect must inform the business team how those request (history and order status) will slow the login process down. Or present options with cost estimates on how this may be achieved. It might require more hardware, faster network connections, different caching options, etc. The business then makes the choice.
So, you have a goal of 1 second, what percentile should the goal be for? Typically, it is 90th or 95th percentile. Under what conditions? Have you defined a reference workday? If you are in Internet Retailer, is this 1 second goal for the day after Thanksgiving? Does the business want to invest in a system or plan to accommodate the business day of the year? Is this 1 second goal relaxed during the peak day or peak hour?
The answers to these questions must be communicated frequently to the development and testing teams. The team that works on the login, account history and current order status must be very aware of the 1 second goal. This will greatly determine the approach taken to implement the 1 second transaction(s).