Archive

Application Performance

Selling the value of your APM investment to your business

 

The Digital business is focused on the customer journey and real-time performance monitoring is a critical success factor on your Digital business journey. The digital business is connecting more and more internal systems and third party partner systems to deliver the customer journey. Creating a complex environment.

Application performance management practices and tools provide a great value to the digital business; from accelerating the root-cause analysis process, to proactively measuring the customer experience across the mobile and web channels, and providing business transaction dashboards. When the full set of APM tools are used properly they can provide alerting and trending information so you find out whether the change to the application or system did no harm, some harm or really caused a problem. You can quickly react to the latest digital marketing event.

In my position, as a CTO and a Sr. performance engineer, I have used many different APM tools and diagnostic tools. Unfortunately, these tools are brought in to a company as a reaction to a problem and not brought in before there is a problem. These tools are usually brought into the company to help solve an immediate production performance or stability problem.  It starts with the developer, engineer, or operations person or team, who have to solve the problem. The system is slow (what, where, when, who, how??).  To quickly get an APM tool in place for such a scenario, the free download option is used for a short period or a vendor is used for a proof-of-concept and the problem is solved.  However, to keep the tool and to switch from a reactive to a proactive position, there is no budget at the time because the tool was not identified last year. So, the people on the ground are stuck, no APM tool.

No one really knows why the system is slow, in fact, they usually can’t even tell you what they mean by slow, is it 5 seconds, 10, or 42 seconds. It is amazing that this is still the rule in most fortune 1,000 companies. Often times the Corporate desktop or laptop is locked-down, no changes unless approved. The issue is that a Enterprise  end-user cannot download and install any software on the corporate laptop or desktop.  With web performance tools (Appdynamics, New Relic, YSlow, Dynatrace, HTTPWatch, et al) having the ability to download and instantly measure the response times of web transaction, SLOW can be measured quickly.

IndBench

Figure 1 – Industry Benchmark – Dynatrace

 

Know your business

Let’s talk about business value and business benefit.  What are the goals of your business this year? If you are in the Insurance business; Is it to capture the business of more Independent Agents by improving their customer experience? Wouldn’t there be value you knew your digital customer experience? Also, if you sell on the web then you know that faster response times mean increased revenue. What if you run a distribution center, what is the goal there? The business goal is to increase revenue and the Distribution Center must move more orders. That requires a highly available environment and a responsive environment. Don’t forget call centers.

Understand the digital business motivation for performance and user experience dashboards.  They help proactively monitoring the customer journey. There are a number of third party companies that monitor the performance of web sites across industries to establish benchmarks. The business maybe motivated to move up in the industry Benchmark and understand that some investment is needed to move from the 20th position to the top five.

Your homework: Go find your company business goals; there should be 4-6 of them.

Payback period, direct benefit and indirect benefit

Technology often gets caught in the budgeting process when you cannot clearly link the investment (purchase) to a business goal and benefit.  Many technologies are far away from revenue or cost reduction, and how do you quantify flexibility? The ability to invest in technology and the willingness to invest in technology varies greatly by industry and by the type of application. Online web retailers relentlessly work to improve the user experience and user performance, the business understands the connection to performance and revenue. They can quickly justify the investment in APM solutions.

Direct benefit: The APM investment you are making will improve the application performance and customer experience for the Independent Agents. This is lined up with the company goal to attract more Independent  Agents and increase revenue.  If the business can attract 15% more IA then revenue increases by XXX%.

Indirect benefit: The APM investment will defer technology (Server) purchases for two quarters.  If the business has a cost containment goal then this will align with it.

Payback period: There will be an initial investment required ($100K), recurring charges ($15K), and one time training expenses.  Based on your direct and in-direct benefits, how long is the payback period?

Digital business and the customer journey: An Agile digital business introduces new features and functions to the marketplace, frequently.  Are all your customer channels providing the same experience? Creating a real-time dashboard of the channels and the critical business transactions provides value to the business by providing leading indicators of their experience.

Your homework: Does your APM investment provide a direct benefit or an in-direct benefit?  How long will the payback period be?

 

 

 

 

 

Advertisements

Software Performance Engineering

Agile and PEYour are not Agile

Integrating Software performance engineering with Agile software development methods.  This should be easy, right? These two methods are not naturally suited for each other;

Performance engineering has rigorous and defined methods for defining non-functional requirements in the development cycle, and performance testing requires production like test systems with a proper transaction workload mix, and a large database. Performance testing can introduce more time in a release schedule when the application was designed and developed without well defined performance and scalability requirements.  The value in performance engineering methods is to manage risk and to design and build highly scalable systems that support the busines growth.

Agile methods start with stories and themes, a defined system architecture, and a partial list of required features. The team leader defines a series of releases comprised of a series of scrums.  Each release will have partial features and functions, Each scrum may…

View original post 934 more words

Architecture and technology decision making.

Making decisions

Making decisions

Lets hide behind the chain saws

In the spirit of Halloween, Chain saws.    What a great commercial from Geico to highlight the subject of poor decision making; when you’re in a horror movie, you make poor decisions.  There is a running car that is ready to go, but they decide not to take it. How many of us realize that after the architecture has been defined and the development starts, that maybe, in hind sight,  we choose to hide behind the chain saws. Another familiar scenario is after your business selected a software product that met the functional requirements,  however it wasn’t even close meeting the performance and scalability requirements. The product was selected, and then later in the SDLC it was very apparent that the software product was not going to meet the performance goals without a significant amount of rework from the vendor and three times the original capacity plan, from the vendor.

What is the decision making process? How do we make the right or wrong decision? The more difficult it is to undo the decision, the more information we need. To help make the choices, we have our personal experience, there are best practices, we have mentors and peers we consult, you have situational information and often times not enough information.  How much information do you need to be comfortable to commit?  One quote comes to mind, from General George Patton, “Perfect is the enemy of the good”.  His approach being the most important thing you can do is to make the decision quickly and move on, then adjust when new information becomes available.  Though, some decisions are tough to undo. Like committing to a new software product without the proper information.

When selecting a product that will be part of the key business function, you should make sure these items are part of your decision making process;

Understand the risks from the User population.

    1. Who is using the solution?
    2. Review the user population, the business growth plans, and volume peaking factors
  1. Understand the associated risk with type of application.
    1. What type of Application is supporting the business function? Is is a messaging application, a reporting and analysis, etc.
    2. For instance, messaging, ERP module(s), reporting and analysis, Business or consumer portal.
  2. Understand the associated risk with the technology the application or solution is dependent upon
    1. Is this a new technology platform or solution for the Enterprise?
    2. Has the technology been demonstrated to work at the expected load?
    3. Is there a critical technology required from a third party?
    4. Will part or all of the solution be hosted externally?
  3. Understand the risks associated with the Application release strategy
    1. Expected frequency of releases
    2. Degree of change per Release
    3. Added business units at each Release increasing volume
    4. Will the application be on a brand new release of the product
  4. Understand the risks associated with the entire Solution Architecture (logical and Physical layers)
    1. Is there a critical new component required for application
    2. What is the configuration testing environment
    3. What reference can the vendor introduce you to, are they using the version and features you will be using?
    4. What is the team depth of the vendor
    5. Who will be making the changes for you, the consulting organization or the development organization?
  5. Understand the risks associated with the team organization and structure
    1. Has the Solution architect done this before
    2. How distributed is the team?
    3. Does the team understand the development methodology
    4. Is the technology new and requires a scarce skillset?

Hopefully this will help you get to the running car and not the chain saws.

 

DevOpsProblem

DevOps is a popular trend in IT and have been gathering steam. The challenges with DevOps for large companies are plentiful. One of the key roadblocks comes from the Vendor Management and Vendor Contracting groups.  Nothing to do with the actual task at hand. Large companies often outsource development to one or more large System Integrators.  They develop the code or assemble the solution and hand it off to another company doing the operations support. The contracts these companies write are usually very specific on who is responsible for what.  If you want one Integrator to support your performance testing, that must be in the contract and part for the price or bid.

How do we get the two or more SI’s to work towards a DevOps model? The customer must adjust their contracting terms to accommodate this and provide a path.

Starting the DevOps culture in a Large fortune 500 company is no small task. You can define and start a pilot within your Performance testing team. A meaningful performance test requires multiple disciplines to execute and analyze results and for root cause analysis.
The Performance engineer drives the process, the application architect can see the application under load or stress to see how their design is working, the operations team can see the installation and configuration process, can help with the automation and deployment scripts or approach.  All can benefit from the APM monitoring that is required for critical transactions. I think it might a smaller challenge to get some SI buying for this type of approach.  Then perhaps more onto a Production application.

 The SPE Body of Knowledge

 

How can we benefit from the using a Body of Knowledge approach for Software Performance Engineering. The goals is to address the career path, the organization, the industry practices and to enable you to build a BoK within your company, for the performance engineer. There are five knowledge areas for performance engineering.

This is an overview presentation I gave at the Greater Boston Computer Measurement Group and the National meeting. There are two documents here, the Powerpoint and the Detailed document.  It is a work in progress.

PresentationSPEBoK CMG National V4

Paper with the details of the SPEBoK: The Guide to the Software Performance engineering body of knowledge V4

Please send me your comments..

 

Walter

 

 

Frustrated User

Bring Your Own Response Time.

The consumers’ expectations have greatly influenced the demands and expectations on Enterprise IT departments. The consumer and IT customer brought their own devices, expected more self service and at a much faster pace. One of the key tasks that a Performance Engineer must do is to help the business and IT set expectations on the response times of corporate systems. The history of performance requirements for corporate facing systems and even call centers has been problematic. Often times ignored and certainly deferred. The typical approach is to see just how slow the system can be before the users completely revolt. This tends to be the case because its not a revenue generating system, however, many of the Corporate IT systems directly touch the customer or business partner after the sale or contract is signed.

Response time or performance goals for Internet retailers is well defined and measured, there are many Industry specific benchmarks that compare the response times of web pages against competitors in the industry. The Internet business models demand faster and faster response times for transactions. Benchmarks can be found at Compuware (www.compuware.com), Keynote (www.keynote.com), among others. However, there is not a benchmark for Corporate systems. The users of Corporate systems are starting to voice their concerns and displeasure more loudly.  They are expecting speeds comparable to Internet Retailer speeds. Their expectations are for less than five seconds and often two seconds for simple transactions.

Our studies have shown and are in alignment with the research done by Jakob Nielson (www.nngroup.com) on usability, A guide to setting user expectations must consider three barriers;

1) 0.100 Seconds: The user perceives the system to respond in real time with out any noticeable delay

2) 1.0 Seconds: the User starts to perceive a slight delay with the system, but us very happy with response time

3) 10.0 Seconds: the user will greatly notice the delay and start to be distracted and attempt to do other things while waiting

So, just as the consumer has brought their own devices, they are bringing their own Response times to Corporate systems.

 

Increase Sales

Selling the value of your APM investment to your business

Application performance management tools provide a great value to the business; from accelerating the root-cause analysis process, to measuring the customer experience, and providing business transaction dashboards. When used properly they can provide alerting and trending information so you find out whether the change to the application or system did no harm, some harm or really caused a problem.

In my position, as a performance engineer and a cranky one sometimes, I have used many different APM tools and diagnostic tools. These tools are usually brought into the company to help solve an immediate production performance or stability problem.  It starts with the developer, engineer, or operations person or team, who have to solve the problem. The system is slow. An APM tool is used during a free-download period or a vendor is used for a proof-of-concept and the problem is solved.  However, there is no budget at the time as the tool was not identified last year. So, the people on the ground are stuck, no APM tool. I think there is an opportunity here to enhance the standard corporate desktop or laptop to add an end-user monitoring or measurement tool, such as YSlow, HTTPWatch or Dynatrace to name a few.

In these cases, no one really knows why the system is slow, in fact, they usually can’t even tell you what they mean by slow, is it 5 seconds, 10, or 42 seconds. It is amazing that this is still the rule in most fortune 1,000 companies. The issue is that a Corporate  end-user cannot download and install any software on the corporate laptop or desktop.  With web performance tools (YSlow, Dynatrace, HTTPWatch, et al) having the ability to download and instantly measure the response times of web transaction, SLOW can be measured.

As a technologist, you know implicitly that the APM investment  will provide value, save time, allow for better use of all resources, and provide some great looking dashboards. So, just buy the tool. However, there are budget keepers who are not as informed a you, the performance engineer, and they don’t understand the value.  So, with hat in hand, you need to find the money.

Know your business

Let’s talk about business value and business benefit.  What are the goals of your business this year? If you are in insurance, Is it to capture the business of more Independent Agents? If you sell on the web then you know that faster response times mean increased revenue. What if you run a distribution center, what is the goal there? The business goal is to increase revenue and the Distribution Center must move more orders. That requires a highly available environment and a responsive environment. Don’t forget call centers.

Understand the business motivation for performance and user experience dashboards.  There a number of third party companies that monitor the performance of web sites across industries to establish benchmarks. The business maybe motivated to move up in the Benchmark and understand that some investment is needed.

Your homework: Go find your company business goals; there should be 4-6 of them.

Payback period, direct benefit and indirect benefit

Technology often gets caught in the budgeting process when you cannot clearly link the investment (purchase) to a business goal and benefit.  Many technologies are far away from revenue or cost reduction, and how do you quantify flexibility? The ability to invest in technology and the willingness to invest in technology varies greatly by industry and by the type of application. Online web retailers relentlessly work to improve the user experience and user performance, the business understands the connection to performance and revenue. They can quickly justify the investment in APM solutions.

Direct benefit: The APM investment you are making will improve the application performance and customer experience for the Independent Agents. This is lined up with the company goal to attract more Independent  Agents and increase revenue.  If the business can attract 15% more IA then revenue increases by XXX%.

Indirect benefit: The APM investment will defer technology (Server) purchases for two quarters.  If the business has a cost containment goal then this will align with it.

Payback period: There will be an initial investment required ($100K), recurring charges ($15), and one time training expenses.  Based on your direct and in-direct benefits, how long is the payback period?

Your homework: Does your APM investment provide a direct benefit or an in-direct benefit?  How long will the payback period be?

Thanks.