Who are Performance Analysts, Performance Engineers, and Performance Testers? All these definitions are quite rare in the information space and combine one very useful profession, which our colleague Vadzim Volkau told us about.
Following the question “What do you do for a living?” people usually ask: “What is it? What exactly do you do?”. To people who are not involved in the IT sphere, I explain my role with real examples. Probably, many of you encountered a situation when an online store arranged a big sale, or a long-awaited update of your favorite online game was released, or registration at the embassy opened to apply for a visa. You want to buy, play, or register, but you cannot do it because the websites go down or do not work at all due to a large number of people interested. Such cases annoy the user, and the business incurs losses and loses its loyalty.
The goal of my profession is to avoid such system failures. A Performance Engineer helps build the most efficient computer systems that run fast and consistently.
Performance is one of the key characteristics of high-quality software. The user will not appreciate the convenience and general beauty of the website, the range of products in the online store, graphics, and the plot of the game if the system is slow or does not work at all.
I would like to define the terminology I will be using further. Sometimes, there is a distinction between performance testing and performance analysis (optimization). In the first case, the profession involves only measuring how the system works and giving this data to someone who will study why it does not work as it should. In the second case, it is not only measuring but also figuring out why the system is slow, or at least help to do it. However, my colleagues and I believe that performance testing alone will be sufficient for the beginners to the Performance Engineer profession.
Further development of the performance analyst assumes that you can independently find problem areas in the system you study. Further, I will use all the terms (analyst, tester, and performance engineer), referring to the same role.
Origins of the profession
We thought about performance problems at a time when computers were just beginning to appear. In 1968, Response time in man-computer conversational transactions by Robert Miller, a classic article about the impact of the computer systems speed on the user, was published. Computers have changed since then, but humans have remained the same, and the requirements collected in the article still apply in estimating the performance.
Responsibilities of the Performance Engineer
I suppose this is one of the most diverse professions in IT. In most cases, even on a large project, there is only one Performance Analyst. On the one hand, the analyst has freedom in choosing methods, technologies, and tools. On the other hand, this is a big responsibility. If you chose some approach, but it did not work, then only you are responsible for the result. Besides, a Performance Engineer is immediately responsible for many areas from the start of the project to its end.
1. The job of a Performance Engineer begins at the stage of collecting business requirements. Yes, business analysts usually do this, but a good engineer can improve requirements taking into consideration how they will be validated later.
Imagine a customer who wants to create a service selling flight tickets. In my experience, the request for the performance of the final product is as follows: “We want the service to work quickly and not “drop” under load.” Such requirements poorly define what characteristics the system should have. Therefore, a Performance Analyst asks additional questions and clarifies the nuances. “How many users do you want to serve at the same time?”, “What can they do: just search for tickets, book, make payments (these are all different load on the system)?”, “How many key user actions are expected per unit of time?”, “For how many seconds should one page load?” and other questions. If the business analyst does not understand what “fast” is, then a Performance Engineer helps to define specific requirements based on human-computer interaction research and industry standards.
2. Sometimes, a Performance Analyst can participate in discussions about the future architecture of a solution. Let us say we create a part of the system that should process messages from users and send them somewhere else. The architect will be developing design languages based on performance requirements as well. However, it certainly will be useful if a Performance Engineer is around and asks: “What kind of message intensity do you expect? Does your architecture take it into account?” A very skillful PE will be able to evaluate the architecture of the system and suggest the necessary changes.
3. The main goal of performance tests is to understand and fix the causes of slow system performance. For this purpose, hardware and software are monitored. Infrastructure monitoring is often set up by a Performance Engineer, although DevOps Engineers can also do the monitoring.
4. Next goes the development process and testing of the final product parts. Using iterative approaches, you can study performance (speed, stability, and scalability of a product) even at the stage when some minimal code is ready. This is a very good scenario. It is bad practice to do performance tests just before the release. Of course, it is better than nothing, but fixing performance problems often falls into 2/3 of Artemy Lebedev’s studio slogan “long and expensive” (and you cannot be sure that everything will be ok in the end).
5. In my opinion, the most interesting period is data analysis, which begins after receiving the test results. This is a very significant part of the responsibilities of a productivity analyst, which, on average, takes half the time. Here, the main task for the engineer will be to determine what reduces the system performance and to fix this part. If after that the product does not meet the requirements, you need to continue looking for a reason and remove the restrictions. “Lather, rinse, repeat.” Sometimes it happens that alone it is impossible to understand the cause of poor performance. In this case, you need to complete the minimum task: gather other team members to find the problem and do your best to help colleagues solve it.
In short, the responsibilities of a Performance Engineer include not only product tests but also a lot of other preparatory and analytical work. At the same time, the main goal is to take care of how comfortable the end-user will be able to work with the system.
Required skills
Now, we will talk about the competencies of a Senior Performance Engineer and what you should strive for, not what you should know as a Junior Performance Engineer.
The main qualities of a Performance Engineer are an analytical mind or the ability to draw conclusions from facts. If you do not have these qualities, then you will not feel very comfortable in the profession since, in principle, all work is based on the analysis of information and conclusions. In general, the specialty of performance analyst lies between three IT professions:
- A part of testing, but, as I have already said, quite a specific one. Therefore, the performance analyst needs to know and correctly apply the load testing methodology.
- A significant part of the administration. To analyze performance, an engineer should know the entire stack of systems, from networking and hardware, clouds and virtualization to browser rendering. Also, you need to understand how operating systems, web and app servers, databases, and runtime high-level languages work, and how to configure all of this to achieve the required performance results.
- Developer skills are also indispensable: a performance engineer must analyze the code and work with application memory. As for writing code, load scripts are still not about programming, although many of them are written in “serious” languages (C in LoadRunner, C# in Visual Studio). However, sometimes it happens that the existing tools are not enough, and you need to write extensions (or your own tools) from scratch—this you can surely call “real” development.
Self-management and some project management. A Performance Analyst often works alone on a project and does not have a team lead to help. Therefore, it is important to be able to plan your employment, understand and meet the deadlines. Knowledge of the development process will come in handy too, and a competent Performance Analyst can also make it better.
Communication skills. To bring product performance to perfection, a Performance Engineer communicates with a large number of people. It is important for you to clearly express your thoughts and interpret the obtained results to colleagues. Also, you must communicate with the team and customers in English.
Business analysis skills will also be useful to understand the customer and business needs and how to turn them into requirements.
Basics of statistics and data processing are required. A Performance Engineer always works with data, sometimes a lot of it. Methods of collection and processing, principles of working with data—all this is a must-have.
Desire to learn and develop. The Performance Engineer’s technical horizon is infinite. To be able to analyze the operation of computer systems, ideally, you need to know everything about everything. Of course, a deep understanding of all domains is impossible but, at the same time, you can choose the part of the technology stack that you like most and dive into it. Someone likes to optimize their work with databases, someone may be a client-side performance specialist, someone goes deep into the “clouds”. In my opinion, these are the delights of the profession: when you have a feeling that you would like to develop further, you can always do that. This diversity defines the level of a Performance Analyst: the more you know, the better you can identify problem areas in the system and fix them more effectively.
Career path
The stories are very different, but there are patterns. Several years ago, we conducted research in the department and found out that a third of people come to us from development and administration, another third from testing, and the rest “start” with performance engineering. I started with functional testing and then moved on to performance testing.
In my opinion, starting as a Performance Engineer is very cool, because the specialty is multidisciplinary, you can understand the work of different disciplines and gain a good technical base. Those who then want to try themselves in another profession go mainly to developers or the popular Site Reliability Engineering.
Domains
Most often, PEs work with domains where there is a multi-user load (e-commerce, streaming media like Netflix, etc.) or on the projects where speed is critical for the system to work. By the way, the multi-user load is not always about people. For example, an IoT with a large number of devices and a stream of data that “flows down” from sensors and needs processing.
Education
I have not met educational institutions where they teach students specifically for this specialty because the profession is very complex. As one of the options, you can get the education closest to the profession in BSUIR at the faculty of Computer Systems and Networks. They teach you about the work of hardware, networks, and operating systems, how to optimize the code. In general, any IT background will be useful here, but you still must learn and gain experience.
Books to read
- You can learn the basics of performance testing methodology from The Art of Application Performance Testing and Performance Testing Guidance for Web Applications.
- To learn about scripting on the most popular load testing tool, read Getting Started with JMeter — A Basic Tutorial.
- For Network Performance Basics, read High Performance Browser Networking.
- For experienced analysts, I recommend Systems Performance — Enterprise and the Cloud and Guerrilla Capacity Planning.
Another training option is the courses. For example, in addition to project work, I and my colleagues teach a course on the basics of Performance Optimization. We talk only about the basics of the profession, and after that students complete their studies within the company for another 3–6 months before getting on the project. Of course, the success of such training largely depends on the background: the more you initially know, the faster you will understand the nuances and subtleties.
I see the need for performance engineers is growing. I can draw an analogy with automated testing. I remember the time when everyone thought that automated testing is unnecessary because there are manual tests. However, then it became clear that automation is a must, now it is a trend, now they are trying to automate as much testing as possible, the profession is in great demand. I see similar changes in my specialty.
Working with performance is important because it greatly affects the success of the system. An application “crash” under load, even for a short time, can lead to significant financial losses, slow system work—to a loss of users and profits. This is especially evident in the field of e-commerce: even a slight acceleration of the website leads to an increase in conversion. Fixing performance problems on an already running system can be time-consuming and costly.
