6 Steps to Improve Performance in Software Engineering
Optimizing for performance requires essential steps that are usually missed or skipped. Read this article to never skip any.
Improving the performance in Software Engineering is a duty that most developers, if not all, have. Developers are in charge of making the platform faster. I have seen many projects and initiatives where a team or individuals made the platform X times faster by implementing a Macro or Micro optimization. Although the metrics show success, teams usually do only 80% of the work needed.
Read this post to learn the six steps required when implementing performance optimizations.
The six steps required when working on performance optimizations
1. Understand the scope
The first step is to define the area and scope we want to improve. This could be an endpoint or a particular Job in the case of a Micro optimization, or it could be an entire API in the case of a Macro optimization.
2. Establish a baseline and a target
At this step, we have to measure the current performance of the area we want to improve. For example, if we're going to make the Users API faster, we have to get some metrics such as:
- Percentiles of the server request time for the different endpoints in the Users API if we want to improve latency.
- Throughput of the Users API if we want to improve throughput.
Now that we have a baseline, we have to define a target. This is how much we want to improve the area, of course, this depends on the technical details, but it will give you an idea. As a project manager colleague says, "An optimization without a target is like a project without a goal."
The target is an estimation that can be adjusted in the following step.
3. Explore and investigate
This step takes a significant amount of time. In the case of Micro Optimizations, it could be investigating specific queries of an endpoint. In the case of Macro Optimizations, it could be exploring different technologies, such as a new database or prototyping a new data model.
For this step, developers rely on different observability tools such as profiling, stress tests, unit tests, traces, monitoring systems, loggers, etc.
After discovering different solutions, developers must evaluate the effort, impact, and complexity added to the platform to move to the implementation step.
In some cases, we have reached a bottleneck, and after evaluating, the solution is not worth implementing. In that case, developers should either search for a new solution or stop working on it.
At this step, developers could tweak the target established in the previous step with a more precise value.
4. Implementation
In the case of Micro optimizations, this is the easiest part of the steps. This could be a single line that could have a massive impact. It could be as easy as removing a line, but the impact can be tremendous. In Macro Optimizations, the solution is usually complicated and requires a multi-month/year project.
5. Measure the impact
Now that we applied the changes, it's time to measure the impact by comparing the baseline, the target and the performance after the optimization.
5.1 Repeat Previous Steps
A first iteration is usually never the final and best iteration. It usually requires multiple repetitions of the previous steps.
6. Prevent performance degradation
Unfortunately, this step is usually skipped. The team should make sure that tools are in place so that the performance is not degraded in the future. This could be:
- Tests: For example, if the performance change removed N+1 queries, the developer could add a regression test checking the number of queries executed.
- Dashboards and metrics: The team could invest in observability so that new devs can see the performance in Dashboards.
- Monitor/Alerts: The team can add monitors to alert them when there is an anomaly in the performance. e.g. If we add a new caching layer, we can have alerts that monitor anomalies on #misses, #hits and #latency.
If we skip this step, the performance would likely be degraded to the baseline or below, which requires a new optimization change. In the worst scenario, the change is the same applied before! Check this post if you want to learn more about this step.
Do things, tell people!
Did you learn something? Share it! You might skip sharing, but never skip the rest of the steps!
This is something that my lead from a few years ago taught me, and I will be forever grateful to her! If there is something someone can learn, then share it!
Time spent in each step
The time spent in each step depends on the type of optimizations. As a general idea, we can see how the time varies between Macro and Micro Optimizations in the following charts:
Conclusion
Make sure the problem and goals are clear; if it's not, don't move to step 2. Always establish a baseline and a target. It will help you measure the impact and convince others that the effort is worthwhile.
Always prevent the system from degrading its performance! Read the Performance Part III: Measuring Performance Improvement post, where we dig into this step.
The exploration and implementation vary depending on the type of optimization.
If you follow each step, you will guarantee that:
- You work on the right thing, even if the target is not met.
- The platform is faster!
- You prevent performance degradation in the future.
Finally, never skip a step!
Thanks to Alex Watt and Siddhant Bajaj for reviewing this post.
