A critique of metrics


 

Metrics are measurements or indicators used to evaluate the performance or effectiveness of something, such as a business, a program, or a system. There are several type of metrics that you might encounter

  1. Financial metrics: These metrics are used to measure the financial impact, eg income, expense, profits, GMV, Unit metrics etc
  2. Operational metrics: These metrics are used to measure the operational performance of a business, such as CSAT, KISS Metrics, NPS, earning per employee etc.
  3. Performance metrics: These metrics are used to measure the performance of an individual, a team, or an organization, such as time to launch, time from idea to landing, lines of code etc. For software systems it might also be time it takes for a user to complete a task in the system, system load for the transaction, number of concurrent transactions per second, users severed per second etc.
  4. Quality metrics: These metrics are used to measure the quality of a product, service, or process, such as bugs per kLOC, customer satisfaction, and compliance with standards.
But as with any tool, you need to be aware of its limitations :
  1. Metrics can be misleading: The use of metrics can be subject to various forms of bias, such as selection bias, measurement bias, or interpretation bias. This can lead to inaccurate or incomplete conclusions about the performance or effectiveness of something.
  2. Metrics can be oversimplified: The use of metrics can often lead to oversimplification of complex issues or situations. This can result in incomplete or distorted understandings of the underlying phenomena being measured.
  3. Metrics can be used for the wrong reasons: The only things that get measured get improved. The use of metrics can be motivated by the desire to achieve a specific outcome or result, rather than by a genuine desire to understand the underlying phenomena. This can lead to the misuse or abuse of metrics for ulterior purposes.
  4. Metrics can be subject to gaming: Anything that can be gamed, will be gamed. The use of metrics can incentivize people or organizations to engage in behaviors that improve their metrics, even if these behaviors are not beneficial in the long term. This can lead to short-term gains at the expense of long-term success.

So I would argue that while metrics can be a useful tool for evaluating performance or effectiveness, they should be used with caution and consideration of their limitations and potential biases.

Comments

Post a Comment

Popular posts from this blog

[24th November 2024] Interesting Things I Learnt This Week

Image
1. AI Makes Tech Debt More Expensive - The core argument is that generative AI significantly widens the gap between codebases with low and high tech debt. Companies with clean code can leverage generative AI to write code much faster, while companies with messy legacy codebases struggle to adopt these new tools. This widens the gap between the two codebases, making tech debt even more expensive for companies with outdated systems. My Take : Legacy systems inevitably accumulate technical debt. A common analogy for this is the state of urinals over time: missed targets lead to increasing mess, keeps pushing people away from urinals and leading to more mess in turn. AI-powered coding tools could potentially accelerate the accumulation of tech debt. However, skilled engineers can leverage AI to reduce and even eliminate it. Ultimately, AI is just a tool, and human judgment remains crucial for making informed decisions. I hope AI will be used responsibly to drive positive outcomes. 2. Make...
Read more

[17th November 2024] Interesting Things I Learnt This Week

Image
 1. Booleans are a trap : Booleans, while seemingly simple, can introduce complexity and limitations into domain modeling. When used to represent complex states, booleans can lead to unclear code and potential errors. To avoid these pitfalls, consider using enums or enum sets. These approaches provide a more expressive and maintainable way to model complex states in your applications. My Take : I wish I had learned this earlier in life. When working with Google Docs, I realized the limitations of booleans. Expanding boolean-based behaviors often led to adding more fields, making the system complex. Enums would have provided a simpler solution. The problem wasn't just the booleans themselves, but also the need to pass them through multiple layers of the stack, often requiring additional parameters. Another important aspect of enums is the inclusion of an " enum not specified " state. This state explicitly indicates that an enum value hasn't been set, which can be usef...
Read more

[1st December 2024] Interesting Things I Learnt This Week

Image
1. C++ Pitfalls - this article, explains some of the pitfalls you can fall into when programming in C++ and shares their experience with them. This includes Operator precedence, Arithmetic conversion rules, Right bit-shift, Implicit conversions and Order of evaluation.  My Take : We've all read about JavaScript's quirks, leading to unexpected outputs. But let's remember that such behaviors aren't unique to JavaScript. Even languages like C and C++ have their share of ambiguities and compiler-specific behaviors. While it's tempting to dive deep into these intricacies, I've found that focusing on clear, consistent, and unambiguous code is often more practical. However, understanding these nuances is essential for any experienced programmer. It helps us avoid pitfalls and write robust, reliable code. Remember, mastering a language isn't just about writing "Hello World." It involves delving into its intricacies, including its counterintuitive and ambi...
Read more