threat intel

On the Diminishing Marginal Utility of IOCs

Prometheus

— 5 min read

Cross-pollination of knowledge and learning from one field of study to another is essential for driving innovation, solving complex problems, and creating new solutions. Or enabling me to defeat what I think are bad ideas.

To that end, I've spent a lot of time thinking about how horrible IOC feeds are... not just the open source ones with millions of IPs and hashes, but the vendor provided ones as well. I've wanted a way to express this thought beyond "this is a bad idea, and you should feel bad for pursuing it", and I think I've arrived on a solution! In the end, my goal (an on-going one, at that) is to provide an Incident Responder's view of the valuation of IOCs, one that is outside the traditional context of dollar-value ROI.

A fair warning - I am, obviously, not an economist... neither in reality, nor in dreams. I know enough to be dangerous.

In economics, there is a concept used to understand consumer behavior called utility. Utility is the satisfaction or pleasure a consumer derives from consuming a good or service. It's a measure of how much happiness or fulfillment we get from using something. Further, there are two derivative concepts: marginal utility and diminishing marginal utility. Marginal utility is the additional satisfaction or pleasure derived from consuming additional units of a good or service. It measures the change in utility when an extra unit of aforementioned good or service is consumed. As we consume more and more of of a prospective good or service, the additional satisfaction derived from each additional unit decreases. This is known as In economics, there is a concept used to understand consumer behavior called utility. Utility is the satisfaction or pleasure a consumer derives from consuming a good or service. It's a measure of how much happiness or fulfillment we get from using something. Further, there are two derivative concepts: marginal utility and diminishing marginal utility. Marginal utility is the additional satisfaction or pleasure derived from consuming additional units of a good or service. It measures the change in utility when an extra unit of aforementioned good or service is consumed. As we consume more and more of of a prospective good or service, the additional satisfaction derived from each additional unit decreases. This is known as diminishing marginal utility.

While we can't directly observe utility, economists have come up with a few ways to estimate it indirectly... 

  1. Ordinal Utility assumes that consumers can rank their preferences for different goods and services but cannot assign a numerical value to those preferences. For example, a consumer might say they prefer A to B, B to C, and C to A, but not how much more they prefer C over A.
  2. Cardinal Utility assigns a numerical value to the utility gained from consuming a good or service.

In the context of a SOC, we can think of IOCs as goods consumed by our logging tools and detection platforms, which can be used to generate alerts for Incident Response Analysts to investigate. IOC feeds are notorious for including observables that represent benign infrastructure (i.e. Google Public DNS), known web crawlers, internet discovery services (Shodan, Censys, Silent Push), mail servers, cloud PaaS providers, expired observables (i.e. observables that, at one point did, but no longer represent a threat), etc… Including these benign observables has the potential to generate false positive alerts, resulting in increased irritation and frustration.

If we consider Incident Response Analysts to be the downstream consumer of IOCs vis-à-vis alerts , we can use the concepts of utility, marginal utility, and diminishing marginal utility as a proxy for analyst morale, job satisfaction, or a rough model for potential burnout scenarios. 

Because we don’t have the time to ask each analyst to rank each IOC feed, we’ll proceed with the assumption that Cardinal Utility is the preferred method to estimate utility. As such, a simple mathematical expression to represent this concept is:

\(MU(Q) = \frac{100}{Q^b}\)

Where

  • MU(Q) is the marginal utility derived from consuming the Qth unit.
  • a is a constant representing the initial marginal utility from consuming the first unit.
  • Q is the quantity of units consumed.
  • b is a constant greater than 0 that controls the rate at which marginal utility diminishes as Q increases.

Suppose the marginal utility of consuming a product can be represented by the following:

  • a=100: The marginal utility when Q=1 (the first unit consumed) is 100.
  • b=2: The rate at which marginal utility diminishes as more units are consumed.

Using the above parameters, we can calculate Marginal Utility as such:

  • For Q=1 (the first unit consumed): \(MU(1) = \frac{100}{1^2} = 100\)
  • For Q=2 (the second unit consumed): \(MU(2) = \frac{100}{2^2} = \frac{100}{4} = 25\)
  • For Q=3 (the third unit consumed): \(MU(3) = \frac{100}{3^2} = \frac{100}{9} ≈ 11.11\)

You can see that with each additional unit consumed, utility (i.e. satisfaction) decreases.

In practice, diminishing marginal utility of IOCs does not follow a constant/uniform decrease. Analyst irritation and frustration can vary. So, let’s adapt the simple equation above and implement it as a Python function:

import random

def random_accelerating_diminishing_marginal_utility(initial_utility, base_rate_of_decrease, quantity):
    
    """
    Calculate the diminishing marginal utility with a random rate of decrease, where the initial rate of decrease is 10%, and each subsequent rate of decrease is multiplied by a random number between 1.01 and 1.5 for each additional unit consumed.

    Parameters:
    initial_utility (float): The initial utility value from consuming the 
    first unit.
    
    base_rate_of_decrease (float): The initial rate at which marginal utility decreases for the first additional unit.
    
    quantity (int): The number of units consumed.

    Returns:
    list: A list of marginal utilities for each unit consumed.

    """
    
    marginal_utilities = []
    current_utility = initial_utility
    current_rate_of_decrease = base_rate_of_decrease

    for i in range(quantity):
        marginal_utilities.append(current_utility)
        current_utility *= (1 - current_rate_of_decrease)
        # Generate a random multiplier between 1.01 and 1.5
        current_rate_of_decrease *= random.uniform(1.01, 1.5)

    return marginal_utilities

# Example usage:
initial_utility = 100  # Utility from the first unit
base_rate_of_decrease = 0.1  # Initial 10% decrease in utility for the first additional unit
quantity = 10  # Number of units consumed

utilities = random_accelerating_diminishing_marginal_utility(initial_utility, base_rate_of_decrease, quantity)

print(utilities)

For an initial utility of 100, a base rate of decrease of 10%, and consuming 10 IOCs that lead to a false positive, the output might look something like this:

[100, 90.0, 78.5, 61.25, 43.4375, 27.14875, 11.7227, 1.758405, -1.803906, -2.473289]
  • Initially, the utility decreases by 10%, from 100 to 90.
  • For each subsequent investigation, the rate of decrease is multiplied by a random value between 1.01 and 1.5. This causes the rate of decrease to fluctuate unpredictably, leading to varying reductions in utility.
  • Some investigations cause a smaller reduction in utility (maybe they're more fun?), while others cause a more significant reduction (maybe your top analyst is investigating an alert triggered by a DNS query to 8.8.8.8?).

What conclusion should you draw from the above exercise? Considering the general lack of context in many IOC feeds and the routine inclusion of benign observables, there is a reasonable likelihood that Incident Response Analysts will perform an investigation that leads to an outcome of False Positive. Simply stated, as more IOCs are ingested and generate more alerts, Incident Response Analysts will derive less job satisfaction due to greater irritation and frustration, leading to burnout and analyst churn.