compass.services.usage.UsageTracker#

class UsageTracker(label, response_parser)[source]#

Bases: UserDict

Rate or API usage tracker

Parameters:

  • label (str) – Top-level label to use when adding this usage information to another dictionary.

  • response_parser (callable()) – A callable that takes the current usage info (in dictionary format) and an LLm response as inputs, updates the usage dictionary with usage info based on the response, and returns the updated dictionary. See, for example, compass.services.openai.usage_from_response().

Methods

add_to(other)

Add the contents of this usage information to another dict

update_from_model([model, response, sub_label])

Update usage from a model response

Attributes

UNKNOWN_MODEL_LABEL

Label used in the usage dictionary for unknown models

totals

Compute total usage across all sub-labels
UNKNOWN_MODEL_LABEL = 'unknown_model'#

Label used in the usage dictionary for unknown models

add_to(other)[source]#

Add the contents of this usage information to another dict

The contents of this dictionary are stored under the label key that this object was initialized with.

Parameters:

other (dict) – A dictionary to add the contents of this one to.

property totals#

Compute total usage across all sub-labels

Returns:

dict – Dictionary containing usage information totaled across all sub-labels.

update_from_model(model=None, response=None, sub_label='default')[source]#

Update usage from a model response

Parameters:

  • model (str, optional) – Name of model that usage is being recorded for. If None or empty string, the usage will be placed under the UsageTracker.UNKNOWN_MODEL_LABEL label.

  • response (object, optional) – Model call response, which either contains usage information or can be used to infer/compute usage. If None, no update is made. By default, None.

  • sub_label (str, optional) – Optional label to categorize usage under. This can be used to track usage related to certain categories. By default, "default".