compass.llm.calling.LLMCaller#

class LLMCaller(llm_service, usage_tracker=None, **kwargs)[source]#

Bases: BaseLLMCaller

Simple LLM caller, with no memory and no parsing utilities.

Parameters:

  • llm_service (compass.services.base.Service) – LLM service used for queries.

  • usage_tracker (compass.services.usage.UsageTracker, optional) – Optional tracker instance to monitor token usage during LLM calls. By default, None.

  • **kwargs – Keyword arguments to be passed to the underlying service processing function (i.e. llm_service.call(**kwargs)). Should not contain the following keys:

    • usage_tracker

    • usage_sub_label

    • messages

    These arguments are provided by this caller object.

Methods

call(sys_msg, content[, usage_sub_label])

Call LLM.
async call(sys_msg, content, usage_sub_label=LLMUsageCategory.DEFAULT)[source]#

Call LLM.

Parameters:

  • sys_msg (str) – The LLM system message.

  • content (str) – Your chat message for the LLM.

  • usage_sub_label (str, optional) – Label to store token usage under. By default, "default".

Returns:

str or None – The LLM response, as a string, or None if something went wrong during the call.