compass.llm.calling.BaseLLMCaller

class BaseLLMCaller(llm_service, usage_tracker=None, **kwargs)

Bases: object

Class to support LLM calling functionality

Purpose:

Helper classes to call LLMs.

Responsibilities:

1. Use a service (e.g. OpenAIService) to query an LLM.

2. Maintain a useful context to simplify LLM query.

   • Typically these classes are initialized with a single LLM model (and optionally a usage tracker)

   • This context is passed to every Service.call invocation, allowing user to focus on only the message.

3. Track message history (ChatLLMCaller) or convert output into JSON (StructuredLLMCaller).

Key Relationships:

Delegates most of work to underlying Service class.

Parameters:

• llm_service (compass.services.base.Service) – LLM service used for queries.

• usage_tracker (compass.services.usage.UsageTracker, optional) – Optional tracker instance to monitor token usage during LLM calls. By default, None.

• **kwargs – Keyword arguments to be passed to the underlying service processing function (i.e. llm_service.call(**kwargs)). Should not contain the following keys:

  • usage_tracker

  • usage_sub_label

  • messages

  These arguments are provided by this caller object.

Methods