Skip to content

comet_llm

init

comet_llm.init(api_key: Optional[str] = None,
    workspace: Optional[str] = None,
    project: Optional[str] = None) -> None

An easy, safe, interactive way to set and save your settings.

Will ask for your api_key if not already set. Your api_key will not be shown.

Will save the config to .comet.config file. Default location is "~/" (home) or COMET_CONFIG, if set.

Args:

  • api_key: str (optional) comet API key.
  • workspace: str (optional) comet workspace to use for logging.
  • project: str (optional) project name to create in comet workspace.

log_prompt

comet_llm.log_prompt(prompt: str, output: str,
    workspace: Optional[str] = None, project: Optional[str] = None,
    tags: Optional[List[str]] = None, api_key: Optional[str] = None,
    prompt_template: Optional[str] = None,
    prompt_template_variables: Optional[
        Dict[str, Union[str,
    bool, float, None]]
    ] = None, metadata: Optional[Dict[str, Union[str,
    bool, float, None]]] = None, timestamp: Optional[float] = None,
    duration: Optional[float] = None) -> llm_result.LLMResult

Logs a single prompt and output to Comet platform.

Args:

  • prompt: str (required) input prompt to LLM.
  • output: str (required), output from LLM.
  • workspace: str (optional) comet workspace to use for logging.
  • project: str (optional) project name to create in comet workspace.
  • tags: List[str] (optional), user-defined tags attached to a prompt call.
  • api_key: str (optional) comet API key.
  • prompt_template: str (optional) user-defined template used for creating a prompt.
  • prompt_template_variables: Dict[str, str] (optional) dictionary with data used in prompt_template to build a prompt.
  • metadata: Dict[str, Union[str, bool, float, None]] (optional) user-defined dictionary with additional metadata to the call.
  • timestamp: float (optional) timestamp of prompt call in seconds
  • duration: float (optional) duration of prompt call

Example:

log_prompt(
    prompt="Answer the question and if the question can't be answered, say \"I don't know\"\n\n---\n\nQuestion: What is your name?\nAnswer:",
    metadata={
        "input.type": "completions",
        "input.model": "text-davinci-003",
        "input.provider": "openai",
        "output.index": 0,
        "output.logprobs": None,
        "output.finish_reason": "length",
        "usage.prompt_tokens": 5,
        "usage.completion_tokens": 7,
        "usage.total_tokens": 12,
    },
    prompt_template="Answer the question and if the question can't be answered, say \"I don't know\"\n\n---\n\nQuestion: {{question}}?\nAnswer:",
    prompt_template_variables={"question": "What is your name?"},
    output=" My name is [your name].",
    duration=16.598,
)

Returns: LLMResult.

log_user_feedback

comet_llm.log_user_feedback(id: str, score: float,
    api_key: Optional[str] = None) -> None

Note: This method is deprecated, instead use the API.get_trace_by_name or API.get_trace_by_key to get the LLMTraceAPI object and then use the LLMTraceAPI.log_user_feedback method.

Logs user feedback for the provided Prompt or Chain ID. This will overwrite any previously set value.

Args:

  • id: str (required) the ID of the Prompt or Chain.
  • score: float (required) the feedback score, can be either 0, 0.0, 1, or 1.0.
  • api_key: str (optional) comet API key.

start_chain

comet_llm.start_chain(inputs: Dict[str, JSONEncodable],
    api_key: Optional[str] = None, workspace: Optional[str] = None,
    project: Optional[str] = None, metadata: Optional[Dict[str, Dict[str,
    JSONEncodable]]] = None, tags: Optional[List[str]] = None) -> None

Creates global Chain object that tracks created Spans.

Args:

  • inputs: Dict[str, JSONEncodable] (required) chain inputs.
  • workspace: str (optional) comet workspace to use for logging.
  • project: str (optional) project name to create in comet workspace.
  • tags: List[str] (optional), user-defined tags attached to a prompt call.
  • api_key: str (optional) comet API key.
  • metadata: Dict[str, Dict[str, JSONEncodable]] (optional) user-defined dictionary with additional metadata to the call.
  • tags: List[str] (optional) user-defined tags attached to the chain

Span

class comet_llm.Span(self, inputs: JSONEncodable, category: str, name: Optional[str] = None, metadata: Optional[Dict[str, JSONEncodable]] = None)

A single unit of Chain that has its own context. Spans can be nested, in that case inner one exist in the context of the outer(parent) one. Outer Span is considered to be a parent for an inner one.

Span.init

__init__(inputs: JSONEncodable, category: str, name: Optional[str] = None,
    metadata: Optional[Dict[str, JSONEncodable]] = None)

Args:

  • inputs: JSONEncodable (required), span inputs.
  • category: str (required), span category.
  • name: str (optional), span name. If not set will be generated automatically.
  • metadata: Dict[str, JSONEncodable] (optional), span metadata.

Span.set_outputs

set_outputs(outputs: Dict[str, JSONEncodable], metadata: Optional[Dict[str,
    JSONEncodable]] = None) -> None

Sets outputs to span object.

Args:

  • outputs: Dict[str, JSONEncodable] (required), outputs
  • metadata: Dict[str, JSONEncodable] (optional), span metadata. If metadata dictionary was passed to init method, it will be updated.

end_chain

comet_llm.end_chain(outputs: Dict[str, JSONEncodable],
    metadata: Optional[Dict[str,
    JSONEncodable]] = None) -> llm_result.LLMResult

Commits global chain and logs the result to Comet.

Args:

  • outputs: Dict[str, JSONEncodable] (required) chain outputs.
  • metadata: Dict[str, Dict[str, JSONEncodable]] (optional) user-defined dictionary with additional metadata to the call. This metadata will be deep merged with the metadata passed to start_chain if it was provided.
  • tags: List[str] (optional) user-defined tags attached to the chain

Returns: LLMResult

is_ready

comet_llm.is_ready() -> bool

True if comet API key is set.

Jul. 12, 2024