Ragas Ragas
stable · 中文译文
中文译文 · 原文：https://docs.ragas.io/en/stable/howtos/integrations/_helicone/ · 许可证 Apache-2.0

Helicone

本 notebook 演示如何将 Helicone 与 Ragas 集成，用于监控和评测 RAG（Retrieval-Augmented Generation）系统。

前置条件

开始之前，请确保你已有 Helicone 账号和 API key：

  1. 登录 Helicone，如果还没有账号请先创建。
  2. 登录后，前往 Developer 部分 生成 API key。

注意：请生成 write-only API key。关于 Helicone 认证的更多信息，请参阅 Helicone Auth 文档。

请妥善保存 Helicone API key，集成时会用到。

设置

首先，安装所需包并配置环境。

!pip install datasets ragas openai

import os

from datasets import Dataset

from ragas import evaluate
from ragas.integrations.helicone import helicone_config  # import helicone_config
from ragas.metrics import answer_relevancy, context_precision, faithfulness

# Set up Helicone
HELICONE_API_KEY = (
    "your_helicone_api_key_here"  # Replace with your actual Helicone API key
)
helicone_config.api_key = HELICONE_API_KEY
os.environ["OPENAI_API_KEY"] = (
    "your_openai_api_key_here"  # Replace with your actual OpenAI API key
)

# Verify Helicone API key is set
if HELICONE_API_KEY == "your_helicone_api_key_here":
    raise ValueError(
        "Please replace 'your_helicone_api_key_here' with your actual Helicone API key."
    )

准备数据

让我们为 RAG 系统评测准备一些示例数据。

data_samples = {
    "question": ["When was the first Super Bowl?", "Who has won the most Super Bowls?"],
    "answer": [
        "The first Super Bowl was held on January 15, 1967.",
        "The New England Patriots have won the most Super Bowls, with six championships.",
    ],
    "contexts": [
        [
            "The First AFL–NFL World Championship Game, later known as Super Bowl I, was played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles, California."
        ],
        [
            "As of 2021, the New England Patriots have won the most Super Bowls with six championships, all under the leadership of quarterback Tom Brady and head coach Bill Belichick."
        ],
    ],
    "ground_truth": [
        "The first Super Bowl was held on January 15, 1967.",
        "The New England Patriots have won the most Super Bowls, with six championships as of 2021.",
    ],
}

dataset = Dataset.from_dict(data_samples)
print(dataset)

使用 Ragas 评测

现在，用 Ragas 评测我们的 RAG 系统。Helicone 会自动记录这次评测期间发出的 API 调用。

# Evaluate using Ragas
score = evaluate(dataset, metrics=[faithfulness, answer_relevancy, context_precision])

# Display results
print(score.to_pandas())

在 Helicone 中查看结果

Ragas 评测期间发出的 API 调用会自动记录到 Helicone。你可以在 Helicone dashboard 中查看这些日志，从而了解 RAG 系统的表现与行为。

查看结果：

  1. 打开 Helicone dashboard
  2. 进入 'Requests' 部分
  3. 你应该能看到 Ragas 评测期间发出的 API 调用

你可以分析这些日志以了解：

  • 评测期间发出的 API 调用次数
  • 每次调用的表现（延迟、使用的 token 等）
  • 评测过程中出现的任何错误或问题

该集成让你可以把 Ragas 在 RAG 系统评测上的能力，与 Helicone 强大的监控和分析能力结合起来。

← Athina AIOpenlayer →