Zeno

用 Zeno 可视化 Ragas 结果

你可以使用 Zeno 评测平台，轻松可视化和探索 Ragas 评测结果。

查看本教程最终效果请点 这里

首先，安装 zeno-client 包：

pip install zeno-client

接下来，在 hub.zenoml.com 创建账号，并在你的 account page 上生成 API key。

现在可以从 Getting Started 指南停下的地方继续评测：

import os

import pandas as pd
from datasets import load_dataset
from zeno_client import ZenoClient, ZenoMetric

from ragas import evaluate
from ragas.metrics import (
    answer_relevancy,
    context_precision,
    context_recall,
    faithfulness,
)

# Set API keys
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["ZENO_API_KEY"] = "your-zeno-api-key"

fiqa_eval = load_dataset("vibrantlabsai/fiqa", "ragas_eval")
result = evaluate(
    fiqa_eval["baseline"],
    metrics=[
        context_precision,
        faithfulness,
        answer_relevancy,
        context_recall,
    ],
)

df = result.to_pandas()
df.head()

现在可以把包含数据和结果的 df 上传到 Zeno。

我们首先创建一个项目，配置自定义 RAG 视图规范，以及我们要跨列评测的指标列：

client = ZenoClient(os.environ["ZENO_API_KEY"])

project = client.create_project(
    name="Ragas FICA eval",
    description="Evaluation of RAG model using Ragas on the FICA dataset",
    view={
        "data": {
            "type": "vstack",
            "keys": {
                "question": {"type": "markdown"},
                "texts": {
                    "type": "list",
                    "elements": {"type": "markdown"},
                    "border": True,
                    "pad": True,
                },
            },
        },
        "label": {
            "type": "markdown",
        },
        "output": {
            "type": "vstack",
            "keys": {
                "answer": {"type": "markdown"},
                "ground_truth": {
                    "type": "list",
                    "elements": {"type": "markdown"},
                    "border": True,
                    "pad": True,
                },
            },
        },
        "size": "large",
    },
    metrics=[
        ZenoMetric(
            name="context_precision", type="mean", columns=["context_precision"]
        ),
        ZenoMetric(name="faithfulness", type="mean", columns=["faithfulness"]),
        ZenoMetric(name="answer_relevancy", type="mean", columns=["answer_relevancy"]),
        ZenoMetric(name="context_recall", type="mean", columns=["context_recall"]),
    ],
)

接下来，上传包含问题和 ground truths 的基础数据集：

data_df = pd.DataFrame(
    {
        "data": df.apply(
            lambda x: {"question": x["question"], "texts": list(x["contexts"])}, axis=1
        ),
        "label": df["ground_truth"].apply(lambda x: "\n".join(x)),
    }
)
data_df["id"] = data_df.index

project.upload_dataset(
    data_df, id_column="id", data_column="data", label_column="label"
)

最后，上传 RAG 输出和 Ragas 指标。

在做对比和迭代时，你可以对任意数量的模型执行此操作：

output_df = df[
    [
        "context_precision",
        "faithfulness",
        "answer_relevancy",
        "context_recall",
    ]
].copy()

output_df["output"] = df.apply(
    lambda x: {"answer": x["answer"], "ground_truth": list(x["ground_truth"])}, axis=1
)
output_df["id"] = output_df.index

project.upload_system(
    output_df, name="Base System", id_column="id", output_column="output"
)

如有任何问题，请通过 Discord 或 hello@zenoml.com 联系 Zeno 团队！