使用预分块数据生成测试集

当你已经有一套完善的分块策略时，Ragas 允许你绕过其内部文档拆分机制，直接使用你自己的 chunks。这在以下情况特别有用：

• 你已针对特定领域优化了分块策略
• 你希望在 RAG pipeline 和评测之间保持一致性
• 你有带自定义 metadata 的预处理文档
• 你需要确保 chunks 与特定业务逻辑或文档结构对齐

概述

TestsetGenerator 的 generate_with_chunks 方法接受预分块数据，并将每个 chunk 直接视为 NodeType.CHUNK，跳过内部拆分 transforms。这意味着你的 chunks 会完全按你提供的样子保留，同时保持内容和 metadata 的完整性。

工作原理

当你使用 generate_with_chunks 时，Ragas：

1. 按原样接受你的 chunks（可以是 Document 对象或字符串）
2. 应用 extractors，如 SummaryExtractor、ThemesExtractor、NERExtractor 和 EmbeddingExtractor，为每个 chunk 丰富额外属性
3. 构建 chunks 之间的关系，使用 CosineSimilarityBuilder 和 OverlapScoreBuilder
4. 基于内容主题生成 personas
5. 为不同查询类型创建 scenarios（单跳、多跳）
6. 合成测试样本，包括问题、contexts 和参考答案

示例：使用预分块 Documents

你可以传入 LangChain Document 对象列表。这种方法会保留 chunks 的 metadata，这对跟踪源文档或其他自定义信息很有用。

import os
from langchain_core.documents import Document
from ragas.testset.synthesizers.generate import TestsetGenerator
from ragas.llms import llm_factory
from ragas.embeddings import OpenAIEmbeddings
from openai import OpenAI

# Initialize OpenAI client
client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

# Initialize generator with your preferred models
generator = TestsetGenerator(
    llm=llm_factory("gpt-4o-mini", client=client),
    embedding_model=OpenAIEmbeddings(client=client)
)

# Your pre-chunked documents
chunks = [
    Document(
        page_content="""The Eiffel Tower (Tour Eiffel) is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It is named after the engineer Gustave Eiffel, whose company designed and built the tower. Locally nicknamed "La Dame de Fer" (French for "The Iron Lady"), it was constructed from 1887 to 1889 as the centerpiece of the 1889 World's Fair. Although initially criticized by some of France's leading artists and intellectuals for its design, it has since become a global cultural icon of France and one of the most recognizable structures in the world.""", 
        metadata={"source": "doc1", "chunk_id": 1}
    ),
    Document(
        page_content="""The tower is 330 metres (1,083 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest man-made structure in the world, a title it held for 41 years until the Chrysler Building in New York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the Chrysler Building by 5.2 metres (17 ft).""", 
        metadata={"source": "doc1", "chunk_id": 2}
    )
]

# Generate testset
testset = generator.generate_with_chunks(
    chunks=chunks,
    testset_size=10
)

# Save to CSV
output_file = "testset.csv"
testset.to_csv(output_file)
print(f"Testset saved to {output_file}")
print(testset.to_pandas().head())

生成过程

生成过程中，你会看到显示各个转换和合成阶段的进度日志：

Applying SummaryExtractor: 100%|████████████████████████████████| 2/2 [00:07<00:00,  3.67s/it]
Applying CustomNodeFilter: 100%|█████████████████████████████| 2/2 [00:00<00:00, 2226.87it/s]
Applying EmbeddingExtractor: 100%|███████████████████████████| 2/2 [00:02<00:00,  1.19s/it]
Applying ThemesExtractor: 100%|██████████████████████████████| 2/2 [00:06<00:00,  3.07s/it]
Applying NERExtractor: 100%|█████████████████████████████████| 2/2 [00:06<00:00,  3.10s/it]
Applying CosineSimilarityBuilder: 100%|█████████████████████| 1/1 [00:00<00:00, 613.29it/s]
Applying OverlapScoreBuilder: 100%|████████████████████████| 1/1 [00:00<00:00, 1491.57it/s]
Generating personas: 100%|███████████████████████████████████| 2/2 [00:05<00:00,  2.77s/it]
Generating Scenarios: 100%|██████████████████████████████████| 2/2 [00:08<00:00,  4.19s/it]
Generating Samples: 100%|████████████████████████████████| 11/11 [00:45<00:00,  4.13s/it]
Testset saved to testset.csv

测试集包含不同类型的查询：

• 单跳查询：可以从单个 chunk 回答的问题
• 多跳查询：需要来自多个 chunks 的信息的问题（当存在关系时）

示例：使用纯字符串

如果你不需要保留 metadata，也可以直接传入纯字符串：

from ragas.testset.synthesizers.generate import TestsetGenerator
from ragas.llms import llm_factory
from ragas.embeddings import OpenAIEmbeddings
from openai import OpenAI

# Initialize models
client = OpenAI()
generator = TestsetGenerator(
    llm=llm_factory("gpt-4o-mini", client=client),
    embedding_model=OpenAIEmbeddings(client=client)
)

# Simple text chunks
text_chunks = [
    "Artificial Intelligence (AI) is the simulation of human intelligence by machines. It involves machine learning, natural language processing, and computer vision.",
    "Machine Learning is a subset of AI that enables systems to learn from data without explicit programming. Popular algorithms include neural networks and decision trees.",
    "Deep Learning uses neural networks with multiple layers to process complex patterns in large datasets. It powers modern applications like image recognition and language translation."
]

# Generate testset
testset = generator.generate_with_chunks(
    chunks=text_chunks,
    testset_size=5
)

# Save to CSV
output_file = "testset.csv"
testset.to_csv(output_file)
print(f"Testset saved to {output_file}")
print(testset.to_pandas())

处理边界情况

• 空内容：page_content 为空或仅含空白的 chunks 会被自动过滤掉。
• 空序列：如果你提供空的 chunks 序列，生成会产出空测试集。