Prompt API 参考

Ragas 的 prompt 系统提供一种灵活且类型安全的方式，为基于 LLM 的指标及其他组件定义 prompt。本页记录核心 prompt 类及其用法。

概览

Ragas 基于 BasePrompt 类采用模块化 prompt 架构。Prompt 可以是：

• Input/Output Models：定义 prompt 输入与输出结构的 Pydantic BaseModel 类
• Prompt Classes：继承 BasePrompt，用于定义指令、示例以及 prompt 生成逻辑
• String Prompts：面向向后兼容的简单文本 prompt

核心类

InputModel

InputModel = TypeVar('InputModel', bound=BaseModel)

OutputModel

OutputModel = TypeVar('OutputModel', bound=BaseModel)

BasePrompt

BasePrompt(name: Optional[str] = None, language: str = 'english', original_hash: Optional[str] = None)

基类：ABC

源代码位于 src/ragas/prompt/base.py

def __init__(
    self,
    name: t.Optional[str] = None,
    language: str = "english",
    original_hash: t.Optional[str] = None,
):
    if name is None:
        self.name = camel_to_snake(self.__class__.__name__)

    self.language = language
    self.original_hash = original_hash

generate

generate(llm: BaseRagasLLM, data: Any, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Callbacks = []) -> Any

从 prompt 生成单次补全。

源代码位于 src/ragas/prompt/base.py

@abstractmethod
async def generate(
    self,
    llm: BaseRagasLLM,
    data: t.Any,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: Callbacks = [],
) -> t.Any:
    """
    Generate a single completion from the prompt.
    """
    pass

generate_multiple

generate_multiple(llm: BaseRagasLLM, data: Any, n: int = 1, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Callbacks = []) -> Any

从 prompt 生成多次补全。

源代码位于 src/ragas/prompt/base.py

@abstractmethod
def generate_multiple(
    self,
    llm: BaseRagasLLM,
    data: t.Any,
    n: int = 1,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: Callbacks = [],
) -> t.Any:
    """
    Generate multiple completions from the prompt.
    """
    pass

save

save(file_path: str)

将 prompt 保存到文件。

源代码位于 src/ragas/prompt/base.py

def save(self, file_path: str):
    """
    Save the prompt to a file.
    """
    data = {
        "ragas_version": __version__,
        "language": self.language,
        "original_hash": self.original_hash,
    }
    if os.path.exists(file_path):
        raise FileExistsError(f"The file '{file_path}' already exists.")
    with open(file_path, "w", encoding="utf-8") as f:
        json.dump(data, f, indent=2, ensure_ascii=False)
        print(f"Prompt saved to {file_path}")

load

load(file_path: str) -> 'BasePrompt'

从文件加载 prompt。

源代码位于 src/ragas/prompt/base.py

@classmethod
def load(cls, file_path: str) -> "BasePrompt":
    """
    Load the prompt from a file.
    """
    with open(file_path, "r", encoding="utf-8") as f:
        data = json.load(f)

    ragas_version = data.get("ragas_version")
    if ragas_version != __version__:
        logger.warning(
            "Prompt was saved with Ragas v%s, but you are loading it with Ragas v%s. "
            "There might be incompatibilities.",
            ragas_version,
            __version__,
        )

    prompt = cls(
        language=data.get("language", "english"),
        original_hash=data.get("original_hash"),
    )

    return prompt

StringPrompt

StringPrompt(name: Optional[str] = None, language: str = 'english', original_hash: Optional[str] = None)

基类：BasePrompt

可用 f-string 语法与附加数据一起格式化的简单 prompt。

对于希望更灵活、不需要 Pydantic 模型的用户，这是 PydanticPrompt 的更简单替代。

参数：

示例：

>>> from ragas.prompt import string_prompt
>>> await prompt.generate(llm=llm, data={"category": "commerce"})

源代码位于 src/ragas/prompt/base.py

def __init__(
    self,
    name: t.Optional[str] = None,
    language: str = "english",
    original_hash: t.Optional[str] = None,
):
    if name is None:
        self.name = camel_to_snake(self.__class__.__name__)

    self.language = language
    self.original_hash = original_hash

generate

generate(llm: BaseRagasLLM, data: str, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Callbacks = []) -> str

根据指令和提供的数据生成文本。

参数：

返回：

源代码位于 src/ragas/prompt/base.py

async def generate(
    self,
    llm: BaseRagasLLM,
    data: str,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: Callbacks = [],
) -> str:
    """
    Generate text based on the instruction and provided data.

    Parameters
    ----------
    llm : BaseRagasLLM
        The language model to use for text generation.
    data : Optional[Dict[str, Any]], optional
        The data to format the instruction with, by default None.
    n : int, optional
        The number of completions to generate, by default 1.
    temperature : Optional[float], optional
        The temperature for text generation, by default None.
    stop : Optional[List[str]], optional
        The stop sequences for text generation, by default None.
    callbacks : Callbacks, optional
        The callbacks to use during text generation, by default [].

    Returns
    -------
    str
        The generated text.
    """
    llm_result = await llm.agenerate_text(
        StringPromptValue(text=data),
        n=1,
        temperature=temperature,
        stop=stop,
        callbacks=callbacks,
    )
    return llm_result.generations[0][0].text

generate_multiple

generate_multiple(llm: BaseRagasLLM, data: str, n: int = 1, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Callbacks = []) -> List[str]

根据指令和提供的数据生成多个互不相同的文本输出。

参数：

返回：

说明

• 启用缓存时，每个输出都会被唯一缓存以防止重复。
• 这确保同一输入的多个输出互不相同。
• 此前缓存返回重复输出的问题已修复。

源代码位于 src/ragas/prompt/base.py

async def generate_multiple(
    self,
    llm: BaseRagasLLM,
    data: str,
    n: int = 1,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: Callbacks = [],
) -> t.List[str]:
    """
    Generate multiple distinct text outputs based on the instruction and provided data.

    Parameters
    ----------
    llm : BaseRagasLLM
        The language model to use for text generation.
    data : str
        The data to format the instruction with.
    n : int, optional
        The number of completions to generate, by default 1.
    temperature : Optional[float], optional
        The temperature for text generation, by default None.
    stop : Optional[List[str]], optional
        Stop sequences for text generation, by default None.
    callbacks : Callbacks, optional
        Callbacks to use during text generation, by default [].

    Returns
    -------
    List[str]
        A list containing `n` generated outputs.

    Notes
    -----
    - When caching is enabled, each output is uniquely cached to prevent duplicates.
    - This ensures that multiple outputs for the same input are distinct.
    - Previous issues where caching returned duplicate outputs have been fixed.
    """
    llm_result = await llm.agenerate_text(
        StringPromptValue(text=data),
        n=n,
        temperature=temperature,
        stop=stop,
        callbacks=callbacks,
    )

    # flatten the generations
    return [gen.text for gen in llm_result.generations[0]]

PydanticPrompt

PydanticPrompt(name: Optional[str] = None, language: str = 'english', original_hash: Optional[str] = None)

基类：BasePrompt、Generic[InputModel, OutputModel]

源代码位于 src/ragas/prompt/base.py

def __init__(
    self,
    name: t.Optional[str] = None,
    language: str = "english",
    original_hash: t.Optional[str] = None,
):
    if name is None:
        self.name = camel_to_snake(self.__class__.__name__)

    self.language = language
    self.original_hash = original_hash

generate

generate(llm: Union[BaseRagasLLM, InstructorBaseRagasLLM, BaseLanguageModel], data: InputModel, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Optional[Callbacks] = None, retries_left: int = 3) -> OutputModel

使用提供的语言模型和输入数据生成单个输出。

该方法是 generate_multiple 的特例，只生成一个输出。

参数：

返回：

说明

该方法内部以 n=1 调用 generate_multiple，并返回第一个（也是唯一一个）结果。

源代码位于 src/ragas/prompt/pydantic_prompt.py

async def generate(
    self,
    llm: t.Union[BaseRagasLLM, InstructorBaseRagasLLM, BaseLanguageModel],
    data: InputModel,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: t.Optional[Callbacks] = None,
    retries_left: int = 3,
) -> OutputModel:
    """
    Generate a single output using the provided language model and input data.

    This method is a special case of `generate_multiple` where only one output is generated.

    Parameters
    ----------
    llm : BaseRagasLLM
        The language model to use for generation.
    data : InputModel
        The input data for generation.
    temperature : float, optional
        The temperature parameter for controlling randomness in generation.
    stop : List[str], optional
        A list of stop sequences to end generation.
    callbacks : Callbacks, optional
        Callback functions to be called during the generation process.
    retries_left : int, optional
        Number of retry attempts for an invalid LLM response

    Returns
    -------
    OutputModel
        The generated output.

    Notes
    -----
    This method internally calls `generate_multiple` with `n=1` and returns the first (and only) result.
    """
    callbacks = callbacks or []

    # this is just a special case of generate_multiple
    output_single = await self.generate_multiple(
        llm=llm,
        data=data,
        n=1,
        temperature=temperature,
        stop=stop,
        callbacks=callbacks,
        retries_left=retries_left,
    )
    return output_single[0]

generate_multiple

generate_multiple(llm: Union[BaseRagasLLM, InstructorBaseRagasLLM, BaseLanguageModel], data: InputModel, n: int = 1, temperature: Optional[float] = None, stop: Optional[List[str]] = None, callbacks: Optional[Callbacks] = None, retries_left: int = 3) -> List[OutputModel]

使用提供的语言模型和输入数据生成多个输出。

参数：

返回：

抛出：

源代码位于 src/ragas/prompt/pydantic_prompt.py

async def generate_multiple(
    self,
    llm: t.Union[BaseRagasLLM, InstructorBaseRagasLLM, BaseLanguageModel],
    data: InputModel,
    n: int = 1,
    temperature: t.Optional[float] = None,
    stop: t.Optional[t.List[str]] = None,
    callbacks: t.Optional[Callbacks] = None,
    retries_left: int = 3,
) -> t.List[OutputModel]:
    """
    Generate multiple outputs using the provided language model and input data.

    Parameters
    ----------
    llm : BaseRagasLLM
        The language model to use for generation.
    data : InputModel
        The input data for generation.
    n : int, optional
        The number of outputs to generate. Default is 1.
    temperature : float, optional
        The temperature parameter for controlling randomness in generation.
    stop : List[str], optional
        A list of stop sequences to end generation.
    callbacks : Callbacks, optional
        Callback functions to be called during the generation process.
    retries_left : int, optional
        Number of retry attempts for an invalid LLM response

    Returns
    -------
    List[OutputModel]
        A list of generated outputs.

    Raises
    ------
    RagasOutputParserException
        If there's an error parsing the output.
    """
    callbacks = callbacks or []

    processed_data = self.process_input(data)
    prompt_rm, prompt_cb = new_group(
        name=self.name,
        inputs={"data": processed_data},
        callbacks=callbacks,
        metadata={"type": ChainType.RAGAS_PROMPT},
    )
    prompt_value = PromptValue(text=self.to_string(processed_data))

    # Handle different LLM types with different interfaces
    # 1. LangChain LLMs have agenerate_prompt() for async with specific signature
    # 2. BaseRagasLLM have generate() with n, temperature, stop, callbacks
    # 3. InstructorLLM has generate()/agenerate() with only prompt and response_model
    if is_langchain_llm(llm):
        # This is a LangChain LLM - use agenerate_prompt() with batch for multiple generations
        langchain_llm = t.cast(BaseLanguageModel, llm)
        # LangChain doesn't support n parameter directly, so we batch multiple prompts
        prompts = t.cast(t.List[t.Any], [prompt_value for _ in range(n)])
        resp = await langchain_llm.agenerate_prompt(
            prompts,
            stop=stop,
            callbacks=prompt_cb,
        )
    elif isinstance(llm, InstructorBaseRagasLLM):
        # This is an InstructorLLM - use its generate()/agenerate() method
        # InstructorLLM.generate()/agenerate() only takes prompt and response_model parameters
        from ragas.llms.base import InstructorLLM

        instructor_llm = t.cast(InstructorLLM, llm)
        if instructor_llm.is_async:
            result = await llm.agenerate(
                prompt=prompt_value.text,
                response_model=self.output_model,
            )
        else:
            result = llm.generate(
                prompt=prompt_value.text,
                response_model=self.output_model,
            )
        # Wrap the single response in an LLMResult-like structure for consistency
        from langchain_core.outputs import Generation, LLMResult

        generation = Generation(text=result.model_dump_json())
        resp = LLMResult(generations=[[generation]])
    else:
        # This is a standard BaseRagasLLM - use generate()
        ragas_llm = t.cast(BaseRagasLLM, llm)
        resp = await ragas_llm.generate(
            prompt_value,
            n=n,
            temperature=temperature,
            stop=stop,
            callbacks=prompt_cb,
        )

    output_models = []
    parser = RagasOutputParser(pydantic_object=self.output_model)

    # Handle cases where LLM returns fewer generations than requested
    if is_langchain_llm(llm) or isinstance(llm, InstructorBaseRagasLLM):
        available_generations = len(resp.generations)
    else:
        available_generations = len(resp.generations[0]) if resp.generations else 0

    actual_n = min(n, available_generations)

    if actual_n == 0:
        logger.error(
            f"LLM returned no generations when {n} were requested. Cannot proceed."
        )
        raise ValueError(f"LLM returned no generations when {n} were requested")

    if actual_n < n:
        logger.warning(
            f"LLM returned {actual_n} generations instead of requested {n}. "
            f"Proceeding with {actual_n} generations."
        )

    for i in range(actual_n):
        if is_langchain_llm(llm) or isinstance(llm, InstructorBaseRagasLLM):
            # For LangChain LLMs and InstructorLLM, each generation is in a separate batch result
            output_string = resp.generations[i][0].text
        else:
            # For Ragas LLMs, all generations are in the first batch
            output_string = resp.generations[0][i].text
        try:
            # For the parser, we need a BaseRagasLLM, so if it's a LangChain LLM, we need to handle this
            if is_langchain_llm(llm) or isinstance(llm, InstructorBaseRagasLLM):
                # Skip parsing retry for LangChain LLMs since parser expects BaseRagasLLM
                answer = self.output_model.model_validate_json(output_string)
            else:
                ragas_llm = t.cast(BaseRagasLLM, llm)
                answer = await parser.parse_output_string(
                    output_string=output_string,
                    prompt_value=prompt_value,
                    llm=ragas_llm,
                    callbacks=prompt_cb,
                    retries_left=retries_left,
                )
            processed_output = self.process_output(answer, data)  # type: ignore
            output_models.append(processed_output)
        except RagasOutputParserException as e:
            prompt_rm.on_chain_error(error=e)
            logger.error("Prompt %s failed to parse output: %s", self.name, e)
            raise e

    prompt_rm.on_chain_end({"output": output_models})

    # Track prompt usage
    track(
        PromptUsageEvent(
            prompt_type="pydantic",
            has_examples=len(self.examples) > 0,
            num_examples=len(self.examples),
            has_response_model=True,  # PydanticPrompt always has response model
            language=self.language,
        )
    )

    return output_models

adapt

adapt(target_language: str, llm: Union[BaseRagasLLM, InstructorBaseRagasLLM], adapt_instruction: bool = False) -> 'PydanticPrompt[InputModel, OutputModel]'

将 prompt 适配到新语言。

源代码位于 src/ragas/prompt/pydantic_prompt.py

async def adapt(
    self,
    target_language: str,
    llm: t.Union[BaseRagasLLM, InstructorBaseRagasLLM],
    adapt_instruction: bool = False,
) -> "PydanticPrompt[InputModel, OutputModel]":
    """
    Adapt the prompt to a new language.
    """

    strings = get_all_strings(self.examples)
    translated_strings = await translate_statements_prompt.generate(
        llm=llm,
        data=ToTranslate(target_language=target_language, statements=strings),
    )

    translated_examples = update_strings(
        obj=self.examples,
        old_strings=strings,
        new_strings=translated_strings.statements,
    )

    new_prompt = copy.deepcopy(self)
    new_prompt.examples = translated_examples
    new_prompt.language = target_language

    if adapt_instruction:
        translated_instruction = await translate_statements_prompt.generate(
            llm=llm,
            data=ToTranslate(
                target_language=target_language, statements=[self.instruction]
            ),
        )
        new_prompt.instruction = translated_instruction.statements[0]

    new_prompt.original_hash = hash(new_prompt)

    return new_prompt

save

save(file_path: str)

将 prompt 保存到文件。

源代码位于 src/ragas/prompt/pydantic_prompt.py

def save(self, file_path: str):
    """
    Save the prompt to a file.
    """
    data = {
        "ragas_version": __version__,
        "original_hash": (
            hash(self) if self.original_hash is None else self.original_hash
        ),
        "language": self.language,
        "instruction": self.instruction,
        "examples": [
            {"input": example[0].model_dump(), "output": example[1].model_dump()}
            for example in self.examples
        ],
    }
    if os.path.exists(file_path):
        raise FileExistsError(f"The file '{file_path}' already exists.")
    with open(file_path, "w", encoding="utf-8") as f:
        json.dump(data, f, indent=2, ensure_ascii=False)
        print(f"Prompt saved to {file_path}")

BoolIO

基类：BaseModel

StringIO

基类：BaseModel

PromptMixin

供拥有 prompt 的类使用的 Mixin，例如 BaseSynthesizer、MetricWithLLM

get_prompts

get_prompts() -> Dict[str, PydanticPrompt]

返回该类的 prompt 字典。

源代码位于 src/ragas/prompt/mixin.py

def get_prompts(self) -> t.Dict[str, PydanticPrompt]:
    """
    Returns a dictionary of prompts for the class.
    """
    prompts = {}
    for _, value in self._get_prompts().items():
        prompts.update({value.name: value})
    return prompts

set_prompts

set_prompts(**prompts)

设置该类的 prompt。

抛出：

源代码位于 src/ragas/prompt/mixin.py

def set_prompts(self, **prompts):
    """
    Sets the prompts for the class.

    Raises
    ------
    ValueError
        If the prompt is not an instance of `PydanticPrompt`.
    """
    available_prompts = self.get_prompts()
    name_to_var = {v.name: k for k, v in self._get_prompts().items()}
    for key, value in prompts.items():
        if key not in available_prompts:
            raise ValueError(
                f"Prompt with name '{key}' does not exist. Use get_prompts() to see available prompts."
            )
        if not isinstance(value, PydanticPrompt):
            raise ValueError(
                f"Prompt with name '{key}' must be an instance of 'ragas.prompt.PydanticPrompt'"
            )
        setattr(self, name_to_var[key], value)

adapt_prompts

adapt_prompts(language: str, llm: Union[BaseRagasLLM, InstructorBaseRagasLLM], adapt_instruction: bool = False) -> Dict[str, PydanticPrompt]

使用给定 LLM，将该类中的 prompt 适配到给定语言。

说明

请务必使用可用的最佳 LLM 来适配 prompt，然后用 save_prompts 和 load_prompts 方法保存并加载 prompt。

源代码位于 src/ragas/prompt/mixin.py

async def adapt_prompts(
    self,
    language: str,
    llm: t.Union[BaseRagasLLM, InstructorBaseRagasLLM],
    adapt_instruction: bool = False,
) -> t.Dict[str, PydanticPrompt]:
    """
    Adapts the prompts in the class to the given language and using the given LLM.

    Notes
    -----
    Make sure you use the best available LLM for adapting the prompts and then save and load the prompts using
    [save_prompts][ragas.prompt.mixin.PromptMixin.save_prompts] and [load_prompts][ragas.prompt.mixin.PromptMixin.load_prompts]
    methods.
    """
    prompts = self.get_prompts()
    adapted_prompts = {}
    for name, prompt in prompts.items():
        adapted_prompt = await prompt.adapt(language, llm, adapt_instruction)
        adapted_prompts[name] = adapted_prompt

    return adapted_prompts

save_prompts

save_prompts(path: str)

将 prompt 保存到目录，格式为 {name}_{language}.json

源代码位于 src/ragas/prompt/mixin.py

def save_prompts(self, path: str):
    """
    Saves the prompts to a directory in the format of {name}_{language}.json
    """
    # check if path is valid
    if not os.path.exists(path):
        raise ValueError(f"Path {path} does not exist")

    prompts = self.get_prompts()
    for prompt_name, prompt in prompts.items():
        # hash_hex = f"0x{hash(prompt) & 0xFFFFFFFFFFFFFFFF:016x}"
        if self.name == "":
            file_name = os.path.join(path, f"{prompt_name}_{prompt.language}.json")
        else:
            file_name = os.path.join(
                path, f"{self.name}_{prompt_name}_{prompt.language}.json"
            )
        prompt.save(file_name)

load_prompts

load_prompts(path: str, language: Optional[str] = None)

从路径加载 prompt。文件格式应为 {name}_{language}.json

源代码位于 src/ragas/prompt/mixin.py

def load_prompts(self, path: str, language: t.Optional[str] = None):
    """
    Loads the prompts from a path. File should be in the format of {name}_{language}.json
    """
    # check if path is valid
    if not os.path.exists(path):
        raise ValueError(f"Path {path} does not exist")

    # check if language is supported, defaults to english
    if language is None:
        language = "english"
        logger.info(
            "Language not specified, loading prompts for default language: %s",
            language,
        )

    loaded_prompts = {}
    for prompt_name, prompt in self.get_prompts().items():
        if self.name == "":
            file_name = os.path.join(path, f"{prompt_name}_{language}.json")
        else:
            file_name = os.path.join(
                path, f"{self.name}_{prompt_name}_{language}.json"
            )
        loaded_prompt = prompt.__class__.load(file_name)
        loaded_prompts[prompt_name] = loaded_prompt
    return loaded_prompts

Metrics Collections Prompts

Ragas 中的现代指标使用专用 prompt 类。每个指标模块包含：

• Input Model：定义 prompt 需要的数据（例如 FaithfulnessInput）
• Output Model：定义期望的 LLM 响应结构（例如 FaithfulnessOutput）
• Prompt Class：继承 BasePrompt，用于生成带有示例和指令的 prompt 字符串

示例：Faithfulness 指标 Prompt

from ragas.metrics.collections.faithfulness.util import (
    FaithfulnessPrompt,
    FaithfulnessInput,
    FaithfulnessOutput,
)

# The prompt class combines input/output models with instructions and examples
prompt = FaithfulnessPrompt()

# Create input data
input_data = FaithfulnessInput(
    response="The capital of France is Paris.",
    context="Paris is the capital and most populous city of France."
)

# Generate the prompt string for the LLM
prompt_string = prompt.to_string(input_data)

# The output will be structured according to FaithfulnessOutput model

可用指标 Prompt

各指标 prompt 的细节见对应指标文档：

• Faithfulness
• Context Recall
• Context Precision
• Answer Correctness
• Factual Correctness
• Noise Sensitivity

自定义

关于为指标自定义 prompt 的详细指南，见 Modifying prompts in metrics。