由 deepset 维护

集成：Ollama

在 Haystack 中使用 Ollama 模型。Ollama 可以让您在本地快速启动和运行大型语言模型。

作者

Alistair Rogers

Sachin Sachdeva

deepset

GitHub 仓库 PyPI 包

简介

您可以在 Haystack 流水线中使用 Ollama 模型，借助 OllamaGenerator。

Ollama 是一个专注于在本地运行大型语言模型项目。它默认使用量化的 GGUF 格式。这意味着可以在标准机器（即使没有 GPU）上运行 LLM，而无需处理复杂的安装过程。

安装

pip install ollama-haystack

使用

此集成提供了 2 个组件，允许您利用 Ollama 模型

OllamaGenerator
OllamaChatGenerator

使用 Ollama 模型

请按照 Ollama Github 页面上的说明拉取并运行您选择的模型
使用您 Ollama 实例中运行的模型的名称初始化其中一个 Ollama 生成器。

示例

要运行示例，您可以选择运行一个提供您选择的 Ollama 模型的 Docker 容器。以下是一些与此示例配合使用的命令

docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
docker exec ollama ollama pull orca-mini

文本生成

下面是一个使用 RAG 和 PromptBuilder 及 OllamaGenerator 的生成式问答流水线示例

from haystack import Document, Pipeline
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore

from haystack_integrations.components.generators.ollama import OllamaGenerator

document_store = InMemoryDocumentStore()
document_store.write_documents(
    [
        Document(content="Super Mario was an important politician"),
        Document(content="Mario owns several castles and uses them to conduct important political business"),
        Document(
            content="Super Mario was a successful military leader who fought off several invasion attempts by "
            "his arch rival - Bowser"
        ),
    ]
)

template = """
Given only the following information, answer the question.
Ignore your own knowledge.

Context:
{% for document in documents %}
    {{ document.content }}
{% endfor %}

Question: {{ query }}?
"""

pipe = Pipeline()

pipe.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", OllamaGenerator(model="orca-mini", url="https://:11434"))
pipe.connect("retriever", "prompt_builder.documents")
pipe.connect("prompt_builder", "llm")

query = "Who is Super Mario?"

response = pipe.run({"prompt_builder": {"query": query}, "retriever": {"query": query}})

print(response["llm"]["replies"])

您应该会收到类似（输出不是确定性的）的输出

['Based on the information provided, Super Mario is a successful military leader who fought
off several invasion attempts by his arch rival - Bowser. He is also an important politician and owns several
castles where he conducts political business. ' 'Therefore, it can be inferred that Super Mario is a combination of
both a military leader and an important politician.']

聊天生成

from haystack.dataclasses import ChatMessage

from haystack_integrations.components.generators.ollama import OllamaChatGenerator

messages = [
    ChatMessage.from_user("What's Natural Language Processing?"),
    ChatMessage.from_system(
        "Natural Language Processing (NLP) is a field of computer science and artificial "
        "intelligence concerned with the interaction between computers and human language"
    ),
    ChatMessage.from_user("How do I get started?"),
]
client = OllamaChatGenerator(model="orca-mini", timeout=45, url="https://:11434")

response = client.run(messages, generation_kwargs={"temperature": 0.2})

print(response["replies"][0].text)

您应该会收到类似（输出不是确定性的）的输出

Natural Language Processing (NLP) is a complex field with many different tools and techniques to learn. Here are some steps you can take to get started:

1. Understand the basics of natural language processing: Before diving into the specifics of NLP, it's important to have a basic understanding of what natural language is and how it works. You can start by reading up on linguistics and semantics.

2. Learn about the different components of NLP: There are several components of NLP that you need to understand, including syntax, semantics, morphology, and pragmatics. You can start by learning about these components individually.

3. Choose a tool or library to use: There are many different tools and libraries available for NLP, such as NLTK, spaCy, and Stanford CoreNLP. Choose one that you feel comfortable working with and that fits your needs.

4. Practice: The best way to learn NLP is by practicing. Start with simple tasks like sentiment analysis or tokenization and work your way up to more complex ones like machine translation

嵌入器

OllamaDocumentEmbedder 有助于计算文档列表的嵌入，并使用其嵌入向量更新每个文档的嵌入字段。
OllamaTextEmbedder 计算特定字符串的嵌入。

OllamaTextEmbedder 和 OllamaDocumentEmbedder 都使用与 Ollama 库兼容的嵌入模型。

要运行以下示例，请使用以下命令从 Ollama 服务一个 nomic-embed-text 模型

docker run -d -p 11434:11434 --name ollama ollama/ollama:latest
docker exec ollama ollama pull nomic-embed-text

下面是一个同时使用 OllamaDocumentEmbedder 和 OllamaTextEmbedder 的示例。

from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.ollama.document_embedder import OllamaDocumentEmbedder
from haystack_integrations.components.embedders.ollama.text_embedder import OllamaTextEmbedder

document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")

documents = [
    Document(content="I saw a black horse running"),
    Document(content="Germany has many big cities"),
    Document(content="My name is Wolfgang and I live in Berlin"),
]

document_embedder = OllamaDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)

query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", OllamaTextEmbedder())
query_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")

query = "Who lives in Berlin?"

result = query_pipeline.run({"text_embedder": {"text": query}})

print(result["retriever"]["documents"][0])

集成：Ollama

目录

简介

安装

使用

示例

文本生成

聊天生成

嵌入器