{
 "cells": [
  {
   "attachments": {
    "4015a2e2-3400-4539-bd93-0d987ec5a44e.png": {
     "image/png": "[encoded data, 177696 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "d8da6094-30c7-43f3-a608-c91717b673db",
   "metadata": {},
   "source": [
    "# Nomic Embeddings\n",
    "\n",
    "Nomic has released a new embedding model with strong performance for long context retrieval (8k context window).\n",
    "\n",
    "The cookbook walks through the process of building and deploying (via LangServe) a RAG app using Nomic embeddings.\n",
    "\n",
    "![Screenshot 2024-02-01 at 9.14.15 AM.png](attachment:4015a2e2-3400-4539-bd93-0d987ec5a44e.png)\n",
    "\n",
    "## Signup\n",
    "\n",
    "Get your API token, then run:\n",
    "```\n",
    "! nomic login\n",
    "```\n",
    "\n",
    "Then run with your generated API token \n",
    "```\n",
    "! nomic login < token > \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f737ec15-e9ab-4629-b54c-24be69e8b60b",
   "metadata": {},
   "outputs": [],
   "source": [
    "! nomic login"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ab7434a-2930-42b5-9164-dc2c03abe232",
   "metadata": {},
   "outputs": [],
   "source": [
    "! nomic login token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3501e2a-4686-4b95-8a1c-f19e035ea354",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install -U langchain-nomic langchain-chroma langchain-community tiktoken langchain-openai langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85cecf42-0144-425b-86c8-219ff17c0195",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Optional: LangSmith API keys\n",
    "import os\n",
    "\n",
    "os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n",
    "os.environ[\"LANGSMITH_ENDPOINT\"] = \"https://api.smith.langchain.com\"\n",
    "os.environ[\"LANGSMITH_API_KEY\"] = \"api_key\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "134475f2-f256-4c13-9712-c55783e6a4e2",
   "metadata": {},
   "source": [
    "## Document Loading\n",
    "\n",
    "Let's test 3 interesting blog posts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01c4d270-171e-45c2-a1b6-e350faa74117",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "urls = [\n",
    "    \"https://lilianweng.github.io/posts/2023-06-23-agent/\",\n",
    "    \"https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/\",\n",
    "    \"https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/\",\n",
    "]\n",
    "\n",
    "docs = [WebBaseLoader(url).load() for url in urls]\n",
    "docs_list = [item for sublist in docs for item in sublist]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "75ab7f74-873c-4d84-af5a-5cf19c61239d",
   "metadata": {},
   "source": [
    "## Splitting \n",
    "\n",
    "Long context retrieval "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f512e128-629e-4304-926f-94fe5c999527",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_text_splitters import CharacterTextSplitter\n",
    "\n",
    "text_splitter = CharacterTextSplitter.from_tiktoken_encoder(\n",
    "    chunk_size=7500, chunk_overlap=100\n",
    ")\n",
    "doc_splits = text_splitter.split_documents(docs_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2a69cf0-e3ab-4c92-a1d0-10da45c08b3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tiktoken\n",
    "\n",
    "encoding = tiktoken.get_encoding(\"cl100k_base\")\n",
    "encoding = tiktoken.encoding_for_model(\"gpt-3.5-turbo\")\n",
    "for d in doc_splits:\n",
    "    print(\"The document is %s tokens\" % len(encoding.encode(d.page_content)))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c58d1e9b-e98e-4bd9-b52f-4dfc2a4e69f4",
   "metadata": {},
   "source": [
    "## Index \n",
    "\n",
    "Nomic embeddings [here](https://docs.nomic.ai/reference/endpoints/nomic-embed-text). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76447866-bf8b-412b-93bc-d6ea8ec35952",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "from langchain_chroma import Chroma\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnableLambda, RunnablePassthrough\n",
    "from langchain_nomic import NomicEmbeddings\n",
    "from langchain_nomic.embeddings import NomicEmbeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15b3eab2-2689-49d4-8cb0-67ef2adcbc49",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add to vectorDB\n",
    "vectorstore = Chroma.from_documents(\n",
    "    documents=doc_splits,\n",
    "    collection_name=\"rag-chroma\",\n",
    "    embedding=NomicEmbeddings(model=\"nomic-embed-text-v1\"),\n",
    ")\n",
    "retriever = vectorstore.as_retriever()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "41131122-3591-4566-aac1-ed19d496820a",
   "metadata": {},
   "source": [
    "## RAG Chain\n",
    "\n",
    "We can use the Mistral `v0.2`, which is [fine-tuned for 32k context](https://x.com/dchaplot/status/1734198245067243629?s=20).\n",
    "\n",
    "We can [use Ollama](https://ollama.ai/library/mistral) -\n",
    "```\n",
    "ollama pull mistral:instruct\n",
    "```\n",
    "\n",
    "We can also run [GPT-4 128k](https://openai.com/blog/new-models-and-developer-products-announced-at-devday). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1397de64-5b4a-4001-adc5-570ff8d31ff6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_ollama import ChatOllama\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "# Prompt\n",
    "template = \"\"\"Answer the question based only on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)\n",
    "\n",
    "# LLM API\n",
    "model = ChatOpenAI(temperature=0, model=\"gpt-4-1106-preview\")\n",
    "\n",
    "# Local LLM\n",
    "ollama_llm = \"mistral:instruct\"\n",
    "model_local = ChatOllama(model=ollama_llm)\n",
    "\n",
    "# Chain\n",
    "chain = (\n",
    "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "    | prompt\n",
    "    | model_local\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1548e00c-1ff6-4e88-aa13-69badf2088fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Question\n",
    "chain.invoke(\"What are the types of agent memory?\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5ec5b4c3-757d-44df-92ea-dd5f08017dd6",
   "metadata": {},
   "source": [
    "**Mistral**\n",
    "\n",
    "Trace: 24k prompt tokens.\n",
    "\n",
    "* https://smith.langchain.com/public/3e04d475-ea08-4ee3-ae66-6416a93d8b08/r\n",
    "\n",
    "--- \n",
    "\n",
    "Some considerations are noted in the [needle in a haystack analysis](https://twitter.com/GregKamradt/status/1722386725635580292?lang=en):\n",
    "\n",
    "* LLMs may suffer with retrieval from large context depending on where the information is placed."
   ]
  },
  {
   "attachments": {
    "0afd4ea4-7ba2-4bfb-8e6d-57300e7a651f.png": {
     "image/png": "[encoded data, 171088 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "de7e6f9e-0c69-47a7-be8a-0ae9233e036c",
   "metadata": {},
   "source": [
    "## LangServe\n",
    "\n",
    "Create a LangServe app. \n",
    "\n",
    "![Screenshot 2024-02-01 at 10.36.05 AM.png](attachment:0afd4ea4-7ba2-4bfb-8e6d-57300e7a651f.png)\n",
    "\n",
    "```\n",
    "$ conda create -n template-testing-env python=3.11\n",
    "$ conda activate template-testing-env\n",
    "$ pip install -U \"langchain-cli[serve]\" \"langserve[all]\"\n",
    "$ langchain app new .\n",
    "$ poetry add langchain-nomic langchain_community tiktoken langchain-openai chromadb langchain\n",
    "$ poetry install\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "Add above logic to new file `chain.py`.\n",
    "\n",
    "---\n",
    "\n",
    "Add to `server.py` -\n",
    "\n",
    "```\n",
    "from app.chain import chain as nomic_chain\n",
    "add_routes(app, nomic_chain, path=\"/nomic-rag\")\n",
    "```\n",
    "\n",
    "Run - \n",
    "```\n",
    "$ poetry run langchain serve\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b4f8022-8aa2-4df4-be7c-635568ef8e24",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
