ASDRP · Agentic RAG Track · Module 05

Stack Migration +
First Real RAG

Swap the local embedder for cloud Ollama and build your first complete end-to-end RAG answer.

Retrieve → Augment Prompt → Generate Grounded Answer

ChatOllama OllamaEmbeddings QdrantVectorStore find_dotenv

Built by mui-group @ ASDRP· Advanced Student Directed Research Program

What we'll cover

The Migration

Why leave the local embedder
Embedding space consistency
The shared tutorials/.env
find_dotenv() pattern

Building the Stack

ChatOllama — generator LLM
OllamaEmbeddings — embedder
Corpus → chunks → Qdrant
In-memory vector store

First RAG Answer

RAG_PROMPT template
rag_answer() function
Retrieve → select top-3 → generate
Grounding vs correctness

The full picture

The RAG Architecture — completed in M05

Modules 01–04 built retrieval. Module 05 completes the loop: cloud embedder + Augment → Generate.

Track 1 · The Migration

Why leave the local embedder?

Embedding space consistency
Cosine similarity only works when query and corpus vectors live in the same space. The index model must equal the query model — always.

RAGAS alignment
RAGAS (Module 06) scores retrieval using its own embedding calls. It must match your retrieval embedder or the scores are meaningless.

Representation quality
qwen3-embedding:0.6b produces better representations for financial/commodity text than a 384-dim general model.

Track 1 · Shared keys

The shared `tutorials/.env` pattern

One file. All twelve modules.
Create tutorials/.env once. Every module notebook calls load_dotenv(find_dotenv()), which walks up the directory tree and resolves to that single shared file.

Already gitignored.
tutorials/.gitignore ignores .env at the parent level. No per-module gitignore line needed — and no risk of accidentally committing real keys.

⚠ NEVER hard-code a key in notebook source. Rotate any key that leaks immediately.

import os
from dotenv import load_dotenv, find_dotenv

# Walks UP → finds tutorials/.env automatically
load_dotenv(find_dotenv())

HAVE_KEYS = bool(os.environ.get("OLLAMA_API_KEY"))

if not HAVE_KEYS:
    print("No key found — using frozen/ results.")

tutorials/.env format:

OLLAMA_API_KEY=<your cloud Ollama key>
# COHERE_API_KEY=  ← added in Module 10
# METALS_API_KEY=  ← added in Module 11

Track 2 · Building the stack

Cloud Ollama LLM + Embedder

from langchain_ollama import ChatOllama, OllamaEmbeddings

os.environ.setdefault("OLLAMA_API_BASE", "http://localhost:11434")

chat_llm = ChatOllama(
    model="nemotron-3-super:cloud",
    base_url=os.environ["OLLAMA_API_BASE"],
    temperature=0.0,          # deterministic answers for reproducibility
)
lc_embeddings = OllamaEmbeddings(
    model="qwen3-embedding:0.6b",
    base_url=os.environ["OLLAMA_API_BASE"],
)

Object	Model	Role
`chat_llm`	`nemotron-3-super:cloud`	Generates the grounded answer
`lc_embeddings`	`qwen3-embedding:0.6b`	Encodes corpus chunks and queries

⚠ Caution — model names are cloud-specific. The :cloud suffix routes through the cloud relay. Substitute locally-served names if running a local daemon.

Track 2 · Indexing

Corpus → chunks → Qdrant `:memory:`

from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_qdrant import QdrantVectorStore
from langchain_core.documents import Document
from pathlib import Path

raw_docs = [{"source": p.name, "page_content": p.read_text()}
            for p in sorted(Path("corpus").glob("*.md"))]  # 8 files

splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=60)
lc_docs  = [Document(page_content=piece, metadata={"source": d["source"]})
            for d in raw_docs
            for piece in splitter.split_text(d["page_content"])]

vector_store = QdrantVectorStore.from_documents(
    lc_docs, embedding=lc_embeddings,
    location=":memory:", collection_name="metals_kb")

⚠ Cost — from_documents sends one embedding call per chunk (~50–80 calls total). Small, but it does consume quota. Skip if no key — use frozen output.

Track 3 · First RAG answer

The `RAG_PROMPT` template — why it matters

Constraint drives grounding
Telling the model to answer only from the context passages — and to say "I don't know" when the answer isn't there — is what separates RAG from pure generation.

If context is wrong, answer is wrong
Grounding constrains the LLM, but doesn't fix bad retrieval. Module 09 measures Faithfulness — how strictly the answer stays in the context.

from langchain_core.prompts import ChatPromptTemplate

RAG_PROMPT = ChatPromptTemplate.from_template(
    "You are a precise metals-markets tutor. "
    "Answer using ONLY the context passages. "
    "If the context does not contain the answer, "
    "say you do not know.\n\n"
    "Context:\n{context}\n\n"
    "Question: {question}\n"
    "Answer:"
)

Track 3 · First RAG answer

`rag_answer()` — retrieve → select → generate

def rag_answer(question: str, k: int = 10, top_n: int = 3) -> dict:
    retriever  = vector_store.as_retriever(search_kwargs={"k": k})
    candidates = [d.page_content for d in retriever.invoke(question)]
    # Module 10 replaces this slice with a Cohere rerank call:
    contexts   = candidates[:top_n]
    block      = "\n\n".join(f"[{i}] {c}" for i, c in enumerate(contexts, 1))
    response   = chat_llm.invoke(
        RAG_PROMPT.format_messages(context=block, question=question)
    ).content.strip()
    return {"response": response, "retrieved_contexts": contexts}

k=10 retrieve 10 candidates from Qdrant

top_n=3 take first 3 (no rerank yet)

⚠ RAG is retrieval-limited. If the right chunk isn't in the top-k, no generator can fix it. Module 07 measures retrieval precision so you can tune k empirically.

Track 3 · Safety net

Frozen fallback — follow along without a key

import json

if HAVE_KEYS:
    result = rag_answer("What factors drive gold prices higher?")
else:
    with open("frozen/rag_answer.json") as fh:
        result = json.load(fh)
    print("(using cached illustrative result — "
          "set OLLAMA_API_KEY in tutorials/.env to run live)")

print(result["response"])

frozen/rag_answer.json is hand-authored, clearly labelled as illustrative, and has the same JSON shape as a live result — {"response": "...", "retrieved_contexts": [...]}. It lets you read sensible output and keep going.

Always know when the number lies

Three cautions for Module 05

⚠ Grounding ≠ correctness. Instructing the LLM to stay in-context reduces hallucination but does not eliminate it. If retrieved passages are inaccurate, the answer can still be wrong. Module 09 measures this gap with Faithfulness and Factual Correctness.

⚠ RAG quality is retrieval-limited. If the right passage is not in the top-k candidates, even a perfect generator cannot produce a correct answer. The parameter k is critical — too small misses passages, too large floods the prompt. Module 07 measures retrieval precision to tune it.

⚠ Cost note. Every rag_answer() call sends one embedding request + one chat-completion request. Indexing sends ~50–80 embedding calls. Use the frozen fallback if you are on a free-tier key and want to preserve quota.

Recap

Summary

The Migration

Local → cloud embedder for quality + RAGAS alignment
Shared tutorials/.env via find_dotenv()
Frozen fallback for keyless students

Building the Stack

ChatOllama + OllamaEmbeddings
Corpus chunked into Qdrant :memory:
Same stack as the capstone — from here forward

First RAG Answer

RAG_PROMPT constrains the LLM to context
rag_answer(): retrieve → top-3 → generate
Grounding ≠ correctness — metrics come in M06–M09

Run the notebook: 05_first_real_rag.ipynb · Next: Module 06 — Why Evaluate? + RAGAS Setup

Stack Migration +First Real RAG

Contents

The Migration

Building the Stack

First RAG Answer

The RAG Architecture — completed in M05

Why leave the local embedder?

The shared tutorials/.env pattern

Cloud Ollama LLM + Embedder

Corpus → chunks → Qdrant :memory:

The RAG_PROMPT template — why it matters

rag_answer() — retrieve → select → generate

Frozen fallback — follow along without a key

Three cautions for Module 05

Summary

The Migration

Building the Stack

First RAG Answer

Stack Migration +
First Real RAG

The shared `tutorials/.env` pattern

Corpus → chunks → Qdrant `:memory:`

The `RAG_PROMPT` template — why it matters

`rag_answer()` — retrieve → select → generate