crewdspy-doc-agent

A Document Q&A agent that solves the same problem as doc-agent — answer questions about your documents — but using CrewAI and DSPy instead of LangChain and LangGraph.

This is a deliberate side-by-side exercise: same goal, different tools, different mental model.

The mapping: doc-agent → crewdspy-doc-agent

| Concept | doc-agent | crewdspy-doc-agent | |---|---|---| | Orchestration | LangGraph StateGraph | CrewAI Crew + Process.sequential | | Prompt engineering | LangChain f-string prompts | DSPy Signature + ChainOfThought | | State management | AgentState TypedDict | CrewAI task context=[] chaining | | Flow control | Explicit graph edges | Agent roles + task delegation | | Retrieval | LlamaIndex (same) | LlamaIndex (same) | | Routing | Score-based (same learned pattern) | Score-based (same) | | Tool calls | LangChain tools | CrewAI BaseTool | | Answer generation | HumanMessage + llm.invoke() | dspy.ChainOfThought(Signature) |

Architecture

question
   │
   ▼
LlamaIndex retriever → top similarity score
   │
   ├── score ≥ 0.62 (docs are relevant)
   │       │
   │       ▼
   │   CrewAI Crew  (crew.py)
   │       │
   │       ├── ResearchAgent (has DocumentSearchTool)
   │       │     · Autonomously searches the index
   │       │     · May call the tool multiple times
   │       │     · Returns a structured research summary
   │       │
   │       └── WriterAgent (no tools)
   │             · Receives ResearchAgent's output
   │             · Writes a draft answer
   │             │
   │             ▼
   │   DSPy RAGModule  (signatures.py)
   │         · Takes crew's draft + question
   │         · ChainOfThought: reasons → answers
   │         · Returns final structured answer
   │
   └── score < 0.62 (docs don't cover this)
           │
           ▼
       DSPy DirectModule  (signatures.py)
             · ChainOfThought from general knowledge
             · Returns final structured answer

What makes CrewAI different from LangGraph

LangGraph is a graph programming model — you define every node and every edge explicitly. The developer controls the entire flow.

# doc-agent: developer controls every transition
builder.add_edge(START, "retrieve_and_route")
builder.add_conditional_edges("retrieve_and_route", decide_route)
builder.add_edge("generate", END)

CrewAI is a role-based delegation model — you define what each agent is responsible for, and the agents decide how to accomplish their tasks.

# crewdspy: developer defines roles, agents control their steps
researcher = Agent(role="Document Research Specialist", tools=[search_tool], ...)
writer     = Agent(role="Technical Answer Writer", ...)
# The researcher will call the search tool as many times as it needs
# The writer will use the researcher's full output

The tradeoff: LangGraph is more predictable and debuggable; CrewAI is more flexible and handles multi-step research better.

What makes DSPy different from LangChain prompts

LangChain (doc-agent): you hand-write the prompt as an f-string.

# doc-agent: manual prompt engineering
response = llm.invoke([HumanMessage(content=(
    "You are a helpful assistant. Use the document excerpts below to "
    "answer the question. Quote or summarize directly from the text...\n\n"
    f"DOCUMENT EXCERPTS:\n{context}\n\nQUESTION: {question}\n\nANSWER:"
))])

DSPy (crewdspy): you declare the signature (inputs + outputs) and DSPy generates and optimizes the prompt.

# crewdspy: declarative signature — no prompt string to maintain
class AnswerFromDocs(dspy.Signature):
    """Answer a question using research extracted from indexed documents."""
    context:  str = dspy.InputField(desc="research summary from the crew")
    question: str = dspy.InputField(desc="the user's question")
    answer:   str = dspy.OutputField(desc="a complete, grounded answer")

predictor = dspy.ChainOfThought(AnswerFromDocs)
result    = predictor(context=context, question=question)

DSPy's ChainOfThought automatically adds a reasoning step before the answer. And with a few labeled examples, DSPy's optimizer (MIPRO, BootstrapFewShot) can rewrite the entire prompt to maximize a metric — no hand-tuning required.

Project structure

crewdspy-doc-agent/
├── docs/              ← your documents (txt, md, pdf)
├── storage/           ← LlamaIndex index (created by ingest.py)
│
├── config.py          · CrewAI LLM + DSPy LM + LlamaIndex embed config
├── ingest.py          · LlamaIndex: load docs → embed → persist (unchanged pattern)
├── signatures.py      · DSPy: Signatures + RAGModule + DirectModule
├── tools.py           · CrewAI: DocumentSearchTool (wraps LlamaIndex)
├── crew.py            · CrewAI: ResearchAgent + WriterAgent + Crew
├── main.py            · CLI: routing → crew → DSPy → answer
│
├── Dockerfile
├── docker-compose.yml
├── entrypoint.sh
├── requirements.txt
└── .env.example

Quick start — Docker

cd crewdspy-doc-agent

# Add a document
cp ../doc-agent/docs/mangolab_knowledge_base.txt docs/

# Start everything
docker compose up --build -d

# Watch startup logs
docker compose logs -f crewdspy-doc-agent

# Connect
docker compose attach crewdspy-doc-agent

Quick start — Local

cd crewdspy-doc-agent
cp .env.example .env

pip install -r requirements.txt

# Add documents, then index
cp ../doc-agent/docs/mangolab_knowledge_base.txt docs/
python ingest.py

python main.py

Key commands

| Action | Command | |---|---| | Rebuild after code changes | docker compose down && docker compose up --build -d | | Re-index after doc changes | docker compose exec crewdspy-doc-agent python ingest.py | | Connect to agent | docker compose attach crewdspy-doc-agent | | View CrewAI reasoning | Set CREW_VERBOSE=true in docker-compose.yml |

Going further with DSPy: optimization

The biggest DSPy feature not yet used here is prompt optimization. Once you have labeled examples, you can auto-tune the signatures:

import dspy
from dspy.teleprompt import BootstrapFewShot

# Define a metric
def answer_metric(example, prediction, trace=None):
    return example.answer.lower() in prediction.answer.lower()

# Build a training set
trainset = [
    dspy.Example(question="Who is Jorge Herrera?", answer="Founder of MangóLab").with_inputs("question"),
    # ... more examples
]

# Optimize
optimizer = BootstrapFewShot(metric=answer_metric)
optimized_module = optimizer.compile(DirectModule(), trainset=trainset)

# Save the optimized prompts
optimized_module.save("optimized_direct.json")

After optimization, DSPy rewrites DirectAnswer's prompt to include few-shot examples that maximize answer_metric. The module is now provably better than the hand-written signature — and you can prove it with a held-out eval set.