🎓 AI Study Assistant Using CrewAI & Pinecone

An end-to-end, multi-agent Retrieval-Augmented Generation (RAG) web application that empowers students to upload course materials, index them into a high-performance vector database, and ask questions with 100% fact-checked, cited answers powered by CrewAI and Pinecone.

---

📌 1. Project Overview

Students often struggle to find exact definitions, explanations, and formulas hidden inside dense lecture slides, textbooks, and PDF notes. Standard LLMs often hallucinate or blend outside general knowledge with course-specific syllabi.

AI Study Assistant solves this by pairing:

1. Dense Vector Retrieval (Pinecone + Sentence Transformers): Fast semantic search across course PDFs with page-level provenance.
2. CrewAI Multi-Agent Coordination: A team of three specialized AI agents (Researcher, Analyst, and Reviewer) working sequentially to guarantee that every answer is 100% grounded in the student's uploaded notes.

---

✨ 2. Key Features

• 📄 Drag-and-Drop PDF Upload: Ingest textbooks, syllabus notes, and slide decks.
• ✂️ Smart Semantic Chunking: 800-character chunks with 100-character overlaps preserving page-level metadata.
• 🧠 Local Embedding Engine: Generates 384-dimensional dense vectors using sentence-transformers/all-MiniLM-L6-v2.
• 🌲 Pinecone Cloud Vector Storage: Scalable serverless index with cosine similarity search.
• 🤖 CrewAI 3-Agent Workflow:
  • Research Agent: Scans retrieved passages for relevant facts and extracts evidence.
  • Analysis Agent: Synthesizes structured, student-friendly explanations with definitions and lists.
  • Review Agent: Audits every sentence against source text to eliminate hallucinations.
• 📚 Document Library: Track all uploaded materials, page counts, chunk counts, and indexing status.
• 📊 Dynamic Dashboard: Live counts for documents, chunks, vectors, and questions asked.
• 🏷️ Page-Level Source Citations: Every answer displays the exact document name and page number.
• 🛡️ Zero API Key Leakage: Strict .env isolation and frontend sanitization.

---

🏗️ 3. System Architecture & Workflow

+-----------------------------------------------------------------------------------+
|                                 STUDENT (WEB UI)                                  |
|  [ Dashboard ]  [ Upload PDF ]  [ Ask Question / Chat ]  [ Documents ]  [ Flow ]  |
+------------------------------------------+----------------------------------------+
                                           | HTTP (REST API)
                                           v
+-----------------------------------------------------------------------------------+
|                             FLASK APPLICATION (app.py)                            |
|  - POST /api/upload      - POST /api/ask       - GET /api/documents               |
|  - GET /api/stats        - GET /api/health     - Web Templates & Static Assets    |
+-------------------+--------------------------------------+------------------------+
                    |                                      |
                    v                                      v
+--------------------------------------+   +----------------------------------------+
|             RAG PIPELINE             |   |        CREWAI MULTI-AGENT CREW         |
|                                      |   |                                        |
| 1. PDF Processor (pypdf text & page) |   | 1. Study Material Researcher Agent     |
| 2. Chunker (800 chars, 100 overlap)  |   |    - Extracts factual evidence         |
| 3. Embeddings (all-MiniLM-L6-v2)     |   | 2. Study Material Analyst Agent        |
| 4. Pinecone Manager (index & search) |   |    - Synthesizes clear, student answer |
+-------------------+------------------+   | 3. Answer Reviewer Agent               |
                    |                      |    - Verifies facts & removes fluff    |
                    v                      +-------------------+--------------------+
+--------------------------------------+                       |
|           PINECONE CLOUD             |                       v
|  Vector Index (384-dim, Cosine)      |              +-----------------+
|  Metadata: filename, page, chunk_text|              | Final Answer    |
+--------------------------------------+              | + Source Pages  |
                                                      +-----------------+

---

🤖 4. The 3 Specialized CrewAI Agents

| Agent | Role | Objective | Backstory & Constraints | | :--- | :--- | :--- | :--- | | Research Agent | Study Material Researcher | Search top-k Pinecone passages for facts matching the student's question. | Academic researcher that extracts definitions, formulas, and facts while rejecting irrelevant noise. Never invents facts. | | Analysis Agent | Study Material Analyst | Combine research findings into a clear, structured study response. | Passionate college tutor. Explains complex topics simply. If facts are absent, outputs: "I could not find enough information in the uploaded study material to answer this question." | | Review Agent | Answer Reviewer | Fact-check draft against original passages and polish formatting. | Quality assurance auditor. Cross-references every claim against source chunks, deletes ungrounded text, and finalizes markdown. |

---

🛠️ 5. Technologies Used

• Backend: Python 3.10+, Flask, Flask-CORS, Werkzeug
• Multi-Agent Orchestration: CrewAI
• LLM Provider: Groq API (llama-3.3-70b-versatile / llama3-70b-8192)
• Vector Database: Pinecone Serverless (aws / us-east-1)
• Embedding Model: sentence-transformers/all-MiniLM-L6-v2 (384 Dimensions)
• PDF Extraction: pypdf
• Frontend: HTML5, CSS3, JavaScript (ES6+), Bootstrap 5.3, FontAwesome 6, Marked.js
• Testing: Python unittest

---

📁 6. Project Structure

AI-Study-Assistant/
├── app.py                      # Flask application factory and main entry point
├── requirements.txt            # Python dependencies
├── README.md                   # Project documentation and viva guide
├── .env.example                # Environment variables template
├── .gitignore                  # Git exclusions for secrets and uploads
│
├── config/
│   ├── __init__.py
│   └── config.py               # Central configuration, constants, and health checks
│
├── agents/
│   ├── __init__.py
│   ├── research_agent.py       # Study Material Researcher Agent
│   ├── analysis_agent.py       # Study Material Analyst Agent
│   └── review_agent.py         # Answer Reviewer Agent
│
├── crew/
│   ├── __init__.py
│   ├── tasks.py                # Sequential CrewAI task definitions
│   └── crew_manager.py         # Crew coordinator and execution pipeline
│
├── rag/
│   ├── __init__.py
│   ├── pdf_processor.py        # pypdf text extraction & metadata parser
│   ├── chunker.py              # Semantic text chunker (800 chars / 100 overlap)
│   ├── embeddings.py           # SentenceTransformer singleton wrapper (384-dim)
│   └── pinecone_manager.py     # Pinecone index management, upsert & vector search
│
├── routes/
│   ├── __init__.py
│   ├── upload_routes.py        # POST /api/upload endpoint
│   ├── question_routes.py      # POST /api/ask endpoint
│   └── document_routes.py      # GET /api/documents, /api/stats, /api/health
│
├── templates/
│   ├── base.html               # Shared layout, navbar, sidebar, toast system
│   ├── index.html              # Main dashboard with live statistics
│   ├── upload.html             # Drag & drop upload with 5-stage live progress bar
│   ├── chat.html               # Interactive multi-turn Q&A with source citations
│   ├── documents.html          # Document library table
│   └── how-it-works.html       # 10-step visual pipeline and viva cheat sheet
│
├── static/
│   ├── css/
│   │   └── style.css           # Custom modern design system
│   └── js/
│       ├── dashboard.js        # Dynamic stats & quick ask logic
│       ├── upload.js           # Multi-stage upload animation & drag-drop
│       └── chat.js             # Chat feed, markdown rendering & citations
│
├── data/
│   └── doc_store.json          # Persistent document registry and stats
│
├── uploads/
│   └── .gitkeep                # Temporary upload cache directory
│
└── tests/
    ├── __init__.py
    ├── test_rag.py             # Unit tests for text cleaning and chunking
    ├── test_upload.py          # Unit tests for upload validation
    └── test_api.py             # Integration tests for REST endpoints

---

🚀 7. Installation & Setup

Step 1: Clone or Navigate to the Project Directory

cd "C:\Users\Prathik SBN\Downloads\ai assgint"

Step 2: Create and Activate a Python Virtual Environment

# Create virtual environment
python -m venv venv

# Activate on Windows PowerShell:
.\venv\Scripts\Activate.ps1

# Or on Command Prompt:
.\venv\Scripts\activate.bat

Step 3: Install Required Dependencies

pip install -r requirements.txt

---

🔑 8. Environment Configuration (.env)

1. Copy the .env.example file to create your .env file:

copy .env.example .env

1. Open .env and fill in your API credentials:

# Groq API Key (Get from https://console.groq.com/keys)
GROQ_API_KEY=gsk_your_actual_groq_api_key_here

# Pinecone API Configuration (Get from https://app.pinecone.io/)
PINECONE_API_KEY=pcsk_your_actual_pinecone_api_key_here
PINECONE_INDEX_NAME=ai-study-assistant
PINECONE_ENVIRONMENT=us-east-1

# Model & Chunking Defaults
GROQ_MODEL=llama-3.3-70b-versatile
EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
CHUNK_SIZE=800
CHUNK_OVERLAP=100
TOP_K=5

🔒 Security Notice: The .env file is listed in .gitignore and must never be pushed to public repositories.

---

💻 9. Running the Application

Start the Flask server:

python app.py

Open your browser and navigate to:

http://127.0.0.1:5000

---

🧪 10. Running the Test Suite

Run the automated test suite with Python's built-in unittest:

python -m unittest discover -s tests -p "test_*.py" -v

---

📡 11. REST API Endpoints

1. Upload Study Material

• Endpoint: POST /api/upload
• Content-Type: multipart/form-data
• Parameters: file (PDF file)
• Response:

{
  "success": true,
  "message": "Document uploaded and indexed successfully.",
  "filename": "Machine_Learning_Unit_1.pdf",
  "doc_id": "doc_1725000000_a1b2c3",
  "pages": 12,
  "chunks": 48,
  "indexed_vectors": 48,
  "file_size": "1.45 MB"
}

2. Ask Question

• Endpoint: POST /api/ask
• Content-Type: application/json
• Body:

{
  "question": "What is supervised learning and what are common regression algorithms?"
}

• Response:

{
  "success": true,
  "question": "What is supervised learning and what are common regression algorithms?",
  "answer": "### Supervised Learning\nSupervised learning is a machine learning paradigm...",
  "sources": [
    {
      "filename": "Machine_Learning_Unit_1.pdf",
      "page": 4,
      "score": 0.882
    },
    {
      "filename": "Machine_Learning_Unit_1.pdf",
      "page": 5,
      "score": 0.841
    }
  ],
  "chunks_used": 5
}

3. List Documents

• Endpoint: GET /api/documents
• Response:

{
  "success": true,
  "count": 1,
  "documents": [
    {
      "doc_id": "doc_1725000000_a1b2c3",
      "filename": "Machine_Learning_Unit_1.pdf",
      "pages": 12,
      "chunks": 48,
      "file_size": "1.45 MB",
      "status": "Indexed",
      "uploaded_at": "2025-08-28 22:30:00"
    }
  ]
}

4. Delete Document

• Endpoint: DELETE /api/documents/<doc_id>

5. System Statistics

• Endpoint: GET /api/stats

6. Health Check

• Endpoint: GET /api/health

---

🎯 12. College Project Viva Q&A Guide

Q1: What is RAG (Retrieval-Augmented Generation)?

Answer: RAG is an AI framework where relevant factual data is dynamically retrieved from an external knowledge base (Pinecone) and supplied as context to the Large Language Model (Groq LLM), preventing hallucinations and ensuring answers reflect proprietary course material.

Q2: Why use Sentence Transformers (all-MiniLM-L6-v2)?

Answer: It runs locally with high speed on standard CPUs, converting text into 384-dimensional dense vectors that capture semantic meaning. This allows the system to match concepts even when the student uses different vocabulary than the textbook.

Q3: Why divide reasoning into 3 CrewAI Agents?

Answer: Traditional single-prompt LLM interactions often mix up fact extraction, drafting, and fact-checking. By splitting into a Research Agent (data extraction), Analysis Agent (pedagogical explanation), and Review Agent (fact-checking), each agent has a focused prompt and role, maximizing answer quality and preventing hallucinations.

---

🔮 13. Future Enhancements

1. Support for additional file types (.docx, .pptx, .txt, .epub).
2. OCR support (using Tesseract or EasyOCR) for scanned handwritten lecture notes.
3. Flashcard and quiz generation mode from uploaded PDFs.
4. Voice input and audio readout of explanations for accessibility.