AI Operations Assistant

A multi-agent AI system built following CrewAI-inspired Agent Architecture patterns that accepts natural-language tasks, automatically plans execution steps, calls real-world APIs/tools, verifies results, and returns structured answers through an interactive Streamlit interface.

Note
This project follows the CrewAI architectural pattern (Planner → Executor → Verifier) using a custom lightweight implementation.
It mirrors CrewAI’s design principles while remaining framework-agnostic and ideal for educational and assignment purposes.

---

Features

• Multi-Agent Architecture
  • Planner, Executor, and Verifier agents working together
• LLM-Powered Reasoning
  • Supports Gemini, Groq (FREE), and OpenRouter
• Tool-Augmented Intelligence
  • GitHub, Weather, Currency, News, StackOverflow APIs
• Conversational Memory
  • Understands follow-up questions and context
• Streamlit UI
  • Clean, modern UI with step-by-step transparency
• Robust Error Handling
  • Graceful failures, partial results, validation checks

---

🧠 Architecture Overview

• ai_ops_assistant/
  • agents/ #consists of agents
  • llm/ #consists of LLMs (Gemini, Groq, OpenRouter)
  • memory/ #consists of memory logic for context
  • tools/ #consists of third party tools for LLMs
  • .env.example
  • main.py
  • README.md
  • requirements.txt
• venv/

---

Architecture Explanation

The AI Operations Assistant is designed as a layered, agent-based architecture inspired by CrewAI principles.
Instead of treating the AI as a single monolithic chatbot, the system decomposes intelligence into specialized components, each with a clear responsibility.

This architecture improves reasoning quality, debuggability, extensibility, and real-world reliability.

---

1. High-Level Architecture

At a high level, the system converts natural language → structured plan → real execution → verified output.

---

2. Architectural Layers

2.1 Presentation Layer (Streamlit UI)

Responsibility

• Accept user input
• Display execution plans, intermediate steps, and final results
• Provide configuration options (LLM provider, API status)

Why Streamlit

• Rapid prototyping
• Single-command deployment
• Clean chat-based interaction
• Ideal for demos and assignments

The UI itself is stateless, while conversation state is managed via Streamlit session state.

---

2.2 Memory Layer

Component

• build_conversation_context()

Responsibility

• Construct short-term conversational memory
• Provide context for follow-up questions
• Maintain continuity across turns

Design Choice

• Session-based memory (no database)
• Lightweight and fast
• Avoids long-term storage complexity

This layer ensures the system understands queries like:

“Tell me more about that”
“Do the same for London”

---

2.3 Agent Orchestrator Layer

Component

• AgentOrchestrator

Responsibility

• Coordinate the entire agent workflow
• Pass outputs between agents
• Track execution state and errors
• Maintain separation of concerns

The orchestrator acts as the control plane of the system.

---

3. Agent-Based Reasoning Layer

3.1 Planner Agent

Role

• Converts unstructured user input into a structured execution plan

Responsibilities

• Understand intent using LLM reasoning
• Break tasks into ordered steps
• Decide which tools are required
• Output a JSON-based execution plan

Why This Matters

• Prevents hallucinated answers
• Makes AI reasoning explicit and inspectable
• Enables deterministic execution

---

3.2 Executor Agent

Role

• Executes the plan created by the Planner Agent

Responsibilities

• Call external APIs and tools
• Handle retries and failures
• Collect raw results
• Execute steps sequentially

Design Benefits

• Tool-agnostic execution
• Easy to add new tools
• Clear separation between reasoning and action

---

3.3 Verifier Agent

Role

• Validate and finalize results

Responsibilities

• Check completeness and correctness
• Detect partial or failed executions
• Assign execution status (complete, partial, failed)
• Format the final user-facing output

Why Verification Is Critical

• Prevents misleading results
• Improves trustworthiness
• Makes the system more production-like

---

4. LLM Abstraction Layer

Component

• BaseLLMClient

Purpose

• Abstract away differences between LLM providers
• Allow runtime switching of models

Supported Providers

• Gemini (Google)
• Groq (FREE, ultra-fast)
• OpenRouter (multi-model gateway)

Architectural Advantage

• Vendor-agnostic
• Easy experimentation
• Future-proof design

---

5. Tooling Layer

Each tool implements a standard interface, enabling the Executor Agent to use them interchangeably.

Characteristics

• Modular
• Extensible
• Isolated from agent logic

Examples

• GitHub Tool → Repository & user data
• Weather Tool → Forecasts (Open-Meteo)
• Currency Tool → Exchange rates
• News Tool → Headlines & articles
• StackOverflow Tool → Developer Q&A

This design allows new tools to be added without modifying agent logic.

---

6. Data Flow Summary

1. User submits a natural-language request
2. Memory context is built from previous turns
3. Planner Agent generates a structured plan
4. Executor Agent performs real API calls
5. Verifier Agent validates results
6. Structured response is returned to UI

This ensures the system behaves as an autonomous reasoning pipeline, not a simple text generator.

---

This architecture demonstrates how modern agentic AI systems are built in practice — combining LLM reasoning, tools, memory, and verification into a cohesive system.

Agent Workflow

1. User Input

   • Natural language task submitted via Streamlit UI

2. Planner Agent

   • Understands user intent
   • Generates a structured JSON execution plan
   • Decides which tools are required

3. Executor Agent

   • Executes each step sequentially
   • Calls external APIs and tools
   • Collects raw results

4. Verifier Agent

   • Validates outputs
   • Detects missing or partial data
   • Produces final structured response with status

---

LLM Providers

The system abstracts LLMs behind a common interface (BaseLLMClient).

1️⃣ Gemini (Google AI)

• High-quality reasoning
• API Key: GEMINI_API_KEY
• Get key: https://makersuite.google.com/app/apikey

2️⃣ Groq (FREE OpenAI Alternative)

• Ultra-fast inference
• Completely free (no credit card)
• Uses LLaMA 3 models
• API Key: GROQ_API_KEY
• Get key: https://console.groq.com

3️⃣ OpenRouter

• Access 300+ models (GPT-4, Claude, Gemini, etc.)
• API Key: OPENROUTER_API_KEY
• Get key: https://openrouter.ai

---

Memory System

• Builds short-term conversational memory from previous messages
• Enables:
  • Follow-up questions
  • Contextual understanding
  • Natural dialogue flow
• Memory is session-based (no long-term persistence)

---

Integrated Tools / APIs

GitHub Tool

• Search repositories
• Fetch repository details
• Get user profiles
  Optional: GITHUB_TOKEN for higher rate limits

Weather Tool

• Current weather
• 5-day forecasts
• Powered by Open-Meteo
• No API key required

Currency Tool

• Currency conversion
• Live exchange rates

News Tool

• Search news articles
• Fetch top headlines
• Requires GNEWS_API_KEY

StackOverflow Tool

• Search programming questions
• Retrieve answers
• Useful for debugging and learning

---

⚡ Quick Start (On LocalHost using CLI)

1. Clone this repo and install dependencies

git clone https://github.com/rahil1801/ai_ops_assistant.git

python -m venv venv (only if virtual environment is not created)

# MACOS
source venv/bin/activate  

# Windows
venv\Scripts\activate

cd ai_ops_assistant

pip install -r requirements.txt

2. Setup Environment Variables

# Create an .env file and copy environment variables from .env.example

GEMINI_API_KEY = your-api-key
GROQ_API_KEY = your-api-key
OPENROUTER_API_KEY = your-api-key (Make sure prompts are available to train their models to use for free)

GITHUB_TOKEN = your-api-key (Make sure to give appropriate permissions)
GNEWS_API_KEY = your-api-key

3. We are almost there. Now run the project

streamlit run main.py

⚡ Quick Start (From Deployed project on Streamlit)

This project is deployed on streamlit so that users can check it to avoid all the necessary setup required to run it from terminal.

Link to project: https://aioperationsassistant.streamlit.app/

---

🧪 Example Tasks

• “Find top Python ML repositories on GitHub”

• “What’s the weather in New York and London?”

• “Convert 100 USD to EUR”

• “Search StackOverflow for Python async issues”

• “Get latest AI news”

• Follow-up queries are supported naturally.

---

🧯 Error Handling

• Automatic retries for tool/API failures

• Partial result support

• Verifier agent detects inconsistencies

• User-friendly error messages

---

📈 Future Improvements

• Parallel tool execution

• Caching (Redis)

• Cost & token tracking

• More tools (Stocks, Maps, Finance)

• User authentication and history persistence

⚠️ Limitations and Trade-offs

While the AI Operations Assistant demonstrates a robust multi-agent architecture, certain limitations and trade-offs were intentionally accepted to keep the system lightweight, understandable, and suitable for educational use.

---

1. No True Parallel Agent Execution

Limitation

• Agents execute sequentially (Planner → Executor → Verifier)
• Tool calls are not parallelized

Trade-off

• ✔ Simpler execution flow
• ✔ Easier debugging and traceability
• ❌ Slower for multi-tool or large tasks

Reasoning Parallel execution adds complexity (async orchestration, race conditions) and was avoided to prioritize clarity and correctness.

---

2. Session-Based Memory Only

Limitation

• Memory exists only for the current session
• No long-term or persistent memory

Trade-off

• ✔ No database or storage overhead
• ✔ Faster and simpler design
• ❌ Context is lost on refresh or restart

Reasoning Persistent memory requires storage, embeddings, and retrieval logic, which was out of scope for this implementation.

---

3. Dependency on LLM Output Quality

Limitation

• Planner reasoning depends heavily on LLM accuracy
• Incorrect plans may lead to suboptimal execution

Trade-off

• ✔ Flexible and intelligent planning
• ❌ Non-deterministic behavior

Reasoning This reflects real-world agentic systems, where verification mitigates but does not eliminate LLM uncertainty.

---

4. Limited Tool Coverage

Limitation

• Only a fixed set of tools is available
• Cannot handle domains without a defined tool

Trade-off

• ✔ Clear and controlled execution environment
• ✔ Easier to validate outputs
• ❌ Reduced domain coverage

Reasoning Each tool requires careful validation; adding more tools was deferred in favor of architectural soundness.

---

5. Sequential Tool Execution

Limitation

• Tools are executed one step at a time

Trade-off

• ✔ Predictable results
• ✔ Easier error handling
• ❌ Increased latency

Reasoning Sequential execution simplifies orchestration and ensures deterministic step ordering.

---

Summary

These limitations reflect conscious design decisions, not architectural weaknesses.
The project focuses on demonstrating agentic reasoning, tool grounding, and architectural clarity, which are foundational concepts for building production-grade AI systems.

Many of these trade-offs can be addressed in future iterations without changing the core architecture.