🥗 AI NourishBot — Multi-Agent Food Intelligence System

📌 Project Overview

AI NourishBot is a production-ready multimodal multi-agent AI system that analyzes food images to detect ingredients, filter by dietary restrictions, perform nutritional analysis, and suggest personalized recipes — all powered by IBM Watsonx Vision AI and CrewAI orchestration.

Upload a photo of any food item and NourishBot's 4 specialized AI agents collaborate to deliver complete nutritional intelligence — from ingredient detection to recipe suggestions tailored to your dietary needs.

Domain: Multimodal Agentic AI — Food Intelligence
Vision Model: Meta Llama 3.2 90B Vision Instruct (IBM Watsonx)
Dietary Filter LLM: IBM Granite 4 H Small
UI: Gradio
Orchestration: CrewAI

🏗️ System Architecture

┌─────────────────────────────────────────────────────────┐
│              Gradio Web Interface (app.py)               │
│   Upload food image + Enter dietary restrictions        │
│   Choose: Recipe Mode OR Nutritional Analysis Mode      │
└──────────────────────┬──────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────┐
│              CrewAI Orchestration (crew.py)              │
│                                                         │
│  NourishBotRecipeCrew     │  NourishBotAnalysisCrew     │
│  (Sequential Process)     │  (Sequential Process)       │
└──┬───────────────────────────────────────┬──────────────┘
   │                                       │
   ▼  RECIPE PIPELINE                      ▼ ANALYSIS PIPELINE
┌──────────────────┐              ┌────────────────────────┐
│ Agent 1          │              │ Agent 3                │
│ Vision AI        │              │ Nutrition Analyst      │
│ Specialist       │              │                        │
│ Tools:           │              │ Tool:                  │
│ ExtractIngredients│             │ NutrientAnalysisTool   │
│ FilterIngredients │             │ (Llama 3.2 90B Vision) │
│ (Llama 3.2 90B) │              └────────────────────────┘
└────────┬─────────┘
         │
┌────────▼─────────┐
│ Agent 2          │
│ Nutritionist AI  │
│ Specialist       │
│ Tool:            │
│ DietaryFilter    │
│ (Granite 4 H)    │
└────────┬─────────┘
         │
┌────────▼─────────┐
│ Agent 4          │
│ Recipe Chef AI   │
│ (No tools —      │
│  Pure LLM)       │
│ Output: JSON     │
│ RecipeSuggestion │
└──────────────────┘

📂 Project Structure

AI-NourishBot/
│
├── app.py                    # Gradio web UI — Recipe & Analysis modes
├── src/
│   ├── crew.py               # CrewAI crew definitions (2 crews, 4 agents)
│   ├── tools.py              # 4 custom LangChain tools
│   ├── models.py             # Pydantic output schemas
│   └── config/
│       ├── agents.yaml       # Agent roles, goals & backstories
│       └── tasks.yaml        # Task descriptions & expected outputs
├── examples/                 # Sample food images (4 images)
├── requirements.txt
└── README.md

🛠️ Tech Stack

| Component | Technology | |---|---| | Agent Orchestration | CrewAI | | Vision LLM | Meta Llama 3.2 90B Vision Instruct (IBM Watsonx) | | Dietary Filter LLM | IBM Granite 4 H Small (IBM Watsonx) | | Platform | IBM Watsonx.ai | | Web UI | Gradio | | Tool Framework | LangChain @tool decorator | | Data Validation | Pydantic BaseModel | | Image Processing | PIL (Pillow) + Base64 encoding | | HTTP Client | requests |

🤖 4 Specialized AI Agents

| Agent | Role | Tools | LLM | |---|---|---|---| | Vision AI Specialist | Detect ingredients from food image | ExtractIngredientsTool + FilterIngredientsTool | Llama 3.2 90B Vision | | Nutritionist AI Specialist | Filter ingredients by dietary restrictions | DietaryFilterTool | Granite 4 H Small | | Nutrition Analysis Specialist | Calorie + nutrient breakdown from image | NutrientAnalysisTool | Llama 3.2 90B Vision | | Recipe Generation Specialist | Creative recipes from filtered ingredients | None (Pure LLM) | IBM Watsonx |

🔧 4 Custom LangChain Tools

1. ExtractIngredientsTool

@tool("Extract ingredients")
def extract_ingredient(image_input: str):
    """Extract ingredients from food image using Llama 3.2 90B Vision"""
    # Base64 encode image → send to Llama Vision → return ingredients list

2. FilterIngredientsTool

@tool("Filter ingredients")
def filter_ingredients(raw_ingredients: str) -> List[str]:
    """Clean and normalize raw ingredient list from LLM output"""

3. DietaryFilterTool

@tool("Filter based on dietary restrictions")
def filter_based_on_restrictions(ingredients, dietary_restrictions) -> List[str]:
    """Use Granite 4 H Small to filter ingredients by vegan/gluten-free/keto etc."""

4. NutrientAnalysisTool

@tool("Analyze nutritional values and calories of the dish from uploaded image")
def analyze_image(image_input: str):
    """Comprehensive nutritional analysis using Llama 3.2 90B Vision"""
    # Returns: portion size, calories, protein, carbs, fats, vitamins, minerals

📊 Pydantic Output Schemas

class Recipe(BaseModel):
    title: str
    ingredients: List[str]
    instructions: str
    calorie_estimate: int

class RecipeSuggestionOutput(BaseModel):
    recipes: List[Recipe]

class NutrientAnalysisOutput(BaseModel):
    dish: str
    portion_size: str
    estimated_calories: int
    nutrients: NutrientBreakdown   # protein, carbs, fats, vitamins, minerals
    health_evaluation: str

🎯 Two Operating Modes

🍽️ Recipe Mode (NourishBotRecipeCrew)

Upload food image
      ↓
Agent 1: Detect ingredients via Llama Vision
      ↓
Agent 2: Filter by dietary restrictions via Granite
      ↓
Agent 4: Generate creative recipes → JSON output

🥗 Nutritional Analysis Mode (NourishBotAnalysisCrew)

Upload food image
      ↓
Agent 3: Full nutritional breakdown via Llama Vision
      ↓
Outputs: Calories, Protein, Carbs, Fats, Vitamins, Minerals, Health Evaluation

⚙️ How to Run

Step 1 — Install dependencies:

pip install -r requirements.txt

Step 2 — Set Watsonx credentials (if running outside Skills Network):

export WATSONX_API_KEY="your-api-key"
export WATSONX_URL="https://us-south.ml.cloud.ibm.com"

Step 3 — Launch the app:

python app.py

Step 4 — Use the Gradio UI:

• Upload a food photo (or use provided examples)
• Enter dietary restrictions (optional): vegan, gluten-free, keto
• Choose Recipe Mode or Nutritional Analysis Mode
• Get AI-powered results! 🍽️

🎓 Skills Demonstrated

• CrewAI multi-agent orchestration (sequential + parallel crews)
• Multimodal AI — vision LLM (Llama 3.2 90B) on food images
• IBM Watsonx.ai integration — Granite + Llama Vision models
• Custom LangChain tool creation with @tool decorator
• Base64 image encoding for vision model API calls
• Pydantic structured output schemas for AI responses
• YAML-based agent/task configuration
• Gradio web UI with image upload + markdown output
• Agent role specialization (Vision, Nutrition, Recipe)
• Dietary restriction filtering via LLM reasoning
• JSON output formatting for recipes and nutrition data
• Production-ready agentic AI application design

📜 Certifications

| Certification | Issuer | Platform | |---|---|---| | IBM Data Science Professional Certificate | IBM | Coursera | | IBM Generative AI Professional Certificate | IBM | Coursera | | IBM RAG and Agentic AI Professional Certificate | IBM | Coursera |

🤝 Connect with Me