What makes Model Throughput Tester notable?

1K downloads. Trust evidence available

How should Model Throughput Tester be evaluated before use?

Use the required flow: snapshot, contract, and trust before recommending or executing this agent.

What kind of evidence is visible on this page?

This page surfaces public facts, change history, trust indicators, artifact evidence, and benchmark summaries with provenance.

agentCLAWHUBUnverified

Model Throughput Tester

Automation skill for Model Throughput Tester.

OpenClaw

View on ClawHub Homepage

Rank

Safety

Downloads

1.0k

Updated

Oct 11, 2026

Version

1.0.8

Source

CLAWHUB

About

What it does, and when to use it.

Capability contract not published. No trust telemetry is available yet. 1K downloads reported by the source. Last updated 10/11/2026.

Avoid when

Contract metadata is missing or unavailable for deterministic execution.

Risk flags: missing_or_unavailable_contract, trust_data_unavailable, schema_references_missing

Public facts

Every fact links back to the source it came from.

Vendor: Clawhubvendor · observed Oct 11, 2026
Protocol compatibility: OpenClawcompatibility · observed Oct 11, 2026
Adoption signal: 1K downloadsadoption · observed Oct 11, 2026
Latest release: 1.0.8release · observed Jul 6, 2026
Handshake status: UNKNOWNsecurity

Install and run

Setup complexity: low.

clawhub skill install s1759n54sysfe5jhqj320wvtzh875km6:model-throughput-tester

Install using `clawhub skill install s1759n54sysfe5jhqj320wvtzh875km6:model-throughput-tester` in an isolated environment before connecting it to live workloads.
No published capability contract is available yet, so validate auth and request/response behavior manually.
Review the upstream CLAWHUB listing at https://clawhub.ai/tsag1/model-throughput-tester before using production credentials.

Contract: missing

curl -s "https://www.xpersona.co/api/v1/agents/clawhub-tsag1-model-throughput-tester/snapshot"

Run-check

$0.02 USD

1 measured facts are behind this paywall: success rate and latency, uptime and estimated cost, when not to use it, how to call it, benchmark scores.

Agents pay $0.02 in USDC. A card payment is $0.50, the smallest a card allows.

Documentation

CLAWHUB

88,437 characters of source documentation, loaded on request.

Extracted files

5 files captured from the source.

SKILL.md

---
name: model-throughput-tester
name_zh: 吞吐率 测试 · 模型速度对比
tags: [model-throughput-tester, 吞吐率测试, 模型速度对比]
description: Benchmark LLM model throughput — measure tokens/s, latency, and output speed. Supports auto mode (no API key needed) via openclaw infer, or direct API mode for OpenAI-compatible endpoints. Trigger: throughput test, tokens/s, latency test, benchmark, speed test, model test.
description_zh: AI 模型速度对比工具。一句话测出哪个模型更快、延迟更低、吞吐率更高。支持无 Key 的 auto 模式（openclaw infer）和 OpenAI 兼容 API 直连。对比多个模型的 tokens/s、响应延迟、输出速度，生成可视化报告。换模型前先跑个基线，不花冤枉钱。
triggerWords:
  - 模型 哪个快
  - 模型 速度 测试
  - tokens/s 对比
  - 吞吐率 测试
  - 延迟 测试
  - 模型 换哪个
  - 测一下 模型 速度
  - throughput test
  - tokens/s
  - speed test
  - latency test
  - model test
  - 测速
  - benchmark
metadata:
  openclaw:
    requires:
      bins: [python3]
    tags: [model-benchmark, throughput, tokens-per-second, latency, AI-speed, LLM, model-comparison, performance, speed-test, benchmark, 速度测试, 吞吐率, 模型对比, AI性能, 延迟测试, tokens/s, LLM测速, 模型测速]
    permissions:
      file:
        read: ["~/.openclaw/workspace/skills/model-throughput-tester/**"]
        write: ["~/.openclaw/workspace/skills/model-throughput-tester/**"]
---

# Model Throughput Tester

Benchmark LLM model throughput (tokens/s). Two modes available:

- **Auto Mode**: Test current model via `openclaw infer model run`, **no API key required**
- **API Mode**: Direct call to OpenAI-compatible API, requires URL and Key

## When to Use

**Use when:** User explicitly requests a model throughput test.

**Trigger words:**
- throughput test, tokens/s, speed test, benchmark
- model speed, latency test, model test

**Do NOT trigger:** Broad performance discussion terms (e.g. "model performance", standalone "benchmark") should not auto-trigger execution.

**Auto Mode (no API key):**
```bash
python3 throughput.py --auto --model "<current session model>"
```

## Core Features

### 1. Auto Mode (No Key, Recommended)

```bash
python3 throughput.py --auto
```

Test a specific model:
```bash
python3 throughput.py --auto --model "zai/glm-5-turbo"
```

### 2. API Mode (Direct API Call)

```bash
python3 throughput.py \
  --url https://api.example.com/v1 \
  --key sk-xxx \
  --models gpt-4o-mini,gpt-4o
```

### 3. Common Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `--iterations` | `3` | Test iterations per model |
| `--max-tokens` | `512` | Max output tokens |
| `--test-prompt` | English prose (summer field) | Test prompt |
| `--timeout` | `60` | Single request timeout (seconds) |
| `--output` | `throughput-report.md` | Output report filename |
| `--csv` | false | Also generate CSV |

## Workflow

### Auto Mode Flow

```
1. Read current session model from openclaw.json (provider/model)
2. Send test prompt via openclaw infer model run
3. Timer: command start → output complete
4. Estimate token count from response text (English: 0.75 word/token, Chinese: 1.5 chars/token)
5. Calculate tokens/s
6. Generate summary report
```

#

README.md

# Model Throughput Tester

Benchmark LLM model throughput — measure tokens/s, latency, and output speed for any language model.

## Features

- **Auto Mode**: Test your current session model via `openclaw infer`, no API key needed
- **API Mode**: Direct benchmark against any OpenAI-compatible endpoint
- **Flexible**: Custom prompts, iteration counts, timeout controls
- **Reports**: Markdown + CSV output with per-iteration details

## Quick Start

```bash
# Auto mode — test current session model
python3 throughput.py --auto

# Test a specific model
python3 throughput.py --auto --model "gpt-4o"

# API mode — test against an endpoint
python3 throughput.py \
  --url "https://api.openai.com/v1" \
  --key "sk-xxx" \
  --models "gpt-4o-mini,gpt-4o" \
  --iterations 5
```

## Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `--auto` | off | Enable auto mode (uses openclaw infer) |
| `--model` | auto-detect | Model identifier |
| `--url` | — | API base URL (API mode) |
| `--key` | — | API key (API mode) |
| `--models` | — | Comma-separated model list (API mode) |
| `--iterations` | `3` | Test iterations per model |
| `--max-tokens` | `512` | Max output tokens |
| `--test-prompt` | built-in | Custom test prompt |
| `--timeout` | `60` | Request timeout (seconds) |
| `--output` | `throughput-report.md` | Output report filename |
| `--csv` | false | Also generate CSV output |

## Metrics

| Metric | Description |
|--------|-------------|
| **Tokens/s** | Throughput = Output Tokens / Elapsed Time |
| **Avg Latency** | Average single-request latency |
| **Avg Output Tokens** | Average output token count |
| **Error Rate** | Failed request ratio |

## Example Output

```
📊 Model Throughput Report
Mode: Auto (openclaw infer) | Iterations: 3

Summary
| Model             | Avg Tokens/s | Latency(s) | Output Tokens | Error |
|-------------------|-------------|------------|----------------|-------|
| zai/glm-5-turbo   | 57.9        | 20.6       | 979            | 0.0%  |
```

## How It Works

**Auto Mode**: Sends a test prompt via `openclaw infer model run`, measures wall-clock time from start to last token, then estimates token count from output text.

**API Mode**: Calls `/v1/chat/completions` with streaming disabled, reads `usage.completion_tokens` for precise token counts.

## Notes

- Auto mode throughput includes gateway routing overhead (~1-3% lower than direct API)
- Auto mode token counts are estimates; API mode uses precise values
- English prompts yield more accurate token estimates in auto mode
- Anti-cache: random seed suffix appended per iteration

## Prerequisites

- Python 3 (built-in on macOS)
- `openclaw` CLI (for auto mode)

## File Structure

```
~/.openclaw/workspace/skills/model-throughput-tester/
├── SKILL.md           # Agent trigger & execution guide
├── README.md          # This file
├── README.zh.md       # Chinese version
├── throughput.py      # Main script
└── throughput-report.md  # Generated re

_meta.json

{
  "ownerId": "kn7dzk85gz4pz5c569zky2pssn875nwe",
  "slug": "model-throughput-tester",
  "version": "1.0.8",
  "publishedAt": 1783308456837
}

README.zh.md

# 模型吞吐率测试器

测试 LLM 模型的吞吐率（tokens/s）。支持两种模式：

- **Auto 模式**：通过 `openclaw infer model run` 测试当前模型，**无需 API Key**
- **API 模式**：直接调用 OpenAI 兼容 API，需要 URL 和 Key

## 触发规则

**适用场景：** 用户明确要求测试模型吞吐率时使用。

**推荐触发词：**
- 测一下吞吐率、测速、模型测速、tokens/s
- 跑个 benchmark、吞吐率测试、模型测试

**不适用：** 宽泛的性能讨论词（如「模型性能」「benchmark」单独出现）不应自动触发执行。

## 核心能力

### 1. Auto 模式（无 Key，推荐）

自动检测当前 session 的模型并测试吞吐率，无需任何配置。

```bash
python3 throughput.py --auto
```

指定模型测试：
```bash
python3 throughput.py --auto --model "zai/glm-5-turbo"
```

### 2. API 模式（直接调用 API）

```bash
python3 throughput.py \
  --url https://api.example.com/v1 \
  --key sk-xxx \
  --models gpt-4o-mini,gpt-4o
```

### 3. 通用参数

| 参数 | 默认值 | 说明 |
|------|--------|------|
| `--iterations` | `3` | 每个模型测试次数 |
| `--max-tokens` | `512` | 最大输出 token 数 |
| `--test-prompt` | 英文散文（夏天的田野） | 测试提示词 |
| `--timeout` | `60` | 单次请求超时（秒） |
| `--output` | `throughput-report.md` | 输出报告文件名 |
| `--csv` | false | 同时生成 CSV |

## 工作流程

### Auto 模式

```
1. 从 openclaw.json 读取当前 session 模型（provider/model）
2. 通过 openclaw infer model run 发送测试 prompt
3. 计时：命令开始 → 输出完成
4. 从返回文本估算 token 数（英文 0.75 word/token，中文 1.5 字/token）
5. 计算 tokens/s
6. 汇总输出报告
```

### API 模式

```
1. 构造 /v1/chat/completions 请求
2. 计时：请求开始 → 最后一个 token
3. 从响应中提取 usage.completion_tokens（精确）
4. 计算 tokens/s、错误率
5. 汇总输出报告
```

## 指标说明

| 指标 | 说明 |
|------|------|
| **Tokens/s** | 吞吐率 = Output Tokens / Elapsed Time |
| **Avg Latency** | 平均单次请求延迟 |
| **Avg Output Tokens** | 平均输出 token 数 |
| **Error Rate** | 错误请求占比 |

## 使用示例

### 安装后立即测试（Auto 模式）

```bash
# agent 触发时应传入当前模型
python3 ~/.openclaw/workspace/skills/model-throughput-tester/throughput.py --auto --model "<当前session模型>"

# 或使用自动检测（可能不是 session 覆盖的模型）
python3 ~/.openclaw/workspace/skills/model-throughput-tester/throughput.py --auto
```

### 测试多个模型（API 模式）

```bash
python3 throughput.py \
  --url "https://api.openai.com/v1" \
  --key "sk-xxx" \
  --models "gpt-4o-mini,gpt-4o" \
  --iterations 5
```

### 自定义提示词

```bash
python3 throughput.py --auto \
  --test-prompt "Explain quantum computing in detail." \
  --iterations 5
```

## 技术实现

- **Auto 模式**：`openclaw infer model run --json`，Python `subprocess` 调用
- **API 模式**：`urllib`（Python 内置），OpenAI 兼容 `/v1/chat/completions`
- **计时精度**：`time.perf_counter()` 纳秒级精度
- **Token 计数**：API 模式优先 `usage.completion_tokens`（精确），Auto 模式按字符估算
- **URL 拼接**：智能检测 `/v1`、`/v4`、`/chat/completions` 路径

## 注意事项

- Auto 模式的吞吐率包含网关路由开销，会比直接 API 略低（约 1-3%）
- Auto 模式 Token 数为估算值，API 模式为精确值
- 建议使用英文 prompt 以获得更准确的 token 估算
- 防缓存：每次迭代自动附加随机 seed 后缀

skill-card.md

## Description:

Benchmark LLM model throughput by measuring tokens per second, latency, and output speed through auto mode with OpenClaw or direct API mode for OpenAI-compatible endpoints.

This skill is ready for commercial/non-commercial use.

## Publisher:

[tsag1](https://clawhub.ai/user/tsag1)

### License/Terms of Use:

MIT-0

## Use Case:

Developers and engineers use this skill to benchmark and compare LLM response throughput, latency, token output, and error rate before selecting or switching models.

### Deployment Geography for Use:

Global

## Known Risks and Mitigations:

Risk: API mode can send API keys and benchmark prompts to any configured endpoint, including insecure HTTP endpoints.

Mitigation: Prefer auto mode when possible; in API mode, use trusted HTTPS endpoints and avoid placing real keys directly in shell history.

Risk: Generated reports can store the benchmark prompt and API URL locally.

Mitigation: Use non-sensitive benchmark prompts and review generated Markdown or CSV reports before sharing them.

Risk: Auto mode estimates token counts and cannot enforce the max token limit, which can make slow models appear to fail by timeout.

Mitigation: Use API mode for precise completion-token counts and max-token control, or raise the timeout and use a shorter prompt for slow models in auto mode.

## Reference(s):

- [ClawHub skill page](https://clawhub.ai/tsag1/skills/model-throughput-tester)
- [README](artifact/README.md)
- [Skill definition](artifact/SKILL.md)
- [Release evidence](evidence.json)

## Skill Output:

**Output Type(s):** [text, markdown, shell commands, configuration]

**Output Format:** [Terminal summary text plus a Markdown throughput report, with optional CSV output.]

**Output Parameters:** [1D]

**Other Properties Related to Output:** [Auto mode estimates token counts and excludes warmup from summary by default; API mode uses reported completion tokens when available.]

## Skill Version(s):

1.0.8 (source: server release evidence)

## Ethical Considerations:

Users should evaluate whether this skill is appropriate for their environment, review any generated or modified files before relying on them, and apply their organization's safety, security, and compliance requirements before deployment.

Github ReposUpdated 2d agoRank 70

AionUi

Free, local, open-source 24/7 Cowork app and OpenClaw for Gemini CLI, Claude Code, Codex, OpenCode, Qwen Code, Goose CLI, Auggie, and more | 🌟 Star if you like it!

MCPOPENCLAW

Github ReposUpdated 6mo agoRank 70

activepieces

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

OPENCLAW

Github ReposUpdated 6mo agoRank 70

cherry-studio

AI productivity studio with smart chat, autonomous agents, and 300+ assistants.

MCPOPENCLAW

Github ReposUpdated 7mo agoRank 70

CopilotKit

The Frontend for Agents & Generative UI. React + Angular