Rust NAPI core, 1-RTT Valkey Server Functions, cluster-native hash slots, workflows and DAGs. Plus built-in AI primitives - cost tracking, token streaming, human-in-the-loop, model failover, budget caps, and vector search.
Benchmarked on AWS ElastiCache Valkey 8.2 (r7g.large) with TLS:
| Concurrency | glide-mq | Leading Alternative | Delta |
|---|---|---|---|
| c=5 | 10,754 j/s | 9,866 j/s | +9% |
| c=10 | 18,218 j/s | 13,541 j/s | +35% |
| c=15 | 19,583 j/s | 14,162 j/s | +38% |
Every primitive AI orchestration needs - built into the queue, not a plugin or middleware.
| Primitive | API | What it does |
|---|---|---|
| Cost tracking | job.reportUsage() / queue.getFlowUsage() | Record model, tokens, cost, latency per job. Aggregate across entire flows. |
| Token streaming | job.stream() / queue.readStream() | Stream LLM output tokens to consumers in real time. SSE proxy endpoint included. |
| Human-in-the-loop | job.suspend() / queue.signal() | Pause for approval. Resume with a named signal and payload. Zero compute while suspended. |
| Budget caps | FlowProducer.add(flow, { budget }) | Cap total tokens or cost across all jobs in a flow. Pre-dispatch + post-completion enforcement. |
| Fallback chains | opts.fallbacks / job.currentFallback | Ordered model/provider alternatives tried automatically on failure. |
| Rate limiting | tokenLimiter + limiter | RPM (requests/min) + TPM (tokens/min) with per-queue and per-worker scopes. |
| Vector search | queue.createJobIndex() / queue.vectorSearch() | KNN similarity search over jobs via Valkey Search. Your jobs are your vector store. |
const worker = new Worker('ai', async (job) => {
const result = await callLLM(job.data.prompt);
await job.reportUsage({ model: 'gpt-5.4', tokens: { input: 50, output: 200 }, costs: { total: 0.003 }, costUnit: 'usd' });
await job.stream({ type: 'token', content: result });
return result;
}, {
connection,
tokenLimiter: { maxTokens: 100000, duration: 60000 },
});Read the full AI-Native guide | 18 runnable examples | Vercel AI SDK integration | LangChain integration
npm install glide-mqRequires Node.js 20+ (also works with Bun and Deno) and Valkey 7.0+ or Redis 7.0+. For vector search, use valkey-bundle.
| Package | Description |
|---|---|
| glide-mq | Core queue library |
| @glidemq/hono | Hono middleware |
| @glidemq/fastify | Fastify plugin |
| @glidemq/nestjs | NestJS module |
| @glidemq/hapi | Hapi plugin |
| @glidemq/dashboard | Express web dashboard |