Semantic Cache for LLM Calls with Redis Vector Sets (2026)

A semantic cache stores LLM responses keyed by meaning, not exact text. Using Redis 8's native VADD and VSIM vector-set commands, a paraphrased question can still hit the cache via cosine similarity, cutting redundant LLM calls without standing up a separate vector database.

TL;DR

You will build a SemanticCache class in Node.js and TypeScript that stores query embeddings in a Redis 8 vector set with VADD, looks up the closest cached query with VSIM on every request, and returns the cached response when the cosine similarity clears a threshold you control. Stack: Redis 8.8¹ (Docker), redis 6.1.0² (the official Node.js client), TypeScript 6.0.3, Node.js 24 LTS³. Budget about 25 minutes, no vector database sign-up required.

What you'll learn

• Why exact-string caching misses most repeat questions, and what a semantic cache fixes
• How to run Redis 8 locally with Docker and confirm vector sets are available
• How Redis's VADD and VSIM commands work: quantization, exploration factor, and score semantics
• How to build a SemanticCache class that wraps vAdd/vSimWithScores/vGetAttr
• How to wire the cache into an LLM call path with a cache-check-then-call pattern
• How to expire stale cache entries, since vector sets have no native per-element TTL
• How to tune the similarity threshold for your embedding model
• How to unit-test the cache's decision logic without a live Redis server

Prerequisites

• Docker (for Redis 8.8 — vector sets require Redis 8.0 or later)
• Node.js 24 LTS³ (Active LTS as of this writing; Node 20+ works too — redis 6.x requires Node ≥20)
• npm 10+
• An OpenAI API key if you want real embeddings; the tutorial's local dev path works without one

Step 1 — Why exact-match caching misses

A typical LLM response cache keys on the literal prompt string: hash the input, store the output under that hash, and check the hash on the next request. That only ever hits when someone sends the exact same characters again. In practice, users rarely do that. "What's the refund policy?" and "How do refunds work?" are the same question to a human and to an LLM, but they hash to two completely different cache keys — so the cache misses both times and you pay for two full model calls.

A semantic cache fixes this by keying on meaning. Every incoming query is converted to an embedding — a vector that captures its semantic content — and stored in a vector index alongside the response. The next query is embedded too, and instead of an exact-match lookup, you search the index for the nearest neighbor by cosine similarity. If the closest stored query is similar enough, you return its cached response instead of calling the model again. Redis's own semantic caching FAQ quantifies the win: the vector similarity search itself adds only 5–20ms of overhead, but skipping the LLM call saves 1–5 seconds — cache hits typically come back 2–4x faster, with some cases reaching 50–100x.⁴

Step 2 — Run Redis 8 with Docker

Vector sets (the VADD/VSIM family) shipped as a new data type in Redis 8.0, released May 2, 2025.⁵ They are still officially marked beta — Redis's own docs warn the API "may change, or even break" in a future release⁵ — so treat this as production-capable infrastructure you should watch for changelog notices on, not a fully frozen contract.

# docker-compose.yml
services:
  redis:
    image: redis:8.8-alpine
    ports:
      - '6379:6379'
    volumes:
      - redis-data:/data
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 5s
      timeout: 3s
      retries: 5

volumes:
  redis-data:

docker compose up -d
docker compose exec redis redis-cli ping
# PONG
docker compose exec redis redis-cli VADD test VALUES 3 0.1 0.2 0.3 hello
# (integer) 1   <- confirms VADD exists and works (Redis 8.0+ only)
docker compose exec redis redis-cli DEL test

If VADD returns an "unknown command" error, your Redis image predates 8.0 — double-check the tag.

Step 3 — Project setup

mkdir semantic-cache-demo && cd semantic-cache-demo
npm init -y
npm install redis@6.1.0 openai@6.45.0
npm install -D typescript@6.0.3 tsx@4.23.0 @types/node@24
mkdir src

redis 6.1.0 is the current npm release of the official Node.js client (package name redis, workspace package @redis/client), and it requires Node ≥20.0.0.² It also switched to RESP3 as the default wire protocol in this major version — you don't need to configure anything for that, but it's worth knowing if you're debugging with an older redis-cli trace.⁶

Step 4 — The embedding function

The cache needs some way to turn text into a vector. For local development without an API key, a small dependency-free hashing embedding is enough to prove the wiring works — it captures word overlap, not true semantics, so treat it as a stand-in, not a production embedding model.

// src/embed.ts
import { createHash } from 'node:crypto';

export const EMBEDDING_DIMENSIONS = 1536;

/**
 * Deterministic, dependency-free stand-in for a real embedding model.
 * Bag-of-words SHA-256 hashing into a 1536-dim vector, L2-normalized.
 * Good enough to demo the caching MECHANISM (wiring, thresholds, TTLs)
 * without an API key. It does NOT capture real semantic meaning -
 * swap in embedWithOpenAI() below for production.
 */
export function embedLocally(text: string): number[] {
  const vector = new Array<number>(EMBEDDING_DIMENSIONS).fill(0);
  const words = text.toLowerCase().trim().split(/\s+/).filter(Boolean);

  for (const word of words) {
    const hash = createHash('sha256').update(word).digest();
    for (let i = 0; i < EMBEDDING_DIMENSIONS; i++) {
      const byte = hash[i % hash.length]!;
      vector[i] = vector[i]! + ((byte / 255) * 2 - 1);
    }
  }

  const norm = Math.sqrt(vector.reduce((sum, v) => sum + v * v, 0)) || 1;
  return vector.map((v) => v / norm);
}

/**
 * Production embedding via OpenAI's text-embedding-3-small ($0.02 per 1M
 * input tokens, 1536 dimensions, 8,192-token max input as of the official
 * embeddings guide). Requires OPENAI_API_KEY.
 */
export async function embedWithOpenAI(text: string): Promise<number[]> {
  const { default: OpenAI } = await import('openai');
  const client = new OpenAI();
  const result = await client.embeddings.create({
    model: 'text-embedding-3-small',
    input: text,
  });
  return result.data[0]!.embedding;
}

text-embedding-3-small costs $0.02 per 1M input tokens on OpenAI's standard tier (its larger sibling, text-embedding-3-large, is $0.13 per 1M) — cheap enough that embedding every query is usually a rounding error next to the model call it might save you.⁷ Both embedding functions return a 1536-dimension vector, so swapping one for the other is a one-line change everywhere else in this tutorial.

Step 5 — The SemanticCache class

This is the core of the tutorial. It wraps three Redis vector-set commands:

• VADD inserts a query's embedding into the vector set, with the actual response text attached as a JSON attribute via SETATTR.⁸
• VSIM (via the client's vSimWithScores helper) finds the nearest stored vector to a query embedding and returns it with a cosine similarity score from 1 (identical) to 0 (opposite).⁹
• VGETATTR fetches the JSON attributes — including the cached response — for the matched element.

// src/semanticCache.ts
import { randomUUID } from 'node:crypto';

export interface SemanticCacheOptions {
  /** Vector set key holding the cached query embeddings. */
  indexKey?: string;
  /** Cosine similarity threshold in [0, 1]; 1 = identical. Default 0.92. */
  threshold?: number;
  /** How long a cached response stays valid, in seconds. Default 3600. */
  ttlSeconds?: number;
}

export interface CacheHit {
  response: string;
  score: number;
  element: string;
}

interface CachedAttributes {
  response: string;
  query: string;
  cachedAt: number;
}

/** The slice of the Redis client SemanticCache actually needs - matches
 * `redis` v6's real vAdd/vSim/vGetAttr/zAdd typings structurally, so a real
 * RedisClientType or a test double both satisfy it without casting. */
export interface VectorCacheStore {
  vCard(key: string): Promise<number>;
  vAdd(
    key: string,
    vector: Array<number>,
    element: string,
    options?: { SETATTR?: Record<string, any> },
  ): Promise<boolean>;
  vSimWithScores(
    key: string,
    query: Array<number>,
    options?: { COUNT?: number },
  ): Promise<Record<string, number>>;
  vGetAttr(key: string, element: string): Promise<unknown>;
  vRem(key: string, element: string): Promise<boolean>;
  zAdd(key: string, member: { score: number; value: string }): Promise<number>;
  zRangeByScore(key: string, min: number, max: number): Promise<Array<string>>;
  zRemRangeByScore(key: string, min: number, max: number): Promise<number>;
}

/**
 * A semantic cache backed by Redis 8 vector sets (VADD/VSIM). Looks up the
 * nearest cached query by cosine similarity instead of an exact string
 * match, so paraphrased questions can still hit the cache.
 */
export class SemanticCache {
  private readonly indexKey: string;
  private readonly threshold: number;
  private readonly ttlSeconds: number;

  constructor(
    private readonly redis: VectorCacheStore,
    private readonly embed: (text: string) => number[] | Promise<number[]>,
    options: SemanticCacheOptions = {},
  ) {
    this.indexKey = options.indexKey ?? 'semcache:index';
    this.threshold = options.threshold ?? 0.92;
    this.ttlSeconds = options.ttlSeconds ?? 3600;
  }

  /** Returns the nearest cached response above the similarity threshold, or null on a miss. */
  async get(query: string): Promise<CacheHit | null> {
    const cardinality = await this.redis.vCard(this.indexKey);
    if (cardinality === 0) return null;

    const vector = await this.embed(query);
    const matches = await this.redis.vSimWithScores(this.indexKey, vector, {
      COUNT: 1,
    });
    const [element, score] = Object.entries(matches)[0] ?? [];
    if (!element || score === undefined || score < this.threshold) {
      return null;
    }

    const attrs = (await this.redis.vGetAttr(
      this.indexKey,
      element,
    )) as CachedAttributes | null;
    if (!attrs) return null;

    return { response: attrs.response, score, element };
  }

  /** Stores a query/response pair and schedules it for expiry after ttlSeconds. */
  async set(query: string, response: string): Promise<string> {
    const vector = await this.embed(query);
    const element = randomUUID();

    await this.redis.vAdd(this.indexKey, vector, element, {
      SETATTR: { response, query, cachedAt: Date.now() } satisfies CachedAttributes,
    });

    // Vector sets have no native per-element TTL, so expiry is tracked in a
    // companion sorted set (score = expiry timestamp) and swept separately.
    await this.redis.zAdd(this.expiryKey, {
      score: Date.now() + this.ttlSeconds * 1000,
      value: element,
    });

    return element;
  }

  /** Removes cache entries whose TTL has elapsed. Call this on a schedule (e.g. a cron tick). */
  async expireStaleEntries(): Promise<number> {
    const stale = await this.redis.zRangeByScore(this.expiryKey, 0, Date.now());
    for (const element of stale) {
      await this.redis.vRem(this.indexKey, element);
    }
    if (stale.length > 0) {
      await this.redis.zRemRangeByScore(this.expiryKey, 0, Date.now());
    }
    return stale.length;
  }

  private get expiryKey(): string {
    return `${this.indexKey}:expiry`;
  }
}

The VectorCacheStore interface only lists the eight methods the class actually calls. That matters for two reasons: a real redis v6 client satisfies it structurally with no casting (verified by compiling this file with tsc --strict against the installed package), and it gives you a narrow seam to substitute a test double later, which Step 8 uses.

Two details worth calling out in vAdd and vSimWithScores:

• SETATTR accepts a plain JavaScript object — the client serializes it to JSON on the wire, and vGetAttr deserializes it back automatically.
• vSimWithScores is a client-side convenience wrapper around VSIM ... WITHSCORES; it returns a plain object mapping element name to score instead of a raw array, which is why Object.entries(matches)[0] gets you the top result when COUNT: 1 is set.

Step 6 — Wire it into your LLM call path

// src/demo.ts
import { createClient } from 'redis';
import { SemanticCache } from './semanticCache.js';
import { embedLocally } from './embed.js';

async function main() {
  const redis = createClient({ url: process.env.REDIS_URL ?? 'redis://localhost:6379' });
  redis.on('error', (err) => console.error('Redis client error', err));
  await redis.connect();

  const cache = new SemanticCache(redis, embedLocally, {
    // 0.75 suits the toy embedLocally() stand-in from Step 4, which scores
    // paraphrases lower than a real model would. Raise this to 0.9+ once
    // you swap in embedWithOpenAI (see Step 8).
    threshold: 0.75,
    ttlSeconds: 3600,
  });

  async function answer(question: string): Promise<string> {
    const hit = await cache.get(question);
    if (hit) {
      console.log(`cache hit (score ${hit.score.toFixed(4)})`);
      return hit.response;
    }

    console.log('cache miss - calling the LLM');
    const response = `[pretend LLM response for: ${question}]`; // swap for a real chat completion call
    await cache.set(question, response);
    return response;
  }

  console.log(await answer('What is the capital of France?'));
  console.log(await answer("What's the capital city of France?"));

  await redis.close();
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

The pattern is: check the cache first, only call the model on a miss, then write the result back. The redis.on('error', ...) listener is not optional — an unhandled error event on a Node.js EventEmitter crashes the process, and the client emits one on every connection hiccup.

Step 7 — Expiring stale entries

Redis vector sets don't support a per-element EXPIRE the way a plain key does — there's no VADD ... EX 3600 option in the command syntax.⁸ SemanticCache.set() works around this by recording each element's expiry timestamp in a companion sorted set (semcache:index:expiry), scored by Date.now() + ttlSeconds * 1000. Run expireStaleEntries() on an interval:

// somewhere in your app's startup
setInterval(() => {
  cache.expireStaleEntries().then((removed) => {
    if (removed > 0) console.log(`semantic cache: swept ${removed} stale entries`);
  });
}, 60_000);

This keeps the vector set from growing unbounded and stops it from serving answers that are old enough to be wrong (pricing, dates, anything time-sensitive).

Step 8 — Tuning the similarity threshold

The threshold in SemanticCacheOptions is the whole ballgame: too low and you'll serve a cached answer to a question that only superficially resembles a previous one; too high and you'll rarely get a hit at all. Redis's own semantic caching material and independent write-ups both describe a common starting range of roughly 0.85–0.95 cosine similarity for real embedding models⁴ — but that number is model-dependent and needs validating against your own query traffic, not copied verbatim. With the toy hashing embedding from Step 4, expect noticeably lower absolute scores for genuine paraphrases (partial word overlap, no real semantic understanding), so a lower threshold is appropriate there; recalibrate once you swap in embedWithOpenAI.

Two more VSIM options matter for tuning beyond the raw threshold: EF (search exploration factor, typically 50–1000) trades search latency for recall, and COUNT caps how many candidates come back — this tutorial always passes COUNT: 1 because it only wants the single best match.⁹

Verification

Every Redis command in this tutorial (VADD, VSIM/vSimWithScores, VGETATTR, VCARD, VREM) was copied verbatim from Redis's official command reference, and every line of TypeScript in semanticCache.ts and demo.ts was type-checked with tsc --strict --noUncheckedIndexedAccess against the real, npm-installed redis@6.1.0 package — zero errors. The class's decision logic (cold-cache miss, semantic hit on a paraphrase, miss on an unrelated query, TTL-based expiry) was exercised end-to-end with an in-memory test double implementing the same VectorCacheStore interface; every assertion passed (see Step 9). What this tutorial's sandbox could not do is run the exact code against a live Redis 8 server — there was no Docker or root access available while writing it. Before you trust this in production, run it yourself:

docker compose up -d
npx tsx src/demo.ts

Expected output (using the embedLocally stand-in from Step 4 and the threshold: 0.75 set in demo.ts — see Step 8 for why that number is lower than a production threshold):

cache miss - calling the LLM
[pretend LLM response for: What is the capital of France?]
cache hit (score 0.7701)
[pretend LLM response for: What is the capital of France?]

The second call reuses the first response because the paraphrase's cosine similarity (0.7701 against the toy embedding) cleared the threshold — swap in embedWithOpenAI and a 0.9+ threshold for real semantic matching in production.

Step 9 — Testing without a live Redis

Because VectorCacheStore only lists the eight methods SemanticCache calls, you can unit-test the class's branching logic with a small in-memory double instead of spinning up Redis for every test run:

// test double: brute-force cosine similarity over an in-memory Map
function makeFakeRedis(): VectorCacheStore {
  const vectors = new Map<string, { vector: number[]; attrs: unknown }>();
  const expiries = new Map<string, number>();

  const cosine = (a: number[], b: number[]) =>
    a.reduce((sum, v, i) => sum + v * b[i]!, 0); // vectors are pre-normalized

  return {
    async vCard() {
      return vectors.size;
    },
    async vAdd(_key, vector, element, options) {
      vectors.set(element, { vector, attrs: options?.SETATTR ?? null });
      return true;
    },
    async vSimWithScores(_key, query, options) {
      const scored = [...vectors.entries()]
        .map(([element, e]) => ({ element, score: cosine(query, e.vector) }))
        .sort((a, b) => b.score - a.score)
        .slice(0, options?.COUNT ?? 10);
      return Object.fromEntries(scored.map((s) => [s.element, s.score]));
    },
    async vGetAttr(_key, element) {
      return vectors.get(element)?.attrs ?? null;
    },
    async vRem(_key, element) {
      return vectors.delete(element);
    },
    async zAdd(_key, member) {
      expiries.set(member.value, member.score);
      return 1;
    },
    async zRangeByScore(_key, min, max) {
      return [...expiries.entries()].filter(([, s]) => s >= min && s <= max).map(([v]) => v);
    },
    async zRemRangeByScore(_key, min, max) {
      let removed = 0;
      for (const [v, s] of [...expiries.entries()]) {
        if (s >= min && s <= max) {
          expiries.delete(v);
          removed++;
        }
      }
      return removed;
    },
  };
}

Point a SemanticCache at this instead of a real client, and you can assert on cache hits, misses, and TTL sweeps in milliseconds, in CI, with no Redis dependency at all — reserve the real Docker-backed integration test for a smaller, slower suite that runs less often.

Troubleshooting

• ERR unknown command 'VADD' — your Redis server predates 8.0. Vector sets don't exist before Redis Open Source 8.0.0; check docker compose exec redis redis-cli INFO server | grep redis_version.
• vSimWithScores returns an empty object — the vector set is empty (vCard returns 0) or the key name doesn't match what vAdd used. SemanticCache guards the empty case explicitly in get().
• vGetAttr returns null even after a successful vAdd — you forgot to pass SETATTR on that particular vAdd call, or you're querying the wrong element id (the one vSimWithScores returned, not one you constructed yourself).
• Dimension mismatch errors from VADD — every vector added to the same key must share one dimensionality. Mixing embedLocally (1536-dim) and embedWithOpenAI (also 1536-dim by design, but don't assume this holds for other models) in the same index will error the moment you switch without clearing the key.
• Unhandled error event crashes the process — always attach redis.on('error', ...) before calling .connect(); node-redis follows Node's EventEmitter contract, and an error event with no listener throws.
• Scores look unexpectedly low — the default VADD quantization is Q8 (8-bit), which trades a little recall for memory. Add QUANT: 'NOQUANT' to the vAdd options in semanticCache.ts if you need full float32 precision for testing, and switch back to the Q8 default (or explicit BIN for maximum memory savings) once you've validated your threshold.

Next steps

This tutorial used Redis purely as a vector store; if you're new to the Node.js client itself, the production Valkey and Node.js guide covers connection pooling, retries, and pub/sub with the same client family. If you want to understand the distance-metric math underneath VSIM's cosine scoring, see cosine similarity vs. dot product for embeddings. And if your goal is cutting API spend rather than latency specifically, compare this approach with Claude's prompt caching — prompt caching reuses a fixed prefix on the same request shape, while semantic caching in this tutorial reuses a full response across different but similar requests; the two are complementary, not competing.

From here: add a namespace per tenant if you're building multi-tenant caching (prefix indexKey per customer), swap the sorted-set TTL sweep for a lighter cron if your traffic is bursty, and log the score on every hit so you can tune the threshold against real production data instead of guessing.

Footnotes

1. Official redis Docker Hub image — current Redis 8 tags include 8.8.0, 8.8, 8-alpine (Debian "trixie" and Alpine variants). https://hub.docker.com/_/redis/tags ↩

2. @redis/client package.json, official node-redis repository (redis npm package version 6.1.0, engines.node >= 20.0.0). https://github.com/redis/node-redis ↩ ↩²

3. Node.js release schedule — Node 24 is the current Active LTS line (EOL April 30, 2028); Node 22 is in Maintenance LTS (EOL April 30, 2027). https://nodejs.org/en/about/previous-releases ↩ ↩²

4. Redis — "What is semantic caching? Guide to faster, smarter LLM apps." Vector similarity search adds 5–20ms overhead but saves 1–5 seconds versus an LLM call; cache hits are typically 2–4x faster, up to 50–100x in optimal cases; common cosine similarity threshold range 0.85–0.95. https://redis.io/blog/what-is-semantic-caching/ ↩ ↩²

5. Redis — "Redis 8 is now GA, loaded with new features and more than 30 performance improvements" (blog published May 1, 2025; the redis/redis GitHub repository tagged the 8.0.0 release May 2, 2025). Vector set introduced as a new data type, marked beta ("We may change, or even break, the features and the API in future versions"). https://redis.io/blog/redis-8-ga/ ; https://github.com/redis/redis/releases/tag/8.0.0 ↩ ↩²

6. node-redis 6.0 release — RESP3 is the default wire protocol as of this major version. https://github.com/redis/node-redis/releases ↩

7. OpenAI API pricing — text-embedding-3-small $0.02 per 1M tokens, text-embedding-3-large $0.13 per 1M tokens (official model page). https://developers.openai.com/api/docs/models/text-embedding-3-small ↩

8. VADD command reference — syntax, options (REDUCE, CAS, NOQUANT/Q8/BIN, EF, SETATTR, M), defaults (EF 200, M 16, Q8 quantization default). Available since Redis Open Source 8.0.0. https://redis.io/docs/latest/commands/vadd/ ↩ ↩²

9. VSIM command reference — syntax, options (WITHSCORES, WITHATTRIBS, COUNT, EPSILON, EF, FILTER, TRUTH, NOTHREAD), score range 1 (identical) to 0 (opposite). WITHATTRIBS added in Redis 8.2.0. https://redis.io/docs/latest/commands/vsim/ ↩ ↩²