Llm Cache
LLM request/response caching with tiered TTL, time travel, and request normalization for Bijection.
git clone https://github.com/bijectionhq/llm-cache.git
cd llm-cache
git checkout b764f9330cc8d400c55752bad509505ebe56e24aBijection repository access is required. Request access.
Description
LLM request/response caching with tiered TTL, time travel, and request normalization for Bijection.
Explore the package README for API concepts, configuration, and examples. Check the Bijection source for adaptation-specific setup.
Review the source and its dependencies before adopting this package. Source availability does not establish npm publication or runtime qualification.
Getting started
- Get the source at the revision above and read its package documentation.
- Check the required Bijection SDK version, dependencies, and external service credentials.
- Follow the package setup instructions and the guide to using components.
- Exercise the component in your deployment according to your project’s verification rules.
Open source
Maintained source: bijectionhq/llm-cache. Based on raymond-UI/llm-cache. License: Apache-2.0. Consult the repository for license terms, notices, and contributor attribution.
From the upstream publisher
LLM request/response caching with tiered TTL, time travel, and request normalization for Convex.
Explore the demo ↗Upstream package FAQ
How does request normalization work in LLM caching?
The @mzedstudio/llm-cache component normalizes LLM requests by standardizing parameters like whitespace, capitalization, and prompt formatting before checking the cache. This ensures that functionally identical requests hit the cache even with minor formatting differences, maximizing cache efficiency.
What is tiered TTL in LLM caching?
Tiered TTL allows you to set different expiration times for different types of cached LLM responses. For example, you might cache static prompt responses for 24 hours but dynamic user-specific responses for only 1 hour, optimizing both performance and data freshness.
Can I use this component with any LLM provider?
Yes, @mzedstudio/llm-cache is provider-agnostic and works with OpenAI, Anthropic, Google, or any other LLM API. It caches the request/response data structure regardless of the underlying LLM service, making it flexible for multi-provider setups.
How does time travel debugging work for LLM requests?
The time travel feature stores all cached LLM interactions with timestamps in your Convex database. You can query historical request/response pairs, replay specific interactions, and analyze how your LLM responses have changed over time, making it easier to debug and optimize prompts.
Use with agents and CLI
Give your coding agent the component’s source identity and setup guidance.
Help me integrate Llm Cache with Bijection.
Read: https://bijection.com/components/llm-cache/SKILL.md
Check SDK compatibility and dependency availability.
Follow this project's execution and verification rules.Upstream README.md
View repository ↗Documentation from raymond-UI/llm-cache at its recorded upstream revision. APIs and setup may differ in the Bijection adaptation.
Loading package documentation…