support for context caching

root_agent = LlmAgent(
    name="question_answer_agent",
    model="gemini-2.0-flash-001",
    description=(
        "QA Agent Answers Questions about cached content. Its system instructions have already been cached."
    ),
    generate_content_config=types.GenerateContentConfig(
        cached_content=cache.name
    )
)

google.genai.errors.ClientError: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'Tool config, tools and system instruction should not be set in therequest when using cached content.', 'status': 'INVALID_ARGUMENT

:(

@jheym

@jheym could you share a full example to reproduce the issue?

@hangfei

@hangfei This is a feature request. I'm hoping to be able to use context caching out of the box, in a similar manner as the sample code above.

The method shown in the OP isn't supposed to work (I assume), it's just something I wanted to try out. If this is possible another way, please do let me know!

content caching can not be used in agents currently due to how we format the request. This is in our backlog to tackle.

Any ETA for this feature?

I have been looking for this too. Hope it will be available soon.

Also would love this feature for very long system instructions. Glad to see it on the feature roadmap. Is there a plan to add any expected dates to the feature tickets?

I have been looking for this feature as well.

+1

+1

@jheym

thank you.
I feel like static instruction can't be changed dynamically after deployed (ex. agent engine).

I am looking for feature like @jheym where you generate cache content and inject to the agent
.

@thanab-gscalejp

@thanab-gscalejp , the way @jheym is doing probably need to manage the cache using gen client outside of the llm_agent. you can't dynamically change that once the llm_agent is created. but there is a workaround, you can wrap cache resolving and then create the llm_agent inside a customised parent agent by overriding the _run_async_impl.

@thanab-gscalejp

Okay @thanab-gscalejp, I found another way to do this, use before_model_callback to manipulate the llm_request.config.system_instruction or llm_request.contents.

@ejfn

@ejfn I also want to cache certain system instruction in agent across different sessions. Have you found the way to handle this case ?

@chuoibo

@chuoibo , get llm_response.cache_metadata from after_model_callback, save it in a global state or somewhere, and then inject it into llm_request.cache_metadata via before_model_callback and then manipulate the llm_request.config.system_instruction to ensure it's same as the cached content. in this way, you can avoid accessing cache_metadata from the session state.

also, you will need at least one conversation for caching (a pair of user + model content), which is same as using gen client to generate cache content.

@ejfn

@ejfn sounds interesting. thank you.

sorry i cannot understand some concepts.

May I ask why I need to manipulate the system instruction after injecting the cache_metadata back to the agent?

If it is necessary to manipulate the system instruction, could you please elaborate about the term manipulation? does it mean concatenate the original system instruction's value with cache_metadata?

@thanab-gscalejp

@thanab-gscalejp , the cache manager will compare the current cache's signature in the metadata with the content prefix in the new request to decide whether reuse the cache or recreate the cache.

If you have any coding agent in hand, ask it to read this file to help you understand.
src/google/adk/models/gemini_context_cache_manager.py

@ejfn

Confirmed context caching is working in 1.51.0, there is no doc yet, but by reading the commit I figured this works.
1. Add context caching config to the app (the one used by runner)

2. Use llm_agent's static_instruction for the content needs to be cached.
The cache will be created at the first call, and will be used from following calls within the conversation.

If you need a cache across different sessions, I haven't tried but I believe it should be able to be done by manually replace the system_instruction with the cache name in the gen config, but you will need to manage the cache meta outside of the agent via gen client. A bit hacky but possible.

Hey @ejfn ,

Do you have a working code? We are trying to implement based on the samples but struggling.

@varun-gupta13

@varun-gupta13 try this

app_name = os.path.basename(os.path.dirname(__file__))  
agent = YourAgent()

app = App(
    name=app_name,
    root_agent=agent,
    context_cache_config=ContextCacheConfig(
        cache_intervals=10, # refresh cache every 10 invocations
        ttl_seconds=1800, # 30 min TTL
        min_tokens=1024,
    )
)

You need to expose the app instead of root_agent for adk web to load with your cache config. also the app_name has to be the folder name of the agent, otherwise it won't load. then you start a conversation in adk web, and watch the cache_name and the content in each request. tweak the cache_intervals to 3 or 5 see the difference.

the current adk web cli in 1.51.1 has a number of issues won't load your cache config, that has been fixed in the main branch. you need to replace the adk-web-server.py with the newer version from the main branch.

@ejfn

@varun-gupta13 try this
app_name = os.path.basename(os.path.dirname(__file__))  
agent = YourAgent()

app = App(
    name=app_name,
    root_agent=YourAgent(),
    context_cache_config=ContextCacheConfig(
        cache_intervals=10, # refresh cache every 10 invocations
        ttl_seconds=1800, # 30 min TTL
        min_tokens=1024,
    )
)
You need to expose the app instead of root_agent for adk web to load. also the app_name has to be the folder name of the agent, otherwise it won't load. then you start a conversation in adk web, and watch the cache_name and the content in each request. tweak the cache_intervals to 3 or 5 see the difference.

the current adk web cli in 1.51.1 has a number of issues won't load your cache config, that has been fixed in the main branch. you need to replace the adk-web-server.py with the newer version from the main branch.

@ejfn ,

Thank you so much for your reply. I will try and confirm.

It's supported now. Closing.

@jheym

i think it still doesn't support what @jheym would like to do which is to explicitly create a particular cache content and inject...

We cannot support using arbitrary cached content because ADK has specific requirement on what have to be cached.

Navigation Menu

Repository navigation

support for context caching #211

25 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

support for context caching #211

Description

Activity

hangfei commented on Apr 17, 2025

jheym commented on Apr 17, 2025

boyangsvl commented on Apr 19, 2025

gisdakis commented on Apr 30, 2025

heyphat commented on Jun 9, 2025

arneyjfs commented on Jun 20, 2025

ypenn21 commented on Jun 28, 2025

jhammarstedt commented on Jun 30, 2025

weisiang9678 commented on Jul 9, 2025

25 remaining items

thanab-gscalejp commented on Sep 30, 2025

eric-alcidion commented on Sep 30, 2025

eric-alcidion commented on Sep 30, 2025

chuoibo commented on Oct 1, 2025

ejfn commented on Oct 1, 2025

thanab-gscalejp commented on Oct 1, 2025

ejfn commented on Oct 1, 2025

varun-gupta13 commented on Oct 9, 2025

ejfn commented on Oct 9, 2025

varun-gupta13 commented on Oct 9, 2025

hangfei commented on Oct 24, 2025

thanab-gscalejp commented on Oct 24, 2025

boyangsvl commented on Oct 29, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions