LangGraph Instantiation Performance Evaluation

Measure LangGraph create_react_agent instantiation over 1000 iterations with PerformanceEval, using ChatOpenAI gpt-4o and a weather tool.

Demonstrates agent instantiation benchmarking with LangGraph.

langgraph_instantiation.py

"""
LangGraph Instantiation Performance Evaluation
==============================================

Demonstrates agent instantiation benchmarking with LangGraph.
"""

from typing import Literal

from agno.eval.performance import PerformanceEval
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent


# ---------------------------------------------------------------------------
# Create Benchmark Tool
# ---------------------------------------------------------------------------
@tool
def get_weather(city: Literal["nyc", "sf"]):
    """Use this to get weather information."""
    if city == "nyc":
        return "It might be cloudy in nyc"
    elif city == "sf":
        return "It's always sunny in sf"
    else:
        raise AssertionError("Unknown city")


tools = [get_weather]


# ---------------------------------------------------------------------------
# Create Benchmark Function
# ---------------------------------------------------------------------------
def instantiate_agent():
    return create_react_agent(model=ChatOpenAI(model="gpt-4o"), tools=tools)


# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
langgraph_instantiation = PerformanceEval(func=instantiate_agent, num_iterations=1000)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    langgraph_instantiation.run(print_results=True, print_summary=True)

Run the Example

1. Set up your virtual environment

   Mac

   uv venv --python 3.12
   source .venv/bin/activate
   
2. Install dependencies

   uv pip install -U agno langchain-core langchain-openai langgraph memory-profiler
   
3. Export your API keys

   Mac/Linux

   export OPENAI_API_KEY="your_openai_api_key_here"
   
4. Run the example

   Save the code above as langgraph_instantiation.py, then run:

   python langgraph_instantiation.py
   

Full source: cookbook/09_evals/performance/comparison/langgraph_instantiation.py