ChatAnthropic in LangChain: import, base_url, streaming, tools and errors

ChatAnthropic from the langchain-anthropic package is LangChain's Claude chat model. It wraps the official Anthropic Python SDK, so pointing it at a gateway takes one argument, base_url, set to the host root. Everything else in your chains, agents and tools stays the same.

Install and import

pip install -U langchain-anthropic

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-5",
    base_url="https://aiprimetech.io",           # host root: the SDK adds /v1/messages
    api_key="your_gateway_key",   # or set ANTHROPIC_API_KEY
)

print(llm.invoke("Say hello in one word.").text)

Configure it from the environment

base_url is an alias of anthropic_api_url, and api_key an alias of anthropic_api_key. When you don't pass them, ChatAnthropic reads ANTHROPIC_API_URL first, then ANTHROPIC_BASE_URL, and the key from ANTHROPIC_API_KEY. Explicit arguments always win.

export ANTHROPIC_API_KEY=your_gateway_key
export ANTHROPIC_API_URL=https://aiprimetech.io

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(model="claude-sonnet-5")   # picks up the gateway from the environment

Streaming

for chunk in llm.stream("Explain backpressure in two sentences."):
    print(chunk.text, end="", flush=True)

.text collects only the text blocks of each chunk, so it stays clean when the model also emits thinking or tool-use blocks. Use streaming for long answers: a long non-streaming request is the usual cause of timeouts.

Tools and structured output

from langchain_core.tools import tool

@tool
def get_time(city: str) -> str:
    """Return the current local time in a city."""
    return "12:00"

msg = llm.bind_tools([get_time]).invoke("What time is it in Tokyo?")
print(msg.tool_calls)

from pydantic import BaseModel

class Weather(BaseModel):
    city: str
    temperature_c: float

print(llm.with_structured_output(Weather).invoke("It is 21 degrees in Lisbon today."))

Retries, timeouts and max_tokens

max_retries defaults to 2, the same as the Anthropic SDK, which retries connection errors, 429 and 5xx responses. timeout (alias of default_request_timeout) is in seconds. If you leave max_tokens unset, it comes from the model's profile.

llm = ChatAnthropic(
    model="claude-sonnet-5",
    base_url="https://aiprimetech.io",
    max_retries=4,
    timeout=120,
    max_tokens=8000,
)

For batches, cap parallelism so you stay inside your key's concurrency: llm.batch(inputs, config={"max_concurrency": 3}).

The ChatOpenAI alternative

If your code is standardised on ChatOpenAI, the gateway's OpenAI-compatible endpoint serves Claude too. This route wants /v1 on the end, the opposite of ChatAnthropic.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="claude-sonnet-5", base_url="https://aiprimetech.io/v1", api_key="your_gateway_key")

Prefer ChatAnthropic for Claude: it keeps native features such as prompt caching and thinking blocks.

Troubleshooting

Frequently asked questions

AI Prime Tech is an independent API gateway. It is not affiliated with, endorsed by, or a reseller of Anthropic. Claude and related model names are trademarks of their respective owners.