Best LLM tools in 2026 (for building, not running locally)

Updated September 2026

Best LLM tools in 2026 (for building, not running locally)

An "LLM tool" in this guide means software for building, connecting to, and monitoring large language models, not software for running a model on your own hardware. If you came here to run a model locally, our best local LLM tools guide covers Ollama, LM Studio, vLLM, and llama.cpp instead. This guide covers the other half of the stack: orchestration frameworks that wire an LLM into an application, hubs that give you access to models and datasets, observability platforms that show you what your app actually did in production, and the API layer that sits between your code and whichever model you're calling.

TL;DR: the 8 tools, by category

• Orchestration and agent frameworks: LangChain for chaining calls, tools, and memory into an application; LlamaIndex for connecting an LLM to your own documents and data.
• Model access and hubs: Hugging Face for hosting, downloading, and running open models and datasets; OpenRouter for a single API key across 400+ hosted models with no markup on inference.
• Observability and evaluation: LangSmith for tracing and evaluating LangChain (and non-LangChain) apps; Langfuse for the open-source, self-hostable version of the same job; Weights & Biases for training-run tracking that extends into LLM evaluation.
• Prompt and API tooling: llm, Simon Willison's command-line tool for hitting any model provider from a terminal without writing a script first.

Orchestration and agent frameworks

This is the layer that turns "call an LLM API" into "an application." If you want the deeper comparison across eight frameworks including LangGraph, CrewAI, and AutoGen, our best open-source AI agent frameworks roundup covers that ground in full. Here we cover the two most commonly reached-for pieces specifically in the LLM-tools context: LangChain and LlamaIndex.

LangChain

LangChain is the framework most people mean when they say "I'm building on top of an LLM." It gives you chains, tool-calling, memory, and a large library of integrations for connecting a model to vector stores, APIs, and other data sources, all under an MIT license. The project's own GitHub description now frames it as "the agent engineering platform," reflecting where most new development has moved: less linear prompt chains, more agents that call tools and make decisions across multiple steps. The repository sits at roughly 146,000 GitHub stars as of this writing (checked September 2026).

LangGraph, built by the same team, is the lower-level graph-based orchestration layer for anyone who needs explicit control over state and branching rather than LangChain's higher-level chain abstractions. If you're choosing between the two, or comparing either against CrewAI or AutoGen, that comparison lives in the frameworks roundup linked above rather than repeated here.

Pricing. LangChain the open-source framework is free. LangSmith, the paid observability layer built by the same company, is covered separately below.

LlamaIndex

LlamaIndex is the framework people reach for when the job is specifically retrieval-augmented generation: connecting an LLM to your own documents, databases, or knowledge base so it answers from your data instead of its training set. The open-source core (llama-index, MIT-licensed) handles indexing, retrieval, and query pipelines, with over 300 integration packages for different LLM, embedding, and vector store providers.

Separate from the open-source framework, LlamaIndex the company runs LlamaCloud and LlamaParse, a paid enterprise platform for document parsing, agentic OCR, and structured extraction. That's a genuinely different product from the free framework, worth knowing before you assume "LlamaIndex" always means the free option.

Pricing. The open-source framework is free to self-host. LlamaCloud/LlamaParse pricing is usage-based and quoted directly on their site; check current rates before committing since this is the part of LlamaIndex most likely to have usage fees attached.

Model access and hubs

Hugging Face

Hugging Face is the default place to find, download, and share open models, datasets, and demo apps (Spaces). The transformers library alone sits at roughly 165,000 GitHub stars at last check, and the platform is where nearly every open-weight model release, from small fine-tunes to frontier-scale open releases, ends up mirrored.

Beyond the free public hub, Hugging Face sells PRO accounts at $9/month for individuals (extra storage, private repos, and inference credits), Team plans at $20/month per user, and Enterprise at $50/month per user with SSO and centralized billing (confirm current rates on their pricing page before budgeting; these move). Compute is separate and metered: Spaces hardware and dedicated Inference Endpoints are billed by the hour, starting near $0.03/hour for basic CPU and scaling up through GPU tiers.

Best for. Teams that need to find, host, or fine-tune an open model, or that want a Spaces demo up without standing up their own infrastructure.

OpenRouter

OpenRouter is a single API and a single key that routes to whichever model you ask for, currently 400+ models across every major provider (431 at the time we checked their public model list in September 2026, and that number moves constantly as providers ship new releases). The pitch is switching providers without rewriting your integration: same request format, same billing dashboard, and per OpenRouter's own FAQ, no markup on the underlying inference price. You pay the same per-token rate you'd pay the provider directly.

The fee is elsewhere: OpenRouter charges when you purchase credits (a listed fee applies to crypto payments specifically), and if you bring your own provider API keys (BYOK) instead of buying OpenRouter credits, usage above a plan-dependent free allowance is billed as a percentage of what the same call would have cost through OpenRouter directly. Read their current pricing page before assuming a specific number, since both the model list and the BYOK terms are the parts most likely to change.

Best for. Teams that want to A/B test models from different providers, or that need a fallback path when one provider has an outage, without maintaining separate SDK integrations for each.

Observability and evaluation

Once an LLM app is live, the hard part stops being "does it work in the demo" and starts being "what did it actually do for this user, and was the answer right." That's what this category is for.

LangSmith

LangSmith is LangChain's commercial platform for tracing, debugging, and evaluating LLM applications, and it works whether or not the app is built with LangChain. It logs every step of a chain or agent run so you can see the exact prompts, tool calls, and outputs that led to a given response, then layer evaluation datasets and automated grading on top.

Pricing at last check: a Developer tier at $0 for one seat plus usage-based billing on LangChain Compute Units and LangChain Storage Units, a Plus tier at $39 per seat per month plus usage, and custom Enterprise pricing. The usage units (LCU at roughly $1.50 each, LSU at roughly $1.00 each) mean your actual bill depends on trace volume, not just seat count, so check the current calculator against your expected volume rather than budgeting off the seat price alone.

Best for. Teams already on LangChain or LangGraph who want the observability layer built by the same company, with the tightest integration.

Langfuse

Langfuse is the open-source alternative in this category: an LLM engineering platform for tracing, evaluation, prompt management, and a playground, with a permissive-enough license to self-host for free indefinitely, plus a hosted cloud version if you don't want to run it yourself. The GitHub repository is at roughly 34,000 stars and integrates with OpenTelemetry, LangChain, the OpenAI SDK, and LiteLLM out of the box.

Hosted pricing at last check: a free Hobby tier, a Core plan at $29/month including 100,000 units with additional usage at $8 per 100,000 units, a Pro plan at $199/month with unlimited history and higher rate limits, and Enterprise at $2,499/month for large-scale teams. Self-hosting removes the usage-based fees entirely at the cost of running the infrastructure yourself.

Best for. Teams that want observability and eval without a per-unit bill, or that have a self-hosting requirement LangSmith's hosted-only model doesn't fit.

Weights & Biases

Weights & Biases (W&B) started as an experiment-tracking tool for training runs and has extended into LLM evaluation and tracing through its Weave product, so it's a real option if your team already tracks fine-tuning or model training experiments in W&B and wants the LLM app layer in the same place rather than a second tool.

Pricing at last check: a Personal tier that's free for non-commercial use, and a Pro tier starting at $60/month billed monthly with a 30-day free trial, plus usage-based storage and data ingestion beyond included quotas. Enterprise is custom. The core wandb client library is MIT-licensed and sits at roughly 11,000 GitHub stars.

Best for. Teams that already run W&B for model training and fine-tuning and want LLM app evaluation in the same dashboard instead of adopting a second observability platform from scratch.

Prompt and API tooling

llm (Simon Willison)

llm is a command-line tool and Python library, built by Simon Willison, for running prompts against any LLM provider directly from a terminal, without writing a throwaway script first. It supports both hosted APIs (OpenAI, Anthropic, Gemini, and others through a plugin system) and local models, logs every prompt and response to a local SQLite database automatically, and handles things like piping file contents into a prompt or chaining a model's output into the next command. The project is Apache 2.0-licensed and sits at roughly 12,500 GitHub stars.

Best for. Developers who want to test a prompt, compare providers, or script a one-off LLM task without standing up an application around it.

How to pick, by what you're actually trying to do

If you're building an application that calls an LLM and needs memory, tool-calling, or multi-step logic, start with LangChain (or LangGraph if you need lower-level control), and add LlamaIndex specifically if the job involves retrieving answers from your own documents.

If you need access to models rather than a way to orchestrate them, Hugging Face is the default for open weights and datasets, and OpenRouter is the practical choice when you want one integration across several hosted providers instead of separate SDKs for each.

If your app is past the demo stage and you need to know what it's actually doing for real users, add an observability layer before you need it, not after an incident. LangSmith is the obvious choice if you're already on LangChain; Langfuse is the better fit if self-hosting or avoiding per-unit fees matters to you; Weights & Biases makes sense specifically if your team already lives there for training runs.

If you just need to test a prompt against a few providers before writing any application code, llm from the terminal is faster than standing up a script.

None of this is a substitute for actually running the thing in production against your real traffic and your real data. Picking the right framework and the right observability tool gets you the primitives. Wiring them into your actual systems, your data, and a permission model that doesn't hand an agent more access than the task needs, is the part that takes real engineering time. If you want that built and maintained by a team that's done it before, our AI agent development work covers exactly that gap, and our AI automation agency work covers the wider automation layer around it.

FAQ

What counts as an "LLM tool"?

In this guide, an LLM tool is software for building, connecting to, or monitoring applications built on large language models: orchestration frameworks (LangChain, LlamaIndex), model access layers (Hugging Face, OpenRouter), observability and evaluation platforms (LangSmith, Langfuse, Weights & Biases), and prompt/API tooling (llm). It does not include software for running a model on your own hardware; that's covered in our separate best local LLM tools guide.

What's the difference between LangChain and LlamaIndex?

LangChain is a general-purpose framework for building LLM applications: chains, tool-calling, memory, and agent logic. LlamaIndex is more specifically built for retrieval-augmented generation, connecting an LLM to your own documents or data so it can answer from that data. Many real projects use both: LangChain (or LangGraph) for the application logic, LlamaIndex for the retrieval layer underneath it.

Is LangChain free?

The open-source LangChain framework is free under the MIT license. LangSmith, LangChain's commercial observability and evaluation platform, has a free Developer tier for one seat plus usage-based billing, and a paid Plus tier at $39 per seat per month plus usage.

What's the difference between LangSmith and Langfuse?

Both trace, debug, and evaluate LLM application runs. LangSmith is LangChain's own commercial product, hosted-only, with the tightest integration if you're already building on LangChain or LangGraph. Langfuse is open-source and self-hostable for free, with a separate hosted cloud tier if you'd rather not run the infrastructure yourself. If self-hosting or avoiding per-unit usage fees matters to you, Langfuse is the more direct fit.

Does OpenRouter charge more than going directly to a model provider?

No, per OpenRouter's own documentation there's no markup on the underlying inference price; you pay the provider's listed rate. OpenRouter's fees are separate: a charge when you purchase credits (with a listed fee on crypto payments specifically), and for bring-your-own-key usage above a plan-dependent free allowance, a percentage-based fee on top of the provider's own cost.

Do I need Hugging Face if I'm only using hosted APIs like OpenAI or Anthropic?

Not necessarily. Hugging Face's core value is open models, datasets, and self-hosted or fine-tuned deployments. If your stack is entirely closed, hosted APIs, a unified API layer like OpenRouter or a direct SDK integration covers your needs without touching Hugging Face at all.

What's the fastest way to test a prompt against multiple providers without writing an app?

Simon Willison's llm command-line tool is built for exactly this: install it, add API keys through plugins for whichever providers you want, and run a prompt from the terminal. It logs every request and response locally, so you can compare outputs across providers without building anything around it first.