How Things Work

LangChain / Semantic Kernel

Frameworks for composing LLM apps — chains and LCEL, prompt templates, memory, plugins/tools, and the move toward stateful agent graphs (LangGraph).

How It Works

Orchestration frameworks like LangChain and Semantic Kernel provide reusable building blocks — prompt templates, models, output parsers, memory, retrievers, and tools — so you compose LLM applications instead of hand-wiring API calls. LangChain pipes components with LCEL into uniform Runnables; Semantic Kernel registers plugins on a Kernel with automatic function calling. Both increasingly favor stateful agent graphs (LangGraph) for complex, multi-step workflows.

Why Orchestration Frameworks Exist

Building an LLM app from raw API calls means hand-wiring prompts, history, retrieval, tool calls, parsing, retries, and streaming yourself. Orchestration frameworks (LangChain, LlamaIndex, Semantic Kernel) provide reusable, composable building blocks so you assemble apps from parts instead of plumbing — with consistent patterns for models, memory, and tools.

Composable Components

The core pieces are prompt templates, models, output parsers, memory, retrievers, and tools. LangChain composes them with LCEL — literally piping components together (prompt | model | parser) into a single Runnable that supports invoke, stream, batch, and async uniformly. Swap any component without rewriting the flow.

Memory & Tools

Memory persists conversation state across turns so the app 'remembers' context. Tools/functions let the model take actions; Semantic Kernel registers them as plugins ([KernelFunction]) and uses automatic function calling (FunctionChoiceBehavior.Auto) to invoke them, while LangChain wires tool-calling agents. This is how chains graduate into agents.

From Chains to Graphs

Simple linear chains aren't enough for stateful, branching, multi-step agent workflows, so the ecosystem moved toward graph runtimes — LangGraph for LangChain, and agent frameworks in Semantic Kernel — that model an app as nodes and edges with explicit state, loops, and human-in-the-loop checkpoints. These give the control and observability production agents need.

Key Concepts

🧩Orchestration Framework

Libraries (LangChain, Semantic Kernel, LlamaIndex) that compose LLM apps from reusable building blocks.

⛓️Chain / LCEL

Composing components into a pipeline; LangChain pipes them (prompt | model | parser) into one Runnable.

📝Prompt Template

A reusable, parameterized prompt with roles and placeholders (system/human, {input}, chat_history).

🧠Memory

Persisted conversation state injected into the prompt so the app remembers prior turns.

🛠️Plugins / Tools

Functions the model can call; Semantic Kernel marks them [KernelFunction] and auto-invokes them.

🕸️Agent Graph (LangGraph)

A stateful node/edge runtime for branching, looping, multi-step agent workflows with checkpoints.

LangChain (LCEL) + Semantic Kernel

tsx

1# --- LangChain (Python) — compose with LCEL: prompt | model | parser ---
2from langchain_core.prompts import ChatPromptTemplate
3from langchain_core.output_parsers import StrOutputParser
4from langchain_openai import ChatOpenAI
5
6prompt = ChatPromptTemplate.from_messages([
7    ("system", "You translate English to French."),
8    ("human", "{text}"),
9])
10chain = prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()   # one Runnable
11chain.invoke({"text": "good morning"})        # also .stream() / .batch() / async
12
13# Tool-using agent (LangGraph powers stateful agents under the hood):
14from langchain.agents import create_tool_calling_agent, AgentExecutor
15agent = create_tool_calling_agent(llm, [search_tool], agent_prompt)
16AgentExecutor(agent=agent, tools=[search_tool]).invoke({"input": "..."})
17
18// --- Semantic Kernel (C#/.NET) — Kernel + plugins + auto function calling ---
19var kernel = Kernel.CreateBuilder()
20    .AddOpenAIChatCompletion("gpt-4o", apiKey)
21    .Build();
22kernel.Plugins.AddFromObject(new WeatherPlugin());   // [KernelFunction]-tagged methods
23
24OpenAIPromptExecutionSettings settings = new()
25    { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() };   // auto-invoke tools
26var answer = await kernel.InvokePromptAsync(
27    "What should I wear in Paris today?", new(settings));

💡

Why This Matters

Orchestration frameworks are how most production LLM apps are built — they standardize prompts, memory, retrieval, tools, and streaming, and power the agent runtimes (LangGraph, SK agents) behind modern AI products. Knowing their building blocks, the LCEL/plugin model, and when abstraction helps vs hurts is core to practical AI engineering and a common interview topic.

Common Pitfalls

⚠Over-abstracting a simple app — heavy framework layers can hide the actual prompt and obscure debugging.

⚠Not inspecting the final prompt/tokens sent to the model, making wrong answers hard to diagnose.

⚠Using linear chains for stateful, branching agent flows that really need a graph runtime (LangGraph).

⚠Coupling to fast-moving framework APIs without pinning versions — these libraries evolve quickly.

⚠Letting the framework manage memory/tools you don't understand, causing surprising cost and behavior.

Real-World Use Cases

1Swapping Models and Adding RAG Without a Rewrite

Scenario

A team prototyped a chatbot with direct OpenAI API calls. Now they need to add conversation memory, retrieval over docs, and the ability to switch model providers — and the hand-rolled code is tangled and hard to extend.

Problem

Bespoke API plumbing couples the app to one provider and one flow. Adding memory, retrieval, and parsing means rewriting glue code, and changing models touches everything — slow and error-prone.

Solution

Adopt an orchestration framework: express the flow as composed components (prompt template → retriever → model → output parser) with a memory module. Swapping providers is a one-line model change, adding RAG is inserting a retriever, and streaming/batching come for free from the framework's runnable interface.

💡

Takeaway: Orchestration frameworks turn LLM apps into composable pipelines, so memory, retrieval, parsing, and model choice become swappable parts — far more maintainable than hand-wired API calls.

2When the Framework Becomes the Problem

Scenario

A team wrapped everything in deep framework abstractions. Debugging a wrong answer meant digging through layers of chains and callbacks, and a simple custom step fought the framework's conventions.

Problem

Over-relying on heavy abstractions hid what was actually sent to the model, made control flow opaque, and added indirection that outweighed the benefit for a relatively simple app — a common critique of early LangChain usage.

Solution

Use the framework deliberately: lean on it for genuinely reusable pieces (prompt templates, retrievers, memory, agent loops), keep the prompts and control flow inspectable, and drop to plain API calls or a graph runtime (LangGraph) where explicit state and debuggability matter more than abstraction.

💡

Takeaway: Frameworks help when they remove real plumbing, not when they hide what's happening. Keep prompts and flow observable, and choose the level of abstraction that fits the app's complexity.

Join the discussion

Comments

Loading comments...

Cookie preferences

We use essential cookies for sign-in and preferences. With your permission, we also use basic analytics to improve lessons.