Day 4: Schema-First Sentiment Analysis

Not quite. Unlike spam detection, which is binary and mostly about what a message is, sentiment analysis is about how a message feels. And as soon as human tone enters the picture—sarcasm, double negations, lukewarm shrugs, and mixed complaints—a naive prompt-and-pray chain falls apart.

Let’s walk through how to build a deterministic sentiment pipeline in LangChain, why “neutral” is a trap, and how a subtle change to the order of fields in your Pydantic schema can dramatically improve your model’s accuracy.

Sentiment as a Typed Contract

If you ask an LLM for the sentiment of a sentence using a standard prompt, it will happily reply with something like: “Overall, the tone leans slightly positive, though the author expresses some frustration at the end.”

That’s pleasant to read in a chat window, and completely useless in a backend pipeline. You can’t write an if/else statement against an essay. You want a typed object you can route on, store in a database, and trust.

Schema-first sentiment: Define the shape of the answer before you write the prompt. The LLM’s job becomes filling out a strict data contract, not improvising prose.

Using LangChain’s with_structured_output and Pydantic’s Literal type, we can bind that contract directly to our model:

from typing import Literal
from pydantic import BaseModel, Field

class NaiveSentimentResult(BaseModel):
    label: Literal["positive", "negative", "neutral"] = Field(
        description="The overall sentiment of the text"
    )
    confidence: float = Field(
        ge=0.0, le=1.0,
        description="How confident you are in this label, from 0 to 1"
    )
    explanation: str = Field(
        description="A one-sentence justification for the label"
    )

When you pass this class into llm.with_structured_output(NaiveSentimentResult), LangChain uses the model’s native structured-output or tool-calling capabilities to guarantee the response parses into a validated NaiveSentimentResult instance. If the model tries to invent a label like "somewhat_positive", Pydantic raises a validation error immediately instead of letting bad data slip downstream.

(Note: If you ever run into an older model that doesn’t support with_structured_output, LangChain’s PydanticOutputParser gives you the exact same contract by injecting the JSON schema directly into the prompt instructions.)

2. Why Field Ordering Matters in Pydantic

Look closely at NaiveSentimentResult above. The fields are ordered:

1. label
2. confidence
3. explanation

Remember how LLMs generate JSON: autoregressively, one token at a time, from left to right. When label is the first key in your schema, the model has to commit to "positive", "negative", or "neutral" on its very first output field before it, has generated a single word of reasoning. By the time it reaches the "explanation" field, it isn’t analyzing the text anymore; it is rationalizing the label it already locked in (Structured Outputs Playbook).

Watch what happens when you feed tricky sentences into a label-first schema on a smaller local model like llama3.2:

3. Reason-First Schema + Addressing the “Neutral” Trap

We can fix both problems, impulsive labeling and the "neutral" trap—right inside our Pydantic schema, without writing a massive prompt:

1. Put reasoning fields before label: Ask the model to extract surface_cues, flag has_contrast_or_sarcasm, and write a short reasoning_trace first. Because those tokens are generated first, the model’s self-attention attends to its own analysis when it finally picks the label.
2. Separate "mixed" from "neutral": "The package arrived on Tuesday" is genuinely neutral (zero emotion). "The weather is nice, but the traffic was a nightmare" is not neutral, it’s mixed. Adding "mixed" to our Literal stops the model from averaging +1 and -1 into 0.

Here is the upgraded contract and chain:

from typing import Literal
from pydantic import BaseModel, Field
from langchain_core.prompts import ChatPromptTemplate
from langchain_ollama import ChatOllama

# --- 1. The Reason-First Contract ---
class HardenedSentimentResult(BaseModel):
    surface_cues: str = Field(
        description="Key opinion words, factual timelines, or tone markers in the text."
    )
    has_contrast_or_sarcasm: bool = Field(
        description="True if the text has conflicting clauses ('but', 'although'), "
                    "lukewarm hedging ('fine, I guess'), or sarcasm/irony."
    )
    reasoning_trace: str = Field(
        description="One-sentence analysis weighing literal words against actual intent "
                    "BEFORE choosing the final sentiment label."
    )
    label: Literal["positive", "negative", "neutral", "mixed"] = Field(
        description="Final sentiment classification based on the reasoning_trace above."
    )
    confidence: float = Field(
        ge=0.0, le=1.0,
        description="Estimated confidence from 0.0 to 1.0."
    )

# --- 2. Prompt & Structured Model ---
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a strict sentiment classifier. "
               "Extract surface cues and write your reasoning trace BEFORE selecting "
               "a label (`positive`, `negative`, `neutral`, or `mixed`). "
               "Reserve `neutral` strictly for factual statements with no emotional charge."),
    ("human", "{text}"),
])

llm = ChatOllama(model="llama3.2", temperature=0)
sentiment_chain = prompt | llm.with_structured_output(HardenedSentimentResult)

How the Code Works Step-by-Step

• surface_cues & has_contrast_or_sarcasm: These act as a lightweight checklist. When the input is "Oh brilliant, another update that wipes all my local settings", the model extracts "Oh brilliant" alongside "wipes all my local settings" and flips has_contrast_or_sarcasm to True.
• reasoning_trace: With the contrast flag set to True, the model writes: “Calling a destructive software bug ‘brilliant’ is clear sarcasm; the actual sentiment is negative.”
• label: Now that the explanation is sitting right there in the context window, choosing "negative" is trivial.

4. Triage Routing for Borderline Cases

Even with a reason-first schema, some inputs are genuinely ambiguous. What should your pipeline do with "It's fine, I guess"? Is it reluctant neutrality or mild negativity?

Instead of pretending the model is 100% certain on every edge case, we can use our structured fields to triage borderline predictions—just like we did with spam probabilities on Day 2:

def triage(result: HardenedSentimentResult, threshold: float = 0.65) -> str:
    if result.confidence < threshold:
        return "needs_review"
    if result.has_contrast_or_sarcasm and result.label == "neutral":
        return "needs_review"
    return result.label

If confidence drops below 0.65, or if the model detects hedging/contrast (has_contrast_or_sarcasm=True) yet still lands on "neutral", we route the item to "needs_review" rather than polluting our downstream analytics.

Grab the Full Code

I’ve put the complete, runnable script—including the side-by-side benchmark comparing the Naive vs. Reason-First schemas, the PydanticOutputParser fallback, and a multi-vote self-consistency helper—into a GitHub Gist so you can test it locally:

👉 View the complete Day 4 Sentiment Analysis code on GitHub Gist

5. What we learned, Whats next?

We now have a structured, reason-first LLM chain that handles sarcasm, double negations, and mixed feelings cleanly. Problem solved, but now we will try a dedicated specialist models are built for speed and precision on their home turf.

Spinning up llama3.2 to generate a five-field JSON object takes hundreds of milliseconds per sentence. That’s great when you need a written explanation for a tricky customer support ticket, but what if you need to score 50,000 tweets or product reviews in a few minutes?

Hugging Face has dedicated sentiment encoders like cardiffnlp/twitter-roberta-base-sentiment-latest that are smaller, run in a few milliseconds, and are trained specifically for sentiment classification.

In the next article, we’re going to put our structured LLM chain against a dedicated Hugging Face sentiment model, and then wire them together into a hybrid pipeline where the fast specialist handles the easy 80% of traffic and escalates only the tricky, sarcastic, or mixed edge cases to the LLM. Stay tuned!