Skip to main content

07_Designing a Langgraph-Based Broadcast Workflow

· 7 min read
sbin
sbin
SceneMakerAI 팀

Introduction​

This article details the LangGraph process for finding segments of broadcast clips using natural language queries.

It demonstrates the process of structuring the data into a clips[] JSON format.

The search process can be summarized most simply as follows

1

: Determine which episode (scope) 2

. Compile a list of what to search for (plan) 3

. Retrieve similar segments (retrieve) 4

. Examine the surrounding context (expand) 5

. Determine the exact start and end points (select) 6

. Merge and adjust the length (assemble)

. The LangGraph fixed graph is run within the agent-search environment.

The code implementation is as follows.

# agent-search/lib/graph/graph.py

ainvoke(request)
 ├ Selecting episodes for the scope (specify v_ids or voting + LLM)
 ├ Plan Route + Beats (by session)
 ├ Retrieve   Multi-query vector search
 ├ expand     Hit ±12 seconds context
 ├ select     start–end confirmed (or one instance of the material)
 └ assemble   Overlap Merge · budget · clips[] JSON

Requests are received via POST /api/v1/search.

service is currently under development only up to compilation / shortform / trailer / ad_slot.

1. scope — Which episode it is​
  • Since this is a series, we first determine “which episode we’re talking about.”
  • If v_ids exists, use it as is. If not, conduct a chapter vector vote → finalize with LLM.
SCOPE_MAX_VIDEOS = 5 # Upper limit for episode selection
SCOPE_VOTE_LIMIT = 40     # Top-k chapters for voting

# lib/graph/scope.py

> async def run(state: dict, deps) - dict:
    req = state["request"]

    # ① User specifies the episode → As is
    if req.get("v_ids"):
        videos = await asyncio.to_thread(db.select_videos, req["v_ids"])
        scope = {"mode": "user", "reason": "Specify v_ids in the request", "votes": []}

    # ② Automatic Selection: Chapter Voting → LLM Finalization
    else:
        qvec = (await asyncio.to_thread(embed.embed_query, [qtext]))[0]
        votes = await asyncio.to_thread(vdb.vote_videos, deps.vdb, qvec)
        # ... The LLM determines v_ids (up to SCOPE_MAX_VIDEOS)
        scope = {"mode": "vote+llm", "reason": ..., "votes": votes}

    return {"videos": videos, "scope": scope}
  • Sample input/output
Input:  query="Goal-scoring moments from Game 7 of the Korean Series", v_ids=None

Output:  # mode=vote+llm
{
  "videos": [{"v_id": 1, "name": "Korean Series: KIA vs. SK", ...}],
  "scope": {
    "mode": "vote+llm",
    "reason": "The request terms 'Game 7 of the Korean Series' and 'scoring play' match v_id=1 ...",
    "votes": [[1, 12], [4, 7], [3, 7], [5, 6], [6, 4], [2, 4]]
  }
}

A total of 40 votes are spread across 6 episodes. Since the top choice received only 12 votes, it’s difficult to decide based on the votes alone, so the LLM finalizes it as one option.

2. plan — Compiling a list of what to look for​
  • For each round, review the chapter timeline and define search units (beats).
  • Two routes: pinpoint (1–2 iconic scenes) / structural (multiple clips from the narrative).
# lib/graph/plan.py

# route:
#   pinpoint   — Highlight clips, memorable quote cards
#   Structural — Episode Recaps, Trailers, Character Highlights

# Beat 1:
#   title, queries[2~3], time_hint, want=["segment","dialogue"]
  • Sample Input/Output
Input:  v_id=3 Winter Sonata, preset=Emotional_Montage

Output:
{
  "route": "structural",
  "beats": [
    {
      "title": "A Walk by the Lake and a Longing Glance at the Bus Stop",
      "queries": [
        "A scene where the male and female leads take a quiet walk by the lake and gaze at each other,"
        "Two people standing awkwardly face to face at a bus stop, conscious of the stares from their friends nearby,"
        "A close-up of an exchange of glances that captures the flutter of excitement from the early episodes of *Winter Sonata*"
      ],
      "time_hint": [0, 600],
      "want": ["segment", "dialogue"],
      "v_id": 3
    }
    # ... multiple beats
  ]
}

The reason there are 2–3 queries entries: To cast a wider net for the same scene—covering visual descriptions, events, and dialogue. Vector search is cost-effective.

3. retrieve — Fetching Similar Clips​
  • Perform Milvus vector search using queries based on beats. No LLM used.
  • Merge multi-queries → Remove duplicates → Top SEARCH_TOPK (10).
  • Cast is a soft filter (since there are many empty cast rows, a hard filter would kill recall).
SEARCH_TOPK = 10

# lib/graph/retrieve.py

> def search_beat(client, beat: dict, cast_filter: list[str] | None) - list[dict]:
    queries = beat.get("queries") or [...]
    qvecs = embed.embed_query(queries)
    # Soft filter steps: [cast+event] → [event] → [cast] → [none]
    # "is_ad" is excluded during post-processing
    return sorted(...)[:SEARCH_TOPK]
  • Sample Input/Output
Input:  queries=["The male and female protagonists are by the lake...", ...]

Output:  candidates (partial)
[
  {
    "ref_type": "segment", "ref_id": 26,
    "start_sec": 150, "end_sec": 156,
    "text": "A man and a woman are walking along a path by the lake. Man and woman walking; woman stops; man stops",
    "cast": "",
    "distance": 0.6867   # Cosine similarity (the higher the value, the more similar)
  },
  ...
]
4. expand — Look at the context before and after​
  • A 6-second hit isn’t a clip. Attach the segment+dialogue from ±12 seconds (two segment slots) before and after.
  • select provides the material for snapping the boundaries.
EXPAND_PAD_SEC = 12   # Expansion width before and after the hit
_TOP_HITS = 4 # Number of top hits to expand per beat

# lib/graph/expand.py

> def expand_beat(client, beat: dict) - list[dict]:
    ctxs = []
    for h in beat["candidates"][:_TOP_HITS]:
        s = max(0, h["start_sec"] - EXPAND_PAD_SEC)
        e = h["end_sec"] + EXPAND_PAD_SEC
        ctxs.append({
            "anchor": f"{h['ref_type']}:{h['ref_id']}",
            "rows": vdb.window(client, beat["v_id"], s, e),
        })
    return ctxs
  • Sample Input/Output
Input:  hit segment: 26 (150–156 seconds)

Output:  context (a portion of the time window rows)
[
  {"ref_type": "segment", "ref_id": 25, "start_sec": 144, "end_sec": 150, "text": "..."},
  {"ref_type": "segment", "ref_id": 26, "start_sec": 150, "end_sec": 156, "text": "A man and a woman are standing by the lake..."},
  {"ref_type": "segment", "ref_id": 27, "start_sec": 156, "end_sec": 162, "text": "Two people by the lake..."},
]
5. select — Finalize where to cut​
  • The LLM examines the beat, candidates, and surrounding timeline to finalize a single clip [start_sec, end_sec].
  • If irrelevant, usable=false + retry_query → Re-search and re-expand, then re-evaluate only once (REFINE_MAX=1).
REFINE_MAX = 1

# lib/graph/select.py

# Output JSON:
# {"usable": true|false, "start_sec", "end_sec",
#  "source_refs", "cast", "reason", "caption", "score",
#  "retry_query": "only when usable=false"}
  • Sample Input/Output (Success)
Input:  beat="A Walk by the Lake and a Longing Glance at the Bus Stop" + candidates + context

Output:
{
  "usable": true,
  "start_sec": 150, "end_sec": 162,
  "caption": "Two people pause during a walk by the lake, exchanging awkward yet heart-fluttering glances",
  "score": 0.92,
  "reason": "A 12-second segment connecting the lakeside walk (26) and the exchange of glances immediately afterward (27)"
}
  • Sample Input/Output (Rejected)
Output:
{
  "usable": false,
  "score": 0.0,
  "reason": "The candidate's timeline contains no actual scoring or batting footage, only commentary",
  "retry_query": "KIA's Druw hitting a single / KIA's Yukemi hitting a single / ..."
}

If nothing is found, do not force a fill-in. This is how we prevent hallucinations in video RAG.

6. assemble — Merge and Adjust Length​
  • Collect only usable clips. No LLM used.
  • Merge overlapping clips from the same episode → adjust to duration budget (score-priority) → sort → final JSON.
BUDGET_TOLERANCE = 1.15   # Allowable multiplier for exceeding target_duration_sec

# lib/graph/assemble.py

# Trailers are listed in order of production; the rest are listed in order of broadcast date and time
# The output's `v_id` plus `start_sec/end_sec` is used as-is as the input for the next ffmpeg step
  • Example Input/Output
{
  "service": "compilation",
  "preset": "Emotional_Montage",
  "route": "structural",
  "total_duration_sec": 54,
  "clips": [
    {
      "v_id": 3,
      "v_name": "Winter Sonata",
      "order": 1,
      "beat": "A Walk by the Lake and a Longing Glance at the Bus Stop",
      "start_sec": 150,
      "end_sec": 162,
      "start_time": "00:02:30",
      "end_time": "00:02:42",
      "caption": "Two people pause during a walk by the lake, exchanging awkward yet heart-fluttering glances",
      "score": 0.92
    }
  ],
  "trace": {"llm_calls": 6, "retries": 0, "dropped_beats": []}
}

Intermediate results for each node are dumped to output/<req_id>/<node>.json.

Conclusion​

We’ve covered the search flow and the process of structuring the clips[] array, which

were our goals.

The next steps are cutting the clips into actual video segments (ffmpeg serving) and integrating with the service UI.

Reference Links)

This article presents research results conducted with support from the Ministry of Science and ICT and the National IT Industry Promotion Agency’s “2026 Open Source AI and Software Development and Utilization Support Project.”