Overview
Project Overview
SceneMakerAI is an open-source multimodal AI service that performs integrated analysis of scenes, emotions, people, and dialogue in broadcast content, and automatically repurposes this data into highlights, short-form clips, and compilations. Its goal is to help broadcast and media companies utilize their vast video archives more quickly and cost-effectively.
SceneMakerAI is being developed as a national project under the 2026 NIPA(National IT Industry Promotion Agency) Open-Source AI and Software Development and Utilization Support Project, and its key deliverables will be released under the Apache 2.0 License.
- Project Overview
| Item | Description |
|---|---|
| Project Name | Development and Commercialization of an Intelligent Broadcasting Content Reprocessing Service Based on Open-Source Multimodal AI |
| Program | 2026 NIPA Open-Source AI and Software Development and Utilization Support Program |
| Project Period | May 2026 – December 2026 |
| Lead Organization | Solbox Co., Ltd. |
| Participating Organization | SBSi (Technology subsidiary of SBS Media Group) |
| License | Apache 2.0 |
The pilot will be conducted using SBSi’s actual broadcast content, with the goal of on-premises deployment and commercialization as a B2B SaaS solution.
###What Problem Does It Solve?
Broadcasting and media companies possess vast video archives, including dramas, variety shows, sports, and news. However, repurposing this content into new formats—such as highlight clips, short-form videos, and personality-based compilations—is largely a manual process that requires people to watch the footage and edit it themselves, making it time-consuming and costly.
SceneMakerAI combines open-source multimodal AI—which understands both video and audio—with workflow orchestration to automate the entire process, from scene analysis to repurposing.
Core Services
SceneMakerAI offers AI services across four key areas.
| Service | Description |
|---|---|
| Compilations | Reconstructions tailored to viewing purposes, such as binge-watching guides, character-specific highlights, and emotional montages |
| Remix & Short-Form | Creation of short-form content such as trailers, teasers, highlight clips, memorable quote cards, and multiplatform conversions |
| Ad Insertion Optimization | Matching ads to scene context and optimizing insertion timing based on emotional flow |
| Batch Automation | Batch processing of full pipelines, short-form marketing, compilation videos with ads, and custom workflows |
###How Does It Work?
SceneMakerAI processes broadcast video through a pipeline consisting of preprocessing → multimodal analysis → indexing → repurposing.
- Preprocessing: Organizes signals required for analysis, such as video decoding, speech-to-text (STT) extraction and subtitling, and audio/speaker separation.
- Multimodal Analysis: Uses open-source multimodal AI that simultaneously understands video and audio to analyze scenes, people, dialogue, and emotions.
- Workflow Orchestration: Efficiently handles large volumes of content using LangGraph-based Map-Reduce parallel processing.
- Indexing and Search: Analysis results are indexed using vector search and a relational database, enabling quick retrieval of desired scenes during reprocessing.
Detailed architecture and technologies used are covered in a separate document.
###Target Performance (KPI)
The project’s performance metrics aim to be validated through ETRI-certified testing; the figures below represent the target values we aim to achieve.
- Processing Performance: Processing time for one hour of broadcast content within 20 minutes
- Scene Analysis Accuracy: F1 score of 0.70 or higher (for 40–60-minute segments), 0.65 or higher (for 60–120-minute segments)
###Open Source and Community
SceneMakerAI was built on the open-source ecosystem and aims to give back its achievements to the community. We publish prompt templates, RAG configurations, and datasets tailored to the broadcast domain, and contribute to the open-source projects used by our project through issues and pull requests. We consistently share our progress and technical background on our technical blog.
- GitHub: github.com/SceneMakerAI
- Technical Blog: Visit the Blog