OpenWatch
AI-powered security intelligence platform that transforms fragmented news into structured, real-time incident data across Nigeria's 36 states
Situation
Nigeria deals with a lot of different security threats at once: kidnapping, terrorism, armed robbery, communal violence, and cattle rustling, spread across 36 states and the FCT. Security information is scattered across dozens of news outlets, social media, and word of mouth. Reports are often duplicated, written in a mix of formal English and Pidgin, and lack structured metadata like coordinates, incident categories, or severity assessments.
Journalists, researchers, NGOs, and security professionals have no single source of truth. They manually piece together the picture from fragmented reports, often missing incidents entirely or double-counting the same event reported by five outlets.
Task
Build a real-time intelligence platform that ingests raw news, extracts structured incident data using AI, deduplicates it semantically, and presents it on an interactive tactical map with daily intelligence briefs. The system needed to run fully autonomously, scraping, extracting, classifying, deduplicating, scoring, and publishing without human intervention.
Tech stack: Next.js 16 with React 19, Supabase (PostgreSQL + pgvector + RLS + Realtime), OpenAI GPT for extraction and analysis, Leaflet for mapping, BullMQ for pipeline orchestration, Twitter API v2 for automated posting, and Framer Motion for the intelligence UI.
Action
Automated ingestion pipeline
RSS scrapers pull articles from major Nigerian outlets (Punch, Vanguard, Premium Times, Channels TV, and others) on a cron schedule. A Twitter scraper monitors security-focused accounts for breaking reports. BullMQ orchestrates the pipeline: ingest → extract → classify → deduplicate → geocode → score → publish.
Domain-specific AI extraction
Each article is sent to GPT with a heavily tuned system prompt that understands Nigerian geography (all 36 states + FCT, LGAs, and common landmarks), Nigerian English and Pidgin slang, and a taxonomy of 12 incident types: terrorism (TER), kidnapping (KDN), armed robbery (ARB), communal violence (CMV), bombing (BOM), cattle rustling (CTR), gender-based violence (GBV), piracy (PIR), assassination (ASN), protest (PRT), missing person (MSP), and scam (SCM).
The model returns structured data: location (state, LGA, coordinates), date, severity, fatality counts, perpetrator groups, ransom info, affected infrastructure, and a narrative summary. Strict rules reject personal recounts, aftermath stories, court proceedings, and non-Nigerian incidents.
Semantic deduplication with pgvector
The same incident reported by five outlets shouldn't appear five times. OpenWatch generates vector embeddings for each extracted incident and uses pgvector's cosine distance to find semantically similar records within a 72-hour window. A multi-signal fusion score combines semantic similarity, geographic proximity (geohash comparison), and temporal closeness. Duplicates are merged, and new sources are linked to existing incidents, improving confidence over time.
Interactive tactical map
Leaflet-powered map with CartoDB dark tiles and custom tactical markers colour-coded by incident type. Clicking a marker opens a detail panel with the full incident report, source links, confidence score breakdown, and corroborating articles. Geohash-based spatial indexing enables efficient proximity queries for clustering nearby incidents.
AI-generated daily intelligence briefs
Every day, an AI analyst prompt synthesises the day's incidents into an executive brief: headline, threat level assessment, hotspot regions, key developments, and incident breakdowns by type. Briefs are stored, browsable by date, and update in real time via Supabase Realtime subscriptions.
Composite confidence scoring
Each incident gets a confidence score based on source tier (Tier A outlets like Punch score higher than unverified social media), the number of corroborating sources, extraction clarity, and location specificity. This score determines whether an incident is auto-published or held for review.
Automated Twitter posting
A bot auto-tweets individual high-severity incidents and publishes daily, weekly, and monthly summary images with trend data, threat assessments, and geographic breakdowns. Generated images use consistent branding and are optimised for Twitter's card format.
Responsive intelligence UI
Desktop uses a split-pane layout: map + incident feed + detail panel. Mobile adapts to a bottom sheet interface with the daily brief as a floating pill with a drop-up panel. All transitions use Framer Motion for smooth crossfades between states.
Technical Highlights
- pgvector for semantic deduplication, not just search: multi-signal fusion of embedding similarity, geohash proximity, and temporal closeness
- Domain-specific AI prompts encoding Nigerian geography, Pidgin English, and a 12-type incident taxonomy with strict false-positive rejection rules
- BullMQ pipeline orchestrating scrape → extract → classify → dedup → geocode → score → publish
- Supabase Realtime subscriptions for live incident feed and brief updates
- Row Level Security on every table. Service role keys never touch the client
- Geohash-based spatial indexing for efficient proximity clustering
- Automated social media publishing with generated summary images
Key design decisions
pgvector for dedup, not just search
Most projects use vector databases for RAG or search. Here, embeddings are the core of a deduplication engine, turning a fuzzy “is this the same incident?” question into a quantifiable similarity score fused with spatial and temporal signals.
Classification accuracy through domain specificity
The extraction prompt isn't generic. It encodes Nigerian geography, distinguishes between Nigerian English and Pidgin, enforces a strict incident taxonomy, and includes rules to reject common false positives (celebrity recounts, aftermath stories, court proceedings). Accuracy comes from domain knowledge, not model size.
Security-first data model
Every table uses Supabase Row-Level Security. Public writes route through server-side API routes with validation. This isn't optional when you're handling sensitive security data that could be weaponised if exposed.
Adaptive intelligence UI
The dashboard adapts from a desktop split-pane layout (map + feed + detail panel) to a mobile-first bottom sheet interface. The daily brief works as both an inline feed element and a floating pill with a drop-up panel. Different form factors, same intelligence.
Result
OpenWatch runs fully autonomously, processing dozens of articles daily and producing structured intelligence without human intervention. The hardest problems weren't technical. They were domain problems: teaching an LLM the difference between a bandit attack and an armed robbery in the Nigerian context, handling the ambiguity of “unknown gunmen” reports, and deciding when a confidence score is high enough to publish automatically.
- Fully automated pipeline: scrape → extract → dedup → geocode → score → publish
- Real-time tactical map covering all 36 states + FCT
- Daily AI-generated intelligence briefs with threat assessments
- Automated Twitter presence with generated summary graphics
- Live at openwatch.ng