GridEye
A real-time geospatial intelligence platform. Aggregates breaking events from 10+ OSINT sources including Telegram, RSS feeds, Twitter/X, Reddit, ACLED, and GDACS. Extracts locations with NLP, plots events on an interactive map with live WebSocket updates, AI-powered summarization, confidence scoring, and semantic deduplication. Designed to be self-hosted via Docker Compose.
System Design
Turborepo monorepo with three apps: web, websocket, and workers. Fully containerized with Docker Compose.
Web Application
Next.js 16 App Router with React 19 and TypeScript. Full-screen MapLibre GL JS map with clustered event markers, chronological sidebar feed, saved filter presets, and admin dashboard. Zustand for state, TanStack Query for server cache.
Worker Pipeline
Node.js 22 worker processes orchestrated by BullMQ job queues. Scrapes 10+ sources on configurable intervals: Telegram channels via MTProto, RSS feeds, Twitter/X API v2, Reddit API, ACLED, GDACS, and scanner feeds. Each source is an isolated adapter with retry and rate-limit handling.
WebSocket Server
Standalone real-time event server. Broadcasts new events to all connected map clients the moment they are ingested and geolocated. Handles connection lifecycle, heartbeats, and room-based filtering for regional subscriptions.
Data Layer
PostgreSQL 17 with PostGIS for geospatial indexing and spatial queries. pgvector extension for semantic embedding storage and deduplication. Redis for job queues, caching, and pub/sub. Drizzle ORM for type-safe schema management and migrations.
Under the Hood
The technical details that power real-time global event intelligence.
Ingests from Telegram channels via MTProto API, RSS/Atom feeds, Twitter/X API v2, Reddit API, ACLED conflict data, GDACS disaster alerts, and local scanner feeds. Each adapter handles authentication, pagination, rate limits, and source-specific parsing independently.
Extracts geographic references from unstructured event text using natural language processing. Resolves place names to coordinates via Nominatim geocoding with fuzzy matching, disambiguation, and confidence thresholds. Supports multilingual input via LibreTranslate.
Interactive MapLibre GL JS map with WebGL rendering. Live event markers with category-based icons, density clustering at high zoom levels, popup detail cards, and smooth animated transitions. Filter by source, category, time range, and saved presets.
Automated event summarization that distills raw source data into structured intelligence briefs. Includes confidence scoring based on source reliability, cross-reference count, and geolocation precision. Highlights key entities, locations, and temporal references.
Generates vector embeddings for each event using Transformers.js. Stores embeddings in pgvector and performs cosine similarity searches to detect and merge duplicate reports from different sources, preventing map clutter while preserving unique information.
Full Docker Compose deployment with Caddy reverse proxy for automatic HTTPS. MinIO for S3-compatible media storage. All services containerized with health checks, volume persistence, and environment-based configuration. Zero vendor lock-in.