Borderlex Datahub

Since 2023, I have designed and built the Borderlex Datahub as a comprehensive web platform that unifies multiple specialized tools for trade policy research under a single, integrated interface. Rather than managing separate applications, Borderlex journalists and subscribers now access six interconnected features through one centralized system that provides comprehensive coverage of European trade policy developments and legislative processes.

The Datahub's integrated architecture creates analytical possibilities that standalone tools cannot deliver. Users can seamlessly navigate from newly published documents to votes on tabled documents in the European Parliament or Council of the EU, or trace the evolution of specific policy positions from amendment proposals through final parliamentary votes. This cross-feature connectivity enables journalists to uncover comprehensive stories about the legislative process rather than isolated data points.

Detailed descriptions of several key Datahub features are available elsewhere on this portfolio site:

• Live Document Hub
• European Parliament Voting Tracker
• European Parliament Amendment Tracker

I built the Datahub as a modular Django application designed for extensibility and independent feature development. This architectural approach allows me to add new analytical tools based on evolving editorial requirements while maintaining system stability and shared infrastructure.

The platform operates on multiple PostgreSQL databases, each optimized for specific data types and processing requirements. I leveraged PostgreSQL's advanced text processing capabilities, including full-text search and vector embeddings through the pg_vector extension, to support the diverse analytical needs across features. This database architecture reflects the separate data collection processes I designed for different institutional sources while enabling cross-feature data relationships.

For frontend interactivity, I chose HTMX over traditional JavaScript frameworks to manage complexity while working as a solo developer. This decision enabled me to build sophisticated interactive components without the overhead of maintaining separate frontend and backend codebases.

I deployed machine learning models for text classification and semantic search as independent FastAPI services running in Docker containers. This microservices approach ensures scalable performance for NLP operations while keeping the core Django application lightweight and responsive.

The Datahub's strength lies in harmonizing diverse data sources and formats into a coherent analytical platform. I developed separate command-line tools for different data collection processes—one for real-time document monitoring, another for comprehensive European Parliament legislative data—while ensuring all features share common data standards and relationships.

This architectural separation enables independent scaling and maintenance of data collection processes while providing users with a seamless, integrated experience across all features.

I collaborated with Borderlex's web developer to implement unified authentication between the main Borderlex.net site and the Datahub subdomain. This integration includes IP-based recognition for organizational subscribers, enabling transparent access across both platforms. The harmonized authentication system ensures that journalists can move fluidly between editorial content and analytical tools without multiple login processes.

The user interface maintains consistent navigation and design patterns across all six features, allowing users to apply similar search and filtering approaches whether analyzing amendments, votes, or document streams. This standardization reduces the learning curve for new features while maintaining the specialized functionality each tool requires.

The Datahub serves as Borderlex's centralized repository for trade policy intelligence, consolidating what would otherwise be scattered research processes into a unified analytical workflow. By building a modular, extensible platform, I created a foundation that adapts to changing editorial priorities and emerging data sources in the dynamic field of trade policy reporting.

The integrated approach transforms individual research tasks into comprehensive analytical capabilities, enabling journalists to produce more sophisticated coverage of European trade policy developments while reducing the time spent navigating between disparate information sources.