Multilingual Training Data Web Crawling ML Systems
I build multilingual data pipelines for machine learning: crawling the web, extracting usable text and making processing reliable at scale.
My work spans ParaCrawl and MaCoCu, production NLP, Transformer-based music transcription and open-source systems.
Multilingual data engineering 2017–2023
I led engineering for multilingual web-data pipelines that processed over 1 PB of Common Crawl and other web data on HPC and cloud infrastructure, producing public corpora for machine translation and language-model research.
As a core developer and maintainer of Bitextor and co-developer of Monotextor, I worked on the path from crawling and WARC ingestion to text extraction, language identification, alignment and cleaning. Bitextor produces parallel corpora; Monotextor focuses on monolingual text.
Publications: ParaCrawl (ACL 2020) MaCoCu (EAMT 2023)
Deduplication across implementations
Rust Python
Deterministic cluster IDs
Binary-format compatibility
Historical-index semantics
Training-data tooling Contribution proposal
I proposed cluster-ID output for DataTrove’s Rust MinHash stage so the existing Python pipeline can consume its metadata. The change numbers components deterministically, independently of the union-find root, while preserving historical-index semantics.
The proposal includes Rust tests and Python interoperability checks, with S3 behavior exercised through Moto.
I also isolated two initialization failures in the ReadabilityInscriptis extractor, separating dependency validation from the sandbox’s empty-input warmup.
Read the proposal and review
Transformer-based music transcription
I led engineering for a research project developing Transformer-based models for optical and audio music transcription within a shared end-to-end framework.
The task was to turn score images and musical audio into structured notation, bringing together model experimentation, evaluation and music-domain knowledge.
Project and research
C++ systems with tight memory limits
I maintain open-source firmware for Xteink e-readers, working on EPUB rendering, parsers, image decoding and regression testing.
For EPUB tables, I implemented a bounded row buffer and simple column layout. Complex or narrow rows fall back to a readable flow, preserving their content. The contribution documents memory and firmware comparisons, regression tests and device checks.
Other merged work includes grouping image buffers by lifetime and handling allocation failure, and fixing progressive JPEGs with separate component scans.
Explore the firmware
Production NLP and retrieval 2015–2017
At Prompsit, I contributed to the data and retrieval pipelines behind Reverso Context, improving retrieval of multilingual examples from aligned corpora.
The product served more than seven million queries per day. My contribution was to its data and retrieval infrastructure within the wider Reverso ecosystem.
Visit Reverso Context
Data access and reusable ingestion infrastructure
My work on CARAMEL and CRYPTOTRACK includes authenticated backend access to Trino/Delta Lake and integration with Airflow, Spark and MinIO.
I develop shared Python components for S3 ingestion and scraping, covering paginated discovery, validation and automated tests. The projects address cardiovascular risk assessment and analysis of cryptocurrency transactions.
I teach programming and software engineering at the University of Alicante. Outside work, I make music and build retrocomputing hardware, including tools that connect Game Boys and MIDI.
Game Boy music hardware PCB projects