herbertThe FAIR Signposting Crawler - Building Knowledge Graphs using FAIR Signposting on the Web
This project addresses the critical challenge of bridging human-readable web resources and machine-actionable data by developing a focused crawling system that discovers, analyses, and integrates FAIR (Findable, Accessible, Interoperable, and Reusable) resources. Despite Linked Data providing a framework for structured metadata, significant barriers exist in automatically discovering and accessing these resources across the web. The FAIR Signposting Crawler implements mechanisms that enhance machine-actionability through systematic resource discovery and knowledge graph construction, enabling automated traversal of interconnected resources without requiring human intervention. The system features a modular architecture with a configurable crawler engine that identifies Signposting links in HTTP headers and HTML elements while employing adaptive fallback methods when direct Signposting is unavailable. The implementation includes an RDF processing pipeline that transforms discovered data into a coherent knowledge graph stored in Apache Jena Fuseki, and a FAIR assessment module that evaluates resource compliance. Developed using Python and Flask, the web interface provides interactive visualisations, a SPARQL query interface, and comprehensive assessment reporting. The project demonstrates how Signposting can effectively reduce barriers between traditional web resources and their machine-actionable counterparts, advancing the broader goal of truly machine-actionable research data on the web.
Patrick Hochstenbach
ljc
RADAR