Disparate biomedical databases are fused into a unified disease-gene-drug knowledge graph
A disease-gene-drug knowledge graph begins as raw exhaust from a dozen incompatible systems. ClinVar ships clinical variant XML, DrugBank exposes a REST/XML drug catalog, OMIM maintains a proprietary phenotype-gene text format, and DisGeNET, UniProt, and Open Targets each publish their own TSV, RDF, or JSON dumps. Ingestion is the unglamorous first mile: pulling, parsing, and staging all of it without losing provenance.
No single database captures disease-gene-drug biology completely, so ingestion must federate across a heterogeneous ecosystem:
• ClinVar (NCBI): clinical significance of genomic variants, submitted as structured XML with ClinVarSet records, updated weekly • DrugBank: drug identity, mechanism, and gene targets, distributed as XML/JSON with DrugBank Accession Numbers (DB IDs) • OMIM: curated Mendelian gene-phenotype relationships in free-text "clinical synopsis" fields with MIM numbers • DisGeNET: aggregated gene-disease associations from GWAS catalogs, UniProt, and text mining, scored by a proprietary Gene-Disease Association (GDA) score • UniProt: canonical protein/gene identifiers, functional annotation, and cross-references to HGNC and Ensembl • Open Targets: an integrated evidence platform that already fuses genetics, somatic mutations, and drug data with its own association scores
Each source has its own release cycle — ClinVar weekly, DrugBank quarterly, OMIM continuously — so ingestion pipelines must track per-source version stamps to keep the graph reproducible.
A production ingestion layer is built as a set of source-specific adapters feeding a common staging schema:
1. Connectors: scheduled pullers (FTP for ClinVar VCF/XML dumps, REST polling for Open Targets and DrugBank, SPARQL endpoints for UniProt RDF) 2. Parsers: format-specific extractors that normalize XML/TSV/JSON/RDF into a flat staging table of (source, record_id, raw_payload, fetched_at) 3. Checksumming: content hashes on each raw record detect unchanged rows so downstream stages skip redundant reprocessing 4. Landing zone: raw payloads are retained verbatim (not just parsed fields) so provenance can always be traced back to the original submission
This staging discipline matters because entity resolution and relation extraction downstream are lossy processes — keeping the raw record lets errors be re-diagnosed and re-run without re-fetching from source.
ClinVar alone adds roughly 8,000–10,000 new or revised variant interpretations per week. A staging layer that cannot incrementally ingest deltas would force a full 3.2M-record reprocess on every run — the checksum-based delta approach cuts weekly ingestion volume by over 95%.
The central difficulty at this stage is not volume but disagreement in structure and vocabulary:
• Identifier heterogeneity: the same gene appears as an HGNC symbol in OMIM, an Ensembl Gene ID in UniProt, and an NCBI Gene ID in DisGeNET • Granularity mismatch: ClinVar records variants at the nucleotide level, OMIM records at the gene-phenotype level, DrugBank at the drug-target level — none share a natural join key without a mapping layer • Confidence semantics differ: DisGeNET's GDA score, ClinVar's five-tier clinical significance, and Open Targets' 0–1 association score are not directly comparable and must be re-normalized later • Update asynchrony: a gene renamed by HGNC may take months to propagate into OMIM records, producing stale cross-references
Rather than resolving these mismatches during ingestion, the pipeline defers reconciliation to the dedicated entity resolution stage — ingestion's only job is complete, faithful, versioned capture.
| Product | Indication | Trial Design | Key Result |
|---|---|---|---|
| ClinVar | Variant → disease clinical significance | Weekly XML/VCF release from NCBI | Highest curation rigor; five-tier significance scale |
| DrugBank | Drug identity, targets, mechanism | Quarterly XML/REST release | Richest pharmacological metadata per drug |
| OMIM | Gene → Mendelian phenotype | Continuous curator updates | Deep phenotype text, but free-text heavy |
| DisGeNET | Gene-disease association scoring | Biannual aggregation release | Combines GWAS, curated, and text-mined evidence |
The word "myocardial infarction" in one source and "MI" in another, "TP53" and "tumor protein p53" in two more — without resolution, these become separate, disconnected nodes and the graph fractures into isolated islands per source. Entity resolution collapses every surface form of the same disease, gene, or drug into a single canonical node, anchored to an authoritative identifier.
Resolution begins with deterministic normalization before any fuzzy matching is attempted:
• Case-folding, punctuation stripping, and Unicode normalization on free-text mentions • Synonym expansion against controlled vocabularies: MONDO for diseases, HGNC for genes, RxNorm/DrugBank ID for drugs • Abbreviation resolution using curated alias tables ("MI" → "myocardial infarction", "T2D" → "type 2 diabetes mellitus")
When exact normalized matches fail, approximate string matching (Levenshtein / Jaro-Winkler distance) proposes candidate merges, which are then validated against ontology parent-child relationships — two strings that are lexically similar but sit in unrelated ontology branches are rejected, preventing false merges between superficially similar but biologically distinct concepts (e.g., "breast cancer" vs. "breast cancer 1 gene").
The durable solution to identifier heterogeneity is a maintained crosswalk table mapping every source-local ID to one canonical identifier:
• Genes: HGNC symbol as the canonical anchor, cross-referenced to Ensembl Gene ID, NCBI Gene ID, and UniProt accession • Diseases: MONDO or UMLS CUI as canonical, cross-referenced to OMIM MIM number, ICD-10, and MeSH term • Drugs: DrugBank ID as canonical, cross-referenced to RxNorm CUI, ChEMBL ID, and PubChem CID
These crosswalks are themselves living artifacts — HGNC alone processes hundreds of symbol updates per year as genes are renamed, split, or merged — so resolution pipelines re-run periodically rather than treating crosswalks as static.
UniProt's cross-reference table alone links each human protein entry to more than 150 external databases. Reusing this crosswalk instead of re-deriving gene identity mappings from scratch is what makes sub-100ms resolution latency achievable at ingestion scale.
Not every mention resolves cleanly. A configurable confidence threshold governs how aggressively candidate matches are auto-merged versus held back for manual review:
• High-confidence matches (exact ID crosswalk hit): merged automatically, no review • Medium-confidence matches (fuzzy string + ontology neighbor): merged only if the confidence score clears the threshold slider; otherwise flagged as an unresolved duplicate and kept as a separate provisional node • Low-confidence matches (weak lexical overlap, no ontology support): rejected, entities remain distinct
Raising the confidence threshold reduces false-positive merges (two genuinely different entities collapsed into one) at the cost of leaving more legitimate duplicates unresolved, fragmenting node counts. Lowering it does the reverse. Production pipelines typically settle around 70–80% confidence, tuned against a manually adjudicated gold sample.
Once entities are canonical, the graph still has no edges. Relation extraction recovers the disease-gene and gene-drug associations buried in structured database fields and in free-text literature, attaching each proposed edge to an evidence type, source provenance, and a numeric confidence score before it is allowed into the graph.
The highest-confidence relations require no natural-language inference at all — they are already explicit in structured database fields:
• ClinVar's clinical significance field directly states a variant-disease relationship (Pathogenic, Likely pathogenic, Uncertain significance, etc.), which projects onto a gene-disease edge via the variant's host gene • DrugBank's "Targets" field explicitly lists the gene/protein a drug acts on, along with the pharmacological action (inhibitor, agonist, substrate) • OMIM's gene-phenotype map provides a direct, curator-asserted gene-disease pair with an inheritance pattern annotation
These structured edges are extracted with simple field mapping rather than inference, and inherit the source's own curation confidence — which is why they anchor the highest-precision tier of the extracted relation set.
Structured fields alone miss the majority of known biology, most of which exists only in free-text publications. The literature-mining pipeline layers:
1. Named Entity Recognition (NER): transformer models fine-tuned on biomedical corpora (BioBERT, SciSpaCy) tag disease, gene, and drug mentions in PubMed abstracts and full text 2. Co-occurrence filtering: sentence- or paragraph-level co-mention of a gene and disease is a weak initial signal, used only to generate candidate pairs 3. Relation classification: a fine-tuned classifier scores each candidate sentence for whether it asserts a genuine causal/associative relationship versus incidental co-mention (e.g., a review sentence merely listing unrelated genes) 4. Evidence aggregation: a gene-disease pair supported by many independent publications accumulates a higher aggregate confidence than a pair mentioned once
This pipeline mirrors the approach used by Open Targets' literature evidence channel and DisGeNET's BeFree text-mining module.
Structured-field extraction reaches roughly 96% precision because the source database has already done the curation work — but literature NLP extraction, at ~78% precision, is what supplies the majority of relation volume, since only a small fraction of known gene-disease-drug biology is ever captured in structured fields.
Every extracted relation, regardless of origin, is attached to a provenance record rather than being flattened into an anonymous edge:
• Evidence type: structured-field, curated-literature, or text-mined • Source database and record ID the evidence was drawn from • A normalized confidence score, rescaled from each source's native scoring system onto a common 0–1 scale • Publication references (PubMed IDs) for literature-derived edges
When the same disease-gene pair is independently supported by ClinVar, DisGeNET, and a literature-mined sentence, these are not collapsed into a single anonymous edge — they are retained as parallel evidence attached to one canonical relation, so the graph can later be queried by evidence strength or provenance, and a relation with contradictory evidence can be flagged rather than silently averaged away.
With canonical nodes and scored, provenance-tagged edges in hand, construction is the act of materializing them into an actual queryable graph. A property graph model — typed nodes carrying attributes, typed and weighted relationships — is loaded into a graph database such as Neo4j, indexed for traversal, and made available for downstream analytics and drug-repurposing queries.
The schema decision that shapes everything downstream is how nodes and edges are typed:
• Node labels: :Disease, :Gene, :Drug — each carrying properties such as canonical_id, preferred_name, source_ids (the crosswalk array), and last_updated • Relationship types: ASSOCIATED_WITH (disease–gene), TARGETS (drug–gene), TREATS (drug–disease, often inferred transitively), INTERACTS_WITH (drug–drug) • Relationship properties: confidence, evidence_type, source_count, pubmed_refs — carried on the edge itself rather than modeled as separate nodes, keeping traversal queries simple
This mirrors the modeling approach used by Open Targets' own Neo4j-backed platform graph, where a small set of strongly typed relationship classes carries rich edge properties rather than proliferating relationship types for every evidence nuance.
Two competing representations are common in biomedical knowledge graphs, and construction must choose (or bridge) between them:
• Property graph (Neo4j, Cypher): nodes and edges both carry arbitrary key-value properties directly; traversal queries (e.g., "genes associated with disease X that are targeted by an approved drug") are expressed naturally in Cypher pattern matching and execute fast on indexed relationship types • RDF/OWL triple store (e.g., Virtuoso, GraphDB): every fact is a subject-predicate-object triple; ontology reasoning (subclass inference, transitive closure) is native via OWL semantics, and the model interoperates directly with public linked-data resources published in RDF, including UniProt's own SPARQL endpoint
Many production pipelines maintain both: an RDF layer for ontology-grounded reasoning and semantic validation, and a Neo4j property graph as the fast, application-facing serving layer — with a translation step mapping RDF triples into labeled property-graph edges during construction.
Open Targets Platform Graph, DisGeNET RDF, and Bio2RDF all publish disease-gene-drug knowledge in RDF specifically so that OWL reasoners can check logical consistency — a capability a bare property graph does not have natively, which is why the validation stage often runs against a shadow RDF/OWL representation even when Neo4j is the serving layer.
A graph with hundreds of thousands of nodes and millions of edges is unusable without deliberate indexing built during construction, not bolted on afterward:
• Uniqueness constraints on canonical_id per node label, which Neo4j backs automatically with a B-tree index — required for fast MERGE-based incremental loading • Composite indexes on frequently filtered properties (e.g., Disease.mondo_id, Gene.hgnc_symbol) • Relationship type indexing so that pattern queries filtering on TARGETS or TREATS do not scan unrelated edge types • Bulk import via neo4j-admin import for the initial load (CSV-based, bypassing the transactional layer for throughput), followed by incremental Cypher MERGE operations for subsequent delta updates
Getting index design right at construction time is what keeps multi-hop queries — such as "drugs that target a gene associated with disease X, excluding drugs already indicated for X" (the core drug-repurposing query pattern) — returning in milliseconds rather than seconds as the graph grows.
A knowledge graph that is wrong is worse than no graph at all, because it launders unreliable claims with the appearance of structure. Validation is the final gate: structural constraints, semantic consistency against ontologies, and coverage benchmarking against gold-standard references, run before the graph is released for querying or downstream model training.
The first validation pass never touches biology — it verifies the graph obeys its own declared schema:
• Cardinality constraints: does every :Drug node have at least one TARGETS edge, or is it an orphan introduced by a failed join? • Type constraints: does every ASSOCIATED_WITH edge connect a :Disease to a :Gene and never a :Disease to a :Drug (a modeling error, not biology)? • Uniqueness constraints: are there duplicate canonical_id values that slipped past entity resolution — a symptom of a resolution bug rather than new biology? • Dangling references: do all source_ids in the crosswalk array actually resolve back to a record that exists in the staging layer?
These checks are cheap, deterministic, and catch the majority of pipeline bugs before any biological judgment is required — which is why they run first and block downstream stages on failure.
The second pass checks whether the graph's content, not just its shape, is internally coherent, using formal ontology tooling:
• SHACL shapes (Shapes Constraint Language) validate that RDF-represented nodes conform to expected value ranges, cardinalities, and datatype constraints • OWL reasoners (e.g., HermiT, Pellet) check for logical contradictions — for instance, a drug asserted to both TREAT and be CONTRAINDICATED_FOR the same disease without a documented exception, or a gene assigned to two mutually exclusive MONDO disease branches • Contradiction detection specifically compares evidence across sources for the same relation: if ClinVar-derived evidence and a text-mined relation disagree on pathogenicity direction for the same variant-disease pair, the edge is flagged rather than silently merged
The confidence threshold slider directly governs how much semantic disagreement is tolerated before a relation is downgraded to "unresolved" versus accepted into the validated graph.
Running an OWL reasoner over a graph with over 100,000 disease and gene nodes is computationally expensive at full scale, so most production pipelines validate incrementally — reasoning only over the subgraph touched by each batch of new or updated edges, rather than re-checking the entire graph on every update.
A structurally sound, internally consistent graph can still be incomplete or biased toward well-studied biology. Coverage validation measures this directly:
• Recall against Open Targets' curated association set: what fraction of its high-confidence gene-disease pairs does the constructed graph also contain? • Recall against DisGeNET's curated (non-text-mined) subset, treated as a stricter gold standard • Bias auditing: coverage is disproportionately strong for well-studied diseases (cancers, cardiovascular disease) and weak for rare diseases with sparse literature — a known and expected skew that must be documented, not hidden • Novelty auditing: relations present in the constructed graph but absent from any gold standard are flagged for manual review rather than assumed correct, since novelty is as often a pipeline artifact as it is a genuine discovery
Only after clearing structural, semantic, and coverage checks is the graph promoted to a released version — queryable for drug-repurposing hypothesis generation, GNN-based link prediction training data, or clinical decision-support lookups.