This was highlighted in a recent project that SciBite undertook with the Human Disease Ontology (DO) Knowledgebase group at University of Maryland School of Medicine. The DO group connected with SciBite requesting disease term mapping based on OMIM IDs (Online Mendelian Inheritance in Man) [6] between the Disease Ontology and UniProt Disease (UPDISEASE) entries [7]. The purpose of this effort, for the DO project, was to facilitate timely disease term mapping between UniProt and the DO. Cross-mapping between resources is a timely and curatorially heavy effort. Automated, expert mappings discerned from SciBite greatly facilitated the speed with which this effort could be accomplished.

Combined mapping approach

The disease mapping project involved aligning the approximate 6,500 UniProt Disease classes to the DO classes, utilizing several of SciBite’s tools in addition to expert curators. Throughout this process, SciBite and DO curators noted that the combination of automated and manual approaches was key to a successful mapping project. We found that automated mapping tools alone were unlikely to create a complete and accurate mapping. Thus, it is necessary to employ manual approaches to both refine the automated approach, verify the mappings created by the tool and find mappings that are missed by the tool.

The primary tool used was SciBite’s mapping tool Workbench [8]. Workbench uses SciBite vocabularies (VOCabs) within TERMite (SciBite’s Named Entity Recognition engine) to align data or ontologies to another data standard or ontology. Workbench is designed to assist the user in the time-consuming and error-prone process of matching terms between sources.

In addition to Workbench, SciBite’s ontology management platform CENtree was incorporated into the mapping process to allow the curators to visualize the DO terms, their synonyms, definitions and position within the hierarchy to assist with validation of the suggested mappings (Figure 2).

Leveraging SciBite vocabularies

Workbench is built to utilize SciBite VOCabs and TERMite to provide comprehensive synonym coverage of life sciences domains. The key VOCab for the UPDISEASE:DO mapping project was SciBite’s DOID VOCab. This VOCab is built from the DO ontology but optimized for named entity recognition (NER) by expert curators. This process includes augmenting with additional synonyms from literature review and rules-based synonym generation plus adding context and disambiguation where appropriate to increase the precision and recall in NER.

Workbench requires a TERMite-ready VOCab for each of the terminologies being mapped, which meant a rudimentary VOCab also had to be created for the UniProt Disease entries. Fortunately, a simple three-column file would suffice for the initial mapping procedure (Figure 3).

Figure 3: Building a UniProt Disease Entity VOCab. The ID, Name and Alternative Name fields from the UniProt export were used to build the UPDISEASE VOCab. The “Mnemonic” field wasn’t used for initial mapping as these short acronyms are likely to produce false positive mappings.

The results of the initial mapping in Workbench were promising (Figure 4). Approximately 90% of UniProt disease terms were mapped to a DO term, either as an EXACT or BROADER match. An exact match is where at least one label or synonym matches exactly between the source term and target, whereas a broader match holds a similarity between labels or synonyms and is above the user-defined cut-off score.

Figure 4: A sample of mappings between UPDISEASE and DO from Workbench. Workbench scores mappings based on the lexical similarity between labels and synonyms and measures mapping ambiguity. This score is normalized to a percentage. Generally, mappings over 80% (0.80) are medium to high confidence.

Curator review

Although a large proportion of the UPDISEASE classes have been successfully mapped to DO, there were still 50% that were not exact and required curator review. Many of the broader matches could be verified by a quick eyeball of the corresponding terms, as in Figure 5a. The UPDISEASE term “Dystonia 6, torsion” is clearly the same as the DO term “torsion dystonia 6”, so these are quick for the curator to review. Other reviews may take longer, as in the example in Figure 5b. The UPDISEASE term “Corticosterone methyloxidase 1 deficiency” was reported as having NO mapping, but a curator checking the ontology would find that an EXACT mapping to DO actually does exist, differing only in the order of the terms within the disease name.

Figure 5a: 5a Some mappings are reported as broader but a quick look at the term labels shows they are clearly exact. 5b Exact mappings may exist, as shown by the DOID term found by the curator, but Workbench found no mapping to the UPDISEASE term.

Semi-automated synonym creation

Curator review of the mapping output can also identify possible improvements to the mapping process. One of the major reasons for non-exact matches in this mapping project was the different word order between the labels of UPDISEASE terms and the DO, or the addition of ‘type’ in the DO label, resulting in no mapping being found (Figure 6a and 6b).

Figure 6: 6a Different word order between resources means that the corresponding term cannot be automatically identified. 6b Testing these UniProt labels over the DOID VOCab in TERMite shows that the full names are not identified, only partial matches to broader DO classes.

This could be resolved by augmenting the UPDISEASE VOCab using a semi-automated approach. This involved using regular expressions to create additional synonyms for the UPDISEASE terms that would match the word order in DO (Figure 7).

Figure 7: Creating synonym variations for the UniProt Disease entities allows the NER to align the synonyms to DO.

By taking this approach of creating synonyms with matching word orders, it was possible to map an additional 10-15% of UPDISEASE terms to DO, thus further reducing the manual effort required by the curators.

Unfortunately, there were too many variations of word order and word addition to capture with regular expressions. Figure 8 shows the initial broader mapping of the UPDISEASE term “Muscular dystrophy congenital LMNA-related” to the DO term “muscular dystrophy”. After manual review it was found the exact mapping was in fact to DO term “congenital muscular dystrophy due to LMNA mutation”. It would be extremely time-consuming to construct regular expressions to cover all the different types of synonym variation for little gain (generally only a handful terms follow each particular pattern), therefore manual review of these was essential.

Figure 8: The initial mapping of the UPDISEASE term was too broad, but the different word order plus additional words in the DO term meant that automated mapping was not possible. After manual review, the curator identified the corresponding term to map to.

At this point most of the low hanging fruit had been mapped, but you may well remember that the UniProt Disease entities also had mnemonics or acronyms associated with them. It was decided to add the mnemonics to the UPDISEASE VOCab and perform the mapping process again using only the remaining unmapped UPDISEASE terms. Adding these mnemonics to the VOCab used in the initial mapping would likely have resulted in many false positives due to the short length of the synonyms. This would have meant the curators would have more mappings to manually verify, increasing the time and effort.

Adding the mnemonic synonyms to the UniProt Disease Entities resulted in an additional ~300 terms being mapped.

Getting into the weeds

As with all ontology mapping exercises, there are several hundred or (if you’re unlucky) thousands of terms that cannot be automatically mapped and the only option, until automated approaches improve, is for a curator to search for a matching term within the target source. This is where the curator’s expertise and fastidiousness come into their own (with a little help from an ontology viewer such as CENtree).

Take the example shown in Figure 9. The UniProt Disease term “Crisponi/Cold-induced sweating syndrome 2” has a mnemonic synonym “CISS2”. This term does not exist as an exact match in DO, but the curator has found a similar term “cold-induced sweating syndrome 2”, however it has no synonyms to provide additional evidence to its meaning (Figure 9a). The question is, are these equivalent?

By looking at DO in an ontology browser such as CENtree, the curator could examine the terms surrounding this potential target term. The parent term of the DO term is “cold-induced sweating syndrome”, and among its synonyms is “Crisponi syndrome” (Figure 9b), so the curator can infer from this that the UPDISEASE term is in fact equivalent to the DO term “cold-induced sweating syndrome 2”.