A couple of days ago, an exciting article on various aspects of semantic technologies was posted on ReadWriteWeb: Semantic Web Patterns: A Guide to Semantic Technologies. Alex Iskold, the author of the article, highlights many important problems of information workers which can be solved by means of the semantic web. This is a nice overview of what has been done by the moment in the area of annotation technologies based on meta data, semantic APIs, search technologies, semantic databases, etc. The notion of the Semantic Web itself is considered from various points of view, since as the author notes,

The Semantic Web means many things to different people, because there are a lot of pieces to it. To some, the Semantic Web is the web of data, where information is represented in RDF and OWL. Some people replace RDF with Microformats. Others think that the Semantic Web is about web services, while for many it is about artificial intelligence - computer programs solving complex optimization problems that are out of our reach. And business people always redefine the problem in terms of end user value, saying that whatever it is, it needs to have simple and tangible applications for consumers and enterprises.

In the natural language processing blog, Hal Daume III has recently written an exciting post where he argues that data mark-up is not natural. To capture the fuzzy notion of naturalness he compares parallel French-English data and part-of-speech tagged data. Parallel French-English data are natural, he says, because they “naturally” exist: one shouldn’t have particular knowledge to understand what these data are and how to use/create them. As for POS-tagged data, they are unnatural and useless for non-linguists/non-developers.

The crux of the argument is that if something is not a task that anyone performs naturally, then it’s not a task worth computationalizing.

The further argumentation consists in that there is no external evidence that when a human translates a text, she performs special operations similar to, say, part-of-speech tagging. Mapping the idea onto the Ontos NLP one could say then that there is no evidence that a human piles up some objects like annotations connected with various fragments of the text using a huge number of patterns similar to the JAPE-patterns we are using. Moreover, it is unbelievable that a human reads the same text scores of times to perform a specific operation each time: first to read the whole text to recognize morphological information, then to recognize the named entities, finally to learn the relations between the recognized entities and so on and so forth. Rather, a human somehow interprets the string, applying by turns various subsystems of the Language Faculty, which are required at the moment: e.g., most likely the encyclopedia is not used permanently, but just in the cases it is necessary.

Still, I am not in the position to maintain the idea that a natural language processor must be “natural” in all its aspects. Mark-up is not natural, but it is not a so bad idea. It is noticed long ago that computers are very dissimilar to the human cognitive system. Suppose someone will implement, for instance, all the complex syntactic algorithms proposed in Generative grammar; if such a system is possible at all, it might be too complex and unusable. Computers’ peculiarities sometimes make us to avoid “naturalness” in the NLP, although it is not to say that NLPers should not try to reach it where it is appropriate and possible.

On Read/WriteWeb there was a post about 10 semantic Apps to watch. It seems that the terminology of semantics is used for all sort of kinds. Our understanding is that semantic applications should use some form of ontology in order to describe the meaning of data/text and this includes named entities and relations between the named entities. In my opinion not all of the presented candidates fulfil this requirement. Nevertheless it will be interesting to see if the semantic search engines will leave the beta status and be able to answer questions to all sort of domains. Another point to watch is if the other Apps add more semantic meaning to their solution. At least Ontos with his news portal already uses the semantic meaning and offers first semantic services and therefore we are happy not to be alone in the take off of semantic Apps.

The single spot in our NLP where compositionality is fully abondoned, is Semantic Tagger. Because of the syntactic and semantic complexity of structures we have to analyse, this module can’t extract the interpretation of a given construction (say, a finite sentence about the relation “be employee of”) step by step from the interpretation of its syntactic parts. We can’t do it not just because we don’t have a sufficiently powerfull syntactic analyser, but we simply don’t need it.

To extract a relation like “be employee of” we use a number of patterns relying on entities recognized previously. Look at (1).

(1) In the mid-1990 the former German citizen Heinz Schimmelbuch becomes CEO of Weissel company,

If the processor already knows that mid-1990 is a Date/Period, Heinz Schimmelbuch is a Person, CEO is a Job Title, Weissel company is an Organization, become is a verb of coming into being or smth. like that, and that there is no punctuation between these items, it has enough information to conclude that this sentence might speak about an employment relation, as long as several additional conditions are satisfied. The first condition is the items’ order in this sentence. The sentence (2) might also be an employment-related sentence, but not (3).

(2) The former German citizen Heinz Schimmelbuch becomes CEO of Weissel company in the mid-1990.
(3) In the mid-1990 Heinz Schimmelbuch becomes American citizen and meets the CEO of Weissel company.

The second condition concerns some specific constraints on the key verb. E.g., the verb become has to be finite: with an non-finite verb, the probability that this sentence is irrelevant, increases, cf. (4).

(4) In the mid-1990 the former German citizen Heinz Schimmelbuch dreamed about becoming CEO of Weissel company.

Then, taking into account these observation, we can safely add a pattern using just the recognized items like Person, Job Title, etc. and some punctuation markup (commas, points, etc.), “hiding” all the irrelevant part of the sentence.

(5) … the mid-1990 … Heinz Schimmelbuch … become … CEO … Weissel company.

Now we can compose such a pattern.

({Date} | {StartPoint})? {Person}
{becomeVG.VOICE = “act”, becomeVG.MOOD = “ind”}
{JobTitle} {Organization}

Oh yeap, this pattern is indeed used in our system.

I came accross an exciting paper by Karin Verspoor, George Papcun and Kari Sentz from LANL. The authors suggest a motivation for the so-called shallow approach to Information Extraction.

Indeed, an ongoing discussion between ‘deepers’ (who argue for a deep and presumably full semantic and syntactic analysis) and ’shallowers’ (who argue for a shallow analysis) revealed that “[s]hallower approaches are more robust to the linguistic variance of free text”, but “they are much faster”, while “[d]eeper approaches … are in principle more domain-neutral because they embody general linguistic principles”, but they are obviously much more expensive both in throughput and development time. I’ve got a particular interest in this issue, because in Ontos we use a kind of shallow approach, and we really do not attempt to get a full syntactic and semantic analysis. Our rules and patterns primarily focus on the recognition of objects and relations represented in the domain ontology.

Verspoor et al. 2003’s motivation relies on the Construction Grammar’s (CG) hypothesis that constructions are the essential linguistic units stored in the human mind. The basic notion of construction is defined in the following way:

C is a construction iff C is a form-meaning pair F_i, S_j>, such that some aspect of F_i (form) or some aspect of S_j (semantics) is not strictly predicted from C’s component parts or from other previously established constructions.

The main argument of constructionists is that in natural language there is a wide range of non-compositional expressions (e.g. idioms), which should be stored in the lexicon. As far as I understand, the radical constructionist view is that any linguistic structure (w.r.t. its syntax and semantics) is a construction. Although the authors note that CG does not entirely reject the principle of compositionality adopted in formal semantics, this principle does not in fact play an important role in CG. Furthermore, the authors suggest that gazetteer entries and expressions captured by syntactic patterns for named entities in a shallow NLP, are constructions in the sense of CG.

I would not be so enthusiastic with the issue of (non)compositionality in the field of IE. Presumably, using gazetteers has nearly nothing in common with (non)compositionality of their entries. In fact, most entries of key-gazetteers (those providing context information) are splendidly compositional (cf. international company, regional hospital), and e.g. patterns using any output of the Morphology Component (i.e. specifying just grammatical features and order of their units) produce also compositional phrases.

But still, there is a special subfield in our NLP, where patterns (not gazetteers) do involve a kind of non-compositional phenomena. I’ll write about it in the next post.

A crucial problem for an information extraction system is the disambiguation of the recognized named entities. Suppose you’ve got a vocabulary of phrases and key words for the entity in question, and a grammar containing various structural templates corresponding to different syntactic patterns of the entity and relying on the words and phrases from the vocabulary. This model roughly corresponds to our NLP-system exploited in Ontos. The immediately arising problem is that these templates are applied independently and allow for intersection of entities generated by different templates.
Take the string (1) as an input for such a model.

(1) President of Kimberly Clark Corporation Steven R. Kalmanson

When we apply all the variety of templates from the grammar, we’ll have a piled up tile of annotations subsecting and intersecting with each other.

For sure this picture is not what we’d like to get as an output. Rather, we’d like to see a finely plain structure of annotations without any intersections. Moreover, on the above picture there are some annotations that should remain in the new structure, and annotations (specifically all others) which we wouldn’t like to see any more in our lives. So how to rule out the incorrect annotations?

The solution developed in our team is the following. Recall that each annotation from the above picture arises from a specific template, and every template is motivated by some heuristics. Then some heuristics are relyible in a wide range of cases as they usually give a correct output, and some heuristics are not: we can pin our hopes on them only if we don’t have any stronger one at hand. Consequently we can rank our heuristics by assigning different weights to the templates used in our grammar. The more trustworthy is a template, the higher rank it gets.

Besides, for each type of conflict we will use a resolving rule formulated with respect to the weights of the annotations in conflict. These rules will constitute a special module called Minimization. After Minimization is applied, we get the desired output.

So that’s how it works. Although sometimes we get an incorrect analysis (when the ranking of specific templates occasionally goes wrong or when the correct hypothesis is absent), we never get intersections.

Today we are all flooded with news that we receive by email or RSS feeds. We all have the same problem of filtering the important ones from the less important ones and to identify what is linked together. The internet community has introduced several methods on how to aggregate news. Which of the methods is the best is difficult to say. One method that doesn’t seem to be mentioned is the possibility of automatic aggregation using semantic technologies, especially the functions of NLP. Automatic aggregation is subscribing to different sources and then analyzing the content with natural language processing (NLP) technologies. Doing it this way the system can extract objects of interest (e.g. person, organization, location etc) and the relation between the objects. After the information extraction process the system has to merge the same objects like Hillary Clinton is the same like she or Hillary or Clinton assuming the NLP can distinguish inside the text the meaning and then merge the objects together. This approach is driven and controlled by a clear knowledge model (ontology) which allows growing a semantic network of information. Over the time the more object – relation – object information we have we can start to rank information on his semantic meaning which provides a much better view. This is different to approaches introduced by Mixx or Digg where people create votes and therefore influence the ranking of the news. Another part that seems to be important for a successful start is the up to date information as seen when Tailrank started.
We’ll keep working on our approach and watch the other methods of aggregation and ranking. Would be interesting to hear about experiences from others when trying to understand how information is linked together. Can it be solved by filters or by votes or by using bookmarks?

Welcome to the Ontos blog. We will use it to inform about new trends and features concerning our technologies especially in the area of NLP (Natural Language Processing), aggregating and merging of news and information sources. We will share ideas and concepts on how to use information and enrich them in a semantic way - for example in our new semantic portal.