In the natural language processing blog, Hal Daume III has recently written an exciting post where he argues that data mark-up is not natural. To capture the fuzzy notion of naturalness he compares parallel French-English data and part-of-speech tagged data. Parallel French-English data are natural, he says, because they “naturally” exist: one shouldn’t have particular knowledge to understand what these data are and how to use/create them. As for POS-tagged data, they are unnatural and useless for non-linguists/non-developers.

The crux of the argument is that if something is not a task that anyone performs naturally, then it’s not a task worth computationalizing.

The further argumentation consists in that there is no external evidence that when a human translates a text, she performs special operations similar to, say, part-of-speech tagging. Mapping the idea onto the Ontos NLP one could say then that there is no evidence that a human piles up some objects like annotations connected with various fragments of the text using a huge number of patterns similar to the JAPE-patterns we are using. Moreover, it is unbelievable that a human reads the same text scores of times to perform a specific operation each time: first to read the whole text to recognize morphological information, then to recognize the named entities, finally to learn the relations between the recognized entities and so on and so forth. Rather, a human somehow interprets the string, applying by turns various subsystems of the Language Faculty, which are required at the moment: e.g., most likely the encyclopedia is not used permanently, but just in the cases it is necessary.

Still, I am not in the position to maintain the idea that a natural language processor must be “natural” in all its aspects. Mark-up is not natural, but it is not a so bad idea. It is noticed long ago that computers are very dissimilar to the human cognitive system. Suppose someone will implement, for instance, all the complex syntactic algorithms proposed in Generative grammar; if such a system is possible at all, it might be too complex and unusable. Computers’ peculiarities sometimes make us to avoid “naturalness” in the NLP, although it is not to say that NLPers should not try to reach it where it is appropriate and possible.

The single spot in our NLP where compositionality is fully abondoned, is Semantic Tagger. Because of the syntactic and semantic complexity of structures we have to analyse, this module can’t extract the interpretation of a given construction (say, a finite sentence about the relation “be employee of”) step by step from the interpretation of its syntactic parts. We can’t do it not just because we don’t have a sufficiently powerfull syntactic analyser, but we simply don’t need it.

To extract a relation like “be employee of” we use a number of patterns relying on entities recognized previously. Look at (1).

(1) In the mid-1990 the former German citizen Heinz Schimmelbuch becomes CEO of Weissel company,

If the processor already knows that mid-1990 is a Date/Period, Heinz Schimmelbuch is a Person, CEO is a Job Title, Weissel company is an Organization, become is a verb of coming into being or smth. like that, and that there is no punctuation between these items, it has enough information to conclude that this sentence might speak about an employment relation, as long as several additional conditions are satisfied. The first condition is the items’ order in this sentence. The sentence (2) might also be an employment-related sentence, but not (3).

(2) The former German citizen Heinz Schimmelbuch becomes CEO of Weissel company in the mid-1990.
(3) In the mid-1990 Heinz Schimmelbuch becomes American citizen and meets the CEO of Weissel company.

The second condition concerns some specific constraints on the key verb. E.g., the verb become has to be finite: with an non-finite verb, the probability that this sentence is irrelevant, increases, cf. (4).

(4) In the mid-1990 the former German citizen Heinz Schimmelbuch dreamed about becoming CEO of Weissel company.

Then, taking into account these observation, we can safely add a pattern using just the recognized items like Person, Job Title, etc. and some punctuation markup (commas, points, etc.), “hiding” all the irrelevant part of the sentence.

(5) … the mid-1990 … Heinz Schimmelbuch … become … CEO … Weissel company.

Now we can compose such a pattern.

({Date} | {StartPoint})? {Person}
{becomeVG.VOICE = “act”, becomeVG.MOOD = “ind”}
{JobTitle} {Organization}

Oh yeap, this pattern is indeed used in our system.

I came accross an exciting paper by Karin Verspoor, George Papcun and Kari Sentz from LANL. The authors suggest a motivation for the so-called shallow approach to Information Extraction.

Indeed, an ongoing discussion between ‘deepers’ (who argue for a deep and presumably full semantic and syntactic analysis) and ’shallowers’ (who argue for a shallow analysis) revealed that “[s]hallower approaches are more robust to the linguistic variance of free text”, but “they are much faster”, while “[d]eeper approaches … are in principle more domain-neutral because they embody general linguistic principles”, but they are obviously much more expensive both in throughput and development time. I’ve got a particular interest in this issue, because in Ontos we use a kind of shallow approach, and we really do not attempt to get a full syntactic and semantic analysis. Our rules and patterns primarily focus on the recognition of objects and relations represented in the domain ontology.

Verspoor et al. 2003’s motivation relies on the Construction Grammar’s (CG) hypothesis that constructions are the essential linguistic units stored in the human mind. The basic notion of construction is defined in the following way:

C is a construction iff C is a form-meaning pair F_i, S_j>, such that some aspect of F_i (form) or some aspect of S_j (semantics) is not strictly predicted from C’s component parts or from other previously established constructions.

The main argument of constructionists is that in natural language there is a wide range of non-compositional expressions (e.g. idioms), which should be stored in the lexicon. As far as I understand, the radical constructionist view is that any linguistic structure (w.r.t. its syntax and semantics) is a construction. Although the authors note that CG does not entirely reject the principle of compositionality adopted in formal semantics, this principle does not in fact play an important role in CG. Furthermore, the authors suggest that gazetteer entries and expressions captured by syntactic patterns for named entities in a shallow NLP, are constructions in the sense of CG.

I would not be so enthusiastic with the issue of (non)compositionality in the field of IE. Presumably, using gazetteers has nearly nothing in common with (non)compositionality of their entries. In fact, most entries of key-gazetteers (those providing context information) are splendidly compositional (cf. international company, regional hospital), and e.g. patterns using any output of the Morphology Component (i.e. specifying just grammatical features and order of their units) produce also compositional phrases.

But still, there is a special subfield in our NLP, where patterns (not gazetteers) do involve a kind of non-compositional phenomena. I’ll write about it in the next post.