In this article we will discuss about:- 1. Introduction to Natural Language Processing 2. Need of Natural Understanding 3. Conceptual Dependency Theory 4. CD-Based Programs 5. Applications.
Introduction to Natural Language Processing:
Natural language understanding is one of the most popular applications of artificial intelligence portrayed in fiction and the media. The idea of being able to control computers by talking to them in our own language is very attractive.
But natural language is ambiguous, which makes natural language understanding particularly difficult. By far the largest part of human linguistic communication occurs as speech. Written language is relatively recent invention and still plays a less central role than speech in most activities. But processing written language is generally easier than processing speech.
Thus the entire language processing problem can be divided in to:
ADVERTISEMENTS:
a. Processing written text, using lexical, syntactic and semantic knowledge of the language as well as the required real world information.
b. Processing spoken language, using all information needed plus additional knowledge about phonology as well as enough added information to handle the further ambiguities which arise in speech.
Natural language processing includes in addition to understanding and generation multilingual translation, but we confine ourselves on understanding of written language.
Need of Natural Understanding:
Before we consider how natural language processing can be achieved, we should be clear about the advantages. There are a number of areas which can be helped by the use of natural language. The first is human computer interaction, through the natural language interface which allows the users to communicate with computer in their own language, rather than in a command language or using menus.
ADVERTISEMENTS:
There are advantages and disadvantages to this: it is a natural form of communication which requires no specialised training, but it is inefficient for expert users and less precise than a command language.
Another area of application is information management, where natural language processing could enable automatic management and processing of information, by interpreting its content. If the system could understand the meaning of a document it could, for example, store it with other similar documents.
A third possibility is to provide an intuitive means of database access. At present most database can be accessed through a query language. Some of these are very complex, demanding considerable expertise to generate even relatively common queries. Others are based on forms and menus, providing a simpler access mechanism.
However, these still require the user to have some understanding of the structure of the database. The user, on the other hand, is usually more familiar with the content of the database, of at least its domain. By allowing the user to ask for information using natural language, queries can be framed in terms of the content and domain rather than the structure.
Conceptual Dependency Theory of Natural Language Processing:
ADVERTISEMENTS:
These techniques provide a useful formation for representing more complex structures such as objects, scenes, and multiple-sentence stories. One of the key ideas of the script approach is to reduce a sentence or story to a set of semantic primitives using a formalism called Conceptual Dependency (CD) theory.
The emphasis in this theory of scripts by is to shift the focus of natural language processing from the form (i.e., syntax) of a communication to its content (i.e., semantics). The goal of a CD under stander is to convert sentences describing an event into a deep representation of its meaning.
Let us consider how we would represent an event in CD theory.
Every event has:
ADVERTISEMENTS:
1. An ACTOR
2. An ACTION performed by that actor
3. An OBJECT on which the action is performed upon
4. A DIRECTION in which that action is oriented.
ADVERTISEMENTS:
By reducing the action taking place in CD theory to a set of simple primitives such as:
Schank and Abelson (1977) were able to represent the deeper semantic meaning present in input stories. Conceptual dependency theory could then be used to answer questions not specifically stated in the story, paraphrase the main ideas of the story, and even translate the paraphrased account into other natural languages.
The basic idea in conceptual dependency theory is that it should be possible to reduce any surface sentence into a deeper representation in terms of semantic primitives. The advantage is that such a system will automatically solve many of the mapping problems that is, two distinct sentences with identical meaning should transform into the same representation in terms of semantic primitives. The semantic primitives of CD theory may be classified in terms of primitive acts, primitive states, and the rules relating them.
Some of these primitives with their abbreviations include:
Primitive Actions:
Primitive States:
Schank and abelson defines also conceptual tenses.
Causality Rules:
1. Actions can cause state changes.
2. States can make certain actions possible.
3. States can prevent certain actions from occurring.
4. States (or acts) can cause mental events.
5. Mental events can cause actions.
Through the semantic implications of the semantic primitives, the CD method permits paraphrasing the input information and drawing inferences from it.
As an example, consider the sentence below and its sensation in CD theory and associated paraphrases and inferences:
The CD representation permits this rather impressive performance by using semantic information about the word sell. It resolves the two function verb sell into instances of the one-function verb give. That is, the word sell as used in this sentence implies that Rao gives Kartar the book and Kartar it gives Rao money.
One significant feature of CD is its structural independence on the languages in which it is expressed. The basic difference between semantic network and CD lies in the naming of connecting links. In semantic nets we name the connecting link between the events in accordance to its relevance to the content and consequently the name different for different users. CD, on the other hand, requires a standard assignment of a dependence relationship and is therefore, independent of the user.
The first program to emerge from the research on CD theory was MARGIE- Memory, Analysis, Response Generation in English (and also the name of Ronger Schank’s mother).
Margie:
Schank, then at Stanford University and his students combined a parser, a generator, and an inference engine in the program MARGIE (1973). The parser took an English sentence and converted it into the semantic primitive representation. The generator looked at the internal representation and returned valid English paraphrases of the statement.
The inferencer, using 16 rules of inference, generated the implications of the statement in semantic primitive language and sent this to the generator which converted the inferences into English.
MARGIE could run in two modes:
1. Inference mode or
2. Paraphrase mode.
Sample output from each are shown below:
1. Inference Mode:
INPUT: Rao told Kartar that Sheela wants a book.
OUTPUT 1: A book about what?
OUTPUT 2: Kartar knows that Sheela wants a book.
OUTPUT 3: Sheela wants to come to have a book.
OUTPUT 4: Sheela wants someone to cease to have a book.
OUTPUT 5: Sheela wants to read a book.
INPUT: Rao gave Kartar an aspirin.
OUTPUT 1: Rao believes that Kartar wants an aspirin.
OUTPUT 2: Kartar is sick.
OUTPUT 3: Kartar wants to feel better.
OUTPUT 4: Kartar will ingest the aspirin.
2. Paraphrase Mode:
INPUT: Rao advised Kartar to drink the wine.
OUTPUT 1: Rao told Kartar that drinking the wine would benefit him.
INPUT: Rao killed Kartar by choking Kartar.
OUTPUT 1: Rao strangled Kartar.
INPUT: Rao choked Kartar and he died because he could not breathe.
OUTPUT 1: Kartar died because he was unable to inhale some air and he was unable to inhale some air because Rao grabbed her neck.
INPUT: Rao advised Kartar to drink the wine.
OUTPUT 1: Rao told Kartar that drinking the wine would benefit her.
INPUT: Rao killed Kartar by choking Kartar.
OUTPUT 1: Rao strangled Kartar.
INPUT 2: Rao choked Kartar and he died because he could not breathe.
INPUT 3: Kartar died because he was unable to inhale some air and he was unable to inhale some air because Rao grabbed his neck..’
Again, although the performance of MARGIE is impressive, it had its limitations.
Two of the most serious were:
1. It could process single sentences only.
2. It demonstrated the problem of an “inference explosion”.
That is, inferences could be drawn from the original statement. And inferences could be drawn from the first generation inferences and so on, leading to a combinatorial explosion.
SAM:
By 1974, Schank had moved to Yale University where he and Robert Abelson, Professor of Psychology invented scripts. Scripts helped solve both of the problems with MARGIE mentioned above. A script provides the framework for integrating the individual sentences of a story.
A script also helps reduce the explosion of inferences by providing a context for validating some inferences and eliminating many others. The program which first made use of these ideas was SAM (Script Applier Mechanism), the Ph.D. thesis work of Richard Culling ford (1981).
The internal structure of SAM consisted of four parts:
1. A parser to convert the story sentences to CD formalism. A modified version of MARGIE’s parser was used.
2. A modified version of MARGIE’s generator to convert CD representations back into correct English.
3. A Script applier to check the consistency of the CD representation with that specified in the script.
4. A question-answer module.
Using automobile accident scripts, SAM was capable of reading, paraphrasing, translating, and answering questions on newspaper reports of accidents.
INPUT:
Friday evening a car swerved off Route 69. The vehicle struck a tree. The passenger, a New Jersey man, was killed. David Hall, 27, was pronounced dead at the scene by Dr. Dana Blanchard, medical examiner. Frank Miller, 32, of 593 Foxon Rd., driver, was taken on Milford Hospital by Flanagan Ambulance. He was treated and released. No charges were made. Patrolman Robert Onofrio investigated the accident.
English Summary:
AN AUTOMOBILE HIT A TREE NEAR HIGHWAY 69 FOUR DAYS AGO. DAVID HALL, AGE 27, RESIDENCE IN NEW JERSEY, THE PASSENGER, DIED. FRANK MILLER, AGE 32, RESIDENT AT 593 FOXON ROAD IN NEW HAVEN, CONNECTICUT, THE DRIVER, WAS SLIGHTLY INJURED. THE POLICE DEPARTMENT DID NOT FILE CHARGES.
Spanish Summary:
UN AUTO CHOCO CONTRA UN ARBOL CERCA DE LA CARRETERA 69 EL- VIERNES AL ANOCHECER DAVID HALL, DE 27 ANOS, QUE VIVTA EN NEW JERSEY, EL PASAGERO, SE MURIO. FRANK MILLER, DE 32 ANOS, QUE VIVIA EN 593 CALLE FOXON EN NEW HAVEN, CONNECTICUT, EL CHOFER, QUEDO IN POCO HERIDO. EL DEPARTMENTO DE POLICIA NO ACUSO A NADIE.
Question-Answering Output:
Q1: Was anyone killed?
A1: YES, DAVID HALL DIED.
Q2: Was anyone hurt?
A2: YES, FRANK MILLER WAS SLIGHTLY INJURED.
Q3: Why was Miller hurt?
A3: BECAUSE THE AUTOMOBILE HIT THE TREE.
Q4: Did Miller go to the hospital?
A4: YES, HE WAS IN THE MILFORD HOSPITAL.
Q5: How did Miller get to the hospital?
A5: THE FLANAGAN AMBULANCE COMPANY TOOK HIM TO THE MILFORD HOSPITAL.
The use of semantic primitives gave SAM the Capability. Paraphrasing the article in Chinese, Russian, and Dutch as well as English and Spanish. However, SAM worked for very restrictive scripts and had no knowledge of the plans and goals of the players in a story. The Yale group’s efforts to address these problems culminated in the program, PAM.
PAM:
PAM (Plan Applier Mechanism) by Robert Wilensky (1983) was a hybrid of the ideas from SAM and those from TALE-SPIN. In TALE-SPIN, Schank and his students gave the characters of a story, certain goals and simulated human planning to meet those goals. The program was then turned loose and wrote its own stories.
PAM differed from SAM in using a different generator and special vocabulary to enhance the realism of its conversations. Also, since it had the goals and plans of each of its main actors available, PAM was able to narrate story summaries from various points of view. Several of these features are apparent in the following except from a PAM dialog about a liquor store robbery.
Input Text:
John wanted money. He got a gun and walked into a liquor store. He told the owner he wanted some money. The owner gave John the money and John left.
Output-Story from John’s Point of View:
I needed, to get some dough, so I got myself this gun, and I walked down to the liquor store. I told the shopkeeper that if he didn’t let me have the money then I would shoot him. So he handed it over. Then I left.
Output-Story from the Owner’s Point of View:
I was minding the store when a man entered. He threatened me with a gun and demanded all the cash receipts. Well, I didn’t want to get hurt, so I gave him the money. Then he escaped.
INPUT: Why did John get a gun?
OUTPUT: Because John wanted to rob the liquor store.
INPUT: Why did John threaten the shopkeeper?
OUTPUT: Because John needed to get some money.
INPUT: Why did shopkeeper give John the money?
OUTPUT: Because the shopkeeper didn’t want to get hurt.
INPUT: Why did John leave?
OUTPUT: Because John didn’t want to get caught.
CD-Based Natural Language Processing Programs:
The parsers, generators, and script appliers development in the MARGIE, SAM and PAM programs provided the tools and inspiration for numerous additional investigations.
Among the more prominent of these were:
1. Politics:
This doctoral work of Jaime Carbonell (1989) integrated the features of the previous programs so that beliefs, plans, goals, and inferences were continuously applied during the conversation rather than in separate stages as previously. The program was applied to analysing international issues and could interpret events and advise from either a liberal or conservative point of view.
2. FRUMP:
This was the acronym for Fast Reading, Understanding and Memory Program and was the thesis work of Gerry De-Jong (1982). It was essentially a highly turned and optimised versin of POLITICS which was hooked up to the UPI news service. It rapidly read stories from the wire service and produced summaries of the stories in several languages. FRUMP was Schank’s first attempt to build a commercial program.
3. IPP:
This Integrated Partial Parser was an extension of FRUMP dedicated to the analysis of news stories on terrorism with the additional ability to learn from what it read. While FRUMP used only scripts, IPP used plans, goals, scripts, and had a memory.
4. BORIS:
BORIS was a Better Organised Reasoning and Inference System which was the thesis work of Michael Dyer under the supervision of Professor Wendy Lehnert of Yale University.
As Schank explains it, “We wanted also to see how we could improve our general story understanding capabilities by using a new set of memory structures we were devising, ones more suited to facilitate learning by cross- contextual understanding. We created a more fully integrated program that relied on a model of human belief and interactions that we could use in understanding little melodramas”.
5. CYRUS:
CYRUS stands for Computerised Yale Reasoning and Understanding System. It was written by Janet Kolodner as part of her thesis work and had some remarkable capabilities and accomplishments.
It was the culmination of the previous series of CD-based programs and had the following features:
1. It was an attempt to model the memory of a particular individual, the diplomat Cyrus Vance.
2. It could learn, that is, continuously change on the basis of new experience.
3. It continually recognized itself to best reflect what it knew. This feature resembles, the human capability of ‘self-awareness’.
4. It had the capability of ‘guessing’ about events of which it had no direct knowledge.
To illustrate this last feature, CYRUS was asked if his wife had ever met Mrs. Begin. The program searched for social occasions to which it was likely that both Cyrus Vance and Prime Minister Begin had brought their wives. If such a case were found, it was likely that the wives had met.
It did find such an occasion and answered:
“Yes, most recently at a state dinner in Israel in Jan. 1980”, both the program’s guess and the assumptions on which it was based turned out to be correct.
6. MORRIS:
This program is a Moral and Reminding Inference System written by Michael Dyer as an extension of his previous program, BORIS. MORRIS is intended to read a story in depth and perform a careful analysis of the appropriateness of character actions.
As a result of this analysis it extracts the moral of the story in terms of abstract planning advice, and uses this moral as an indexing structure for storage of the story in long-term episodic memory. Whenever a later story is read which can be analysed in terms of the same planning advice, MORRIS is reminded of the prior story. The goal of MORRIS is the ability to express this advice in terms of an appropriate cultural saying or adage.
Applications of Natural Language Processing:
Our intuition tells us that there are a great many applications in which Natural Language Processing will become the standard I/O technique, particularly in those cases where management personnel and other professionals with minimal computer literacy need direct access to information. Is it correct, then, to assume that almost all applications will eventually shift to natural language interfaces. Not necessarily.
One of the main goals of adding intelligence to machines is to communicate effectively, but not necessarily in natural language. There are a number of applications for which alternative I/O protocols will prove more efficient than natural language processors.
These include:
1. Applications such as word processors and spread sheets with specialised command structure in which key-or menu-driven I/O is far faster and more natural.
2. Sophistically, object driven graphical operating systems such as Windows and LINUX.
3. Graphically oriented application programs such as CAD/CAM systems requiring manual dexterity such as a car driving or playing a video game.
So it, can be said that despite its ambiguity natural language processing will play a very important role in future computer systems but not to the exclusion of more efficient communication techniques.
Natural Language processing techniques will, however most certainly eliminate the more obscure, arcane systems and job control languages as the burden of effective communication between humans and computers shifts towards more intelligent computer interfaces.