In linguistic semantics, cognitive psychology, and knowledge representation most of the necessary data concerning lexical, semantic and/or external world information is still provided introspectively. Researchers are exploring (or make test-persons explore) their own linguistic/cognitive capacities and memory structures to depict their findings (or let hypotheses about them be tested) in various representational formats (lists, arrays, trees, nets, active networks, etc.). It is widely accepted that model structures resulting from these analyses do have a more or less ad hoc character and tend to be confined to their limited theoretical or operational performances within a specified subject domain and/or implemented system. Thus, these approaches - by definition - can only map what of the world's fragment under investigation is already known to the analysts, not, however, what of it might be conveyed in texts unknown to them. Being basically interpretative and in want of operational control, such knowledge representations will not only be restricted quite naturally to undisputed informational structures which consequently can be mapped in accepted and well established (concept-hierarchical, logically deductive) formats, but they will also lack the flexibility and dynamics of more constructive model structures which are needed for automatic meaning analysis and representation from input texts to allow for a component to build up and/or modify a system's own knowledge, however shallow and vague that may appear compared to human understanding.
Other than these more orthodox lines of introspective data acquisition in meaning and knowledge representation research, the present approach has been based on the algorithmic analysis of discourse that real speakers/writers produce in actual situations of performed or intended communication on a certain subject domain. The approach makes essential use of procedural means to map fuzzy word meanings and their connotative interrelations in the format of conceptual stereotypes. Their varying dependencies constitute dynamic dispositions that render only those concepts accessible which may - within differing contexts differently - be considered relevant under a specified perspective or aspect. Thus - under the notion of lexical relevance and semantic disposition - a new meaning relation may operationally be defined between elements in a conceptual representation system which in itself may empirically be reconstructed from natural language discourse. Such dispositional dependency structures would seem to be an operational prerequisite to and a promising candidate for the simulation of content-driven (analogically-associative) instead of formal (logically-deductive) inferences in semantic processing.
In view of an introductory illustration rather than a detailed and qualifying discussion, some of the standard concept and/or word-meaning representational formats in memory models and knowledge systems will be compared (1) in order to motivate our rather strict departure from them in developing and using (2) some statistical means for the analysis of texts and the representation of the data obtained which will briefly be introduced as the semantic space model. Starting from the notion of priming and spreading activation in memory as a cognitive model for comprehension processes, we will (3) deal with our procedural method of representing semantic dispositions by way of inducing a relation of lexical relevance among labeled concept representations in semantic space. Concluding (4), two or three problem areas connected with word meaning and concept processing will be touched which might be tackled anew and perhaps be brought to a more adequate though still tentative solution under an empirically founded approach in procedural semantics.
Here we have a hierarchy of labeled concept nodes with predicates and properties linked to them which are inherited by directly dependent nodes. The hypotheses formulated and tested in experiments predict that testpersons will take more time to identify and decide given propositions with an increasing number of node- and level-transitions to be processed in the course of interpretation. Evaluating a sentence like "A canary can sing" will take less time than to decide whether the sentence "A robin can breathe" is true or not. Thus, reaction-time serves as an indicator for the proposed model structure either to be correct or in need of modification.
In early artificial intelligence research a different type of knowledge representation was developed for question-answering-systems. A fragment of the most common schema of the semantic network type (e.g. Winograd, 1975) is shown in Fig. 1.2. Here again we have labeled concept nodes linked to one another by pointers representing labeled relations which form a network instead of a tree structure. This enables the system to answer questions like: "Is Susy a cat?" correctly by identifying the SUSY-node, its ISA-relation pointer and the CAT-node. Moreover, the pointer structure allows for the processing of paths laid through the network, initiated by questions like: "Susy, cat?" which will prompt the answer "Susy is a cat. Cats eat fish. Cat is an animal. Fish is an animal."
A schematic representation of concept relatedness as envisaged by cognitive theorists who work along more procedural lines of memory models (Collins & Loftus, 1975) is shown in Fig. 1.3. Their distance-relational conception lends itself readily to the notion of stereotype representation for concepts that do not have intersubjectively identifiable sharp boundaries (Rosch, 1975).
Instead of binarily decidable category membership, stereotypical concepts or prototypes are determined by way of their adjacency to other prototypes. Taken as a memory model, stimulation of a concept will initiate spreading activation to prime the more adjacent concepts more intensely than those farther away in the network structure, thus determining a realm of concepts related by their primed semantic affinity. In the given example, the stimulation of the concept-node MANAGEMENT will activate that of BUSINESS first, then INDUSTRY and ORGANISATION with about the same intensities, then ADMINISTRATION and so on, with the intensities decreasing as a function of the activated nodes' distances.
These three schemata of model structures - although obviously concerned with the simulation of symbol understanding processes - are designed to deal primarily with static aspects of meaning and knowledge. Thus, in interpreting input symbols/strings, pre-defined/stored meaning relations and constructions can be identified and their representations be retrieved. Without respective grounding made explicit and represented in that structure, however, possibly distorted and/or modified instantiations of such relations or relevant supplementary semantic information can hardly be recognized or be provided within such representational systems. As the necessary data is not taken from natural language discourse in communicative environments but elicited in experimental settings by either exploring one's own or the testpersons' linguistically relevant cognitive and/or semantic capacities, usage similarities of different and/or contextual variations of identical items are difficult to be ascertained. This is rather unsatisfactory from a linguist's point-of-view who thinks that his discipline is an empirical one and, hence, that descriptive semantics ought to be based upon linguistic data produced by real speakers/hearers in factual acts of communicative performance in order to let new meaning representations (or fragments of them) replace (or improve) older ones and change/update a static memory structure.
The empirical analysis of discourse and the formal representation of vague word meanings in natural language texts as a system of interrelated concepts is based on the Wittgensteinian assumption that a great number of texts analysed for the terms' usage regularities will reveal essential parts of the concepts and hence the meanings conveyed.
The statistics which have been used so far for the quantitative analysis not of propositional strings but of their elements, namely words in natural language texts, is basically descriptive. Developed from and centred around a correlational measure to specify intensities of co-occurring lexical items used in natural language discourse, these analysing algorithms allow for the systematic modelling of a fragment of the lexical structure constituted by the vocabulary employed in the texts as part of the concomitantly conveyed world knowledge.
A correlation coefficient appropriately modified for the purpose has been used as a mapping function (Rieger, 1981a). It allows to compute the relational interdependency of any two lexical items from their textual frequencies. Those items which co-occur frequently in a number of texts will positively be correlated and hence called affined, those of which only one (and not the other) frequently occurs in a number of texts will negatively be correlated and hence called repugnant. Different degrees of word-repugnancy and word-affinity may thus be ascertained without recurring to an investigator's or his testpersons' word and/or world knowledge (semantic competence), but can instead solely be based upon the usage regularities of lexical items observed in a corpus of pragmatically homogeneous texts, spoken or written by real speakers/hearers in actual or intended acts of communication (communicative performance).
Following a holistic approach to system description and taking each word employed as a potential descriptor to characterize any other word's virtual meaning, the modified correlation coefficient can be used to map each lexical item into fuzzy subsets (Zadeh, 1965) of the vocabulary according to its numerically specified usage regularities. Measuring the differences of any one's lexical item's usages, represented as fuzzy subsets of the vocabulary, against those of all others allows for a consecutive mapping of items onto another abstract entity of the theoretical construct. These new operationally defined entities - called an item's meanings - may verbally be characterized as a function of all the differences of all regularities any one item is used with compared to any other item in the same corpus of discourse.
The resulting system of sets of fuzzy subsets constitutes the semantic space. As a distance-relational datastructure of stereotypically formatted meaning representations it may be interpreted topologically as a hyperspace with a natural metric. Its linguistically labeled elements represent meaning points, and their mutual distances represent meaning differences. The position of a meaning point may be described by its semantic environment. This is determined by those other points in the semantic hyperspace which - within a given diameter - are most adjacent to the central one chosen to be illustrated. Fig. 2.1 shows the topological environment EáGESCHAEFTñ, i.e. those points being situated within the hypersphere of a certain diameter of the meaning point GESCHAEFT/business as computed from a corpus of German newspaper texts comprising some 8000 tokens of 360 types in 175 texts from the 1964 editions of the daily DIE WELT.
Having checked a great number of environments, it was ascertained that they do in fact assemble meaning points of a certain semantic affinity. Further investigation revealed that there are regions of higher point density in the semantic space, forming clouds and clusters. These were detected by multivariate and cluster-analyzing methods which showed, however, that the both paradigmatically and syntagmatically related items formed what may be named connotative clouds rather than what is known to be called semantic fields. Although its internal relations appeared to be unspecifiable in terms of any logical deductive or concept hierarchical system, their elements' positions revealed a high degree of stable structures which suggested a regular form on contents-dependent associative relatedness (Rieger, 1981b, 1983).
Taking up the heuristics provided by Spreading Activation Theory in semantic memory, cognitive structures, and concept representation as advanced by Quillian (1968), Olson (1970) and Collins and Loftus (1975), the notion of spreading activation can be employed not only to denote activating related concepts in the process of priming studied in subsequent publications (e.g. Lorch, 1982; Flores d'Arcais & Jarvella, 1983) but - generically somewhat prior to that - may also signify the very procedure which induces these relations between concepts. Originally developed as a procedural model to cope with observed latencies of activated concepts in comprehension processes, priming and spreading activation is based on network-type models or world-knowledge structures as illustrated briefly above. Essentially defined by nodes, representing concepts, meanings or objects, and pointers which relate them conceptually, semantically or logically to one another, these formats have a considerable advantage over the semantic space structure outlined above: one of the problems of distance-like data structures in semantic processing is that - distance being a symmetric relation - well-known search strategies for retrieval, matching and inferencing purposes cannot be applied because these are based upon some non-symmetric relations, as realized by pointer structures in well-known meaning and/or world knowledge representations.
In order to make such procedures operate on the semantic space data, its structure has to be transformed into some hierarchical organisation of its elements. For this purpose, the semantic space model has to be re-interpreted as a sort of conceptual raw data and associative base structure. What appeared to be a disadvantage first, now turns out to be an advantage over more traditional formats of representation. Other than these approaches which have to presuppose the structural format of the semantic memory models that are to be tested in word recall and/or concept recognition experiments, the semantic space provides some of the necessary data for the procedural definition of dynamic, instead of static model structures that allow variable stereotype instead of fixed categorial concept representations. Thus, the concept nodes as abstract mappings of meanings of lexical items are not just linked to one another according to what cognitive scientists supposedly know about the way conceptual information is structured in memory, but it is this very structure that is already considered to be a dynamic format of stereotype concept organization. Defined as procedures that operate on the semantic space data, this is tantamount to a dynamic restructuring of meaning points - and depending on the controlling parameters - the generation of paths between them along which - in case of priming - activation might spread whenever a meaning point is stimulated.
Unlike the ready-set and fixed relations among nodes, an algorithm has been devised which operates on the semantic space data structure as its base to induce dependencies between its elements, i.e. among subsets of the meaning points. The recursively defined procedure detects fragments of the semantic space according to the meaning point it is started with and according to the semantic similarities, i.e. the distance relations it encounters during operation, constituting what we termed semantic relevance. Stop-conditions may deliberately be formulated either qualitatively (naming a target point) or quantitatively (number of points to be processed).
Given one meaning point's position as a start, the algorithm will - other than in Rieger (1982) - first list all its neighbouring points by increasing distances, second provide similar lists for each of these neighbours, and third prime the starting point as dominant node to mark the tree's root. Then, the algorithm's generic procedure will take the first entry from the first list, determine from the appropriate second list its most adjacent neighbour among those points already primed, in order to identify it as the ancestor (mother-node) to which the new descendant (daughter-node) is linked whose label then gets deleted from the first list. Repeated successively for each of the meaning points listed and in turn primed in accordance with this procedure, the algorithm will select a particular fragment of the relational structure latently inherent in the semantic space data under a certain perspective, i.e. the aspect or initially primed meaning point the algorithm is started with. Working its way through and consuming all labeled points in the space structure - unless stopped under conditions of given target points, number of points to be processed, or threshold of maximal distance - the algorithm transforms prevailing similarities of meanings as represented by adjacent points to establish - in the process of priming - a binary, non-symmetric, and transitive relation between them. This relation allows for the hierarchical re-organization of meaning points as descendant nodes under a primed head or root in an n-ary DDS-tree (Rieger, 1984). Weighted numerically as a function of a node's distance values and level of its tree-position, this measure either expresses a concept's dependencies as given by the root's descendants in that tree, or, inversely, it evaluates their criterialities for that concept as specified and determined by that tree's root. Without introducing the algorithms formally, some of their operative characteristics can well be illustrated in the sequel by a few simplified examples. Beginning with the schema of a distance-like data structure as shown in the two-dimensional configuration of 11 points, labeled a to k, in Fig. 3.1 the stimulation of three different starting points a, b and c results in the dependency structures which the algorithm of least distance selects (Fig. 3.2) as distance detection (first row), then as n-ary tree representation (second row) and as its binary-tree transformation (third row) of respective points related to be primed. It is apparent that stimulation of other points within the same configuration of basic data points will result in similar but nevertheless differing trees, depending on the aspect under which the structure is accessed, i.e. the point initially stimulated to start the algorithm with.
Applied to the semantic space data of 360 defined meaning points calculated from the textcorpus of the 1964 editions of the German newspaper DIE WELT, the Dispositional Dependency Structures (DDS) of AUFTRAG/order and GESCHAEFT/business are given in Figs. 3.3 and 3.4 as generated by the procedure described.
Different stop conditions given for the generation of the DDS resulted in different trees: DDSáAUFTRAGñ qualitative stop by target node GESCHAEFT, grade 7, depth 13, 64 nodes; and DDSáGESCHAEFTñ quantitative stop by number of nodes processed, grade 4, depth 10, 60 nodes. In the DDSáAUFTRAGñ (Fig. 3.3) we find only one descendant (LEIT/lead) on level 1, three as connotative alternates on level 2, one of which (ELEKTRON/electronic) has even 7 descendants on level 3, etc. In the DDSáGESCHAEFTñ (Fig. 3.4) there are two descendant connotative alternates (WERB/advertism; KENNTNIS/knowledge) on level 2, each of which has four descendants on level 3, etc. Attention is drawn to the dependencies of the direct descendants (BITTE/request) ® (PERSON/person) ® (HAUS/house). As in DDSáAUFTRAGñ this dependency is found in exactly the same order in the DDSáGESCHAEFTñ but here it is situated farther from the root, starting on the tree's sixth level only, instead of its third. To calculate such differences, a numerical measure of criteriality of a node with respect to its root or aspect was defined as a function of both, the distance values and the tree's levels concerned. For a wide range of purposes in processing DDS-trees, different criterialities of nodes can be used to estimate which paths are more likely being taken against others being followed less likely under priming of certain meaning points.
It goes without saying that generating DDS-trees is a prerequisit to source-oriented, contents-driven search and retrieval procedures which may thus be performed effectively on the semantic space structure. Given the meaning point ERFAHR/experience being stimulated, and GESCHAEFT/business as the target point to be searched for, then, the DDSáERFAHRñ will be generated as illustrated above, providing with decreasing criterialities the range of semantic dispositions inherent in the semantic space data under the aspect of, and triggered by the priming of ERFAHR/experience. The tree generating process being stopped after hitting and incorporating as its last node the target item, its dependency path will be activated. This is to trace those intermediate nodes which determine the associative transitions of any target node under any specifiable aspect. Looking up GESCHAEFT/business as a target node under the aspect of AUFTRAG/order, its dependency path (in Fig. 3.5) consists of WERBUNG/advertism, BITTE/request and TECHNIK/technic, FAEHIG/capable, ELKTRON/electronic, LEIT/lead which - not surprisingly though - proves to be approximately the dependency path of AUFTRAG/order under the aspect of GESCHAEFT/business but in inverted order and FAEHIG/capable replaced by COMPUTER/compute, DIPLOM/diploma, and UNTERRICHT/instruct (in Fig. 3.5).
Using source-oriented search and retrieval processes as described, an analogical, contents-driven form of inference - as opposed to logical deduction - may operationally be devised by way of parallel processing of two (or more) dependency-trees. For this purpose the algorithms are started by the two (or more) meaning points considered to represent the premises, of say, AUFTRAG/order and GESCHAEFT/business. Their DDS-trees will be generated before the inferencing procedure begins to work its way (breadth-first or depth-first) through both (or more) trees, tagging each encountered node. When in either tree the first node is met that has previously been tagged by activation from another priming source, the search procedure stops to activate the dependency paths from this concluding common node - in our case FAEHIG/capable for breadth-first and DIPLOM/diploma for depth-first searches - in the DDS-trees concerned and separately presented in Figs. 3.6 and 3.7.
Applied to any distance-like data structures of knowledge and/or meaning representation systems, the DDS-procedure allows for the generation of possible paths of spreading activation which branch across semantic space, submitting relevant portions of it to associatively guided search strategies and retrieval operations.
Replacing the storage of fixed and ready-set networks by a contents-driven induction of relevance related nodes, the problem of identifying stored meaning constructions with distorted instantiations of them, can be circumvented. Triggered by any identifiable label, the DDS will be generated according to the database provided and the resultant tree-structure will therefore vary according to the possibly varying status of the data in the semantic space structure.
In view of tacit knowledge and implied information the DDS-procedure offers an empirically based approach and a dynamic representation of semantic dispositions which - in language understanding systems - might serve as connotative default values in identifying and/or interpreting input labels and solving ambiguity and/or vagueness problems of input strings.
Among other extensions, it is hoped to develop a numerical expression for measuring the amount of meaning conveyed by any string interpreted by DDS-processing. Other than the probabilities calculated from empirical distributions of sets of symbols that have to be finite in classical information theory, the measure of meaning will have to be based upon structural properties of open sets and dynamically organized systems of symbols instead.
1This paper presents some results of a project in Computational Semantics of the MESY-research group at the German Department of the Technical University of Aachen, West Germany, supported by the NEW Ministry of Science and Research under grant IV A 2-FA 8600. The project is concerned with the development of means for the automatic construction of fuzzy semantic and associative knowledge representation systems from natural language discourse input. As central aspects have also been reported in a paper presented on the 3rd Intern. Conference on Databases in the Humanities and Social Sciences (ICDBHSS/83) at Rutgers University (Rieger, 1984), these aspects are taken up in some parts here again. Published in: Hoppenbrouwers, G./Seuren, P./Weijters, A. (Eds.): Meaning and the Lexicon. Dordrecht (Foris Publications) 1985, pp. 387-400.