1 An ecological approach to semiotics
Life may be understood as the ability to survive by adapting to changing requirements in the real world. Living systems do so by way of processing information they receive or derive from relevant portions of their surrounding environments, of learning from their experience, and of changing their behaviour accordingly. In contrast to other living systems which transmit experiencial results of environmental adaptation only biogenetically1 to their descendants, human information processing systems have additional means to convey their knowledge to others. In addition to the vertical transmission of system specific (intraneous ) experience through (biogenetically successive) generations, mankind has complementally developed horizontal means of mediating specific and foreign (extraneous ) experience and knowledge to (biogenetically unrelated) fellow systems within their own or any later generation. This is made possible by a semiotic move that allows not only to distinguish processes from results of experience but also to convert the latter to knowledge facilitating it to be re-used, modified and improved in learning. Vehicle and medium of this move are representations, i.e. complex sign systems which constitute languages and form structures, called texts which may be realized in communicative processes, called actualisation.In terms of the theory of information systems, texts -whether internal or external to the systems-function like virtual environments2. Considering the system-environment relation, virtuality may be characterized by the fact that it dispenses with the identity of space-time coordinates for system-environment pairs which normally prevails for this relation when qualified to be indexed real.
It appears, that this dispensation of identity (space-time-dispensation, for short) is not only conditional for the possible distinction of (mutually and relatively independent) systems from their environments, but establishes also the notion of representation.
Accordingly, immediate or space-time-identical system-environments
existing in their space-time-identity may well be distinguished from mediate or space-time-dispensed
system-environments whose particular representational form (texts )
corresponds to their particular status both, as language material (being signs ),
and as language structure (having meaning ). This double identity
calls for a particular modus of actualisation (understanding )
that may be characterized as follows:
For systems appropriately adapted and tuned to such environments
actualisation consists essentially in a twofold embedding to realize
the space-time-identity of pairs of immediate system-environment coordinates which will let the system experience the material properties of texts as signs (i.e. by functions of physical access and mutually homomorphic appearance). These properties apply to the percepts of language structures presented to a system in a particular discourse situation, and the representational identity of pairs of mediate system-environment parameters which will let the system experience the semantic properties of texts as meanings (i.e. by functions of emergence, identification, organisation, representation of structures). These apply to the comprehension of language structures recognized by a system to form the described situation.
Hence, according to the theory of information systems, functions like interpreting signs and understanding meanings translate to processes which extend the fragments of reality accesssible to a living (natural and possibly artificial) information processing system. This extension applies to both, the immediate and mediate relations a system may establish according to its own evolved adaptedness or dispositions (i.e. innate and acquired structuredness, processing capabilities, represented knowledge ).
The actualisation of environments, however, does not merely add to the amount of experiencial results, but constitutes instead a significant change in experiencial modus. This change is characterized by the fact that only now the processes of experience may be realized as being different and hence be separated from the results of experience which may thus even be represented, other than in immediate system-environments where result and process of experience appear to be indistinguishable. Splitting up experience in experiencial processes and experiencial results-the latter being representational and in need for actualisation by the former-is tantamount to the emergence of virtual experiences which have not to be made but can instead just be tried, very much like hypotheses in an experimental setting of a testbed. These results -like in immediate system-environments-may become part of a system's adaptive knowledge but may also-different from immediate system-environments-be neglected or tested, accepted or dismissed, repeatedly actualized and re-used without any risk for the system's own survival, stability or adaptedness.
The experimental quality of textual representations which increases the potentials of adaptive information processing immensely, will have to be constrained simultaneously by dynamic structures, corresponding to knowledge. The built-up, employment, and modification of these structural constraints3 is controlled by procedures whose processes determine cognition and whose results constitute adaptation. Systems properly adapted to textual system-environments have acquired these structural constraints (language knowledge) and can perform certain operations efficiently on them (language understanding). These are prerequisites to recognizing mediate (textual) environments and to identify their need for and the systems' own ability to actualize the mutual (and trifold) relatedness constituting what Peirce called semiosis 4.
Systems capable of and tuned to such knowledge-based processes of actualisation will in the sequel be referred to as semiotic cognitive information processing systems (SCIPS).
2 Language and cognition
Perception, identification, and interpretation of (external or internal) structures may be conceived as some form of information processing which (natural or artificial) cognitive systems-due to their own structuredness-are able to perform. Under this unifying paradigm for cognition, research programs in cognitive linguistics and cognitive language processing can roughly be characterized to consist of subtle forms in confronting models of competence theory of language with observable phenomena of communicative language performance to explore the structure of mental activities believed to underlie language learning and understanding by way of modelling these activities procedurally to enable algorithmic implementation and testing by machine simulation.Whereas traditional approaches in artificial intelligence research (AI ) or computational linguistics (CL ) model cognitive tasks or natural language understanding in information processing systems according to the realistic view of semantics, it is argued here that meaning need not be introduced as a presupposition of semantics but may instead be derived as a result of procedural modelling5 as soon as a semiotic line of approaches to cognition will be followed.
2.1 Understanding: situations
The present approach is based upon a phenomenological (re-)interpretation of the formal concept of situation and the analytical notion of language game. The combination of both lends itself easily to operational extensions in empirical analysis and procedural simulation of associative meaning constitution which will grasp essential parts of the process of understanding.According to Situation Semantics any language expression is tied to reality in two ways: by the discourse situation allowing an expression's meaning being interpreted and by the described situation allowing its interpretation being evaluated truth-functionally. Within this relational model of semantics, meaning may be considered the derivative of information processing which (natural or artificial) systems-due to their own structuredness-perform by recognizing similarities or invariants between situations that structure their surrounding realities (or fragments thereof).
By ascertaining these invariants and by mapping them as uniformities across situations, cognitive systems properly attuned to them are able to identify and understand those bits of information which appear to be essential to form these systems' particular views of reality: a flow of types of situations related by uniformities like e.g. individuals, relations, and time-space-locations. These uniformities constrain a system's external world to become its view of reality as a specific fragment of persistent (and remembered) courses of events whose expectability renders them interpretable or even objective.
In semiotic sign systems like natural languages, such uniformities appear to be signalled also by word-types whose employment as word-tokens in texts exhibit a special form of structurally conditioned constraints. Not only allows their use the speakers/hearers to convey/understand meanings differently in different discourse situations (efficiency), but at the same time the discourses' total vocabulary and word usages also provide an empirically accessible basis for the analysis of structural (as opposed to referencial ) aspects of event-types and how these are related by virtue of word uniformities accross phrases, sentences, and texts uttered. Thus, as a means for the intensional (as opposed to the extensional) description of (abstract, real, and actual) situations, the regularities of word-usages may serve as an access to and a representational format for those elastic constraints which underly and condition any word-type's meaning, the interpretations it allows within possible contexts of use, and the information its actual word-token employment on a particular occasion may convey.
2.2 Communicating: language games
The notion of language games "complete in themselves, as complete systems of human communication'' is primarily concerned with the way of how signs are used ''simpler than those in which we use the signs of our highly complicated everyday language". Operationalizing this notion and analysing a great number of texts for usage regularities of terms can reveal essential parts of the concepts and hence the meanings conveyed by them. This approach has also produced some evidence that an analytical procedure appropriately chosen could well be identified also with solving the representational task if based upon the universal constraints known to be valid for all natural languages.The philosophical concept of language game can be combined with the formal notion of situations allowing not only for the identification of an cognitve system's (internal ) structure with the (external ) structure of that system's environment. Being tied to the observables of actual language performance enacted by communicative language useage opens up an empirical approach to procedural semantics. Whatever can formally be analysed as uniformities in Barwiseian discourse situations may eventually be specified by word-type regularities as determined by co-occurring word-tokens in pragmatically homogeneous samples of language games. Going back to the fundamentals of structuralistic descriptions of regularities of syntagmatic linearity and paradigmatic selectivity of language items, the correlational analyses of discourse will allow for a multi-level word meaning and world knowledge representation whose dynamism is a direct function of elastic constraints established and/or modified in language communication.
As has been outlined in some detail elsewhere the meaning function's range may be computed and simulated as a result of exactly those (semiotic) procedures by way of which (representational) structures emerge and their (interpreting) actualisation is produced from observing and analyzing the domain's regular constraints as imposed on the linear ordering (syntagmatics ) and the selective combination (paradigmatics ) of natural language items in communicative language performance. For natural language semantics this is tantamount to (re)present a term's meaning potential by a fuzzy distributional pattern of the modelled system's state changes rather than a single symbol whose structural relations are to represent the system's interpretation of its environment. Whereas the latter has to exclude, the former will automatically include the (linguistically) structured, pragmatic components which the system will both, embody and employ as its (linguistic) import to identify and to interpret its environmental structures by means of its own structuredness.
3 Knowledge and representation
In knowledge based cognitive linguistics and semantics, researchers get the necessary lexical, semantic, or external world information by exploring (or making test-persons explore) their own linguistic or cognitive capacities and memory structures in order to depict their findings in (or let hypotheses about them be tested on the bases of) traditional forms of knowledge representation. Being based upon this pre-defined and rather static concept of knowledge, these representations are confined not only to predicative and propositional expressions which can be mapped in well established (concept-hierarchical, logically deductive) formats, but they will also lack the flexibility and dynamics of re-constructive model structures more reminiscent of language understanding and better suited for automatic analysis and representation of meanings from texts. Such devices have been recognized to be essential for any simulative modelling capable to set up and modify a system's own knowledge structure, however shallow and vague its semantic knowledge and inferencing capacity may appear compared to human understanding. The semiotic approach argued for here appears to be a feasible alternative focussing on the dynamic structures which the speakers'/hearers' communicative use of language in discourse will both, constitute and modify, and whose reconstruction may provide a paradigm of cognition and a model for the emergence of meaning. In a corresponding meaning representation formalism has been defined and tested whose parameters may automatically be detected from natural language texts and whose non-symbolic and distributional format of a vector space notation allows for a wide range of useful interpretations.
3.1 Quantitative text analysis
Based upon the fundamental distinction of natural language items' agglomerative or syntagmatic and selective or paradigmatic relatedness, the core of the representational formalism can be characterized as a two-level process of abstraction. The first (called a-abstraction) on the set of fuzzy subsets of the vocabulary provides the word-types' usage regularities or corpus points, the second (called d-abstraction) on this set of fuzzy subsets of corpus points provides the corresponding meaning points as a function of word-types which are being instantiated by word-tokens as employed in pragmatically homogeneous corpora of natural language texts.The basically descriptive statistics used to grasp these relations on the level of words in discourse are centred around a correlational measure (Eqn. 1 ) to specify intensities of co-occurring lexical items in texts, and a measure of similarity (or rather, dissimilarity) (Eqn. 4) to specify these correlational value distributions' differences. Simultaneously, these measures may also be interpreted semiotically as set theoretical constraints or formal mappings (Eqns. 2 and 5) which model the meanings of words as a function of differences of usage regularities.
ai,j allows to express pairwise relatedness of word-types (xi,xj) Î V ×V in numerical values ranging from -1 to +1 by calculating co-occurring word-token frequencies in the following way
where eit=[(Hi)/L] ltand
ejt=[(Hj)/L] lt, with
the textcorpus
K={ kt } ; t=1,¼,T
having an overall length
L=åt=1T lt; 1 £ lt £ L
measured by the number of word-tokens per text, and a vocabulary
V={ xn } ; n=1,¼,i,j,¼,N
whose frequencies are denoted by
Hi=åt=1Thit ; 0 £ hit £ Hi.
Evidently, pairs of items which frequently either co-occur in, or are both
absent from, a number of texts will positively be correlated and hence called
affined, those of which
only one (and not the other) frequently occurs in a number of texts
will negatively be correlated and hence called repugnant.
As a fuzzy binary relation,
[(a)\tilde] : V×V ® I
can be conditioned on xn Î V which yields a crisp mapping
where the tupels á(xn,1,[(a)\tilde](n,1)),¼,(xn,N,[(a)\tilde](n,N))ñ
represent the numerically specified, syntagmatic
usage regularities that have been observed for each word-type xi
against all other xn Î V. a-abstraction
over one of the components in each ordered pair defines
Hence, the regularities of usage of any lexical item will be
determined by the tupel of its affinity/repugnancy -values towards
each other item of the vocabulary which - interpreted as coordinates -
can be represented by points in a vector space C
spanned by the number of axes each of which corresponds to an entry in the
vocabulary.
where the tupels
Identifying zn Î S with the numerically specified elements of
potential paradigms, the set of possible combinations S ×S
may structurally be constrained and evaluated without (direct
or indirect) recourse to any pre-existent external world.
Introducing a Euclidian metric
the hyperstructure áS,zñ or
semantic hyper space (SHS ) is declared constituting the
system of meaning points as an empirically founded
and functionally derived representation of a lexically labelled
knowledge structure (Tab. 1).
As a result of the two-stage consecutive mappings
any meaning point's position in SHS is determined by all the
differences (d- or distance-values) of all regularities
of usage (a- or correlation-values) each lexical item shows
against all others in the discourse analysed.
Without recurring to any investigator's or his test-persons'
word or world knowledge (semantic competence ), but solely on
the basis of usage regularities of lexical items in discourse resulting
from actual or intended acts of communication (communicative
performance ), text understanding is modelled procedurally the process
to construct and identify the topological
positions of any meaning point zi Î áS,zñ
corresponding to the vocabulary items xi Î V
which can formally be stated as composition of the two restricted relations
[(d)\tilde] | y and [(a)\tilde] | x (Fig. 1).
Processing natural language texts the way these algorithms do would appear
to grasp some interesting portions of the ability to recognize and represent
and to employ and modify the structural information available to and
accessible under such performance. A semiotic cognitive information
processing system (SCIPS) endowed with this ability and able to
perform likewise (Fig. 2) would consequently be said to have constituted some text
understanding. The problem is, however, whether (and if so, how) the
contents of what such a system is said to have acquired can be tested,
i.e. made accessible other than by the language texts in question and/or
without committing to a presupposed semantics determining possible
interpretations.
Figure 2:
Situational setting of SCIP system within
its environment
which is defined to allow for the system's view (Endo-Reality ) to
differ from the external observer's view (Exo-Reality ) by
keeping the system's (non-propositional) faculties of language processing
strictly apart from the (propositional) way of generating the environmental
language data as textual
descriptions. Note, that grammar (lexicon, syntax) and
semantics are not part of the system's knowledge base but are introduced
to specify and formally control the language environment the system is exposed
to as "true" descriptions of the external reality. Thus, the
the system's processing of these language data and its independently
built-up internal representations allow for a semantic
interpretation and visible imaging of the structures the system might have
acquired.
For the purpose of testing semiotic processes, their
situational complexity has to be reduced by abstracting away
irrelevant constituents, hopefully without oversimplifying the
issue and trivializing the problem. Therefore, the propositional
form of natural language predication will be used here only to
control the format of the natural language training material, not,
however, to determine the way it is processed to model
understanding.
Illustrating an example situation, the reference plane (
Fig. 2) shows two object-locations. These
have (automatically) been described in a corpus of language
expressions comprising some 12 432 word tokens of 26 word types in
2 483 sentences and 684 texts generated according to the formal
syntax and semantics specified for all possible system-
positions and orientations. The training set of
language material was then exposed to the SCIP system
which perceived it as environmental data to be processed according
to its system faculties as specified. It is worthwhile noting here
again, that this processing is neither based on, nor does it
involve any knowledge of syntax or semantics on
the system's side.
In the course of processing, the two-level consecutive mappings result
in the semantic hyper space (SHS) whose intrinsic
structure reveal some properties which can be made visible in a
three stage process:
The matrix Endo2m,n contains the data for an external observer's image of the
system's endo- view as computed from the described object
locations relative to system positions. The (two-dimensional)
scattergram of Endo2 gives an overall picture of even
referential likelihood by isoreferentials denoting
potential object locations quite clearly, under
crisp 1.0 (Fig. 1) and under
fuzzy 1.1 interpretation (Fig. 3).
The corresponding 3-dimensional profile representations of the
same patterns show in an even more detailed illustration the
higher referential resolution which fuzzy interpretations
of descriptive hedged core predications gain over crisp
ones.
The development of the above model of semiotic cognitive
information processing, however, will have to be elaborated in
at least three directions before the present SCIP- systems
may justifiably be named semiotic agents :
Some of these ideas are being followed and discussed just now in
an very early state of development however which still is
characteristic of all computational semiotics sofar.
3.2 Distributed meaning representation
Considering C as representational structure of
abstract entities constituted by syntagmatic regularities of
word-token occurrences in pragmatically homogeneous discourse, then
the similarities and/or dissimilarities of these entities will capture
their corresponding word-types' paradigmatic regularities.
These may be calculated by a distance measure d
of, say, Euclidian metric
represents the numerically specified paradigmatic structure
that has been derived for each abstract
syntagmatic usage regularity yj
against all other yn Î C. The distance values can therefore be
abstracted analogous to Eqn. 3, this time, however, over
the other of the
components in each ordered pair, thus defining an element zj Î S
called meaning point by
4 The experimental setting
To enable an intersubjective scrutiny, the (unknown) results of an abstract
system's (well known) acquisition
process is compared against the (well known) traditional interpretations of
the (unknown) processes of natural language meaning constitution6. To achieve
this, it had to be guaranteed
4.1 Positions and locations
The experimental setting consists of a two dimensional environment
with some objects at certain places that a SCIP- system will have to identify on the grounds of
natural language descriptions of system-position and
object-location relations it is exposed to. Although the system's
perception is limited to its (formal) language processing and
as its ability to act (and react) is restricted to pacewise
linear movement, what makes it semiotic is
that-whatever the system might gather from its
environment-it will not apply any coded knowledge
available prior to that process, but will instead only be confined
to the system's own (co- and contextually restricted)
susceptibility and processing capabilities to
(re-)organize the environmental data a n d to
(re-)present the results in some dynamic structure which
determines the system's knowledge (susceptibility), learning
(change) and understanding (representation). It is based on the
assumption that some deeper representational level or core
structure might be identified as a common base for different
notions of meaning developped sofar in theories of
referential and situational semantics as well as some
structural or stereotype semantics.
4.2 Process and result
The strict separation between the process and its result on the
system's side now corresponds to the sharp distinction between the
formal specification to control the propositional generation of
referentially descriptive language material and its
non-propositional processing within the experimental SCIP
setting.
The Endo1i,j data serves as base for the following third step of a line- and
column-wise transform which results in a new mapping
Endo2m,n according to the
summation equation
5 Conclusion
The paradigm of agentive systems seems to be particularly suited
for any multivariate form of dynamic interaction that leaves
traces of and is dependent on the results of such mutually
triggered activity. Natural language communication certainly is an
example for such a phenomenon whose traces in the form of texts
have to be actualized in order to let the processes believed to be
responsible for their production be inverted and experienced as
understanding. The concepts of situation and language
game have proved to be seminal in elucidating the structure and
compounds involved in the constitution of meaning.
References