Concrete Digital Computation: What Does it Take for a Physical System to Compute?

This paper deals with the question: what are the key requirements for a physical system to perform digital computation? Time and again cognitive scientists are quick to employ the notion of computation simpliciter when asserting basically that cognitive activities are computational. They employ this notion as if there was or is a consensus on just what it takes for a physical system to perform computation, and in particular digital computation. Some cognitive scientists in referring to digital computation simply adhere to Turing??s notion of computability. Classical computability theory studies what functions on the natural numbers are computable and what mathematical problems are undecidable. Whilst a mathematical formalism of computability may perform a methodological function of evaluating computational theories of certain cognitive capacities, concrete computation in physical systems seems to be required for explaining cognition as an embodied phenomenon. There are many non-equivalent accounts of digital computation in physical systems. I examine only a handful of those in this paper: (1) Turing??s account; (2) The triviality ??account??; (3) Reconstructing Smith??s account of participatory computation; (4) The Algorithm Execution account. My goal in this paper is twofold. First, it is to identify and clarify some of the underlying key requirements mandated by these accounts. I argue that these differing requirements justify a demand that one commits to a particular account when employing the notion of computation in regard to physical systems. Second, it is to argue that despite the informative role that mathematical formalisms of computability may play in cognitive science, they do not specify the relationship between abstract and concrete computation.     

The Linguistic Subversion of Mental Representation

Embedded and embodied approaches to cognition urge that (1) complicated internal representations may be avoided by letting features of the environment drive behavior, and (2) environmental structures can play an enabling role in cognition, allowing prior cognitive processes to solve novel tasks. Such approaches are thus in a natural position to oppose the ??thesis of linguistic structuring??: The claim that the ability to use language results in a wholesale recapitulation of linguistic structure in onboard mental representation. Prominent examples of researchers adopting this critical stance include Andy Clark, Michael Wheeler, and Mark Rowlands. But is such opposition warranted? Since each of these authors advocate accounts of mental representation that are broadly connectionist, I survey research on formal language computation in artificial neural networks, and argue that results indicate a strong form of the linguistic structuring thesis is true: Internal representational systems recapitulate significant linguistic structure, even on a connectionist account of mental representation. I conclude by sketching how my conclusion can nonetheless be viewed as consistent with and complimentary to an embedded/embodied account of the role of linguistic structure in cognition.     

An Analysis of the Criteria for Evaluating Adequate Theories of Computation

This paper deals with the question: What are the criteria that an adequate theory of computation has to meet? (1) Smith’s answer: it has to meet the empirical criterion (i.e. doing justice to computational practice), the conceptual criterion (i.e. explaining all the underlying concepts) and the cognitive criterion (i.e. providing solid grounds for computationalism). (2) Piccinini’s answer: it has to meet the objectivity criterion (i.e. identifying computation as a matter of fact), the explanation criterion (i.e. explaining the computer’s behaviour), the right things compute criterion, the miscomputation criterion (i.e. accounting for malfunctions), the taxonomy criterion (i.e. distinguishing between different classes of computers) and the empirical criterion. (3) Von Neumann’s answer: it has to meet the precision and reliability of computers criterion, the single error criterion (i.e. addressing the impacts of errors) and the distinction between analogue and digital computers criterion. (4) “Everything” computes answer: it has to meet the implementation theory criterion by properly explaining the notion of implementation.     

Computation and Dynamical Models of Mind

Van Gelder (1995) has recently spearheaded a movement to challenge the dominance of connectionist and classicist models in cognitive science. The dynamical conception of cognition is van Gelder's replacement for the computation bound paradigms provided by connectionism and classicism. He relies on the Watt governor to fulfill the role of a dynamicist Turing machine and claims that the Motivational Oscillatory Theory (MOT) provides a sound empirical basis for dynamicism. In other words, the Watt governor is to be the theoretical exemplar of the class of systems necessary for cognition and MOT is an empirical instantiation of that class. However, I shall argue that neither the Watt governor nor MOT successfully fulfill these prescribed roles. This failure, along with van Gelder's peculiar use of the concept of computation and his struggle with representationalism, prevent him from providing a convincing alternative to current cognitive theories.     

“Words lie in our way”

The central claim of computationalism is generally taken to be that the brain is a computer, and that any computer implementing the appropriate program would ipso facto have a mind. In this paper I argue for the following propositions: (1) The central claim of computationalism is not about computers, a concept too imprecise for a scientific claim of this sort, but is about physical calculi (instantiated discrete formal systems). (2) In matters of formality, interpretability, and so forth, analog computation and digital computation are not essentially different, and so arguments such as Searle's hold or not as well for one as for the other. (3) Whether or not a biological system (such as the brain) is computational is a scientific matter of fact. (4) A substantive scientific question for cognitive science is whether cognition is better modeled by discrete representations or by continuous representations. (5) Cognitive science and AI need a theoretical construct that is the continuous analog of a calculus. The discussion of these propositions will illuminate several terminology traps, in which it's all too easy to become ensnared.     

Levels of description and explanation in cognitive science

The notion of levels has been widely used in discussions of cognitive science, especially in discussions of the relation of connectionism to symbolic modeling of cognition. I argue that many of the notions of levels employed are problematic for this purpose, and develop an alternative notion grounded in the framework of mechanistic explanation. By considering the source of the analogies underlying both symbolic modeling and connectionist modeling, I argue that neither is likely to provide an adequate analysis of processes at the level at which cognitive theories attempt to function: One is drawn from too low a level, the other from too high a level. If there is a distinctly cognitive level, then we still need to determine what are the basic organizational principles at that level.     

Computationalism

Computationalism, the notion that cognition is computation, is a working hypothesis of many AI researchers and Cognitive Scientists. Although it has not been proved, neither has it been disproved. In this paper, I give some refutations to some well-known alleged refutations of computationalism. My arguments have two themes: people are more limited than is often recognized in these debates; computer systems are more complicated than is often recognized in these debates. To underline the latter point, I sketch the design and abilities of a possible embodied computer system.     

How minds can be computational systems

The proper treatment of computationalism, as the thesis that cognition is computable, is presented and defended. Some arguments of James H. Fetzer against computationalism are examined and found wanting, and his positive theory of minds as semiotic systems is shown to be consistent with computationalism. An objection is raised to an argument of Selmer Bringsjord against one strand of computationalism, namely, that Turing-Test-passing artifacts are persons, it is argued that, whether or not this objection holds, such artifacts will inevitably be persons.     

An introduction to learning

At a symbolic level cognition can be modelled as a production system where meaning units are represented as condition-action rules. Anderson (1982, 1987) provides a good example of how learning can occur with this type of knowledge representation. At a subsymbolic level cognition can be modelled with a connectionist network where meaning units are represented as patterns of parallel distributed activity. The work of the McClelland and Rumelhart (1986) group is a prototype of this approach. We elaborate on these two approaches to learning and contrast the symbolic search space paradigm with the connectionist paradigm.     

Connectionist meta-representation for propositional attitudes

Abstract

This paper discusses the representation of propositional attitudes (beliefs, etc.) in connectionist systems that do not implement symbolic representations. One prominent way of symbolically representing attitudes is through meta-representational schemes. These have representational expressions that themselves refer to representational expressions. Meta-representation is one of the most expressively powerful symbolic approaches for attitude representation. Therefore: could non-implementational connectionist systems use an analogous approach? Unfortunately, it is not straightforward to devise a plausible analogy. The paper looks at three main possibilities: (i) the representational activation patterns of the non-implementational connectionist system refer to the system's own activation patterns; (ii) the activation patterns refer to formal symbolic expressions; and (iii) the activation patterns refer to natural-language expressions. These approaches, which are not claimed to be exhaustive, concentrate respectively on the following facts about symbolic meta-representation schemes: (1) they typically refer to their own representational expressions; (2) the schemes typically refer to formal symbolic expressions; although (3) some of the schemes refer to natural language expressions. The article briefly argues that possibility (iii) avoids some of the problems of (i) and (ii). There are also two independent, non-connectionism-derived reasons for considering (iii). One is that it is strongly related to a prevalent commonsense metaphor of beliefs and so on as internal, natural language utterances. (The other is to do with the heightened difficulty of handling vague quantification within propositional attitude contexts, but is discussed elsewhere and not in the present paper.) The paper as a whole highlights the point that even a non-implementational connectionist system must be able to think about complex symbolic constructs such as logic expressions and natural language phrases, even though it does not think with them.     

Extension of grounding mechanism for abstract words: computational methods insights

The attempts to model cognitive phenomena effectively have split the research community in two paradigms: symbolic and connectionist. The extension of grounding phenomenon for abstract words is very important for social interactions of cognitive robots in real scenarios. This paper reviews the strength of symbolic and connectionist methods to address the abstract word grounding problem in cognitive robots. In particular, the presented work is focused on designing and simulating cognitive robotics model to achieve a grounding mechanism for abstract words by using the semantic network approach, as well as examining the utility of connectionist computation for the same problem. Two neuro-robotics models based on feed forward neural network and recurrent neural network are presented to see the pros and cons of connectionist approach. The simulation results and review of attributes of these methods reveal that the proposed symbolic model offers the solution to the problem of grounding abstract words with attributes like high data storage capacity with recall accuracy, structural integrity and temporal sequence handling. Whereas, connectionist computation based solutions give more natural solution to this problem with some shortcomings that include combinatorial ambiguity, low storage capacity and structural rigidity. The presented results are not only important for the advancement in communication system of cognitive robot, also provide evidence for embodied nature of abstract language.     

Interruptibility as a constraint on hybrid systems

It is widely mooted that a plausible computational cognitive model should involve both symbolic and connectionist components. However, sound principles for combining these components within a hybrid system are currently lacking; the design of such systems is oftenad hoc. In an attempt to ameliorate this we provide a framework of types of hybrid systems and constraints therein, within which to explore the issues. In particular, we suggest the use of system independent constraints, whose source lies in general considerations about cognitive systems, rather than in particular technological or task-based considerations. We illustrate this through a detailed examination of an interruptibility constraint: handling interruptions is a fundamental facet of cognition in a dynamic world. Aspects of interruptions are delineated, as are their precise expression in symbolic and connectionist systems. We illustrate the interaction of the various constraints from interruptibility in the different types of hybrid systems. The picture that emerges of the relationship between the connectionist and the symbolic within a hybrid system provides for sufficient flexibility and complexity to suggest interesting general implications for cognition, thus vindicating the utility of the framework.     

Sign use and cognition in automated scientific discovery: are computers only special kinds of signs?

James Fetzer criticizes the computational paradigm, prevailing in cognitive science by questioning, what he takes to be, its most elementary ingredient: that cognition is computation across representations. He argues that if cognition is taken to be a purposive, meaningful, algorithmic problem solving activity, then computers are incapable of cognition. Instead, they appear to be signs of a special kind, that can facilitate computation. He proposes the conception of minds as semiotic systems as an alternative paradigm for understanding mental phenomena, one that seems to overcome the difficulties of computationalism. Now, I argue, that with computer systems dealing with scientific discovery, the matter is not so simple as that. The alleged superiority of humans using signs to stand for something other over computers being merely “physical symbol systems” or “automatic formal systems” is only easy to establish in everyday life, but becomes far from obvious when scientific discovery is at stake. In science, as opposed to everyday life, the meaning of symbols is, apart from very low-level experimental investigations, defined implicitly by the way the symbols are used in explanatory theories or experimental laws relevant to the field, and in consequence, human and machine discoverers are much more on a par. Moreover, the great practical success of the genetic programming method and recent attempts to apply it to automatic generation of cognitive theories seem to show, that computer systems are capable of very efficient problem solving activity in science, which is neither purposive nor meaningful, nor algorithmic. This, I think, undermines Fetzer’s argument that computer systems are incapable of cognition because computation across representations is bound to be a purposive, meaningful, algorithmic problem solving activity.     

Genetic algorithms for data-driven web question answering

We present an evolutionary approach for the computation of exact answers to natural languages (NL) questions. Answers are extracted directly from the N-best snippets, which have been identified by a standard Web search engine using NL questions. The core idea of our evolutionary approach to Web question answering is to search for those substrings in the snippets whose contexts are most similar to contexts of already known answers. This context model together with the words mentioned in the NL question are used to evaluate the fitness of answer candidates, which are actually randomly selected substrings from randomly selected sentences of the snippets. New answer candidates are then created by applying specialized operators for crossover and mutation, which either stretch and shrink the substring of an answer candidate or transpose the span to new sentences. Since we have no predefined notion of patterns, our context alignment methods are very dynamic and strictly data-driven. We assessed our system with seven different datasets of question/answer pairs. The results show that this approach is promising, especially when it deals with specific questions.     

Syntactic systematicity in sentence processing with a recurrent self-organizing network

As potential candidates for explaining human cognition, connectionist models of sentence processing must demonstrate their ability to behave systematically, generalizing from a small training set. It has recently been shown that simple recurrent networks and, to a greater extent, echo-state networks possess some ability to generalize in artificial language learning tasks. We investigate this capacity for a recently introduced model that consists of separately trained modules: a recursive self-organizing module for learning temporal context representations and a feedforward two-layer perceptron module for next-word prediction. We show that the performance of this architecture is comparable with echo-state networks. Taken together, these results weaken the criticism of connectionist approaches, showing that various general recursive connectionist architectures share the potential of behaving systematically.     

An integrated symbolic/connectionist architecture for parsing italian sentences containing pp-attachment ambiguities

This paper describes a hybrid (symbolic/connectionist) system that performs PP-attachment disambiguation by taking advantage of three distinguishing features of neutral networks: distributed representation, functional compositionality, and inductive learning. The connectionist part of the system follows all the steps performed by the symbolic parser, and drives the parser's behavior by inducing a bias towards the most semantically plausible attachment choices. The sentence to be parsed is read one word at a time. When the symbolic parser has more than one production to apply, the connectionist module has already developed an inner representation of the sentence and a distribution of probabilities over the possible choices. The parser continues its work according to such a distribution.     

From Computer Metaphor to Computational Modeling: The Evolution of Computationalism

In this paper, I argue that computationalism is a progressive research tradition. Its metaphysical assumptions are that nervous systems are computational, and that information processing is necessary for cognition to occur. First, the primary reasons why information processing should explain cognition are reviewed. Then I argue that early formulations of these reasons are outdated. However, by relying on the mechanistic account of physical computation, they can be recast in a compelling way. Next, I contrast two computational models of working memory to show how modeling has progressed over the years. The methodological assumptions of new modeling work are best understood in the mechanistic framework, which is evidenced by the way in which models are empirically validated. Moreover, the methodological and theoretical progress in computational neuroscience vindicates the new mechanistic approach to explanation, which, at the same time, justifies the best practices of computational modeling. Overall, computational modeling is deservedly successful in cognitive (neuro)science. Its successes are related to deep conceptual connections between cognition and computation. Computationalism is not only here to stay, it becomes stronger every year.     

What the #$*%! is a Subsymbol?

In 1988, Smolensky proposed that connectionist processing systems should be understood as operating at what he termed the `subsymbolic' level. Subsymbolic systems should be understood by comparing them to symbolic systems, in Smolensky's view. Up until recently, there have been real problems with analyzing and interpreting the operation of connectionist systems which have undergone training. However, recently published work on a network trained on a set of logic problems originally studied by Bechtel and Abrahamsen (1991) seems to offer the potential to provide a detailed, empirically based answer to questions about the nature of subsymbols. In this paper, a network analysis procedure and the results obtained using it are discussed. This provides the basis for an insight into the nature of subsymbols, which is surprising.