On-line sensitivity analysis of Markov chains

Discrete-event systems modeled as continuous-time Markov processes and characterized by some integer-valued parameter are considered. The problem addressed is that of estimating performance sensitivities with respect to this parameter by directly observing a single sample path of the system. The approach is based on transforming the nominal Markov chain into a reduced augmented chain, the stationary-state probabilities which can be easily combined to obtain stationary-state probability sensitivities with respect to the given parameter. Under certain conditions, the reduced augmented chain state transitions are observable with respect to the state transitions of the system itself, and no knowledge of the nominal Markov-chain state of the transition rates is required. Applications for some queueing systems are included. The approach incorporates estimation of unknown transition rates when needed and is extended to real-valued parameters     

基于粗-模糊神经网络的决策控制

通过将模糊集和粗集,神经网络结合,提出了一种基于模糊规则的新的粗模糊神经网络,它通过利用误差反向传播算法实时修正该新型网络中的权值参数,从而能被有效地应用于不确定系统的决策分类与模式识别问题.最后通过对一个不确定决策系统的模式识别的仿真结果表明该粗模糊神经网络能大大提高模式识别决策的准确率.     

Generating transition probabilities to support model‐based software testing

Markov chain usage models support test planning, test automation, and analysis of test results. In practice, transition probabilities for Markov chain usage models are often specified using a cycle of assigning, verifying, and revising specific values for individual transition probabilities. For large systems, such an approach can be difficult for a variety of reasons. We describe an improved approach that represents transition probabilities by explicitly preserving the information concerning test objectives and the relationships between transition probabilities in a format that is easy to maintain and easy to analyze. Using mathematical programming, transition probabilities are automatically generated to satisfy test management objectives and constraints. A more mathematical treatment of this approach is given in References [ 1 ] (Poore JH, Walton GH, Whittaker JA. A constraint‐based approach to the representation of software usage models. Information and SoftwareTechnology 2000; at press) and [ 2 ] (Walton GH. Generating transition probabilities for Markov chain usage models. PhD Thesis, University of Tennessee, Knoxville, TN, May 1995.). In contrast, this paper is targeted at the software engineering practitioner, software development manager, and test manager. This paper also adds to the published literature on Markov chain usage modeling and model‐based testing by describing and illustrating an iterative process for usage model development and optimization and by providing some recommendations for embedding model‐based testing activities within an incremental development process. Copyright © 2000 John Wiley & Sons, Ltd.     

Information theoretic sensor data selection for active objectrecognition and state estimation

We introduce a formalism for optimal sensor parameter selection for iterative state estimation in static systems. Our optimality criterion is the reduction of uncertainty in the state estimation process, rather than an estimator-specific metric (e.g., minimum mean squared estimate error). The claim is that state estimation becomes more reliable if the uncertainty and ambiguity in the estimation process can be reduced. We use Shannon's information theory to select information-gathering actions that maximize mutual information, thus optimizing the information that the data conveys about the true state of the system. The technique explicitly takes into account the a priori probabilities governing the computation of the mutual information. Thus, a sequential decision process can be formed by treating the a priori probability at a certain time step in the decision process as the a posteriori probability of the previous time step. We demonstrate the benefits of our approach in an object recognition application using an active camera for sequential gaze control and viewpoint selection. We describe experiments with discrete and continuous density representations that suggest the effectiveness of the approach     

Additions of interactive fuzzy numbers

This paper provides an account of an approach to modeling unknown data by means of fuzzy-set theory, and addresses the problem of deriving the uncertainty, on a sum of variables whose values lie within fuzzy intervals. The first part is an extensive presentation of the theoretical background of the approach: the extension principle is stated in terms of possibility of an event; the concept of variable interaction is investigated at length. Section II gives new results regarding the effective practical computation of additions of fuzzy numbers. Its originality lies in the introduction of interaction which enables to control the growth of uncertainty, in calculations. Moreover, the problem of computing mathematical expectations with fuzzy probabilities is solved. The results derived in this paper can easily be used in decision problems where values of parameters or decision variables are not yet precisely fixed or assessed. Typical applications could be multicriteria optimization and decision making under uncertainty where fuzzy expected utilities can be obtained out of uncompletely assessed probabilities. More generally, fuzzy arithmetic can be an important tool for sophisticated, computationally tractable sensitivity analysis in systems modeling, computer-aided design and operations research.     

GPU based generation of state transition models using simulations for unmanned surface vehicle trajectory planning

This paper describes GPU based algorithms to compute state transition models for unmanned surface vehicles (USVs) using 6 degree of freedom (DOF) dynamics simulations of vehicle–wave interaction. A state transition model is a key component of the Markov Decision Process (MDP), which is a natural framework to formulate the problem of trajectory planning under motion uncertainty. The USV trajectory planning problem is characterized by the presence of large and somewhat stochastic forces due to ocean waves, which can cause significant deviations in their motion. Feedback controllers are often employed to reject disturbances and get back on the desired trajectory. However, the motion uncertainty can be significant and must be considered in the trajectory planning to avoid collisions with the surrounding obstacles. In case of USV missions, state transition probabilities need to be generated on-board, to compute trajectory plans that can handle dynamically changing USV parameters and environment (e.g., changing boat inertia tensor due to fuel consumption, variations in damping due to changes in water density, variations in sea-state, etc.). The 6 DOF dynamics simulations reported in this paper are based on potential flow theory. We also present a model simplification algorithm based on temporal coherence and its GPU implementation to accelerate simulation computation performance. Using the techniques discussed in this paper we were able to compute state transition probabilities in less than 10 min. Computed transition probabilities are subsequently used in a stochastic dynamic programming based approach to solve the MDP to obtain trajectory plan. Using this approach, we are able to generate dynamically feasible trajectories for USVs that exhibit safe behaviors in high sea-states in the vicinity of static obstacles.     

Properties of a job search problem on a partially observable Markov chain in a dynamic economy

This paper observes a job search problem on a partially observable Markov chain, which can be considered as an extension of a job search in a dynamic economy in [1]. This problem is formulated as the state changes according to a partially observable Markov chain, i.e., the current state cannot be observed but there exists some information regarding what a present state is. All information about the unobservable state are summarized by the probability distributions on the state space, and we employ the Bayes' theorem as a learning procedure. The total positivity of order two, or simply TP₂, is a fundamental property to investigate sequential decision problems, and it also plays an important role in the Bayesian learning procedure for a partially observable Markov process. By using this property, we consider some relationships among prior and posterior information, and the optimal policy. We will also observe the probabilities to make a transition into each state after some additional transitions by empolying the optimal policy. In the stock market, suppose that the states correspond to the business situation of one company and if there is a state designating the default, then the problem is what time the stocks are sold off before bankrupt, and the probability to become bankrupt will be also observed.     

Delay‐dependent state estimation for discrete Markovian jump neural networks with time‐varying delay

The state estimation problem is discussed for discrete Markovian jump neural networks with time‐varying delays in terms of linear matrix inequality (LMI) approach. The considered transition probabilities are assumed to be time‐variant and partially unknown. The aim of the state estimation problem is to design a state estimator to estimate the neuron states and ensure the stochastic stability of the error‐state system. A delay‐dependent sufficient condition for the existence of the desired state estimator is proposed. An explicit expression of the desired estimator is also given. A numerical example is introduced to show the effectiveness of the given result. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Diffusive information accumulation by minimal recurrent neural models of decision making

An important class of psychological models of decision making assumes that evidence is accumulated by a diffusion process to a response criterion. These models have successfully accounted for reaction time (RT) distributions and choice probabilities from a wide variety of experimental tasks. An outstanding theoretical problem is how the integration process that underlies diffusive evidence accumulation can be realized neurally. Wang ( 2001 , 2002 ) has suggested that long timescale neural integration may be implemented by persistent activity in reverberation loops. We analyze a simple recurrent decision making architecture and show that it leads to a diffusive accumulation process. The process has the form of a time-inhomogeneous Ornstein-Uhlenbeck velocity process with linearly increasing drift and diffusion coefficients. The resulting model predicts RT distributions and choice probabilities that closely approximate those found in behavioral data.     

Feature Enhancement for Noisy Speech Recognition With a Time-Variant Linear Predictive HMM Structure

This paper presents a new approach for speech feature enhancement in the log-spectral domain for noisy speech recognition. A switching linear dynamic model (SLDM) is explored as a parametric model for the clean speech distribution. Each multivariate linear dynamic model (LDM) is associated with the hidden state of a hidden Markov model (HMM) as an attempt to describe the temporal correlations among adjacent frames of speech features. The state transition on the Markov chain is the process of activating a different LDM or activating some of them simultaneously by different probabilities generated by the HMM. Rather than holding a transition probability for the whole process, a connectionist model is employed to learn the time variant transition probabilities. With the resulting SLDM as the speech model and with a model for the noise, speech and noise are jointly tracked by means of switching Kalman filtering. Comprehensive experiments are carried out using the Aurora2 database to evaluate the new algorithm. The results show that the new SLDM approach can further improve the speech feature enhancement performance in terms of noise-robust recognition accuracy, since the transition probabilities among the LDMs can be described more precisely at each time point.     

改进型Elman神经网络发酵过程建模研究

依据发酵过程的机理和改进的Elman神经网络动态建模原理,提出了一个新的发酵过程建模分批训练算法。通过发酵过程仿真实验,与传统的BP建模算法比较,改进的Elman神经网络建模算法具有收敛速度快、泛化能力强等特点。此外,利用该算法编制的软件可以内嵌到发酵过程监控系统中,实现发酵过程在线建模与状态参量的在线预估。     

Stability analysis for discrete delayed Markovian jumping neural networks with partly unknown transition probabilities

This paper addresses the analysis problem of asymptotic stability for a class of uncertain neural networks with Markovian jumping parameters and time delays. The considered transition probabilities are assumed to be partially unknown. The parameter uncertainties are considered to be norm-bounded. A sufficient condition for the stability of the addressed neural networks is derived, which is expressed in terms of a set of linear matrix inequalities. A numerical example is given to verify the effectiveness of the developed results.     

Missile defense and interceptor allocation by neuro-dynamicprogramming

This paper proposes a solution methodology for a missile defense problem involving the sequential allocation of defensive resources over a series of engagements. The problem is cast as a dynamic programming/Markovian decision problem, which is computationally intractable by exact methods because of its large number of states and its complex modeling issues. We employed a neuro-dynamic programming framework, whereby the cost-to-go function is approximated using neural network architectures that are trained on simulated data. We report on the performance obtained using several different training methods, and we compare this performance with the optimal approach     

A novel reduction approach to analyzing QoS of workflow processes

Quality of service (QoS) of workflows and workflow‐based applications is given increasing attention by both industry and academic. In this paper, we propose a novel analytical framework to analyze QoS (metrics include make‐span, cost, and reliability) of workflow systems based on GWF‐net, which extends traditional workflow net by associating tasks with generally distributed firing delay and time‐to‐failure. The GFW‐net model is used to model process structure and task organization of workflows at the process level. In contrast with prevailing QoS models based on Markovian process, we introduce a reduction technique to evaluate QoS of GWF‐net process avoiding the state‐explosion problem and tedious mathematical derivation of state‐transition probabilities. Through a case study, we show that our framework is capable of modeling real‐world workflow‐based application effectively. Also, experiments and confidence‐interval analysis in the case study indicate that the reduction methods are verified by real results. We also compare our approach with related research in the text. Copyright © 2008 John Wiley & Sons, Ltd.     

Multi-aspect target discrimination using hidden Markov models and neural networks

This paper presents a new multi-aspect pattern classification method using hidden Markov models (HMMs). Models are defined for each class, with the probability found by each model determining class membership. Each HMM model is enhanced by the use of a multilayer perception (MLP) network to generate emission probabilities. This hybrid system uses the MLP to find the probability of a state for an unknown pattern and the HMM to model the process underlying the state transitions. A new batch gradient descent-based method is introduced for optimal estimation of the transition and emission probabilities. A prediction method in conjunction with HMM model is also presented that attempts to improve the computation of transition probabilities by using the previous states to predict the next state. This method exploits the correlation information between consecutive aspects. These algorithms are then implemented and benchmarked on a multi-aspect underwater target classification problem using a realistic sonar data set collected in different bottom conditions.     

Sensitivity analysis in influence diagrams

The influence diagram framework serves as a powerful modeling tool for symmetric decision problems with a single decision maker. However, one of the main difficulties when representing decision problems using influence diagrams is eliciting the utilities and the probabilities. This makes it desirable to be able to investigate: 1) how sensitive the solution is to variations in some utility or probability parameter, and 2) how robust the solution is to joint variations over a set of parameters. In this paper, we propose a general algorithm for performing these types of analysis.     

Neural Network Modeling of the Level of Observation Decision in an Acute Psychiatric Ward

Patients in an acute psychiatric ward need to be observed with varying levels of closeness. We report a series of experiments in which neural networks were trained to model this “level of observation” decision. One hundred eighty-seven such clinical decisions were used to train and test the networks which were evaluated by a multitrialv-fold cross-validation procedure. One neural network modeling approach was to break down the decision process into four subproblems, each of which was solved by a perceptron unit. This resulted in a hierarchical perceptron network having a structure that was equivalent to a sparsely connected two-layer perceptron. Neural network approaches were compared with nearest neighbor, linear regression, and naive Bayes classifiers. The hierarchical and sparse neural networks were the most accurate classifiers. This shows that the decision process is nonlinear, that neural nets can be more accurate than other statistical approaches, and that hierarchical decomposition is a useful methodology for neural network design.     

Learning dynamic audio-visual mapping with input-output Hidden Markov models

In this paper, we formulate the problem of synthesizing facial animation from an input audio sequence as a dynamic audio-visual mapping. We propose that audio-visual mapping should be modeled with an input-output hidden Markov model, or IOHMM. An IOHMM is an HMM for which the output and transition probabilities are conditional on the input sequence. We train IOHMMs using the expectation-maximization(EM) algorithm with a novel architecture to explicitly model the relationship between transition probabilities and the input using neural networks. Given an input sequence, the output sequence is synthesized by the maximum likelihood estimation. Experimental results demonstrate that IOHMMs can generate natural and good-quality facial animation sequences from the input audio.     

Robust M-ary detection filters and smoothers for continuous-time jump Markov systems

In this paper, we consider a dynamic M-ary detection problem when Markov chains are observed through a Wiener process. These systems are fully specified by a candidate set of parameters, whose elements are, a rate matrix for the Markov chain and a parameter for the observation model. Further, we suppose these parameter sets can switch according to the state of an unobserved Markov chain and thereby produce an observation process generated by time varying (jump stochastic) parameter sets. Given such an observation process and a specified collection of models, we estimate the probabilities of each model parameter set explaining the observation. By defining a new augmented state process, then applying the method of reference probability, we compute matrix-valued dynamics, whose solutions estimate joint probabilities for all combinations of candidate model parameter sets and values taken by the indirectly observed state process. These matrix-valued dynamics satisfy a stochastic integral equation with a Wiener process integrator. Using the gauge transformation techniques introduced by Clark and a pointwise matrix product, we compute robust matrix-valued dynamics for the joint probabilities on the augmented state space. In these new dynamics, the observation Wiener process appears as a parameter matrix in a linear ordinary differential equation, rather than an integrator in a stochastic integral equation. It is shown that these robust dynamics, when discretised, enjoy a deterministic upper bound which ensures nonnegative probabilities for any observation sample path. In contrast, no such upper bounds can be computed for Taylor expansion approximations, such as the Euler-Maryauana and Milstein schemes. Finally, by exploiting a duality between causal and anticausal robust detector dynamics, we develop an algorithm to compute smoothed mode probability estimates without stochastic integrations. A computer simulation demonstrating performance is included.     

A reservoir-driven non-stationary hidden Markov model

In this work, we propose a novel approach towards sequential data modeling that leverages the strengths of hidden Markov models and echo-state networks (ESNs) in the context of non-parametric Bayesian inference approaches. We introduce a non-stationary hidden Markov model, the time-dependent state transition probabilities of which are driven by a high-dimensional signal that encodes the whole history of the modeled observations, namely the state vector of a postulated observations-driven ESN reservoir. We derive an efficient inference algorithm for our model under the variational Bayesian paradigm, and we examine the efficacy of our approach considering a number of sequential data modeling applications.