Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems

In this paper, an integral reinforcement learning (IRL) algorithm on an actor–critic structure is developed to learn online the solution to the Hamilton–Jacobi–Bellman equation for partially-unknown constrained-input systems. The technique of experience replay is used to update the critic weights to solve an IRL Bellman equation. This means, unlike existing reinforcement learning algorithms, recorded past experiences are used concurrently with current data for adaptation of the critic weights. It is shown that using this technique, instead of the traditional persistence of excitation condition which is often difficult or impossible to verify online, an easy-to-check condition on the richness of the recorded data is sufficient to guarantee convergence to a near-optimal control law. Stability of the proposed feedback control law is shown and the effectiveness of the proposed method is illustrated with simulation examples.     

Characterization and methanation activity of supported nickel catalysts

The methanation of carbon monoxide was studied over various supported nickel catalysts in a fixed bed reactor in the temperature range of 250 to 400°C. The hydrogen to carbon monoxide ratio was kept higher than stoichiometric in order to avoid carbon formation. The catalyst particles had an average diameter of 80 pm and nickel contents ranging from 5 to 42 wt %. The performance of laboratory prepared catalysts was compared with commercial ones. The catalysts were characterized by BET surface area, pore volume and carbon monoxide chemisorption. Specific activities for the methanation reaction were obtained and were found to vary with nickel concentration and with crystallite size. There exists a crystallite size range in which the maximum activity was observed. Metal support interactions may also contribute to the vdation in specific activity. The activation energy was found to vary from 58.2 MJ/kmol to 119.4 MJ/kmol. It appears that the methanation reaction over Ni/aIumina catalysts is a structure sensitive reaction. A compensation effect was observed for this reaction.     

Catalytic upgrading of fast pyrolysis oil over hzsm-5

The upgrading of bio-oil, obtained by fast pyrolysis of maple wood, was studied over HZSM-5 in a fixed bed micro-reactor operated at atmospheric pressure and in the temperature range 330-425°C. The objective of upgrading was to maximize the amount of organic distillate product with a high yield of aromatic hydrocarbons. A maximum organic distillate of 38 wt.% of bio-oil, which represented 28.6 wt.% of wood, was obtained at 370°C. The yield of aromatic hydrocarbons was 19.9 wt.% of wood. Above 400°C, nearly 50 wt.% of the bio-oil was converted to coke and char. The conversion of non-volatile components of the bio-oil (pitch) to volatiles was at a maximum of 68 wt. % at 370°C. However, when the bio-oil was co-processed with tetralin, the maximum conversion of non-volatiles increased to 86 wt.% at 410°C and the amounts of coke and char decreased. The yield of aromatic hydrocarbons also decreased to a maximum of 10.3 wt. % of wood. The role of tetralin was mainly as a diluent and not as a hydrogen donor solvent.     

On-line joint QoS routing and channel assignment in multi-channel multi-radio wireless mesh networks

We study the problem of on-line joint QoS routing and channel assignment for performance optimization in multi-channel multi-radio wireless mesh networks, which is a fundamental issue in supporting quality of service for emerging multimedia applications. To our best knowledge, this is the first time that the problem is addressed. Our proposed solution is composed of a routing algorithm that finds up to k but not necessarily feasible paths for each demand and an on-demand channel (re)assignment algorithm that adapts network resources to maintain feasibility of one of the paths. We also study the problem of obtaining an upper bound on the network performance. First, we consider an artificial version of the problem, in which all demands arrive at the same time, and formulate it as a mixed integer linear programming model. To tackle the complexity of the model, it is relaxed that provides a tight upper bound while improves solution time up to 3.0e+5 times. Then, we model the original problem by extending the relaxed model to consider dynamic demands, it leads to a huge model; thus, we develop another model, which is equivalent to the first one and is decomposable. It is broken down by a decomposition algorithm into subproblems, which are solved sequentially. Our extensive simulations show that the proposed solution has comparable performance to the bound obtained from the decomposition algorithm; it efficiently exploits available channels, and needs very few radios per node to achieve high network performance.     

Analysis of the Influence of Element’s Entropy on the Bulk Metallic Glass (BMG) Entropy,Complexity, and Strength

Metallurgical and Materials Transactions A - Bulk metallic glasses (BMGs) are disordered compounds without a long-range crystalline order. In this paper, we study the effect of an element’s...     

Statistical modeling and denoising Wigner–Ville distribution

Studying the properties of the Wigner–Ville distribution (wvd) and its smoothed versions such as smoothed pseudo-WVD (spwvd), we demonstrate that they have significantly non-Gaussian statistics. Also, we investigate the presence of two-dimensional heteroscedasticity in them for different signals based on employing Lagrange multiplier (LM) procedure. Therefore, we employ a heteroscedastic model called two-dimensional generalized autoregressive conditional heteroscedastic (2-D garch) for statistical modeling of these distributions. This modeling captures the characteristics of WVD and SPWVD, such as heavy tailed marginal distribution, and the dependencies among them. Since the performance of WVD and its smoothed versions degrade in the presence of additive noise, we design a novel Bayesian estimator for estimating the clean distributions based on garch modeling. Also, estimating the instantaneous frequency (if) curves of signals in presence of noise based on WVD and its smoothed versions is an interesting topic in the radar domain. So, we apply the denoised distributions for estimating the if. Experimental results demonstrate the efficiency of proposed method in denoising wvd and SPWVD and also performance improvement for if estimation in utilizing the denoised distributions.     

Study on Conveyance Coefficient Influenced by Momentum Exchange Under Steady and Unsteady Flows in Compound Open Channels

Water Resources Management - Many natural compound channels with differential stages play a vital role during high flow events in real-time. When a flood occurs, and water flows into floodplains,...     

Optimization of sewer networks using the mixed-integer linear programming

This paper introduces a method for optimizing sewer networks using the mixed-integer linear programming (MILP) for a given layout. The objective function is defined as the sum of the costs for pipe purchase, pipe-laying, and manhole construction expressed in linear terms and subject to minimum and maximum allowable slopes, velocities, and relative depths for both minimum and maximum sewage discharge rates in each pipe. Additionally, provisions are made as constraints or conditions to ensure that a minimum pipe cover is required, that pipe diameters do not decrease in the flow direction, and that pipes maintain a steady elevation at each manhole. All the non-linear constraints are transformed into the linear format. Pipe slope, binary variables accounting for commercial pipe diameters and average implemented depths have also been considered as decision variables. Finally, the performance of the proposed optimization method is evaluated in a benchmark sewer network from the literature.     

Modeling and Simulation of an Integrated Micro Packed Bed Reactor-Heat Exchanger Configuration for Direct Dimethyl Ether Synthesis

The application of an integrated micro packed bed reactor-heat exchanger for highly exothermic syngas conversion to dimethyl ether was investigated by numerical simulation. A three-dimensional pseudo-homogeneous model was developed to describe concentration and temperature distributions inside a single reaction slit. The model was adopted at low Reynolds numbers (small catalyst particles and long residence times). The simulation results reveal that the micro packed reaction slit is close to isothermal, and a negligible temperature gradient is predicted also for a doubling of the critical reaction slit dimension, i.e. the reaction slit height. The flow in the reaction slit was modelled by Brinkman??s equation and it was found that the pressure drop in wide range of gas velocity was not significant. The reactor simulation showed good agreement with experimental results and could be further developed to assess the potential of microstructured packed bed reactors to intensify large-scale industrial processes.