When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Specification methods for material-handling control algorithms in flexible manufacturing systems

Good methods are needed to specify, test, and debug material-handling control logic. This article surveys a number of representative methods for defining and describing control algorithms for programmable material-handling equipment used in flexible manufacturing systems. The methods are evaluated with regard to their suitability for communication between people and as bases for interfaces to automatic program generators. It is concluded that no single method is entirely satisfactory. Three methods (position diagrams, function block diagrams, and operation networks) have potential to be combined into an effective hybrid approach that minimizes the need for the user to switch between various conceptual models.     

A process oriented semantics of the PRAM-language FORK

The parallel language FORK [1], based on a scalable shared memory model, is a PASCAL-like language with some additional parallel constructs. A PRAM (Parallel Random Access Machine) algorithm can be expressed on a high level of abstraction as a FORK program which is translated into efficient PRAM code guaranteeing theoretically predicted runtimes.

In this paper, we concentrate on those features of the language FORK related to parallelism, such as the group concept, a shared memory access and synchronous or asynchronous execution. We present a trace-based denotational interleaving semantics where processes describe synchronous computations. Processes are created or deleted dynamically and run asynchronously. Interleaving rules reflect the underlying CRCW (concurrent-read-concurrent-write) PRAM model.     

Uncertainty in Quantitative Electron Probe Microanalysis

Quantitative electron probe analysis is based on models based on the physics or x-ray generation, empirically adjusted to the analyses of specimens of known composition. Their accuracy can be estimated by applying them to a set of specimens of presumably well-known composition.     

Ischemic preconditioning prior to myocardial protection with cold blood cardioplegia in coronary surgery

OBJECTIVE: Encouraging results on myocardial preconditioning in experimental models of infarction, stunning or prolonged ischemia raise the question whether preconditioning techniques may enhance conventional cardioplegic protection used for routine coronary surgery. METHODS: A prospective clinical trial was conducted to investigate the effect of additional ischemic normothermic preconditioning prior to cardioplegic arrest applying cold blood cardioplegia in patients scheduled for routine coronary surgery (3 vessel disease, left ventricular ejection fraction > 50%). Two cross clamp periods of 5 min with the hearts beating in sinus rhythm were applied followed by 10 min of reperfusion, each (n = 7, group I). Inducing moderate hypothermia cold blood cardioplegia was delivered antegradely. In control groups, cold intermittent blood cardioplegia (n = 7, group II) was used alone. Coronary sinus effluents were analyzed for release of creatine kinase (CK), CK-MB, lactate, and troponin T at 1, 3, 6, 9, and 12 h. In addition, postoperative catecholamine requirements were monitored. RESULTS: The procedure was tolerated well, and no perioperative myocardial infarction in any of the groups studied occurred. Concentrations of lactate tended to be higher in group I, but this difference was not significant. In addition, no significant differences for concentrations of CK, CK-MB, and troponin T were found. Following ischemic preconditioning an increased dosage of dopamine was required within the first 12 h postoperatively (group I: 2.63 +/- 1.44 microg/kg/min, group II: 0.89 +/- 1.06 microg/kg/min). CONCLUSIONS: Combining ischemic preconditioning and cardioplegic protection with cold blood cardioplegia does not appear to ameliorate myocardial protection when compared to cardioplegic protection applying cold blood cardioplegia alone. Inversely, contractile function seemed to be impaired when applying this protocol of ischemic preconditioning.     

Biocide polymers

Summary Potential polymeric herbicides have been synthesized by reacting poly(acrylic acid)s with 3-aniino-1,2,4-triazole and 1-acetyl-5-amino-1,2,4-triazole, respectively. The produced copolymers contained the active moieties as pendant groups in the range of 3 to 27 per cent of weight and with a content of carboxyl groups up to 95 mol-%. The hydrolytic release of 3-amino-1,2,4-triazole was investigated.Part 12: M. Hartmann, H,-J. Bauer, A. Pritzsche and K. Wermann, Makromol. Chem., in press     

Computer-aided design in mechanisms

The man-machine dialog is very important for the design process. In this paper a program system, based on the minicomputer KRS 4201 (ROBOTRON), is demonstrated by using an interactive graphical screen. This program system provides the opportunity to analyze and synthesize mechanisms up to 16 links. The application is demonstrated by designing a front-loader-system for a truck. Real-time animation of seven-bar linkages and coupler curves, including velocity and acceleration, are shown.     

Injection mold flashing of liquid crystalline polymers

Thermotropic polyesters, such as Vectra (Hoechst Celanese), have excellent moldability for intricate parts that require high precision of form, such as electronic connectors. Two apparently contradictory aspects of molding behavior contribute to the moldability. On the one hand, the low viscosity of the liquid crystalline polymer (LCP) at high shear rates favors ease of filling molds that contain long, thin paths. On the other, parts molded from LCP have little or no flash to interfere with the functioning of the parts. There has apparently been little work on the rheological aspects of flash formation. An approximate analysis is made by considering that the flash is the result of melt being extruded from the mold cavity into a slit at the mold parting line. The driving force for the extrusion is the injection pressure. The flow is assumed to be isothermal until solidification occurs, at a time that depends on the thickness of the slit, on the thermal diffusivity of the melt, the melt and mold temperatures, and on the solidification temperature of the material. The viscosity is assumed to have power-law dependence on shear rate. It is found that when the aspect ratio (length to thickness) of the flash is small, its length is strongly dependent on the magnitude of the pressure drop at the contraction from the cavity to the slit. At the minimum pressure required to fill a mold, the flash length is predicted to be independent of the rheological and thermal properties of the melt, except for the power-law exponent. Differences in end correction can, however, account for different tendencies to flash at equal moldability. Comparison of the model with Richardson's analysis of freezing in a cavity suggests a correlation of the thermal properties of the melt with his parameter c, which is related to mold filling ability. Tests of the model and possible refinements are suggested.     

Influence of particles on recrystallization textures of ferritic stainless steels

Five microalloyed ferritic steels with a Cr content of 10 to 17 % were cold rolled and recrystallized. Steels with a high volume fraction of Nb and Ti precipitations revealed a deviation from the typical {111} fibre texture. The new main component {557}<583> was explained by particle induced growth selection.