Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

1.3 μm InAsyP1-y-InPstrained-layer quantum-well laser diodes grown by metalorganic chemicalvapor deposition

The authors have fabricated 1.3-μm InAsP-InP separate-confinement-heterostructure (SCH) strained-layer double-quantum-well (SL-DQW) laser diodes (LDs) by metalorganic chemical vapor deposition (MOCVD). A low threshold current density of 410 A/cm ² was obtained. The CW threshold current was as low as 1.8 mA at 20°C, and maximum CW operating temperature of 120°C was obtained. These characteristics are almost the same as those of well-designed GaInAsP-InP SL-QW LDs. Further improvement of the characteristics of InAsP-InP LDs is expected by optimizing the device structure     

Lectin-like characteristics of recombinant human interleukin-1beta recognizing glycans of the glycosylphosphatidylinositol anchor

We found that 35S-labeled recombinant human interleukin-1beta (rhIL-1beta) binds phosphatidylinositol-specific phospholipase C-treated human placental alkaline phosphatase, phosphatidylinositol-specific phospholipase C-treated trypanosome surface variant glycoproteins, and urinary uromodulin immobilized on plates or immobilized on CNBr-activated Sepharose 4B. The interaction between rhIL-1beta and these glycoproteins was lectin-like, since it was inhibited in the presence of specific saccharides, i.e. mannose 6-phosphate or synthetic Ac-NH.CH2.CH2. PO4--->6Manalpha1-->(+/-2Manalpha1-->+/-6Manalpha1-->) propyl at about 1 microM. On the other hand, a wide variety of compounds including biantennary sugar chains derived from these glycoproteins as well as ethanolamine phosphate, inositol phosphate, mannose 6-sulfate, mannose 1-phosphate, glucose 6-phosphate, and mannitol 6-phosphate did not show any inhibitory effect at concentrations up to 1 mM. These results indicate that rhIL-1beta interacts with these glycoproteins via the mannose 6-phosphate diester of glycans on the glycosylphosphatidylinositol (GPI) anchor. Furthermore, when monolayers of polarized Madin-Darby canine kidney cells on polycarbonate filter membranes were incubated with 35S-rhIL-1beta in either the apical or basolateral chamber, 35S-interleukin-1beta was found to bind specifically to the apical membranes with a Ka value of 4.6 x 10(7) M-1, and the specific interaction was inhibited by 1 microM mannose 6-phosphate. Since the mannose 6-phosphate diester moiety exists only in the GPI glycans on plasma membranes, it was evident that interleukin-1beta can directly interact with the mannose 6-phosphate diester component of the intact glycan of GPI anchors on plasma membranes.     

Prostaglandin E receptor subtypes in mouse osteoblastic cell line

PGE2 is one of the key molecules in the osteoblast. It is the major prostanoid in the bone, and its production is under the control of both systemic and local factors. PGE2 has been reported to have multiple actions in the osteoblast, such as growth promotion and cell differentiation. To better understand the action of PGE2 in the osteoblast, we determined the PGE receptor subtypes in MC3T3-E1, an osteoblastic cell line derived from the normal mouse calvaria. Northern blot analysis revealed that EP1 and EP4 subtypes are expressed in MC3T3-E1. In contrast, EP3 subtype was not detected by either Northern blot analysis or RT-PCR. The contribution of each subtype was evaluated by studying the effects of subtype-specific analogs on osteoblastic function at confluency and 5 days after confluency. An EP1 agonist, 17-phenyl-omega-trinor PGE2, increased DNA synthesis and decreased alkaline phosphatase activity. 11-Deoxy-PGE1, and EP2 and EP4 agonist, decreased DNA synthesis and increased alkaline phosphatase activity at both stages. Butaprost, an EP2-selective agonist, showed effects similar to those of 11-deoxy-PGE1 only at confluency. Another and more differentiated osteoblastic marker, osteocalcin production, was detectable and was stimulated by 11-deoxy-PGE1 only 5 days after confluency. The exposure of these cells to EP1 agonist changed the cell shape to a more fibroblastic appearance. These results indicate that EP1, EP4, and probably EP2 are present in MC3T3-E1 cells; EP1 promotes cell growth, and EP2 and EP4 mediate differentiation of the osteoblast. Furthermore, the decreased response to EP2-specific agonist 5 days after confluency suggests that the expression of PGE receptor subtype is dependent on the stage of osteoblastic differentiation. This is the first report to determine PGE receptor subtypes in the bone.     

Beam test of the CsI(Tl) calorimeter for the BELLE detector at the KEK-B factory

We studied the performance of a prototype electromagnetic calorimeter for the BELLE detector at the KEK proton synchrotron for an energy range of 0.25–3.5 GeV. The prototype consisted of an array of 6 × 5 CsI(Tl) crystals with 30 cm length (16.2 radiation lengths) and about 6 cm × 6 cm cross section. The scintillation light of each CsI(Tl) crystal was read out by two large-area PIN photodiodes and charge-sensitive preamplifiers attached at the rear face of the crystal. We measured the energy and position resolution for electrons and the e/π separation for two sets of matrix configurations: one corresponded to the center and the other to the edge of the barrel calorimeter. The overall performance measured by the test proves that the prototype calorimeter is satisfactory for the use in the BELLE detector.     

A method for locating fault section by using sequential information at substations

This paper presents a new real-time method for locating fault section at substations. When a fault occurs, considerable information resulting from protective relay operation and circuit-breaker operation is recorded. Using this information an attempt is made to locate the fault section, especially by considering the sequential relationship of the information and by dividing fault areas. Primarily, in this method, the fault area is divided into several sections based on protection area and operation time of protective relays. Next, expecting subsequent operation of backup relays, the “waiting time” for reasoning is given to each divided section. After a lapse of “waiting time,” all the suspected fault sections are extracted and given priority based on the empirical knowledge of experts. A prototype of the expert system was developed for fault section location, which was applied to various complicated fault cases. The effectiveness of the method was proved even in case of multiple faults and no-operation of protective relays the circuit breakers.     

Pulsed KrF excimer laser annealing of silicon solar cell

The pulsed KrF excimer laser annealing of silicon films for solar cell with EBEP-CVD and LP-CVD was studied theoretically and experimentally. Three-dimensional thermal diffusion equation for microcrystalline and amorphous silicon was solved by using the finite difference methods. The results of our heat-flow simulation of laser re-crystallization in a laser irradiation with 50 ns pulse duration almost agree with the experimental results in re-crystallization depth of 0.7 μm for microcrystalline silicon (EBEP-CVD) and 0.4 μm for amorphous silicon (LP-CVD) in a single pulse excimer laser annealing.     

Systematic analysis of neutron yields from thick targets bombarded by heavy ions and protons with moving source model

A simple phenomenological analysis using the moving source model has been performed on the neutron energy spectra produced by bombarding thick targets with high energy heavy ions which have been systematically measured at the Heavy-Ion Medical Accelerator (HIMAC) facility (located in Chiba, Japan) of the National Institute of Radiological Sciences (NIRS). For the bombardment of both heavy ions and protons in the energy region of 100–500 MeV per nucleon, the moving source model incorporating the knock-on process could be generally successful in reproducing the measured neutron spectra within a factor of two margin of accuracy. This phenomenological analytical equation is expressed having several parameters as functions of atomic number Z_p, mass number A_p, energy per nucleon E_p for projectile, and atomic number Z_T, mass number A_T for target. By inputting these basic data for projectile and target into this equation we can easily estimate the secondary neutron energy spectra at an emission angle of 0–90° for bombardment with heavy ions and protons in the aforementioned energy region. This method will be quite useful to estimate the neutron source term in the neutron shielding design of high energy proton and heavy ion accelerators.     

Characterization of calcium-independent cytosolic phospholipase A2 activity in the submucosal regions of rat stomach and small intestine

This study was undertaken to compare the calcium-independent phospholipase A₂ (PLA₂) activities in the cytosols of twelve rat tissues and to determine whether their activities were distinct. 1-O-Alk-1′-enyl-2-[¹⁴C]-oleoyl-sn-glycero-3-phosphocho-line (PlsC) and 1-O-Alk-1′-enyl-2-[¹⁴C]oleoyl-sn-glycero-3-phosphethanolamine (PlsE) were synthesized and used as substrates, instead of phosphatidyl compounds, to exclude hydrolysis by cytosolic PLA₁ activity that could be present in some of the cytosolic preparations. For each tissue, we examined substrate specificity, pH optimum, and effect of adenosine triphosphate (ATP) and ATP analogues. PLA₂ activity was detected in eleven out of the twelve issues examined. Based on substrate specificity and pH optimum, cytosolic calcium-independent PLA₂ were classified in three groups. The first group, which included PLA₂ from small intestine, stomach and spleen, had the highest specific activity with PlsC as substrate (1253, 309 and 75 nmol/mg protein/hour, respectively) and an optimal pH at 6.5. Activity with PlsE as substrate was much lower (20–37%) than with PlsC. The second group of PLA₂ activities included the cytosolic activities from thymus, lung, liver and pancreas that showed lower specific activities for both substrates (14–23 nmol/mg protein/hour with PlsC) and had a broader optimal pH range of 6.1 to 7.5. The cytosols from brain, kidney, heart and muscle comprised the third PLA₂ group that was found to have a higher specific activity with PlsE (5–20 nmol/mg protein/hour) than PlsC and an optimal pH range from 7.4 to 7.9. Since the highest specific activity was found in the cytosol from small intestine, this PLA₂ was examined further. PLA₂ activity was found to be equally distributed in the cytosol of the submucosal portion of duodenum, jejunum and ileum with an optimal pH of 6.1 and a 5-fold higher activity with PlsC than PlsE as substrate. Moreover, this PLA₂ activity was inhibited by treatment with detergents. These results indicate the presence in the submucosal portion of the intestine of a calcium-independent cytosolic PLA₂ with a high specific activity toward PlsC and properties distinct from those described for the PLA₂ found in the intestinal brush-border.     

Carrying capacity and vulnerability of four Asian lakes: A comparative study

To present axes for evaluating the water resources, lake environment and water use in a lake and its watershed, a comparative study was done for four East Asian lakes; lake Kasumigaura (Japan), Lake Laguna (Philippines), Lake Songkhla (Thailand) and Lake Dianchi (China). Two axes were then determined from the relationship between: (1) water use and water supply rate, and (2) water use and water quality. To clarify the meanings ‘carrying capacity’ and ‘vulnerability’, the maximum population density in the lake basin and total nitrogen concentration in the lake water were calculated and discussed to decide the direction of environmentally sound management of the drainage basin.