Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

Geometrical analysis and design of a motion transmitting element for mobile vehicles

The range of speed or torque of motors is limited to some degree. Industrial vehicles working in factories are required to output a large force when they carry loads, and also to move with high velocity when they move to their destination. However, it is impossible to realize both those requirements if a conventional reduction device is used. Velocity variation devices using gears are widely used to change the velocity ratio between the input and output shafts. However, the motion transmission from the input shaft to the output shaft is interrupted during the velocity ratio variation process. In order to solve this problem, a velocity variation method that can transmit motion continuously is proposed, in which a motion transmitting element is used. In this report, a motion transmitting element using the geometrical analysis method is proposed, a method of expressing the curve using dispersed points is proposed, and a method of calculating the inclination and the distances along the curve is developed. Based on the proposed methods, the geometrical form of the transmitting element is analyzed.     

Photoactivable Elimination of Tumorigenic Human Induced Pluripotent Stem Cells by Using a Lectin–Doxorubicin Prodrug Conjugate

Human pluripotent stem cells (hPSCs) are attractive resources for regenerative medicine, but medical applications are hindered by their tumorigenic potential. Previously, a hPSC-specific lectin probe, rBC2LCN, was identified through comprehensive glycome analysis by using high-density lectin microarrays. Herein, a lectin–doxorubicin (DOX) prodrug conjugate, with controllable photolysis activation for the elimination of tumorigenic human induced pluripotent stem cells, has been developed. rBC2LCN was fused with a biotin-binding protein, tamavidin (BC2Tama), and the fusion protein was expressed in Escherichia coli and purified by means of affinity chromatography. BC2Tama was then conjugated with doxorubicin-photocleavable biotin (DOXPCB). The BC2Tama–DOXPCB conjugates were observed to bind to hPSCs followed by internalization. Upon exposure to ultraviolet light, DOX was released inside the cells, which allowed specific killing of the hPSCs. Thus, BC2Tama–DOXPCB should be useful for the targeted elimination of hPSCs contained in hPSC-derived cell therapy products. This is the first report of the generation of lectin–prodrug conjugates. BC2Tama should be applicable for the targeted delivery of various types of biotinylated compounds into hPSCs.     

Determination of urine saturation with computer program EQUIL 2 as a method for estimation of the risk of urolithiasis

Two measures of weight cycling and indexes of psychological functioning were examined in a large sample of dieters. History of weight cycling was assessed to include number of dieting attempts, total lifetime weight lost and regained, and number of weight cycles over 20 lb (9.1 kg). Experience of weight cycling measured perception of being a yo-yo dieter and perceived success at maintaining past weight losses. Experience was more strongly related than history to all psychological measures. Further, when controlling for the effects of age, body mass index, and experience, the relation between history and the psychological variables was nonsignificant. This finding suggests that an individual's perception of being a weight cycler may be more related to psychological problems than the actual number of pounds lost and regained over time.     

Alcohol dehydrogenases that catalyse methyl formate synthesis participate in formaldehyde detoxification in the methylotrophic yeast Candida boidinii

Methyl formate synthesis during growth on methanol by methylotrophic yeasts has been considered to play a role in formaldehyde detoxification. An enzyme that catalyses methyl formate synthesis was purified from methylotrophic yeasts, and was suggested to belong to a family of alcohol dehydrogenases (ADHs). In this study we report the gene cloning and gene disruption analysis of three ADH-encoding genes in the methylotrophic yeast Candida boidinii (CbADH1, CbADH2 and CbADH3) in order to clarify the physiological role of methyl formate synthesis. From the primary structures of these three genes, CbAdh1 was shown to be cytosolic and CbAdh2 and CbAdh3 were mitochondrial enzymes. Gene products of CbADH1, CbADH2 and CbADH3 expressed in Escherichia coli showed both ADH- and methyl formate-synthesizing activities. The results of gene-disruption analyses suggested that methyl formate synthesis was mainly catalysed by a cytosolic ADH (CbAdh1), and this enzyme contributed to formaldehyde detoxification through glutathione-independent formaldehyde oxidation during growth on methanol by methylotrophic yeasts.     

Osteoclast differentiation and characteristic trabecular bone formation during growth plate destruction in osteoprotegerin-deficient mice

Osteoprotegerin (OPG) is an osteoblast-derived secreted member of the tumour necrosis factor receptor superfamily that inhibits osteoclastogenesis. Mice that are OPG-deficient have severe bone loss, including growth plate cartilage destruction. Using OPG-deficient mice as a useful animal model, we attempted to clarify differentiation and ultrastructural features of osteoclasts located on destructed growth plate cartilage and trabecular bone matrix. In the humerus and femur of OPG homozygous (-/-) mice, adjacent to the growth plate cartilage, bone trabeculae without a calcified cartilage core were characteristically formed at the metaphyseal side of the medullary cavity, which resulted in an irregular chondrocyte distribution and arrangement in growth plate cartilage. During growth plate cartilage destruction, osteoclasts positive for tartrate-resistant acid phosphatase showed unusual localization on both type-II collagen-positive cartilage and type-I collagen-positive trabecular bone matrix at the ossification centre of the epiphyseal/metaphyseal border. Although multinucleated osteoclasts were distributed within open lacunar canals in the growth plate, those on uncalcified cartilage matrix lacked a ruffled border. Facing the calcified cartilage matrix within lacunar canals, osteoclasts showed irregularly formed ruffled borders. After growth plate destruction, a thin bone layer was deposited on the remaining cartilage surfaces by invading osteoblasts. Osteoclasts formed prominent ruffled border structures on bone matrix, deposited on the remaining growth plate cartilage. These results suggest that, in OPG (-/-) mice, terminal osteoclast differentiation requires the presence of newly produced bone matrix, as the coupled phenomenon of bone formation and resorption, as well as osteoblast-derived cytokines.     

Anaerobic Biodegradation Tests of Poly(lactic acid) under Mesophilic and Thermophilic Conditions Using a New Evaluation System for Methane Fermentation in Anaerobic Sludge

Anaerobic biodegradation tests of poly(lactic acid) (PLA) powder were done at the thermophilic (55 °C) and mesophilic temperature (35 °C) under aquatic conditions [total solid concentrations of the used sludge were 2.07% (at 55 °C) and 2.24% (at 35 °C)] using a newly developed evaluation system. With this system, the evolved biogas is collected in a gas sampling bag at atmospheric pressure. This method is more convenient than using a pressure transducer or inverted graduated cylinder submerged in water. PLA was degraded about 60% in 30 days, about 80% in 40 days and about 90% in 60 days at 55 °C. On the other hand, the PLA degradation started in 55 days at 35 °C and degradation rate was much slower than at 55 °C.     

Preparation of carbon-coated W18O49 and its photoactivity under visible light

Carbon-coated W₁₈O₄₉ powders were prepared from the mixture of para-ammonium tungstate with poly(vinyl alcohol) by heat treatment in inert atmosphere at a temperature between 750 and 900 °C for 1 h. The synthesized W₁₈O₄₉ crystals had prismatic morphology in small size, less than 0.5 μm in diameter and about 1 μm in length. Carbon-coated W₁₈O₄₉ was shown to have photoactivity under visible light irradiation by comparing the concentration changes of methylene blue, phenol and dimethylsulfoxide with time under the irradiation of visible light to that in the dark. Photoactivity of W₁₈O₄₉ was supposed to be due to the formation of OH radicals on the basis of the degradation of dimethylsulfoxide, its quantitative transformation to methanesulfonic acid. Carbon coating seemed to have various roles: to reduce WO₃ to W₁₈O₄₉, to inhibit the sintering and crystal growth of W₁₈O₄₉ to keep them small size, and also to concentrate pollutants around W₁₈O₄₉ crystal by adsorption.