Reinforcement learning to adjust parametrized motor primitives to new situations

Humans manage to adapt learned movements very quickly to new situations by generalizing learned behaviors from similar situations. In contrast, robots currently often need to re-learn the complete movement. In this paper, we propose a method that learns to generalize parametrized motor plans by adapting a small set of global parameters, called meta-parameters. We employ reinforcement learning to learn the required meta-parameters to deal with the current situation, described by states. We introduce an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. To show its feasibility, we evaluate this algorithm on a toy example and compare it to several previous approaches. Subsequently, we apply the approach to three robot tasks, i.e., the generalization of throwing movements in darts, of hitting movements in table tennis, and of throwing balls where the tasks are learned on several different real physical robots, i.e., a Barrett WAM, a BioRob, the JST-ICORP/SARCOS CBi and a Kuka KR?6.     

Applying EuroWordNet to Cross-Language Text Retrieval

We discuss ways in which EuroWordNet (EWN) can be used in multilingual information retrieval activities, focusing on two approaches to Cross-Language Text Retrieval that use the EWN database as a large-scale multilingual semantic resource. The first approach indexes documents and queries in terms of the EuroWordNet Inter-Lingual-Index, thus turning term weighting and query/document matching into language-independent tasks. The second describes how the information in the EWN database could be integrated with a corpus-based technique, thus allowing retrieval of domain-specific terms that may not be present in our multilingual database. Our objective is to show the potential of EuroWordNet as a promising alternative to existing approaches to Cross-Language Text Retrieval.     

Review of ECSC research work in ironmaking

The Treaty of the European Coal and Steel Community (ECSC), which expired in 2002, states in its Article 55 to promote technical and economic research with respect to the production and growing application of coal and steel. The results of funded research projects shall be made available to all representatives and partners concerned. For the European steel industry Article 55 was the basis for building up a steel research programme covering topics from raw materials to steel application and instigated a European network of researchers from the steel industry, research institutes and universities. On the basis of these research and development activities carried out by the ECSC steel research, the iron and steel industry has shown important advances in technology and environmental compatibility. Furthermore, the ECSC steel research has contributed to nearly all important technological developments. The main objective of the ECSC steel Research and Technology Development (RTD) programme was to sustain and develop the Community steel industry's competitiveness. The following report describes fundamental developments and activities within the framework of ECSC research with particular respect to hot metal production in the last 15 years.     

Mapping of polished coal surfaces by automated reflectance microscopy

The capabilities of Rapid Scan, a system of automated reflectance microscopy, has been extended to the actual mapping of petrographic variation across polished coal surfaces. This is accomplished through computer control of a microscope stepping stage and the storage of binary reflectance data on disc in a matrix form which permits the proper spatial arrangement of values to be re-created. The maps depict lithotype variation, and the distribution of mineral occurrences, cracks and cleat. An image processor can be employed in further map analysis such as the compilation of reflectance profiles from selected areas.     

Improved digital image processing technique to investigate plastic zone formation in steel

An improved digital image processing procedure is developed to extract the size and shape of the plastic zone formation at the tip of a notched metal specimen based on the characteristics of the variation of intensity in a digitized image. The result is compared with the result from finite element analysis, and with that from a previously developed image processing procedure. Comparison shows that this method provides a faster and more accurate result than previously.     

Thermodynamic driving forces for PAH isomerization and growth during thermal treatment of polluted soils

For a limiting case of thermodynamic equilibrium, the importance of two classes of thermal chemical reactions that modify the structure and bioactivity of polycyclic aromatic hydrocarbons (PAH) was assessed computationally. These reactions are molecular weight (MW) growth by acetylene addition, and intramolecular rearrangement (isomerization). Temperatures (300-1100 degrees C), and the chemical environment (C(2)H(2)/H(2) molar ratios) were selected for relevancy to thermal treatment of PAH-contaminated soils under oxygen-free conditions. Molecular mechanics methods [MM3(92)] were used to compute thermochemical properties for calculation of equilibrium constants, i.e., heats of formation, standard entropies, and heat capacities for 30 PAH with empirical formulae C(14)H(10), C(16)H(10), C(18)H(10), C(18)H(12), C(20)H(10), and C(20)H(12). Included were 11 PAH containing only six-membered rings and 19 PAH containing both five- and six-membered rings. For each of these PAH the calculations predict that with increasing temperature, isomerization increases the "complexity" of the PAH mixture, i.e., the relative abundance of each PAH isomer in the mixture other than the most stable isomer, increases. Isomerization also partially transforms non-mutagens to mutagens, e.g., pyrene and benzo[e]pyrene to fluoranthene and benzo[a]pyrene, respectively, and partially converts cyclopenta[c, d]pyrene (CPEP) and chrysene, both human cell mutagens, to one and three additional human cell mutagens, respectively. Acetylene addition transforms the non-mutagens phenanthrene and pyrene to the mutagens triphenylene and CPEP, respectively. Some of the predicted PAH have been observed elsewhere among the products of aromatics pyrolysis. This study elucidates PAH reactivity for comparison with measurements, and identifies PAH reactions to be monitored and avoided in soil thermal decontamination and other waste remediation processes.     

Wakefulness in young and elderly subjects driving at night in a car simulator

Young drivers are over-represented in nighttime traffic accidents and several studies have suggested that many accidents are associated with elevated sleepiness levels. It has been suggested that there may be a connection between lowered wake capacity and functional sensory motor skills on the one hand and sleep deprivation at the circadian low in young drivers on the other.Performance during a 45/min evening and night drive among young (n = 10, age range 18–24 years) and elderly (n = 10, age range 55–64 years) subjects was studied using a moving base driving simulator. EEG was measured continuously. Every 5 min, subjects were rated on the Karolinska Sleepiness Scale (KSS). Saliva cortisol was assessed before and after each drive.The results showed that sleepiness increased across each drive and was higher among young drivers at night. Relative EEG power increased among older drivers for frequencies of 10–16 Hz. The sigma 1 frequency band (12–14 Hz) proved particularly sensitive to sustained driving, and was elevated among subjects in the elderly group. Cortisol levels before and after the evening and night drive showed higher mean levels for elderly subjects.The present study has demonstrated that young drivers were more sleepy while driving at night. The effects could represent a mobilization of effort and a reorganization of brain firing pattern among older subjects, possibly reflecting better ability and effort to resist sleepiness.     

Optimizing Army Sustainability at Fort Bragg: A Case Study Connecting Life-Cycle Cost Analysis with Leadership in Energy and Environmental Design for Existing Buildings

Abstract:

In 2006, the Assistant Secretary of the Army for Installations and Environment announced the Army's desire to transition to the U.S. Green Building Council's Leadership in Energy and Environmental Design for Existing Buildings rating scheme in an effort to institutionalize sustainable practices across all Army facilities. This article describes an engineering management tool developed for project managers at Fort Bragg's Directorate of Housing and Public Works used to determine the most cost-effective route to building certification and to support the Army goal of sustainability. The results expose practicing engineering managers to a significant issue concerning Army sustainability, where numerous engineering management methods are employed for more effective decision-making. Future research is needed to convert retrofit project life-cycle costs from specific location sources to national values.