A Reinforcement Learning-based Approach to Dynamic Job-shop Scheduling

Production scheduling is critical to manufacturing system.Dispatching rules are usually applied dynamically to schedule (?)he job in a dynamic job-shop.Existing scheduling approaches sel- dom address machine selection in the scheduling process.Composite rules,considering both machine selection and job selection,are proposed in this paper.The dynamic system is trained to enhance its learning and adaptive capability by a reinforcement learning(RL)algorithm.We define the concep- tion of pressure to describe the system feature.Designing a reward function should be guided by the scheduling goal to accurately record the learning progress.Competitive results with the RL-based approach show that it can be used as real-time scheduling technology.     

Guidelines for creating a rule-based knowledge learning system and their application to a Chinese business card layout analysis

Rule selection has long been a problem of great challenge that has to be solved when developing a rule-based knowledge learning system.Many methods have been proposed to evaluate the eligibility of a single rule based on some criteria.However,in a knowledge learning system there is usually a set of rules,These rules are not independent,but interactive,They tend to affect each other and form a rulesystem.In such case,it is no longer rasonable to isolate each rule from others for evaluation.A best rule according to certain criterion is not always the best one for the whole system.Furthermore,the data in the real world from which people want to create their learning system are often ill-defined and inconsistent.In this case,the completeness and consistency criteria for rule selection are no longer essential.In this paper,some ideas about how to solve the rule-selection problem in a systematic way are proposed.These ideas have been applied in the design of a Chinese business card layout analysis system and gained a goods result on the training data set of 425 images.The implementation of the system and the result are presented in this paper.     

一种面向作业的快速调度算法S

Job oriented scheduling (JOS) has been the most commonly used technique in actual job shop scheduling. It loads jobs one by one onto machines. In this paper, the authors present a fast scheduling algorithm of computer-based JOS system, the algorithm assigns feasible schedule start and finish times to the operations of a job by loading them forward or backward onto the capacity constrained machines. The computation time to find the feasible time slot on the machine is reduced by log and modify each machine’s feasible time slot. Thus, the computational efficiency is substantially improved. Experimental testing shows that the algorithm has significant merits for large size problems.     

An approach to active learning for classifier systems

In this paper,the active learning mechanism is proposed to be used in classifier systems to cope with complex problems:an intelligent agent leaves its own signals in the environment and later collects and employs them to assist its learning process.Principles and components of the mechanism are outlined,followed by the introduction of its preliminary implementation in an actual system.An experiment wit te system in a dynamic problem is then introduced,together with discussions over its results.The paper is concluded by pointing out some possible improvements that can be made to the proposed framework.     

Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy （HJSS）, and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy （ADHRS）, to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica： first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.     

Performance analysis of active schedules in identical parallel machine

Active schedule is one of the most basic and popular concepts in production scheduling research. For identical parallel machine scheduling with jobs’ dynamic arrivals, the tight performance bounds of active schedules under the measurement of four popular objectives are respectively given in this paper. Similar analysis method and conclusions can be generalized to static identical parallel machine and single machine scheduling problem.     

Performance analysis of active schedules in identical parallel machine

Active schedule is one of the most basic and popular concepts in production scheduling research. For identical parallel machine scheduling with jobs’ dynamic arrivals, the tight performance bounds of active schedules under the measurement of four popular objectives are respectively given in this paper. Similar analysis method and conclusions can be generalized to static identical parallel machine and single machine scheduling problem.     

On-Line Predicting Behaviors of Jobs in Dynamic Load Balancing

A key issue of dynamic load balancing in a lossely coupled distributed system is selecting appropriate jobs to transfer.In this paper,a job selection policy based on on-line predicting behaviors of jobs is proposed.Tracing is used at the beginning of execution of a job to predict the approximate execution time and resource requirements of th job so as to make a correct decision about whether transferring the job is worthwhil.A dynamic load balancer using the job selection policy has been implemented.Experimental measurement results show that the policy proposed is able to improve mean response time of jobs and resource utilization of systems substantially.     

Web-Based Learning and Fault Diagnostic System

This paper presents a novel knowledge-based multi-agent system for remote fault diagnosis, which is composed of learning and diagnostic agents （LDAs）, machine agents （MAs） and a central manageraent agent （CMA）. Machines are remotely diagnosed by the LDAs through the communication channels between the MAs and the LDAs. When faults that cannot be solved by the present knowledge base occur, the LDAS can acquire new knowledge, translate it into rules using a rule builder, and update the rules into the CKB（Central Knowledge Base）. The CKB will become mature through a continuous learning process. A prototype system has been developed and used for remote fault diagnostics of tool wear in computer numerically controlled （CNC） machining.     

机电一体化培训和教育中的电子移动学习课件开发(英文)

Knowledge is essential for the competitiveness of individuals as well as organizations.Thus the application of the latest methodologies and technologies are utilized to support knowledge acquisition,warehousing,distribution,and transfer.Means and methods of web 2.0 are useful to support this procedure.Especially,highly complex and very dynamic knowledge domains have to be accessible and applicable in the framework of learning network communities,including the stakeholders of training and education.Mechatronics for example is such an interdisciplinary,dynamic field of research and application.Based on intelligence,software,and hardware it is requiring special approaches for developing a courseware based learning and knowledge transfer environment.After defining the specifics of mechatronics education and postgraduate training in the context of e-education,the concepts of the development and utilization of mechatronic courseware can be deduced from e-learning 2.0 and mobile learning facilities,possibilities,and abilities.Mechatronic courseware will be developed by using authoring software and embedding the material into learning management systems with respect to general methods and rules of modern system and software development.As an example,the courseware is used for vocational training and further education especially in cooperation networks of educational institutions and SME.     

动态车间作业调度问题中调度规则算法研究综述

调度规则是解决实际生产中的动态车间作业调度问题的有效方法,但它一般只在特定调度环境下性能较好,当环境发生变化时,就需要进行实时选择和评价。对调度规则的实时选择和评价方法进行综述,以研究实际生产中动态车间的实时调度问题。对调度规则的发展、分类以及特点进行了概述,并对调度规则的选择和评价方法进行了总结。详细介绍了调度规则的选择方法,包括使用较多的稳态仿真方法和表现较好的人工智能方法,并给出了仿真方法、专家系统、机器学习方法以及人工神经网络方法,用于调度规则的选择时所取得的研究成果和结论。此外,还介绍了调度规则的评价指标及评价方法。最后针对调度规则存在的不足,指出了未来的研究方向。     

基于合同网协议交互投标的动态调度方法研究

柔性制造系统使生产加工路径有很多可选性，所以调度系统必须考虑机器调度问题。分配规则调度是一种最基本、最具影响力的动态调度方法。然而，分配规则调度方法很少考虑机器顺序选择。兼顾工件选择和机器选择两方面，本文运用交互投标过程，构建基于合同网协议调度的协商规则。研究作业车间动态调度问题，提出并构建了5种合同网规则调度方法。通过实验分析结果表明，基于合同网交互投标模式的规则调度能够大大改善调度系统性能，提高设备的利用率和设备负荷平衡指标。     

Reactive scheduling in a job shop where jobs arrive over time

The paper considers the dynamic job shop scheduling problem (DJSSP) with job release dates which arises widely in practical production systems. The principle characteristic of DJSSP considered in the paper is that the jobs arrive continuously in time and the attributes of the jobs, such as the release dates, routings and processing times are not known in advance, whereas in the classical job shop scheduling problem (CJSSP), it is assumed that all jobs to be processed are available at the beginning of the scheduling process. Reactive scheduling approach is one of the effective approaches for DJSSP. In the paper, a heuristic is proposed to implement the reactive scheduling of the jobs in the dynamic production environment. The proposed heuristic decomposes the original scheduling problem into a number of sub problems. Each sub problem, in fact, is a dynamic single machine scheduling problem with job release dates. The scheduling technique applied in theproposed heuristic is priority scheduling, which determines the next state of the system based on priority values of certain system elements. The system elements are prioritized with the help of scheduling rules (SRs). An approach based on gene expression programming (GEP) is also proposed in the paper to construct efficient SRs for DJSSP. The rules constructed by GEP are evaluated in the comparison of the rules constructed by GP and several prominent human made rules selected from literatures on extensive problem sets with respect to various measures of performance.     

Development of two-level decision tree-based real-time scheduling system under product mix variety environment

Most of the research on machine learning-based real-time scheduling (RTS) systems has been aimed toward product constant mix environments. However, in a product mix variety manufacturing environment, the scheduling knowledge base (KB) is dynamic; therefore, it would be interesting to develop a procedure that would automatically modify the scheduling knowledge when important changes occur in the manufacturing system. All of the machine learning-based RTS systems (including a KB refinement mechanism) proposed in earlier studies periodically require the addition of new training samples and regeneration of new KBs. Hence, previous approaches investigating machine learning-based RTS systems have been confronted with the training data overflow problem and an increase in the scheduling KB building time, which are unsuitable for RTS control. The objective of this paper is to develop a KB class selection mechanism that can be supported in various product mix ratio environments. Hence, the RTS KB is developed by a two-level decision tree (DT) learning approach. First, a suitable scheduling KB class is selected. Then, for each KB class, the best (proper) dispatching rule is selected for the next scheduling period. Here, the proposed two-level DT RTS system comprises five key components: (1) training samples generation mechanism, (2) GA/DT-based feature selection mechanism, (3) building a KB class label by a two-level self-organizing map, (4) DT-based KB class selection module, and (5) DT-based dynamic dispatching rule selection module. The proposed two-level DT-based KB RTS system yields better system performance than that by a one-level DT-based RTS system and heuristic individual dispatching rules in a flexible manufacturing system under various performance criteria over a long period.     

Integration of deep reinforcement learning and multi-agent system for dynamic scheduling of re-entrant hybrid flow shop considering worker fatigue and skill levels

In real-life manufacturing systems, production management is often affected by urgent demands and unexpected interruptions, such as new job insertions, machine breakdowns and operator unavailability. In this context, agent-based techniques are useful and able to respond quickly to dynamic disturbances. The ability of agents to recognize their environment and make decisions can be further enhanced by deep reinforcement learning (DRL). This paper investigates a novel dynamic re-entrant hybrid flow shop scheduling problem (DRHFSP) considering worker fatigue and skill levels to minimize the total tardiness of all production tasks. An integrated architecture of DRL and MAS (DRL-MAS) is proposed for real-time scheduling in dynamic environments. Two DRL models are proposed for different sub-decisions, where a reward-shaping technique combining long-term and short-term returns is proposed for the job sequence and machine selection sub-decisions, and an attention-based network is proposed for the worker assignment sub-decision for efficient feature extraction and decision making. Numerical experiments and case studies demonstrate the superior performance of the proposed DRL models compared with existing scheduling strategies.     

基于Agent的混合流水车间动态调度系统

针对敏捷制造调度环境的不确定性、动态性以及混合流水车间（HFS）调度问题的特点,设计了一种基于多Agent的混合流水车间动态调度系统,系统由管理Agent、策略Agent、工件Agent和机器Agent构成。首先提出一种针对混合流水车间环境的插值排序（HIS）算法并集成于策略Agent中,该算法适用于静态调度和多种动态事件下的动态调度。然后,设计了各类Agent间的协调机制,在生产过程中所有Agent根据各自的行为逻辑独立工作并互相协调。在发生动态事件时,策略Agent调用HIS算法根据当前车间状态产生工件序列,随后各Agent根据生成的序列继续进行协调直到完成生产。最后进行了发生机器故障、订单插入情况下的重调度以及在线调度等动态调度的实例仿真,结果表明对于这些问题,HIS算法的求解效果均优于调度规则,特别是在故障重调度中,HIS算法重调度前后的Makespan一致度达97.6%,说明系统能够灵活和有效地处理混合流水车间动态调度问题。     

车间调度中的数据挖掘技术研究

基于Multi-Agent System（MAS）的人机合作技术适合于解决复杂调度问题。为了使人与机能够更好地合作来完成高效、准确的车间调度,引入C4.5算法,建立并实现了基于机器学习和MAS的人机合作车间调度系统仿真模型。在Java环境下,以Weka、JADE为开发平台,以Eclipse为开发工具,Access为后台数据库,完成了系统的开发。通过实例仿真和结果分析,运用机器学习算法动态调度的结果稍优于最佳的静态调度结果,证明了系统的正确性和优越性。     

Study on shop floor control system in semiconductor fabrication by self-organizing map-based intelligent multi-controller

To confirm semiconductor wafer fabrication (FAB) operating characteristics, the scheduling decisions of shop floor control systems (SFCS) must develop a multiple scheduling rules (MSRs) approach in FABs. However, if a classical machine learning approach is used, an SFCS in FABs knowledge base (KB) can be developed by using the appropriate MSR strategy (this method is called an intelligent multi-controller in this study) as obtained from training examples. A classical machine learning approach main disadvantage is that the classes (scheduling decision variables) to which training examples are assigned must be pre-defined. This process becomes an intolerably time-consuming task. In addition, although the best decision rule can be determined for each scheduling decision variable, the combination of all the decision rules may not simultaneously satisfy the global objective function. To address these issues, this study proposes an intelligent multi-controller that incorporates three main mechanisms: (1) a simulation-based training example generation mechanism, (2) a data preprocessing mechanism, and (3) a self-organizing map (SOM)-based MSRs selection mechanism. These mechanisms can overcome the long training time problem of the classical machine learning approach in the training examples generation phase. Under various production performance criteria over a long period, the proposed intelligent multi-controller approach yields better system performance than fixed decision scheduling rules for each of the decision variables at the start of each production interval.     

周期性和事件驱动的JobShop滚动调度策略

为了适应加工的连续性及环境的变化，借用了预测控制中的滚动优化思想提出了周期性和事件驱动的滚动调度策略。调度算法将遗传算法和分派规则相结合，以此来处理与操作序列有关的工件安装时 间和工件到期时间约束的复杂调度问题。