Routing using reinforcement learning in vehicular ad hoc networks

In vehicular ad hoc networks (VANETs), the frequent change in vehicle mobility creates dynamic changes in communication link and topology of the network. Hence, the key challenge is to address and resolve longer transmission delays and reduced transmission stability. During the establishment of routing path, the focus of entire research is on traffic detection and road selection with high traffic density for increased packet transmission. This reduces the transmission delays and avoids carry-and-forward scenarios; however, these techniques fail in obtaining accurate traffic density in real-time scenario due to rapid change in traffic density. Thus, it is necessary to create a model that efficiently monitors the traffic density and assist VANETs in route selection in an automated way with increased accuracy. In this article, a novel machine learning architecture using deep reinforcement learning (DRL) model is proposed to monitor and estimate the data essential for the routing protocol. In this model, the roadside unit maintains the traffic information on roads using DRL. The DRL predicts the movement of the vehicle and makes a suitable routing path for transmitting the packets with improved transmission capacity. It further uses predicted transmission delays and the destination location to choose the forwarding directions between two road safety units (RSUs). The application of DRL over VANETs yields increased network performance, which provides on-demand routing information. The simulation results show that the DRL-based routing is effective in routing the data packets between the source and destination vehicles than other existing method.     

A flow allocation strategy for routing over multiple flow classes with an application to air cargo terminals

Advances of information technology have enabled the utilization of automated material handling systems in the logistics industry. The increasing costs of labor in developing countries have accelerated this trend. Major cargo terminals are now installing more and more integrated automated shipment handling systems in order to increase their operational efficiency which can be measured by the average shipping times or the facility throughput, for example. Routing is clearly an important decision category that has significant impact on the operational efficiency. In this paper, motivated by a project with one of the busiest air cargo terminals in the world, we investigate a routing optimization problem for multiple flow classes with different levels of priority. We propose a flow allocation (FA) routing strategy in which when a shipment arrives at a decision point, a set of allocation ratios will be employed to direct it to the next location. These ratios are determined by solving a mathematical model that explicitly considers the congestion effect and the characteristics of the multi-commodity network. Comprehensive simulation experiments demonstrate that the proposed FA routing strategy significantly outperforms the one currently in use.     

阻止信息收集的供应链契约

针对内生信息结构下供应链中阻止信息收集的问题,建立一个非线性规划模型.研究发现,存在一个作为给出不同契约参考点的信息收集成本阈值,进而给出了不同信息收集成本下阻止信息收集的契约,分析比较两种不同契约下供销双方的决策和收益,并且证明了当信息收集成本大于信息收集成本阈值时,供应商更愿意阻止信息收集.此时,供应商获得相对更好的收益,而销售商获得保留收益.最后给出的数值算例验证了主要结论.     

Communication reduction in multiple multicasts based on hybrid static-dynamic scheduling

This paper presents a novel approach to reducing the communication costs incurred when performing multiple multicasts on wormhole routed two-dimensional mesh multiprocessor systems. Both unicast and path-based implementations of multicasting incur communication costs due to the inherent message passing and contention for network resources. The start-up time dominates the transmission time when the data volume is small. However, in the presence of multiple multicasts when the data volume is very large, the communication delays due to message blocking and resource contention become very significant. Because of this, we present a hybrid static-dynamic technique to reduce the communication costs incurred when performing multiple multicasts on wormhole routed direct networks. This technique requires a focus on ordering and routing information for the individual message transmissions. At compile time, each message is assigned a priority using the recently developed collision graph model. Then at runtime these priorities are used to arbitrate the message transmissions. As a base, dimension-ordered routing is used. However, to further reduce the communication costs, some messages will be rerouted. This technique is useful either as a stand-alone algorithm or as an embedded procedure into existing algorithms. Furthermore, the techniques can be applied to higher dimension direct networks. For a single multicast, our work performs as well as conventional methods. For multiple multicasts, results show that our approach provides significant improvement over baseline techniques.     

Understanding optimal data gathering in the energy and latency domains of a wireless sensor network

The problem of optimal data gathering in wireless sensor networks (WSNs) is addressed by means of optimization techniques. The goal of this work is to lay the foundations to develop algorithms and techniques that minimize the data gathering latency and at the same time balance the energy consumption among the nodes, so as to maximize the network lifetime. Following an incremental-complexity approach, several mathematical programming problems are proposed with focus on different network performance metrics. First, the static routing problem is formulated for large and dense WSNs. Optimal data-gathering trees are analyzed and the effects of several sensor capabilities and constraints are discussed, e.g., radio power constraints, energy consumption model, and data aggregation functionalities. Then, dynamic re-routing and scheduling are considered. An accurate network model is proposed that captures the tradeoff between the data gathering latency and the energy consumption, by modeling the interactions among the routing, medium access control and physical layers.For each problem, extensive simulation results are provided. The proposed models provide a deeper insight into the problem of timely and energy efficient data gathering. Useful guidelines for the design of efficient WSNs are derived and discussed.     

Markov decision processes with delays and asynchronous cost collection

Markov decision processes (MDPs) may involve three types of delays. First, state information, rather than being available instantaneously, may arrive with a delay (observation delay). Second, an action may take effect at a later decision stage rather than immediately (action delay). Third, the cost induced by an action may be collected after a number of stages (cost delay). We de rive two results, one for constant and one for random delays, for reducing an MDP with delays to an MDP without delays, which differs only in the size of the state space. The results are based on the intuition that costs may be collected asynchronously, i.e., at a stage other than the one in which they are induced, as long as they are discounted properly.     

Improving robustness of rolling stock circulations in rapid transit networks

The rolling stock circulation depends on two different problems: the rolling stock assignment and the train routing problems, which up to now have been solved sequentially. We propose a new approach to obtain better and more robust circulations of the rolling stock train units, solving the rolling stock assignment while accounting for the train routing problem. Here robustness means that difficult shunting operations are selectively penalized and propagated delays together with the need for human resources are minimized. This new integrated approach provides a huge model. Then, we solve the integrated model using Benders decomposition, where the main decision is the rolling stock assignment and the train routing is in the second level. For computational reasons we propose a heuristic based on Benders decomposition. Computational experiments show how the current solution operated by RENFE (the main Spanish train operator) can be improved: more robust and efficient solutions are obtained.     

Management,control, and design of integrated networks with real-time dynamic routing

Integrated real-time dynamic routing (IRR) networks provide dynamic routing features for multiple classes-of-service on an integrated transport network. In this paper it is shown that IRR networks allow reduced network management costs since with real-time dynamic routing a number of network operations are simplified or eliminated. These simplifications include eliminating the storage of voluminous routing tables in the network switches, eliminating the calculation of routing tables in network design, simplifying the routing administration operations which require downloading new routing information to the network, and eliminating the automatic rerouting function in on-line traffic management. A new bandwidth allocation technique is described here which is based on the optimal solution of a network bandwidth allocation model for IRR networks. The model achieves significant improvement in both the average network blocking and node pair blocking distribution when the network is in a congested state such as under peak-day loads. In a paper to appear in the next Journal issue we further describe a new algorithm for the transport design of IRR networks which achieves near-optimal capacity engineering. These optimization techniques attain significant capital cost reductions and network performance improvements by properly modeling the more efficient operation of IRR networks.     

Efficient hierarchical QoS routing in ATM networks

For reducing network information to achieve scalability in large ATM networks, ATM Private Network-to-Network Interface (PNNI) adopts hierarchical routing. Consequently, although routing complexity is significantly reduced, numerous issues in PNNI routing require further study to achieve more efficient, accurate, scalable, and QoS-aware routing.Several methods are adopted herein to achieve efficient, scalable, and QoS-aware ATM PNNI routing. First, an efficient aggregation scheme, referred to as Asymmetric Simple, is proposed. The aggregated routing information includes available bandwidth, delay and cost. Second, two approaches for defining link costs are investigated, namely, the Markov Decision Process (MDP) approach and the Competitive On-Line (COL) routing approach, and these are compared with the Widest Path (WP) approach. Finally, a dynamic update policy, referred to as the dynamic cost-based update (DCU) policy, is proposed to improve the accuracy of the aggregated information and the performance of hierarchical routing, while decreasing the frequency of re-aggregation and information distribution.Simulation results demonstrate that the proposed Asymmetric Simple aggregation scheme yields very good network utilization while significantly reducing the amount of advertised information. Between these two link cost functions, the MDP approach provides a systematic method of defining call admission function and yields better network utilization than the COL approach. The proposed DCU policy also yields an enhanced network utilization while significantly reducing the frequency of re-aggregation and the amount of distributed aggregation information.     

A Value-Driven System for Autonomous Information Gathering

This paper presents a system for autonomous information gathering in an information rich domain under time and monetary resource restrictions. The system gathers information using an explicit representation of the user's decision model and a database of information sources. Information gathering is performed by repeatedly selecting the query with the highest marginal value. This value is determined by the value of the information with respect to the decision being made, the responsiveness of the information source, and a given resource cost function. Finally, we compare the value-driven approach to several base-line techniques and show that the overhead of the meta-level control is made up for by the increased decision quality.     

考虑碳排放的物流网络优化模型研究

考虑碳排放的物流网络优化问题, 分两阶段建立考虑碳排放的配送中心选择与需求匹配模型和碳排放成本最小的配送路径优化模型, 通过两阶段模型的配合使用, 解决了物流网络中配送中心选址、不同物流节点间的需求匹配以及配送车辆路径优化等多种决策问题。对“考虑碳排放”与“不考虑碳排放”情况进行Lingo软件的数值分析, 结果表明：考虑碳排放的物流网络综合物流成本明显降低, 比不考虑碳排放的传统方法更具现实意义。     

A framework for end-to-end deterministic-delay service provisioning in multiservice packet networks

The problem of providing end-to-end delay guarantees for deterministic-delay services in multiservice packet networks is addressed through a combination of dynamic resource reservation and routing. Our model is based on using rate-controlled earliest-deadline-first (RC-EDF) for providing hard bounds on end-to-end delays. With RC-EDF, a certain delay bound has to be allocated for a connection at each node in the selected path. The most commonly used resource reservation policy is uniform allocation which is based on dividing the end-to-end delay bound equally among the nodes in the selected path. This simple allocation policy could lead to nonuniform resource loading and subsequently lead to high blocking rates. Moreover, the most commonly used routing method is shortest-path first routing which is known to lead to network hotspots. We propose a set of dynamic nonuniform resource reservation policies and dynamic routing methods. One of the routing methods is the well-known widest-shortest path method and the other is a dynamic routing method that adaptively adjusts link costs and uses a similar algorithm to shortest-path routing (e.g., Dijkstra's algorithm). We show that for both uniform and nonuniform traffic loading of some example network topologies that the combination of the proposed resource reservation policies and dynamic routing can lead to significant reduction in the connection blocking ratio in all loading conditions except for excessively high loads.     

Variable neighborhood search for the inventory routing and scheduling problem in a supply chain

The inventory, routing and scheduling decisions are three major driving factors for supply chain performance. Since they are related to one another in a supply chain, they should be determined simultaneously to improve the decision quality. In the past, the inventory policy, vehicle routing and vehicle scheduling are determined sequentially and separately. Hence, the total cost (inventory, routing and vehicle costs) would increase. In this paper, an integrated model for the inventory routing and scheduling problem (IRSP) is proposed. Since searching for the optimal solution for this model is a non-polynomial (NP) problem, a metaheuristic, variable neighborhood search (VNS), is proposed. The proposed method was compared with other existing methods. The experimental results indicate that the proposed method is better than other methods in terms of average cost per day.     

Coordinating efficiency and equity in disaster relief logistics via information updates

This paper addresses a multi-supplier, multi-affected area, multi-relief, and multi-vehicle relief allocation problem in disaster relief logistics. A multi-objective optimisation model based on disaster scenario information updates is proposed in an attempt to coordinate efficiency and equity through timely and appropriate decisions regarding issues such as vehicle routing and relief allocation. An optimal stopping rule is also proposed to determine the optimum period of delay before responding to disaster, because decision making requires accurate disaster information. The main contribution of this paper is solving relief allocation problem in a novel way by correlating operational research with statistical decision making and Bayesian sequential analysis. Finally, a case is presented based on the post-disaster rescue in Eastern China after supertyphoon Saomai to test the applicability and show the potential advantages of the proposed model.     

Modular fault-tolerant Boolean n-cubes

The modular design approach we propose for fault-tolerant Boolean n-cube architectures lets us reduce hardware cost and increase system reliability, while assuring high performance during message routing. Use of a specific module size with shared links and switches during reconfiguration helps keep system costs low. Though this strategy may increase the number of routing steps needed to transmit a message between nodes, routing performance nonetheless surpasses competing approaches, if we consider switching delays     

Performance model and evaluation on geographic-based routing

In order to improve the inter-domain routing performance as well as its route aggregation, people have proposed to use geographical information to assist the routing system in both addressing architecture and routing mechanisms. Researches on such mechanism focus on reducing the geographical length of the path selected by routing mechanism. However, to the best knowledge of authors, there is no evaluation about how such reduction influences the actual end-to-end data delivery performance. In this paper, a simple model considering both processing and propagation delays is developed and a method to evaluate the performance of a routing mechanism is presented. Also, a heuristic algorithm to optimize routing performance is presented base on the model. After that, the performance of Geographic-Based Routing (GBR), compared to current practice using Border Gateway Protocol (BGP), is evaluated through a simulation using concrete statistical data accordingly. Result shows that GBR did not show performance improvement in most of the time, and current practice can be improved by more than 50% in performance theoretically. It is believed that this is the first work to model and examine the performance of inter-domain routing in the context of actual packet delivery behavior.     

The rescheduling arc routing problem

In this paper, the rescheduling arc routing problem is introduced. This is a dynamic routing and scheduling problem that considers adjustments to an initial routing itinerary when one or more vehicle failures occur during the execution stage and the original plan must be modified. We minimize the operational and schedule disruption costs. Formulations based on mixed‐integer programming are presented to compare different policies in the rerouting phase. A solution strategy is developed when both costs are evaluated and it is necessary to find a solution quickly. Computational tests on a large set of instances compare the different decision‐maker policies.     

无线传感器网络中一种混合动态自适应路由算法

针对无线传感器网络的节能以及能耗均衡问题，本文提出了一种无线传感器网络混合路由网络模型，将平面路由和层次路由有机地结合在一起，在数据获取阶段采用层次路由，而在数据传输过程中使用平面路由。同时，论文提出了一种基于该模型的动态成簇自适应路由算法HDAR（Hybrid Dynamic Adaptive Routing algorithm）。在算法中设计了基于现场数据的动态成簇机制来完成数据的收集，使用自适应的路由选择算法将数据传输回Sink节点。仿真结果表明HDAR协议在节能和能耗均衡方面达到了良好的效果。     

Routing with multi-level cross-community social groups in mobile opportunistic networks

Mobile opportunistic networks (MONs) are intermittently connected networks, such as pocket switched networks formed by human-carried mobile devices. Routing in MONs is very challenging as it must handle network partitioning, long delays, and dynamic topology. Flooding is a possible solution but with high costs. Most existing routing methods for MONs avoid the costly flooding by selecting one or multiple relays to deliver data during each encounter. How to pick the “good” relay from all encounters is a non-trivial task. To achieve efficient delivery of messages at low costs, in this paper, we propose a novel group-based routing protocol in which the relay node is selected based on multi-level cross-community social group information. We apply a simple group formation method to both historical encounters (social relationships in physical world) and/or social profiles of mobile users (social relationships in social world) and build multi-level cross-community social groups, which summarize the wide range of social relationships among all mobile participants. Our simulations over several real-life data sets demonstrate the efficiency and effectiveness of the proposed method by comparing it with several existing MON routing schemes.