An overview of the competitive and adversarial approaches to designing dynamic power management strategies

Dynamic power management (DPM) refers to the problem of judicious application of various low-power techniques based on runtime conditions in an embedded system to minimize the total energy consumption. To be effective, often such decisions take into account the operating conditions and the system-level design goals. DPM has been a subject of intense research in the past decade driven by the need for low power consumption in modern embedded devices. We present a comprehensive overview of two closely related approaches to designing DPM strategies, namely, competitive analysis approach and model checking approach based on adversarial modeling. Although many other approaches exist for solving the system-level DPM problem, these two approaches are closely related and are based on a common theme. This commonality is in the fact that the underlying model is that of a competition between the system and an adversary. The environment that puts service demands on devices is viewed as an adversary, or to be in competition with the system to make it burn more energy, and the DPM strategy is employed by the system to counter that.     

MapReduce optimization algorithm based on machine learning in heterogeneous cloud environment

We present an approach to optimize the MapReduce architecture, which could make heterogeneous cloud environment more stable and efficient. Fundamentally different from previous methods, our approach introduces the machine learning technique into MapReduce framework, and dynamically improve MapReduce algorithm according to the statistics result of machine learning. There are three main aspects： learning machine performance, reduce task assignment algorithm based on learning result, and speculative execution optimization mechanism. Furthermore, there are two important features in our approach. First, the MapReduce framework can obtain nodes＇ performance values in the cluster through machine learning module. And machine learning module will daily calibrate nodes＇ performance values to make an accurate assessment of cluster performance. Second, with the optimization of tasks assignment algorithm, we can maximize the performance of heterogeneous clusters. According to our evaluation result, the cluster performance could have 19% improvement in current heterogeneous cloud environment, and the stability of cluster has greatly enhanced.     

异构网络中用户关联和功率控制的协同优化

针对正交频分多址(Orthogonal Frequency Division Multiplexing Access,OFDMA)异构网络中用户关联和功率控制协同优化不佳的问题,提出了一种多智能体深度Q学习网络(Deep Q-learning Network,DQN)方法.首先,基于用户关联和功率控制最优化问题,构建了正交频分多址的双层异构网络系统模型,以实现智能决策;其次,根据应用场景和多智能体DQN框架的动作空间,对状态空间和奖励函数进行重构;最后,通过选取具有宏基站(Base Station,BS)和小型BS的两层异构网络,对多智能体DQN算法的性能进行仿真实验.仿真结果表明,相较于传统学习算法,多智能体DQN算法具有更好的收敛性,且能够有效提升用户设备(User Equipment,UE)的服务质量与能效,并可获得最大的长期总体网络实用性.     

Uncertainty-Aware Dynamic Power Management in Partially Observable Domains

This paper tackles the problem of dynamic power management (DPM) in nanoscale CMOS design technologies that are typically affected by increasing levels of process and temperature variations and fluctuations due to the randomness in the behavior of silicon structure. This uncertainty undermines the accuracy and effectiveness of traditional DPM approaches. This paper presents a stochastic framework to improve the accuracy of decision making during dynamic power management, while considering manufacturing process and/or environment induced uncertainties. More precisely, variability and uncertainty at the system level are captured by a partially observable semi-Markov decision process with interval-based definition of states while the policy optimization problem is formulated as a mathematical program based on this model. Experimental results with a RISC processor in 65-nm technology demonstrate the effectiveness of the technique and show that the proposed uncertainty-aware power management technique ensures system-wide energy savings under statistical circuit parameter variations.      

基于MSR模型的动态功耗管理策略

系统级动态功耗管理(Dynamic Power Management,DPM)策略根据系统状态和负载,动态地调整系统配置,从而能够有效降低系统功耗.传统的DPM策略仅从设备的角度考察工作负载状况,忽略了工作负载的应用特征.本文从任务的角度分析负载,提出新颖的多请求源(Multiple Service Requesters,MSR)系统级功耗管理的模型,以及基于该模型的自适应超时策略(Multiple-Service-Requester-Based Timeout Policy,MSRBTP).实验表明,与传统DPM策略相比较,在非平稳的应用环境下,MSRBTP策略具有更好更稳定的节能效果.     

基于空间约束多特征联合稀疏编码的遥感图像标注方法研究

针对稀疏表示分类器不能较好地适应多特征框架的问题,该文提出一种空间约束多特征联合稀疏编码模型,并以此实现遥感影像的自动标注。该方法利用l1,2混合范数正则化多特征编码系数,约束编码系数共享相同的稀疏模式,在保持多特征关联的同时,又不添加过于严格的约束。同时,将字典学习技术扩展到多特征框架中,通过约束字典更新的变换矩阵,解决了字典学习过程丢失多特征关联的问题。另外,针对遥感影像中的空间关系常常被忽略或者利用不充分的不足,还提出了将空间一致性与多特征联合稀疏编码相结合的分类准则,提高了标注性能。在遥感公开数据集与大尺寸卫星影像上的实验证明了该方法的有效性。     

PBALT动态电源管理策略

在嵌入式和便携式系统的低功耗设计中，动态电源管理（Dynamic Power Management，DPM）是一个非常重要的技术。DPM本质上是一种“在线”问题，因为PM（Power Management）策略必须在系统所有输入信息可用之前就能够对系统资源的使用情况做出正确的判断。本文在对自适应学习树（Adaptive Learning Tree，ALT）不足之处进行分析的基础上，提出了一种新颖的DPM策略——PBALT（Probability-Based ALT）。实验结果表明，PBALT具有很强的稳定性；而且在对空闲时段的预测准确性方面，PBALT比ALT具有更高的命中率。     

Toward intelligent resource management in dynamic Fog Computing-based Internet of Things environment with Deep Reinforcement Learning: A survey

Fog computing has already started to gain a lot of momentum in the industry for its ability to turn scattered computing resources into a large-scale, virtualized, and elastic computing environment. Resource management (RM) is one of the key challenges in fog computing which is also related to the success of fog computing. Deep learning has been applied to the fog computing field for some time, and it is widely used in large-scale network RM. Reinforcement learning (RL) is a type of machine learning algorithms, and it can be used to learn and make decisions based on reward signals that are obtained from interactions with the environment. We examine current research in this area, comparing RL and deep reinforcement learning (DRL) approaches with traditional algorithmic methods such as graph theory, heuristics, and greedy for managing resources in fog computing environments (published between 2013 and 2022) illustrating how RL and DRL algorithms can be more effective than conventional techniques. Various algorithms based on DRL has been shown to be applicable to RM problem and proved that it has a lot of potential in fog computing. A new microservice model based on the DRL framework is proposed to achieve the goal of efficient fog computing RM. The positive impact of this work is that it can successfully provide a resource manager to efficiently schedule resources and maximize the overall performance.     

Duty cycle learning algorithm (DCLA) for IEEE 802.15.4 beacon-enabled wireless sensor networks

The current specification of the IEEE 802.15.4 standard for beacon-enabled wireless sensor networks does not define how the fraction of the time that wireless nodes are active, known as the duty cycle, needs to be configured in order to achieve the optimal network performance in all traffic conditions. The work presented here proposes a duty cycle learning algorithm (DCLA) that adapts the duty cycle during run time without the need of human intervention in order to minimise power consumption while balancing probability of successful data delivery and delay constraints of the application. Running on coordinator devices, DCLA collects network statistics during each active duration to estimate the incoming traffic. Then, at each beacon interval uses the reinforcement learning (RL) framework as the method for learning the best duty cycle. Our approach eliminates the necessity for manually (re-)configuring the nodes duty cycle for the specific requirements of each network deployment. This presents the advantage of greatly reducing the time and cost of the wireless sensor network deployment, operation and management phases. DCLA has low memory and processing requirements making it suitable for typical wireless sensor platforms. Simulations show that DCLA achieves the best overall performance for either constant and event-based traffic when compared with existing IEEE 802.15.4 duty cycle adaptation schemes.     

A survey of design techniques for system-level dynamic powermanagement

Dynamic power management (DPM) is a design methodology for dynamically reconfiguring systems to provide the requested services and performance levels with a minimum number of active components or a minimum load on such components. DPM encompasses a set of techniques that achieves energy-efficient computation by selectively turning off (or reducing the performance of) system components when they are idle (or partially unexploited). In this paper, we survey several approaches to system-level dynamic power management. We first describe how systems employ power-manageable components and how the use of dynamic reconfiguration can impact the overall power consumption. We then analyze DPM implementation issues in electronic systems, and we survey recent initiatives in standardizing the hardware/software interface to enable software-controlled power management of hardware components     

Efficient co-simulation framework enhancing system-level power estimation for a platform-based SoC design

This paper proposes a hardware–software (HW-SW) co-simulation framework that provides a unified system-level power estimation platform for analyzing efficiently both the total power consumption of the target SoC and the power profiles of its individual components. The proposed approach employs the trace-based technique that reflects the real-time behavior of the target SoC by applying various operation scenarios to the high-level model of target SoC. The trace data together with corresponding look-up table (LUT) is utilized for the power analysis. The trace data is also used to reduce the number of input vectors required to analyze the power consumption of large H/W designs through the trade-offs between the signal probability in the trace results and its effect on the power consumption. The effect of cache miss on power, occurring in the S/W program execution, is also considered in the proposed framework. The performance of the proposed approach was evaluated through the case study using the SoC design example of IEEE 802.11a wireless LAN modem. The case study illustrated that, by providing fast and accurate power analysis results, the proposed approach can enable SoC designers to manage the power consumption effectively through the reconstruction of the target SoC. The proposed framework maps all hardware IPs into FPGA. The trace based approach gets input vectors at transactor of the each IP and gets power consumption indexing a LUT. This hardware oriented technique reports the power estimation result faster than the conventional ones doing it at S/W level.     

CAPPS: A Framework for Power–Performance Tradeoffs in Bus-Matrix-Based On-Chip Communication Architecture Synthesis

On-chip communication architectures have a significant impact on the power consumption and performance of emerging chip multiprocessor (CMP) applications. However, customization of such architectures for an application requires the exploration of a large design space. Designers need tools to rapidly explore and evaluate relevant communication architecture configurations exhibiting diverse power and performance characteristics. In this paper, we present an automated framework for fast system-level, application-specific, power–performance tradeoffs in a bus matrix communication architecture synthesis (CAPPS). Our study makes two specific contributions. First, we develop energy models for system-level exploration of bus matrix communication architectures. Second, we incorporate these models into a bus matrix synthesis flow that enables designers to efficiently explore the power–performance design space of different bus matrix configurations. Experimental results show that our energy macromodels incur less than 5% average cycle energy error across 180–65 nm technology libraries. Our early system-level power estimation approach also shows a significant speedup ranging from 1000 to 2000 $ times$ when compared with detailed gate-level power estimation. Furthermore, on applying our synthesis framework to three industrial networking CMP applications, a tradeoff space that exhibits up to 20% variation in power and up to 40% variation in performance is generated, demonstrating the usefulness of our approach.      

An efficient actor‐critic reinforcement learning for device‐to‐device communication underlaying sectored cellular network

In this paper, a novel reinforcement learning (RL) approach with cell sectoring is proposed to solve the channel and power allocation issue for a device‐to‐device (D2D)‐enabled cellular network when the prior traffic information is not known to the base station (BS). Further, this paper explores an optimal policy for resource and power allocation between users intending to maximize the sum‐rate of the overall system. Since the behavior of wireless channel and traffic request of users in the system is stochastic in nature, the dynamic property of the environment allows us to employ an actor‐critic RL technique to learn the best policy through continuous interaction with the surrounding. The proposed work comprises of four phases: cell splitting, clustering, queuing model, and channel allocation and power allocation simultaneously using an actor‐critic RL. The implementation of cell splitting with novel clustering technique increases the network coverage, reduces co‐channel cell interference, and minimizes the transmission power of nodes, whereas the queuing model solves the issue of waiting time for users in a priority‐based data transmission. With the help of continuous state‐action space, the actor‐critic RL algorithm based on policy gradient improves the overall system sum‐rate as well as the D2D throughput. The actor adopts a parameter‐based stochastic policy for giving continuous action while the critic estimates the policy and criticizes the actor for the action. This reduces the high variance of the policy gradient. Through numerical simulations, the benefit of our resource sharing scheme over other existing traditional scheme is verified.     

一种新颖的多agent强化学习方法

提出了一种综合了模块化结构、利益分配学习以及对手建模技术的多agent强化学习方法,利用模块化学习结构来克服状态空间的维数灾问题,将Q-学习与利益分配学习相结合以加快学习速度,采用基于观察的对手建模来预测其他agent的动作分布.追捕问题的仿真结果验证了所提方法的有效性.     

Classification with ensembles and case study on functional magnetic resonance imaging

The ensemble is a technique that strategically combines basic models to achieve better accuracy rates. Diversity, combination methods, and selection topology are the main factors determining ensemble performance. Consequently, it is a challenging task to design an efficient ensemble scheme. Even though numerous paradigms have been proposed to classify ensemble schemes, there is still much room for improvement. This paper proposes a general framework for creating ensembles in the context of classification. Specifically, the ensemble framework consists of four stages: objectives, data preparing, model training, and model testing. It is comprehensive to design diverse ensembles. The proposed ensemble approach can be used for a wide variety of machine learning tasks. We validate our approach on real-world datasets. The experimental results show the efficiency of the proposed approach.     

一种基于图注意力网络的异质信息网络表示学习框架

常用的异质信息网络有知识图谱和具有简单模式层的异质信息网络,它们的表示学习通常遵循不同的方法。该文总结了知识图谱和具有简单模式层的异质信息网络之间的异同,提出了一个通用的异质信息网络表示学习框架。该文提出的框架可以分为3个部分：基础向量模型,基于图注意力网络的传播模型以及任务模型。基础向量模型用于学习基础的网络向量;传播模型通过堆叠注意力层学习网络的高阶邻居特征;可更换的任务模型适用于不同的应用场景。与基准模型相比,该文所提框架在知识图谱的链接预测任务和异质信息网络的节点分类任务中都取得了相对不错的效果。     

Novel fuzzy reinforced learning vector quantisation algorithm and its application in image compression

A new approach to the design of optimised codebooks using vector quantisation (VQ) is presented. A strategy of reinforced learning (RL) is proposed which exploits the advantages offered by fuzzy clustering algorithms, competitive learning and knowledge of training vector and codevector configurations. Results are compared with the performance of the generalised Lloyd algorithm (GLA) and the fuzzy K-means (FKM) algorithm. It has been found that the proposed algorithm, fuzzy reinforced learning vector quantisation (FRLVQ), yields an improved quality of codebook design in an image compression application when FRLVQ is used as a pre-process. The investigations have also indicated that RL is insensitive to the selection of both the initial codebook and a learning rate control parameter, which is the only additional parameter introduced by RL from the standard FKM     

认知无线网络中基于模糊逻辑和Q学习的集中式动态频谱分配(英文)

A novel centralized approach for Dynamic Spectrum Allocation (DSA) in the Cognitive Radio (CR) network is presented in this paper. Instead of giving the solution in terms of formulas modeling network environment such as linear programming or convex optimization, the new approach obtains the capability of iteratively on-line learning environment performance by using Reinforcement Learning (RL) algorithm after observing the variability and uncertainty of the heterogeneous wireless networks. Appropriate decisi...     

The Agamid design-space exploration framework

The emergence of many-core processors raises novel demands to system design. Power-limitations and abundant parallelism require for efficient and scalable run-time management. The integration of dedicated hardware to enhance the performance of the run-time management system is gaining an increasing importance. But the design of a run-time manager for many-core generally suffers from exhaustive evaluation time. Previous works do not address for the required flexibility or do not address for reasonable evaluation time of the simulation framework. We propose the novel simulation framework Agamid to foster the development and evaluation of hardware enhanced run-time management for many-core. Our transaction-level framework performs design point evaluation of hardware enhanced run-time management for many-core at the timescale of seconds. We use a hybrid simulation approach considering the run-time management and the user application at different levels of abstraction. The framework provides a generic run-time manager to compare arbitrary management systems and HW/SW partitionings. The implementation of the run-time manager facilitates direct execution at the host machine and a detailed synchronization model. Agamid applies user application workloads by means of transaction-based task graphs. An extendable system-call interface allows arbitrary interaction between the user application and the run-time management system. The thorough calibration of the RTM timing model enables reasonable approximations of the management overhead. Our evaluation considers the accuracy, wall-time and design space exploration capabilities of Agamid. Our findings substantiate the usefulness to integrate the modeling of the run-time management, hardware architecture and user application into a single transaction-level framework.     

无线网络多用户干扰下智能发射功率控制算法

针对无线网络多用户互相干扰的问题,通过对发射功率进行智能控制,实现干扰管理,保证多用户通信服务质量。首先,考虑复杂动态无线信道环境,建立以无线通信系统加权数据速率最大化为目标的发射功率控制模型。其次,设计以深度强化学习"行动器-评判器"为基本架构的智能发射功率控制算法,缩短功率控制决策时间。仿真验证表明,所提算法收敛速度快,在10对收发机场景下,计算时间缩短到传统最优算法的1/4。