网格的数据挖掘*

网格是网络计算、分布式计算和高性能计算技术研究的热点。随着科学计算领域中的数据剧烈增长以及未来网格计算环境下广域分布的海量数据共享成为现实,数据挖掘技术将在挖掘有效的信息、发现新的知识和规律发挥着重要的作用。结合网格的特点,概述了网格数据挖掘的特点和关键技术,重点讨论了网格数据挖掘的体系结构和基本过程,最后给出了基于OGSA的网格数据挖掘的例子。     

An appraisal and design of a multi-agent system based cooperative wireless intrusion detection computational intelligence technique

The deployment of wireless sensor networks and mobile ad-hoc networks in applications such as emergency services, warfare and health monitoring poses the threat of various cyber hazards, intrusions and attacks as a consequence of these networks’ openness. Among the most significant research difficulties in such networks safety is intrusion detection, whose target is to distinguish between misuse and abnormal behavior so as to ensure secure, reliable network operations and services. Intrusion detection is best delivered by multi-agent system technologies and advanced computing techniques. To date, diverse soft computing and machine learning techniques in terms of computational intelligence have been utilized to create Intrusion Detection and Prevention Systems (IDPS), yet the literature does not report any state-of-the-art reviews investigating the performance and consequences of such techniques solving wireless environment intrusion recognition issues as they gain entry into cloud computing. The principal contribution of this paper is a review and categorization of existing IDPS schemes in terms of traditional artificial computational intelligence with a multi-agent support. The significance of the techniques and methodologies and their performance and limitations are additionally analyzed in this study, and the limitations are addressed as challenges to obtain a set of requirements for IDPS in establishing a collaborative-based wireless IDPS (Co-WIDPS) architectural design. It amalgamates a fuzzy reinforcement learning knowledge management by creating a far superior technological platform that is far more accurate in detecting attacks. In conclusion, we elaborate on several key future research topics with the potential to accelerate the progress and deployment of computational intelligence based Co-WIDPSs.     

容器技术在科学计算中的应用研究

作为一种新兴的虚拟化技术,容器能够以低廉的资源开销为应用程序和服务提供隔离的运行环境,近年来在持续集成和持续部署、自动化测试、微服务等多种业务场景中获得了广泛应用。在科学计算领域,容器技术的应用也正获得越来越多的关注。借助自身的打包能力及日益壮大的生态系统,容器技术有望为科学计算领域的生产力提升提供助力。文中对容器技术在科学计算中的应用现状进行了调研分析,并根据现有的应用实例讨论了在科学计算中使用容器及相关技术的多种方式。对不同应用模式的分析研究表明,通过提升应用程序的可移植性、改善研究的可重复性、提供非传统应用部署方案、简化云资源调度管理等多种方式,容器及相关技术可以为科学计算领域带来多方面的效率提升。     

遥感影像的高性能并行处理技术研究

随着空间遥感技术和对地观测技术的不断发展,光学、热红外和微波等不同技术手段可以获取同一地区的多种遥感影像数据（多时相、多光谱、多传感器、多平台和多分辨率等）,每天获取的遥感数据量越来越大。同时,大量的遥感应用需要快速地对这些遥感数据进行处理与分析,提供辅助决策信息。因此,如果不能及时进行数据处理,这些数据就会失去时效性,甚至失去数据本身的价值。高性能计算与并行处理技术,加速了遥感影像数据处理与信息提取的进度,如大规模多处理系统、网格与云计算技术、通用图形处理器（GPGPU）等。文中综述了高性能计算、并行处理及云计算技术应用于遥感领域的最新进展,给出了一些研究与应用范例,并提出了当前高性能遥感影像处理所面临的一些挑战。     

A characterization of workflow management systems for extreme-scale applications

Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today’s computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.     

微服务环境下容器编排可视化实践研究

伴随着云计算的快速发展,海量数据等业务需求的处理无法只依赖单体应用程序。微服务软件架构模式以其模块化、可扩展、高可用的应用优势为应用程序的开发带来了新的设计思路。容器是基于共享Linux内核、面向应用的一种新兴的轻量级虚拟化技术,以Docker为代表的容器技术为微服务提供了理想的载体。同时,以Kubernetes为代表的容器编排工具则极大地简化了容器化微服务创建、集成、部署、运维的整个流程。在开发和运维向“面向容器”的转变中,会带来数量庞大且关系复杂的服务组合,此时微服务的创建与部署则变得尤为重要。从易用性角度出发,提供了一种容器编排的可视化方法,实践分析显示,利用此方法进行的微服务部署不仅为研发人员提供了友好型服务创建界面,而且还便利了服务创建过程,提高了开发效率。     

An analytical study of central and in-network data processing for wireless sensor networks

This paper compares the performance of centralized and in-network data processing for wireless sensor networks (WSNs) under various deployment conditions on the real sensor hardware Sun SPOT from Sun Microsystems. We define several criteria to measure the quality of responses in WSN applications. Guided by an extensive experimental study, we discuss in detail the performance impacts of different deployment factors on algorithms that implement both centralized and in-network computing. Finally, performance guidelines are given to algorithm designers for WSN applications.     

基于RISC-V架构的强化学习容器化方法研究

RISC-V作为近年来最热门的开源指令集架构,被广泛应用于各个特定领域的微处理器,特别是机器学习领域的模块化定制.但是,现有的RISC-V应用需要将传统软件或模型在RISC-V指令集上重新编译或优化,故如何能快速地在RISC-V体系结构上部署、运行和测试机器学习框架是一个亟待解决的技术问题.使用虚拟化技术可以解决跨平台...     

Customizable execution environments for evolutionary computation using BOINC + virtualization

Evolutionary algorithms (EAs) consume large amounts of computational resources, particularly when they are used to solve real-world problems that require complex fitness evaluations. Beside the lack of resources, scientists face another problem: the absence of the required expertise to adapt applications for parallel and distributed computing models. Moreover, the computing power of PCs is frequently underused at institutions, as desktops are usually devoted to administrative tasks. Therefore, the proposal in this work consists of providing a framework that allows researchers to massively deploy EA experiments by exploiting the computing power of their instituions’ PCs by setting up a Desktop Grid System based on the BOINC middleware. This paper presents a new model for running unmodified applications within BOINC with a web-based centralized management system for available resources. Thanks to this proposal, researchers can run scientific applications without modifying the application’s source code, and at the same time manage thousands of computers from a single web page. Summarizing, this model allows the creation of on-demand customized execution environments within BOINC that can be used to harness unused computational resources for complex computational experiments, such as EAs. To show the performance of this model, a real-world application of Genetic Programming was used and tested through a centrally-managed desktop grid infrastructure. Results show the feasibility of the approach that has allowed researchers to generate new solutions by means of an easy to use and manage distributed system.     

IB网上CPU-GPU异构超算平台容器性能评估及优化

为了实现资源和系统环境的隔离,近年来新兴了多种虚拟化工具,容器便是其中之一。在超算资源上运行的问题通常是由软件配置引起的。容器的一个作用就是将依赖打包进轻量级可移植的环境中,这样可以提高超算应用程序的部署效率。为了解基于IB网的CPU-GPU异构超算平台上容器虚拟化技术的性能特征,使用标准基准测试工具对Docker容器进行了全面的性能评估。该方法能够评估容器在虚拟化宿主机过程中产生的性能开销,包括文件系统访问性能、并行通信性能及GPU计算性能。结果表明,容器具备近乎原生宿主机的性能,文件系统I/O开销及GPU计算开销与原生宿主机差别不大。随着网络负载的增大,容器的并行通信开销也相应增大。根据评估结果,提出了一种能够发挥超算平台容器性能的方法,为使用者有针对性地进行系统配置、合理设计应用程序提供依据。     

A study on virtual machine deployment for application outsourcing in mobile cloud computing

In mobile cloud computing, application offloading is implemented as a software level solution for augmenting computing potentials of smart mobile devices. VM is one of the prominent approaches for offloading computational load to cloud server nodes. A challenging aspect of such frameworks is the additional computing resources utilization in the deployment and management of VM on Smartphone. The deployment of Virtual Machine (VM) requires computing resources for VM creation and configuration. The management of VM includes computing resources utilization in the monitoring of VM in entire lifecycle and physical resources management for VM on Smartphone. The objective of this work is to ensure that VM deployment and management requires additional computing resources on mobile device for application offloading. This paper analyzes the impact of VM deployment and management on the execution time of application in different experiments. We investigate VM deployment and management for application processing in simulation environment by using CloudSim, which is a simulation toolkit that provides an extensible simulation framework to model the simulation of VM deployment and management for application processing in cloud-computing infrastructure. VM deployment and management in application processing is evaluated by analyzing VM deployment, the execution time of applications and total execution time of the simulation. The analysis concludes that VM deployment and management require additional resources on the computing host. Therefore, VM deployment is a heavyweight approach for process offloading on smart mobile devices.     

An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2

Workflows are used to orchestrate data-intensive applications in many different scientific domains. Workflow applications typically communicate data between processing steps using intermediate files. When tasks are distributed, these files are either transferred from one computational node to another, or accessed through a shared storage system. As a result, the efficient management of data is a key factor in achieving good performance for workflow applications in distributed environments. In this paper we investigate some of the ways in which data can be managed for workflows in the cloud. We ran experiments using three typical workflow applications on Amazon’s EC2 cloud computing platform. We discuss the various storage and file systems we used, describe the issues and problems we encountered deploying them on EC2, and analyze the resulting performance and cost of the workflows.     

Big data deployment in containerized infrastructures through the interconnection of network namespaces

Big Data applications tackle the challenge of fast handling of large streams of data. Their performance is not only dependent on the data frameworks implementation and the underlying hardware but also on the deployment scheme and its potential for fast scaling. Consequently, several efforts have focused on the ease of deployment of Big Data applications, notably through the use of containerization. This technology was indeed raised to bring multitenancy and multiprocessing out of clusters, providing high deployment flexibility through lightweight container images. Recent studies have focused mostly on Docker containers. Notwithstanding, this article is actually interested in recent Singularity containers as they provide more security and support high-performance computing (HPC) environments and, in this way, they can make Big Data applications benefit from the specialized hardware of HPC. Singularity 2.x, however, does not isolate network resources as required by most Big Data components. Singularity 3.x allows allocating each container with isolated network resources, but their interconnection requires a nontrivial amount of configuration effort. In this context, this article makes a functional contribution in the form of a deployment scheme based on the interconnection of network namespaces, through underlay and overlay networking approaches, to make Big Data applications easily deployable inside Singularity containers. We provide detailed account of our deployment scheme when using both interconnection approaches in the form of a “how-to-do-it” report, and we evaluate it by comparing three Big Data applications based on Hadoop when performing on a bare-metal infrastructure and on scenarios involving Singularity and Docker instances.     

Distributed dynamic load balancing for pipelined computations on heterogeneous systems

One of the most significant causes for performance degradation of scientific and engineering applications on high performance computing systems is the uneven distribution of the computational work to the resources of the system. This effect, which is known as load imbalance, is even more noticeable in the case of irregular applications and heterogeneous distributed systems. This motivated the parallel and distributed computing research community to focus on methods that provide good load balancing for scientific and engineering applications running on (heterogeneous) distributed systems. Efficient load balancing and scheduling methods are employed for scientific applications from various fields, such as mechanics, materials, physics, chemistry, biology, applied mathematics, etc. Such applications typically employ a large number of computational methods in order to simulate complex phenomena, on very large scales of time and magnitude. These simulations consist of routines that perform repetitive computations (in the form of DO/FOR loops) over very large data sets, which, if not properly implemented and executed, may suffer from poor performance. The number of repetitive computations in the simulation codes is not always constant. Moreover, the computational nature of these simulations may be in fact irregular, leading to the case when one computation takes (unpredictably) more time than others. For successful and timely results, large scale simulations require the use of large scale computing systems, which often are widely distributed and highly heterogeneous. Moreover, large scale computing systems are usually shared among multiple users, which causes the quality and quantity of the available resources to be highly unpredictable. There are numerous load balancing methods in the literature for different parallel architectures. The most recent of these methods typically follow the master-worker paradigm, where a single coordinator (master) is responsible for making all the scheduling decisions based on information provided by the workers. Depending on the application requirements, the scheduling policy and the computational environment, the benefits of this paradigm may be limited as follows: (1) its efficiency may not scale as the number of processors increases, and (2) it is quite probable that the scheduling decisions are made based on outdated information, especially on systems where the workload changes rapidly. In an effort to address these limitations, we propose a distributed (master-less) load balancing scheme, in which the scheduling decisions are made by the workers in a distributed fashion. We implemented this method along with other two master-worker schemes (a previously existing one and a recently modified one) for three different scientific computational kernels. In order to validate the usefulness and efficiency of the proposed scheme, we conducted a series of comparative performance tests with the two master-worker schemes for each computational kernel. The target system is an SMP cluster, on which we simulated three different patterns of system load fluctuation. The experiments strongly support the belief that the distributed approach offers greater performance and better scalability on such systems, showing an overall improvement ranging from 13% to 24% over the master-worker approaches.     

Container-based cluster orchestration systems: A taxonomy and future directions

Containers, enabling lightweight environment and performance isolation, fast and flexible deployment, and fine-grained resource sharing, have gained popularity in better application management and deployment in addition to hardware virtualization. They are being widely used by organizations to deploy their increasingly diverse workloads derived from modern-day applications such as web services, big data, and internet of things in either proprietary clusters or private and public cloud data centers. This has led to the emergence of container orchestration platforms, which are designed to manage the deployment of containerized applications in large-scale clusters. These systems are capable of running hundreds of thousands of jobs across thousands of machines. To do so efficiently, they must address several important challenges including scalability, fault tolerance and availability, efficient resource utilization, and request throughput maximization among others. This paper studies these management systems and proposes a taxonomy that identifies different mechanisms that can be used to meet the aforementioned challenges. The proposed classification is then applied to various state-of-the-art systems leading to the identification of open research challenges and gaps in the literature intended as future directions for researchers.     

Simulations of amphiphilic fluids using mesoscale lattice-Boltzmann and lattice-gas methods

We compare two recently developed mesoscale models of binary immiscible and ternary amphiphilic fluids. We describe and compare the algorithms in detail and discuss their stability properties. The simulation results for the cases of self-assembly of ternary droplet phases and binary water-amphiphile sponge phases are compared and discussed. Both models require parallel implementation and deployment on large scale parallel computing resources in order to achieve reasonable simulation times for three-dimensional models. The parallelization strategies and performance on two distinct parallel architectures are compared and discussed. Large scale three-dimensional simulation of multiphase fluids requires the extensive use of high performance visualization techniques in order to enable the large quantities of complex data to be interpreted. We report on our experiences with two commercial visualization products: AVS and VTK. We also discuss the application and use of novel computational steering techniques for the more efficient utilization of high performance computing resources. We close the paper with some suggestions for the future development of both models.     

Service-oriented grid computing system for digital rights management (GC-DRM)

Service-oriented computing and applications have recently gained significant attention since they provide new service infrastructure and development of service-oriented technology. Under such trend and ubiquitous computing requirement, grid computing is becoming popular in scientific and enterprise computing due to its flexible deployment and implementation. In this paper, we proposed a service-oriented digital rights management (DRM) platform based on grid computing (called GC-DRM) which is in the compliance of Grid Portal standards by using porlet. The platform integrates Globus Toolkit 4 and Condor 6.9.2 and uses web 2.0 to construct the web-based user interface for providing job submission, control, management, monitor for DRM services. GC-DRM can provide different categories of services which include watermark embedding and extraction, image scrambling, visible watermark embedding, image tamper-detection and recovery. In addition, GC-DRM has been applied to analyze the robustness of digital watermark by filter bank selection and the performance can be improved in the aspect of speedup, stability and processing time compared with NaradaBrokering based Computing Power Services (NB-CPS) and Web Services based Computing Power Service (WS-CPS). Therefore, GC-DRM can be concluded as a superior service-oriented computing which provides the user friendly environment with efficient DRM service performance based on grid computing architecture.     

Performance evaluation of the SX‐6 vector architecture for scientific computations

The growing gap between sustained and peak performance for scientific applications is a well‐known problem in high‐performance computing. The recent development of parallel vector systems offers the potential to reduce this gap for many computational science codes and deliver a substantial increase in computing capabilities. This paper examines the intranode performance of the NEC SX‐6 vector processor, and compares it against the cache‐based IBM Power3 and Power4 superscalar architectures, across a number of key scientific computing areas. First, we present the performance of a microbenchmark suite that examines many low‐level machine characteristics. Next, we study the behavior of the NAS Parallel Benchmarks. Finally, we evaluate the performance of several scientific computing codes. Overall results demonstrate that the SX‐6 achieves high performance on a large fraction of our application suite and often significantly outperforms the cache‐based architectures. However, certain classes of applications are not easily amenable to vectorization and would require extensive algorithm and implementation reengineering to utilize the SX‐6 effectively. Copyright © 2005 John Wiley & Sons, Ltd.     

Grid-based framework for high-performance processing of scientific knowledge

An essential matter in the knowledge-based information society is how to extract useful information quickly from a large volume of literature. Since most existing data mining frameworks deal with structured input data, many limitations are faced in analyzing unstructured scientific literature and extracting new information. This study proposes a scientific-knowledge processing framework, which offers high performance by using grid computing technology for extracting important entities and their relations from the scientific literature. Since the grid computing provides a large volume of data storage and high-speed computing, the proposed framework can efficiently analyze the massive body of scientific literature and process knowledge. The workflow tool that we have developed for the proposed framework enables users to easily design and execute complicated applications that consist of complicated scientific-knowledge processes. The experimental results showed that the proposed framework reduced working time by approximately 83 % when the number of running nodes was assigned in accordance with the workload ratio of each step in scientific-knowledge processes. As a result, it is possible to effectively process a large volume of scientific literature by flexibly adjusting the number of computing nodes that constitute the grid environment as the number of documents for processing increases.