Astrocomp: web technologies for high performance computing on a network of supercomputers

Astrocomp is a project developed by the INAF-Astrophysical Observatory of Catania, University of Roma La Sapienza and Enea in collaboration with Oneiros s.r.l. The project has the goal of building a web-based user-friendly interface which allows the international community to run some parallel codes on a set of high-performance computing (HPC) resources, with no need for specific knowledge about Unix and Operating Systems commands. Astrocomp provides CPU times, on parallel systems, available to the authorized user. The portal makes codes for astronomy available: FLY code, a cosmological code for studying three-dimensional collisionless self-gravitating systems with periodic boundary conditions [Becciani, Antonuccio, Comput. Phys. Comm. 136 (2001) 54]. ATD treecode, a parallel tree-code for the simulation of the dynamics of self-gravitating systems [Miocchi, Capuzzo Dolcetta, A&A 382 (2002) 758]. MARA a code for stellar light curves analysis [Rodonò et al., A&A 371 (2001) 174]. Other codes will be added to the portal in the future.     

Time-constrained services: a framework for using real-time web services in industrial automation

The use of web services in industrial automation, e.g. in fully automated production processes like car manufacturing, promises simplified interaction among the manufacturing devices due to standardized protocols and increased flexibility with respect to process implementation and reengineering. Moreover, the adoption of web services as a seamless communication backbone within the overall industrial enterprise has additional benefits, such as simplified interaction with suppliers and customers (i.e. horizontal integration) and avoidance of a break in the communication paradigm within the enterprise (i.e. vertical integration). The Time-Constrained Services (TiCS) framework is a development and execution environment that empowers automation engineers to develop, deploy, publish, compose, and invoke time-constrained web services. TiCS consists of four functional layers—tool support layer, real-time infrastructural layer, real-time service layer, and hardware layer—which contain several components to meet the demands of a web service based automation infrastructure. This article gives an overview of the TiCS framework. More precisely, the general design considerations and an architectural blueprint of the TiCS framework are presented. Subsequently, selected key components of the TiCS framework are discussed in detail: the SOAP4PLC engine for equipping programmable logic controllers with a web service interface, the SOAP4IPC engine for processing web services in real-time on industrial PCs, the WS-TemporalPolicy language for describing time constraints, and the TiCS Modeler for composing time-constrained web services into a time-constrained BPEL4WS workflow.     

Remote rendering solutions using web technologies

Remote rendering is a well-known solution to the issue of running high-performance applications requiring complex visualizations on less capable hardware/software platforms or when client access to the data source for visualization is undesired or prohibitive in terms of required bandwidth. Visualizing the output of these remote rendering applications is typically achieved through native applications or, when considering a browser environment, through plug-ins. In this paper, several solutions are presented that enable deployment of these applications on standard web browsers, even those from the pre-HTML5 era. The focus in this paper is on two specific use case scenarios, taking into account that the proposed solutions are generic enough to be applied to a range of similar applications. The technologies presented cover the entire range of sub-processes contained in a complete remote rendering solution, such as the establishment of interaction feedback channels and delivery of images as part of the rendering pipeline. Depending on factors such as application requirements, developer preferences, feature availability in the web browser or raw performance figures, a custom solution can be composed from the options discussed in this paper. This is illustrated by applying them to the two aforementioned use cases, each with specific requirements and challenges, and benchmarking these example setups in terms of performance. A comparison of advantages and disadvantages is presented to guide developers in applying the technologies under real-life conditions.     

Dependency management for digital preservation using semantic web technologies

The preservation of digital objects is a topic of prominent importance for archives and digital libraries. In this article, we focus on the problem of preserving the intelligibility of digital objects. We formalize the problem in terms of dependencies and specify a number of basic intelligibility-related tasks. In parallel, we introduce a preservation scenario as a means for clarifying the pros and cons of various representation and modeling languages that are used for the problem at hand, which reveals the benefits of adopting Semantic Web (SW) languages as a representation framework. To this end, we propose a minimal core ontology for representing intelligibility-related dependencies along with methodological hints for extending it. Finally, we report empirical and experimental results from applying the proposed approach on real data sets. It is worth mentioning that this approach can be used not only on SW-based repositories or archives, but also on those that are based on conventional approaches and languages (like EAST, DEDSL, XFDU/SAFE).     

A denotational framework for data flow analysis

Summary It is shown how to express data flow analysis in a denotational framework by means of abstract interpretation. A continuation style formulation naturally leads to the MOP (Meet Over all Paths) solution, whereas a direct style formulation leads to the MFP (Maximal Fixed Point) solution.     

Using semantic web technologies to access soft AEC data

Building related data tends to be generated, used and retained in a domain-specific manner. The lack of interoperability between data domains in the architecture, engineering and construction (AEC) industry inhibits the cross-domain use of data at an enterprise level. Semantic web technologies provide a possible solution to some of the noted interoperability issues. Traditional methods of information capture fail to take into account the wealth of soft information available throughout a building. Several sources of information are not included in performance assessment frameworks, including social media, occupant communication, mobile communication devices, occupancy patterns, human resource allocations and financial information.The paper suggests that improved data interoperability can aid the integration of untapped silos of information into existing structured performance measurement frameworks, leading to greater awareness of stakeholder concerns and building performance. An initial study of how building-related data can be published following semantic web principles and integrated with other ‘soft-data’ sources in a cross-domain manner is presented. The paper goes on to illustrate how data sources from outside the building operation domain can be used to supplement existing sources. Future work will include the creation of a semantic web based performance framework platform for building performance optimisation.     

RMAP: a system for visualizing data in multidimensional relevance space

exact relevance measures of the objects are not significant. We flatten the multidimensionality of the feature space into a 2D relevance map, capturing the inter-relations among the features. The prototype, extract information from the World Wide Web from query engines, automatically categorizes and clusters the information and allow the user to visualize.     

A unifying passivity framework for network flow control

Network flow control regulates the traffic between sources and links based on congestion, and plays a critical role in ensuring satisfactory performance. In recent studies, global stability has been shown for several flow control schemes. By using a passivity approach, this paper presents a unifying framework which encompasses these stability results as special cases. In addition, the new approach significantly expands the current classes of stable flow controllers by augmenting the source and link update laws with passive dynamic systems. This generality offers the possibility of optimizing the controllers, for example, to improve robustness in stability and performance with respect to time delays, unmodeled flows, and capacity variation.     

Methods and framework for visualizing higher-order finite elements

The finite element method is an important, widely used numerical technique for solving partial differential equations. This technique utilizes basis functions for approximating the geometry and the variation of the solution field over finite regions, or elements, of the domain. These basis functions are generally formed by combinations of polynomials. In the past, the polynomial order of the basis has been low-typically of linear and quadratic order. However, in recent years so-called p and hp methods have been developed, which may elevate the order of the basis to arbitrary levels with the aim of accelerating the convergence of the numerical solution. The increasing complexity of numerical basis functions poses a significant challenge to visualization systems. In the past, such systems have been loosely coupled to simulation packages, exchanging data via file transfer, and internally reimplementing the basis functions in order to perform interpolation and implement visualization algorithms. However, as the basis functions become more complex and, in some cases, proprietary in nature, it becomes increasingly difficult if not impossible to reimplement them within the visualization system. Further, most visualization systems typically process linear primitives, in part to take advantage of graphics hardware and, in part, due to the inherent simplicity of the resulting algorithms. Thus, visualization of higher-order finite elements requires tessellating the basis to produce data compatible with existing visualization systems. In this paper, we describe adaptive methods that automatically tessellate complex finite element basis functions using a flexible and extensible software framework. These methods employ a recursive, edge-based subdivision algorithm driven by a set of error metrics including geometric error, solution error, and error in image space. Further, we describe advanced pretessellation techniques that guarantees capture of the critical points of the polynomial basis. The framework has been designed using the adaptor design pattern, meaning that the visualization system need not reimplement basis functions, rather it communicates with the simulation package via simple programmatic queries. We demonstrate our method on several examples, and have implemented the framework in the open-source visualization system VTK.     

StockProF: a stock profiling framework using data mining approaches

Information Systems and e-Business Management - Analysing stock financial data and producing an insight into it are not easy tasks for many stock investors, particularly individual investors....     

Computer animation for visualizing terrain data

Until recently, the study of terrain data gathered by satellites has meant the time-consuming analysis of hundreds of still images. An attempt to use computer animation to analyze such images more effectively is described. A simulated fly-by created by computer animation allows scientists to study many different perspective views of the data quickly. Using this technique, scientists can often detect features they might otherwise have missed     

STGMN: A gated multi-graph convolutional network framework for traffic flow prediction

Applied Intelligence - Accurate traffic flow prediction is crucial for the development of intelligent transportation. It can not only effectively avoid traffic congestion and other traffic...     

Gneiss: spreadsheet programming using structured web service data

Web services offer a more reliable and efficient way to access online data than scraping web pages. However, interacting with web services to retrieve data often requires people to write a lot of code. Moreover, many web services return data in complex hierarchical structures that make it difficult for people to perform any further data manipulation. We developed Gneiss, a tool that extends the familiar spreadsheet metaphor to support using structured web service data. Gneiss lets users retrieve or stream arbitrary JSON data returned from web services to a spreadsheet using interaction techniques without writing any code. It introduces a novel visualization that represents hierarchies in data using nested spreadsheet cells and allows users to easily reshape and regroup the extracted structured data. Data flow is two-way between the spreadsheet and the web services, enabling people to easily make a new web service call and retrieve new data by modifying spreadsheet cells. We report results form a user study that showed that Gneiss helped spreadsheet users use and analyze structured data more efficiently than Excel and even outperform professional programmers writing code. We further use a set of examples to demonstrate our tool's ability to create reusable data extraction and manipulation programs that work with complex web service data.     

A predictor-corrector technique for visualizing unsteady flow

Presents a method for visualizing unsteady flow by displaying its vortices. The vortices are identified by using a vorticity-predictor pressure-corrector scheme that follows vortex cores. The cross-sections of a vortex at each point along the core can be represented by a Fourier series. A vortex can be faithfully reconstructed from the series as a simple quadrilateral mesh, or its reconstruction can be enhanced to indicate helical motion. The mesh can reduce the representation of the flow features by a factor of 1000 or more compared with the volumetric dataset. With this amount of reduction, it is possible to implement an interactive system on a graphics workstation to permit a viewer to examine, in 3D, the evolution of the vortical structures in a complex, unsteady flow     

Towards a framework of key technologies for drones

The potential applications for drones, especially those in manned areas or into non-segregated airspace, are currently not possible without the development and validation of certain key enabling technologies: “detect and avoid”, “air traffic management”, “command and control (C2) link”, and “security and cyber resilience”. This issue has a high impact on European innovation as identified by SESAR JU and demands R&D investments and incentives for the convergence of shared technologies and markets. In this paper, we discuss the vision of COMP4DRONES project to complement SESAR JU efforts with a particular focus on safe software and hardware drone architectures. COMP4DRONES will bear a holistically designed ecosystem ranging from application to electronic components. The ecosystem aims at supporting (1) efficient customization and incremental assurance of drone-embedded platforms, (2) safe autonomous decision making concerning individual or cooperative missions, (3) trustworthy drone-to-drone and drone-to- ground communications even in presence of malicious attackers and under the intrinsic platform constraints, and (4) agile and cost-effective design and assurance of drone modules and systems. Lead applications driving ecosystem development and benchmarking on the fields of transport, inspection, logistic, precision agriculture, and parcel delivery will be produced.     

behaviorism: a framework for dynamic data visualization

While a number of information visualization software frameworks exist, creating new visualizations, especially those that involve novel visualization metaphors, interaction techniques, data analysis strategies, and specialized rendering algorithms, is still often a difficult process. To facilitate the creation of novel visualizations we present a new software framework, behaviorism, which provides a wide range of flexibility when working with dynamic information on visual, temporal, and ontological levels, but at the same time providing appropriate abstractions which allow developers to create prototypes quickly which can then easily be turned into robust systems. The core of the framework is a set of three interconnected graphs, each with associated operators: a scene graph for high-performance 3D rendering, a data graph for different layers of semantically linked heterogeneous data, and a timing graph for sophisticated control of scheduling, interaction, and animation. In particular, the timing graph provides a unified system to add behaviors to both data and visual elements, as well as to the behaviors themselves. To evaluate the framework we look briefly at three different projects all of which required novel visualizations in different domains, and all of which worked with dynamic data in different ways: an interactive ecological simulation, an information art installation, and an information visualization technique.     

ST-Hadoop: a MapReduce framework for spatio-temporal data

This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for three fundamental spatio-temporal queries, namely, spatio-temporal range, top-k nearest neighbor, and join queries. Extensibility of ST-Hadoop allows others to extend features and operations easily using similar approaches described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.     

Format-independent and metadata-driven media resource adaptation using semantic web technologies

Adaptation of media resources is an emerging field due to the growing amount of multimedia content on the one hand and an increasing diversity in usage environments on the other hand. Furthermore, to deal with a plethora of coding and metadata formats, format-independent adaptation systems are important. In this paper, we present a new format-independent adaptation system. The proposed adaptation system relies on a model that takes into account the structural metadata, semantic metadata, and scalability information of media bitstreams. The model is implemented using the web ontology language. Existing coding formats are mapped to the structural part of the model, while existing metadata standards can be linked to the semantic part of the model. Our new adaptation technique, which is called RDF-driven content adaptation, is based on executing SPARQL Protocol and RDF Query Language queries over instances of the model for media bitstreams. Using different criteria, RDF-driven content adaptation is compared to other adaptation techniques. Next to real-time execution times, RDF-driven content adaptation provides a high abstraction level for the definition of adaptations and allows a seamless integration with existing semantic metadata standards.     

基于数据流的网络入侵实时检测框架

针对计算机网络访问请求具有实时到达以及动态变化的特点,为了实时检测网络入侵,并且适应网络访问数据的动态变化,提出一个基于数据流的网络入侵实时检测框架。首先,将误用检测模式与异常检测模式相结合,通过初始聚类建立由正常模式和异常模式构成的知识库;其次,采用数据点与数据簇之间的不相似性来度量网络访问数据与正常模式和异常模式的相似性,从而判定网络访问数据的合法性;最后,当网络访问数据流发生演化时,通过重新聚类来更新知识库以反映网络访问的最近状态。在入侵检测数据集KDDCup99上进行实验,当初始聚类的样本数为10000,缓冲区聚类的样本数为10000,调节系数为0.9时,召回率达到91.92%,误报率达到0.58%,接近传统非实时检测模式的结果,但整个学习和检测过程只需扫描网络访问数据一次,并引入了知识库的更新机制,在入侵检测的实时性和适应性方面更具有优势。