Web应用下的集群基础技术分析——Session故障恢复

几乎所有的商用应用服务器都对集群提供支持以达到对应用系统的负载均衡、冗余备份、容错的目的，从而得到最大限度的可用性、可靠性及可伸缩性。集群模式下，一组服务器协同工作，当一个服务器失败时，请求会被透明地转发到备份或者同步处理的服务器上．并不会因此而影响客户的访问。对客户最严重的影响可能是一次失败的请求．如果再次发出请求．即可得到继续的服务。[编者按]     

基于Python的新闻聚合系统网络爬虫研究

本文分析了基于Python的新闻聚合系统网络爬虫,指的是根据Python的网络爬虫构建新闻聚合系统,利用爬虫获取新闻聚合系统的新闻数据,不同网站的页面布局是不同的,因此需要创建开源爬虫,可以在不同页面布局中获取数据。在网络爬虫开发过程中需要利用Python语言,而网络爬虫Web提取工具为BeautifulSoup,Web应用程序框架为Laravel,选用的后端语言为PHP。网络爬虫可以根据用户配置文件提取不同页面布局的数据,并且可以向数据库系统中导入提取的数据。     

基于SmarTele语音流程脚本语言构建IVR系统的研究

1 序言 交互式语音应答(IVR)是呼叫中心系统的重要组成部分.通过IVR系统,客户可以在普通的电话上通过按键输入其服务请求及相关信息,而IVR系统则根据客户请求,自动访问呼叫中心或企业后台业务系统中的资源,完成客户的服务请求,并将执行结果通过电话、传真等方式提供给客户.本文主要论述使用强讯的SmarTefe语音流程脚本语言构建IVR系统.     

基于客户端蜜罐的恶意网页检测系统的设计与实现

分析基于网页的恶意行为特点，设计一个基于客户端蜜罐的恶意网页检测系统。该系统将蜜罐与爬虫相结合，通过设计一个网络爬虫来获取url数据源，利用蜜罐内的客户段引擎自动化启动Internet Explorer浏览网页，并监控通过浏览恶意网页下载的恶意软件。最后，分析恶意代码，将恶意网页加入黑名单。     

截取过滤器模式在基于J2EE的Web开发中的应用

在基于J2EE的Web开发中,经常需要对用户请求进行预处理,传统的方法很难实现软件复用,因此需要一种新型、高效的技术来解决这个问题,本文结合截取过滤器模式的特点,提出了使用截取过滤器模式来解决用户请求预处理的问题,并用一个例子来说明如何在开发中运用截取过滤器模式提高软件的可读性、可维护性。     

在Visual FoxPro中实现多台工作站共享多台打印机

在Windows 95中,我们可以通过利用虚拟设备技术(spooling),用共享的大容量磁盘的某些区域来模拟独占打印机,实现将独占打印机变换为共享打印机。即在Windows 95系统中,当一个用户请求打印输出时,spooling系统并不真正把打印机分配给该用户,而是将要打印的数据送入共享的磁盘上形成请求打印队列,多个用户的打印请求依请求时间顺序被插入到请求打印队列中。如果打印机空闲,spooling系统将把位于请求打印队列队首的打印数据送到内存缓冲区,再由打印机进行打印。打印完后,spooling系统再查看请求打印队列中是否还有需要打印的数据,若有,继续打印,直到请     

基于Python爬虫技术的虚假数据溯源与过滤

针对现有网络虚假数据追踪与过滤方法中存在的追踪定位精度低、过滤覆盖范围小的问题,提出基于Python爬虫技术的虚假数据溯源与途中过滤.将虚假数据覆盖的网络划分为相同大小的网格,根据一定概率标记数据包;分析Python爬虫抓取过程,以适用多场景、界面可视化、负载均衡为系统设计目标,设计爬虫管理器、采集器与内嵌浏览器硬件设备;结合系统爬取数据流程描述各软件模块的协作方式;在系统中部署节点,建立协作关系区域,利用布隆过滤器生成数据包,通过共享密钥证明MAC是否合法,启动溯源过程;在溯源途中若节点不存在任何一个密钥,则将其过滤,完成虚假数据溯源与途中过滤.仿真结果证明,上述方对虚假数据的过滤效果较高,且对虚假数据溯源定位精度较高.     

基于Python爬虫技术的虚假数据溯源与过滤

针对现有网络虚假数据追踪与过滤方法中存在的追踪定位精度低、过滤覆盖范围小的问题,提出基于Python爬虫技术的虚假数据溯源与途中过滤.将虚假数据覆盖的网络划分为相同大小的网格,根据一定概率标记数据包;分析Python爬虫抓取过程,以适用多场景、界面可视化、负载均衡为系统设计目标,设计爬虫管理器、采集器与内嵌浏览器硬件设备;结合系统爬取数据流程描述各软件模块的协作方式;在系统中部署节点,建立协作关系区域,利用布隆过滤器生成数据包,通过共享密钥证明MAC是否合法,启动溯源过程;在溯源途中若节点不存在任何一个密钥,则将其过滤,完成虚假数据溯源与途中过滤.仿真结果证明,上述方对虚假数据的过滤效果较高,且对虚假数据溯源定位精度较高.     

基于期限的多请求数据检索问题研究

给定一个包含多条信道的集合以及一个包含多个请求的集合,其中每一个请求包含多个请求数据项并且希望在一定期限内下载到,基于期限的多请求数据检索问题指当客户配有多条天线时寻找一个在期限内下载多个请求的数据检索序列,使得所有天线的最大访问延迟最小化。大多数现有数据检索方法关注于单个请求或者单条天线,很少研究当客户配有多条天线时多请求的数据检索问题,尤其是每一个请求的检索有时间约束。基于此,本文提出一种多请求的数据检索算法,以调度合适地天线检索这些请求并找到关于这些请求的检索序列,从而平衡在各天线上的访问延迟。针对单请求的数据检索,本文采用最大团思想寻找下载该请求中所有请求数据项的访问模式,使得检索该请求的访问延迟以及期限丢失率最小化。     

用VC5.0编写文件传输客户程序

1.文件传输机理 文件传输是基于客户/服务器模式而设计的,客户和服务器之间利用TCP建立联系。因文件传输是一个交互式会话系统,客户每次执行文件传输,都需要与服务器建立双重连接,一个是控制连接,一个是数据连接。其中控制连接负责传输控制信息,尤其是客户的命令(如文件传输命令)。利用控制命令,客户可以向服务器提出无限次的请求(一个请求就是一个客户命令)。客户每提出一个请求,服务器再与客户建立一个数据连接,进行实际的数据传输。一旦数据传输结束,数据连接相继撤消,但控制连接依然存在,客     

Video personalization in heterogeneous and resource-constrained environments

Access to multimedia data and multimedia services is becoming increasingly common in networked mobile environments. In such environments, both the mobile client devices and multimedia servers are typically resource constrained. Moreover, the mobile client population is often heterogeneous in terms of the clients’ preferences with regard to multimedia content, the clients’ quality of service requirements and system-level resource constraints. In order to provide a resource-constrained mobile client with its desired video content, it is often necessary to personalize the requested multimedia content in a manner that satisfies simultaneously the various client-specified content preferences and the system-level resource constraints. Also, in order to simultaneously reduce the client-experienced latency, provide optimal quality of service to the clients and ensure efficient utilization of server and network resources, it is necessary to perform client request aggregation on the server end. To this end, a video personalization strategy is proposed to provide mobile, resource-constrained clients with personalized video content that is most relevant to the clients’ requests while satisfying simultaneously multiple client-side system-level resource constraints. A client request aggregation strategy is also proposed to cluster client requests with similar video content preferences and similar client-side resource constraints such that the number of requests the server needs to process and service, and the client-experienced latency are both reduced simultaneously. The primary contributions of the paper are: (1) the formulation and implementation of a Multiple-choice Multi-dimensional Knapsack Problem (MMKP)-based video personalization strategy; and (2) the design and implementation of a client request aggregation strategy based on a multi-stage clustering algorithm. Experimental results comparing the proposed MMKP-based video personalization strategy to existing 0/1 Knapsack Problem (0/1KP)-based and the Fractional Knapsack Problem (FKP)-based video personalization strategies are presented. It is observed that: (1) the proposed MMKP-based personalization strategy includes more relevant video content in response to the client’s request compared to the existing 0/1KP-based and FKP-based personalization strategies; and (2) in contrast to the 0/1KP-based and FKP-based personalization strategies which can satisfy only a single client-side resource constraint at a time, the proposed MMKP-based personalization strategy is capable of satisfying multiple client-side resource constraints simultaneously. Experimental results comparing the client-experienced latency with and without the proposed server-side client request aggregation strategy are also presented. It is shown that the proposed client request aggregation strategy reduces the mean client-experienced latency without significant reduction in the average relevance of the delivered video content and without significant deviation in the client-side resources actually consumed by the delivered video content from the client-specified resource constraints.     

浏览器缓存污染防御策略

浏览器缓存主要用于提高用户对网络资源的请求速度,然而攻击者可以通过中间人攻击等方式实施缓存污染攻击。传统的缓存污染防御策略无法全面覆盖各种攻击方式,为此提出一种可调控的浏览器缓存污染防御策略。这种策略部署于用户与服务器之间,对用户所请求的缓存资源进行随机数判断、请求相应延时判断、资源代表性判断、哈希验证和众包策略,可以有效防御浏览器缓存污染问题。实验选取200个JavaScript资源文件作为实验样本,利用中间人攻击的方式污染其中100个样本,在访问这些资源的同时启用防御脚本,分析污染样本的检测率和正常样本的误判率。实验结果表明,在松弛条件下,污染样本的命中率达到87%,正常样本误判率为0%;而在严格条件下,污染样本的命中率达到95%,正常样本误判率为4%。同时所有实验样本的请求响应时间差分别为5277ms和6013ms,均小于全部重新加载资源的时间差,在防御了绝大部分的受污染资源的同时还缩短了用户访问的时间。该策略简化了缓存污染攻击防御的流程并可以通过不同的参数在用户体验性和安全性中取得平衡。     

Minimizing bandwidth requirements for on-demand data delivery

Two recent techniques for multicast or broadcast delivery of streaming media can provide immediate service to each client request, yet achieve considerable client stream sharing which leads to significant server and network bandwidth savings. The paper considers: 1) how well these recently proposed techniques perform relative to each other and 2) whether there are new practical delivery techniques that can achieve better bandwidth savings than the previous techniques over a wide range of client request rates. The principal results are as follows: First, the recent partitioned dynamic skyscraper technique is adapted to provide immediate service to each client request more simply and directly than the original dynamic skyscraper method. Second, at moderate to high client request rates, the dynamic skyscraper method has required server bandwidth that is significantly lower than the recent optimized stream tapping/patching/controlled multicast technique. Third, the minimum required server bandwidth for any delivery technique that provides immediate real-time delivery to clients increases logarithmically (with constant factor equal to one) as a function of the client request arrival rate. Furthermore, it is (theoretically) possible to achieve very close to the minimum required server bandwidth if client receive bandwidth is equal to two times the data streaming rate and client storage capacity is sufficient for buffering data from shared streams. Finally, we propose a new practical delivery technique, called hierarchical multicast stream merging (HMSM), which has a required server bandwidth that is lower than the partitioned dynamic skyscraper and is reasonably close to the minimum achievable required server bandwidth over a wide range of client request rates     

基于模拟退火算法的数据网格副本部署策略

副本管理是提高网格性能、降低客户端延迟的有效手段。针对副本部署策略问题,提出基于模拟退火算法的副本部署策略,给出优化模型及算法。OptorSim的仿真结果表明,该策略可减少作业对文件请求的响应时间,提高系统的整体性能。目前该策略已在大庆油田海量数据中心副本部署中得到成功应用。     

Java Web开发中过滤器组件应用及实例解析

Servlet过滤器是Java Web应用中的常用组件,可以拦截请求和响应,以便查看、提取或以某种方式操作客户机和服务器之间交换的数据,具有功能强大、部署简捷的特点。阐述了过滤器的工作原理,组成体系,工作过程,并通过典型实例对过滤器的具体开发和部署进行了详尽说明。     

债券监控模块设计与实现

根据债券业务要求，设计并实现了对债券交易平台的监控。提出了篝中管理债券客户服务器的思想，同时按客户类型和允许连接的最大客户端Client数进行了License的设计，并且实现了客户服务器的远程更新，用实际案例说明该设计思想的主要实现过程。     

Harmonic proportional bandwidth allocation and scheduling for service differentiation on streaming servers

To provide ubiquitous access to the proliferating rich media on the Internet, scalable streaming servers must be able to provide differentiated services to various client requests. Recent advances of transcoding technology make network-I/O bandwidth usages at the server communication ports controllable by request schedulers on the fly. In this article, we propose a transcoding-enabled bandwidth allocation scheme for service differentiation on streaming servers. It aims to deliver high bit rate streams to high priority request classes without overcompromising low priority request classes. We investigate the problem of providing differentiated streaming services at application level in two aspects: stream bandwidth allocation and request scheduling. We formulate the bandwidth allocation problem as an optimization of a harmonic utility function of the stream quality factors and derive the optimal streaming bit rates for requests of different classes under various server load conditions. We prove that the optimal allocation, referred to as harmonic proportional allocation, not only maximizes the system utility function, but also guarantees proportional fair sharing between classes with different prespecified differentiation weights. We evaluate the allocation scheme, in combination with two popular request scheduling approaches, via extensive simulations and compare it with an absolute differentiation strategy and a proportional-share strategy tailored from relative differentiation in networking. Simulation results show that the harmonic proportional allocation scheme can meet the objective of relative differentiation in both short and long timescales and greatly enhance the service availability and maintain low queueing delay when the streaming system is highly loaded.     

A paracasting model for concurrent access to replicated Internet content

In this paper, we develop a model to study how to effectively download a document from a set of replicated servers. We propose a generalized application-layer anycasting protocol, known as paracasting, to advocate concurrent access of a subset of replicated servers to cooperatively satisfy a client's request. Each participating server satisfies the request in part by transmitting a subset of the requested file to the client. The client can recover the complete file when different parts of the file sent from the participating servers are received. This model allows us to estimate the average time to download a file from the set of homogeneous replicated servers, and the request blocking probability when each server can accept and serve a finite number of concurrent requests. Our results show that the file download time drops when a request is served concurrently by a larger number of homogeneous replicated servers, although the performance improvement quickly saturates when the number of servers increases. If the total number of requests that a server can handle simultaneously is finite, the request blocking probability increases with the number of replicated servers used to serve a request concurrently. Therefore, paracasting is effective when a small number of servers, say, up to four, are used to serve a request concurrently.     

面向智慧矿山管控平台的视频点播技术

针对现有的视频点播技术无法直接在智慧矿山管控平台中直接使用的问题, 基于HTTP的自适应码率流媒体传输协议与FFmpeg开源库设计一种视频点播技术. 该技术包括客户端模块、Web请求处理模块、多媒体处理模块. 该技术中客户端模块通过设定的视频源信息向Web请求处理模块发送视频请求; Web请求处理模块利用请求中的视频源...