41 / 2015-10-19 21:44:24

Research on the reliability of network traffic data collection

Hadoop ,Big data,Network traffic,Data collection,Reliability transmission

In this paper, the problem of how to improve the integrity and reliability of the data acquisition system based on Hadoop is introduced, and the distributed network fault detection technology is introduced; And based on this technology, a distributed node monitoring framework (DNMF) is designed, which is suitable for the data acquisition system ,the node fault detection and processing algorithm and node load balancing algorithm are proposed in this framework; At the same time, the load of DNMF is used to balance the load, and the load of some nodes is prevented.