Compared to traditional visual tracking, long-term tracking appears to be more challenging since the target is likely to suffer more severe deformation, occlusion, scale change or move out of view scenarios. It is challenging to develop a robust and efficient target model. In this paper, we propose a robust model for long-term tracking in complex scenes. In order to achieve this goal, firstly, we extract multi-scale feature based on the illumination invariant color space to solve scale and illumination change of the target. For the purpose of reducing time consumption caused by the multi-scale feature, we adopt a random measurement matrix to project the high-dimensional multi-scale features onto a low-dimensional subspace. Secondly, we introduce a tracking Failure Detection Strategy(FDS) to decide whether the tracking is a failure which cause by occlusion, illumination change and situations when the target moves out of camera view. Finally, we proposed a Weighted Random Forest(WRF) classifier to retrieve the target position after the tracking failure situation, and the classifier is updated online, so that the performance of the model improves over time. Our proposed model performs favorably in complex scenes against conventional models in terms of robustness and time consumption.