The fault signal of rotating machinery contains a lot of useful information such as equipment degradation data, but the components are complex. This paper proposes a method of residual life prediction of rotating machinery based on manifold algorithm and time-varying hidden-semi Markov model pp(HSMM). The LLE algorithm is used to fuse the high dimensional features of the running signals of the equipment, and a new low dimensional fusion feature set which covers the linear and nonlinear information of the state signals is obtained. The hidden-semi Markov model is combined with the time-varying state transition probability matrix, and the residual life of rotating machinery is predicted according to the fusion feature set. Finally, the method is verified by the life cycle signal of rolling bearing.