Features based on local self-similarities (LSS features) are proposed for hand gesture recognition according to the fact there exist self-similarities in images. To strengthen the self-similarity features of hand gestures, distribution of skin in YCbCr color space that illumination component varies from different lighting conditions and its chrominance components are limited in a relatively small region is considered. The sum of square differences (SSD) in LSS is modified by increasing the weight of the chrominance components and decreasing that of luminance component. Experiments have been implemented on 1008 pictures of 9 kinds of gesture with different background, focus and luminance. Three kinds of features (improved LSS, LSS and HOG) are separately employed to classify gestures by SVM, SRC and NN classification models. The experiment shows SVM is better than SRC and NN. The results also show that the improved LSS features can achieve higher recognition rate then LSS and HOG.