The leaf can be called the most important part of a plant. If the leaf gets affected by some disease it affects the whole plant. To get a high-quality fruit, the quality of the leaf must be ensured. To ensure the quality of the leaf, early disease detection of the leaf is very effective. Lychee is one of the most profitable fruits in Bangladesh. Every year more than 47,500 metric ton of Lychee is sold in Bangladesh. There are many diseases that affect the growth of the fruit. Most of this disease will show signs on the leaf of the plant. In this paper, we have used two Convolution Neural Network (CNN) architectures VGG16 and VGG19 which is a Deep Learning algorithm to classify Lychee leaf diseases. Also, a basic CNN model (3 convolution layers, 3 max-pooling layers, and 2 dense layers) is used to compare both structures. The aim of this paper is to find out which architecture performs better to identify Lychee leaf disease. Here VGG16 gives 90% accuracy, VGG19 gives 88% accuracy and the basic CNN model gives 82% accuracy with dataset containing 1655 leaf images. Furthermore, an Android Application has been created with the best model. This will make the model more accessible to the farmers and anyone will be able to get benefited. This project can be used to detect early lychee leaf disease and prevent production losses.