Surgical resection is the primary treatment for brain cancer, where maximizing cancer removal while preserving adjacent healthy tissue is crucial. This task is challenging due to the invasive nature of brain tumors. Current intraoperative histopathology methods, such as frozen section and smear preparation, have drawbacks like labeling requirements and being time-consuming and labor-intensive. In this paper, we propose an automatic, label-free method for brain tumor identification using nonlinear optical microscopy (NOM) combined with deep learning. NOM is employed to capture dual-modal images of brain tissue, including two-photon excitation fluorescence (TPEF) and fluorescence lifetime imaging (FLIM) images. These images serve as input for a U-net network, which accurately identifies tumor regions. This method was tested on brain tissue slices from tumor-bearing nude mice and ND2:SmoA1 transgenic mice. Results show that NOM can reveal cerebral structures and metabolic conditions. By extracting structural features and microenvironment information from NOM images, U-net could distinguish between tumor and peritumor regions with an average accuracy of 0.98 and evaluate the granular layer. The deep learning-based, label-free NOM approach promises automatic identification of cancerous versus non-cancerous tissues and is expected to integrate with endoscopy, offering a promising tool for characterizing cancer and cerebral structures during neurosurgery.