Abstract—The production of traditional surface mine has the characteristics of high potential safety hazards and low production efficiency. How to promote the intelligent upgrading and unmanned operation of surface mine for improving the production efficiency and safety of surface mine has become more and more important. In recent years, with the development of automatic driving, the application of low-speed autonomous driving has gradually become a reality. However, there are still many difficulties and challenges in some specific scenarios. Especially, in surface mine, which is unstructured, complex and changeable, it not only brings enormous challenge to the perception system, but also put forwards high requirements for the testing of autonomous driving system.

In order to improve the generalization ability of perception system and testing efficiency of the autonomous driving, we built a digital twin simulation system corresponding to the actual production scenario of automatic driving in surface mine to generate fully labeled, dynamic, and photo-realistic proxy virtual worlds for a series of autonomous tasks and provide a test platform for the testing of automatic driving function modules as shown in Fig. 1. The construction process of this system can be described as the following steps. Firstly, the system performs 3D reconstruction of the surface mine with a high-fidelity and equal proportion, which contain the scene of surface mine, as the foundation of the entire digital twin simulation system. Then, dynamic modeling of various types of mining machinery were conducted consistent with real mining machinery as the basic transportation agents. What’s more, various sensors, including camera, lidar, radar and IMU, were conducted based on the physical characteristics of the relevant sensors and installed in the corresponding vehicles. At last, based on a real-time scheduling platform, a large number of autonomous driving controllers and accurate vehicle dynamics kernel (simulating vehicle suspension, tire deformation and steering hydraulic pressure) conduct au- tomated operations in an orderly manner, the system support the testing of hundreds of virtual mining trucks throughout the entire surface mine.

Our parallel simulation system was built as a parallel system with providing assistance and verification for autonomous driving in surface mine of real world, which include perception model generalization capabilities improvement and functional testing. We have constructed a virtual dataset in the simulation engine with the corresponding obstacle categories and perception tasks. An efficient synthetic-to-real transfer method was proposed to perform domain adaptive conversion from synthetic to real for eliminating the domain gap between two different domains to improve the performance of perception including object detec- tion, semantic segmentation, tracking and so on as shown in Fig. 2. The complete workflow is simulated to test the cyclic operation process including loading, transportation, unloading and other groups. In addition, it provides the function testing of autonomous driving in surface mine with ensuring the safety during the testing of the autonomous driving system and reducing the influence of debugging on production of surface mine. The testing for functional modules including perception, decision- making, planning, control and so on.

In the future, we will explore the application of simulation system in data closed loop and hard scenes prediction. We expect that the simulation system can play a more important role in the landing of autonomous driving.