Current end-to-end deep learning driving models have two problems: (1) Poor
 generalization ability of unobserved driving environment when diversity of train-
 ing driving dataset is limited (2) Lack of accident explanation ability when driving
 models don’t work as expected. To tackle these two problems, rooted on the be-
 lieve that knowledge of associated easy task is benificial for addressing difficult
 task, we proposed a new driving model which is composed of perception module
 for see and think and driving module for behave, and trained it with multi-task
 perception-related basic knowledge and driving knowledge stepwisely.  Specifi-
 cally segmentation map and depth map (pixel level understanding of images) were
 considered as what & where and how far knowledge for tackling easier driving-
 related perception problems before generating final control commands for difficult
 driving task. The results of experiments demonstrated the effectiveness of multi-
 task perception knowledge for better generalization and accident explanation abil-
 ity. With our method the average sucess rate of finishing most difficult navigation
 tasks in untrained city of CoRL test surpassed current benchmark method for 15
 percent in trained weather and 20 percent in untrained weathers. Observing progressive improvement in various fields of pattern recognition with end-to-end deep learning based methods BID13 BID8 , self-driving researchers try to revolutionize autonomous car field with the help of end-to-end deep learning techniques BID3 BID4 . Impressive results have been acquired by mapping camera images directly to driving control commands BID3 with simple structure similar to ones for image classfication task BID19 . Further researches were conducted to improve the performance of deep learning based autonomous driving system, for example, Conditional Imitation Learning approach has been proposed to solve the ambigious action problem.However, two crutial problems failed to be spotted: (1) Poor generalization ability of unobserved driving environment given limited diversity of training scenerios. For example, though addressed the driving direction selection problem, it showed poor generalization ability in unseen test town which has different map and building structure than training town's. This generalization problem is extremely important since collected driving dataset always has limitation of diversity (2) Current end-to-end autonomous approaches lack of accident explanation ability when these models behave unexpectedly. Although saliency map based visualization methods BID20 BID23 BID21 BID2 have been proposed to dig into the 'black box', the only information these methods could bring is the possible attention of the model instead of the perception process of the model.We proposed a new driving approach to solve the two aforementioned problems by using multi-task basic perception knowledge. We argue that when end-to-end model is trained to address a specific difficult task, it's better to train the model with some basic knowledge to solve relevant easier tasks before BID17 ). An analogy for this can be observed when human beings learn a difficult knowledge. For example, to solve a complex integration problem, compared with students without basic math knowledge, students who know about basic knowledge of math are able to learn the core of intergration more quickly and solve other similar integration problems instead of memorizing the solution of the specific problem.Our proposed model consists of two modules: perception module and driving module as in FIG0 . The perception module is used for learning easier driving-related perception knowledge, which we refer as ability of pixel level understanding of input including what & where and how far knowledge. We trained perception module with segmentation map and depth map first, while the former serves as what & where knowledge and the latter serves as how far knowledge. By visualizing inferenced segmentation and depth results whether perception process works well or not could be inferred. After the perception module was trained to have ability of pixel level understanding of its image input, we freezed the perception module weights and trained driving module with driving dataset. This decomposition of end-to-end driving network strucuture is considered to be mediated perception approach BID25 . With our proposed driving structure and stepwise training strategy, the generalization and accident explanation problems were addressed to a certain extent. In this paper we propose a new driving system for better generalization and accident explanation ability by enabling it to do simpler driving-related perception task before generating commands for diffult driving task. Through multiple experiments we empirically proved the effectiveness of the multi basic perception knowledge for better generalization ability of unobserved town when diversity of training dataset is limited. Besides our proposed model has self-explanation ability by visualizing the predicted segmentation and depth maps from the perception module to determine the cause of driving problems when they happen. One interesting result we acquired by comparing different train strategies is that the generalization ability of driving origins from basic knowledge and lies in weights of the perception module which should not be modified during training with driving dataset. We hope our work could movitivate other researches to use multi-task target related perception knowledge for better performance in robot learning. In future we will investigate more effective network structures.
we proposed a new self-driving model which is composed of perception module for see and think and driving module for behave to acquire better generalization  and accident explanation ability. Presents a multitask learning architecture for depth and segmentation map estimation and the driving prediction using a perception module and a driving decision module. A method for a modified end-to-end architecture that has better generalization and explanation ability, is more robust to a different testing setting, and has decoder output that can help with debugging the model. The authors present a multi-task convolutional neural network for end-to-end driving and provide evaluations with the CARLA open source simulator showing better generalization performance in new driving conditions than baselines