Introduction to the CARLA simulator: training a neural network to control a car (Part 2)

入门学习资料
1

Before we start, the source code for this whole project is available here. If you have questions regarding the code, please create an Issue. And if you’d like to gain access to the data to re-produce the results, leave a comment, and I’ll try my best to put it out somewhere.

I used three methods of collecting data for training the models:

  1. The Proportion-Derivative (PD) controller introduced in Part 1;
  2. The Model Predictive Control (MPC) which we’ll discuss shortly;
  3. The Game Pad (GP), i.e. I trained the model to mimic my driving behavior.

Future work and why use depth images instead of RGB

A high-level list of ideas I’m planning to explore:

  1. Get the network to predict the speed, not throttle, and use a controller for throttle to achieve that speed (might not work if changes in speed require the controller to be able to adapt quickly).
  2. Other / better controllers—here’s a good source of implementations; also POLO looks like an interesting avenue, but additionally: I have an idea (partly inspired by POLO) of supplementing the cost function in the MPC with “learnable” terms making the simple kinematic model a bit more realistic.
  3. Find better strategies of evaluating the models offline.
  4. Run CARLA in asynchronous mode.

References

[1] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017). CARLA: An open urban driving simulator. arXiv preprint arXiv:1711.03938.
[2] Pan, Y., Cheng, C. A., Saigol, K., Lee, K., Yan, X., Theodorou, E., & Boots, B. (2017). Agile off-road autonomous driving using end-to-end deep imitation learning. arXiv preprint arXiv:1709.07174.
[3] Yang, L., Liang, X., Wang, T., & Xing, E. (2018). Real-to-Virtual Domain Unification for End-to-End Autonomous Driving. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 530–545).
[4] Bewley, A., Rigley, J., Liu, Y., Hawke, J., Shen, R., Lam, V. D., & Kendall, A. (2018). Learning to Drive from Simulation without Real World Labels. arXiv preprint arXiv:1812.03823.

Other cool resources