Smart Traffic Signal in India using Deep Reinforcement Learning and Computer Vision (Simulator and OpenAI-Gym ENVS included)

Ujwal Tewari (~ujwal)


With a growth in the economic sector since the early 1990s, India has been undergoing a paradigm shift in the domain of transportation as well. With the explosion in the number of road-vehicles, the surge in traffic density (especially in metropolitan cities like Bangalore and Delhi) has become a major cause of concern. It has, therefore, become really important to develop Intelligent traffic signals and systems in order to optimize the escalating traffic flow. For a developing country like India where the majority of motorists are still negligent of traffic laws and are often seen breaking them, for e.g, driving in opposite lanes, violating an indicator near turns and turning in a wrong direction, and many more such instances which eventually lead to traffic jams. In such Indian scenarios, the power of reinforcement learning applied on traffic behavior can be leveraged to greatly reduce these traffic jams. For a detailed setup and pipeline of the project visit the repository here.

The proposed outline of the talk shall be as stated below :-

  • What exactly is Reinforcement Learning? (5 min)
  • OpenAI gym and Environments (2 min)
  • SUMO simulator (1 min)
  • AIMSUN simulator (2 min)
  • Environment Creation (10 min)
  • Deploying RL models on the simulation (10 min)

In the end FST-60 or 40 (Fixed green time to each signal) can be used as control to compare the results-

enter image description here

For traffic simulator SUMO is a python controlled GUI simulator along with AIMSUN which offers a much more realistic approach and a much simpler way to build real time maps and simulations. However, it is not python emulated, and hence needs an API-Environment class to be built in order for it to implement RL models. A couple of images for traffic simultion environment are given below as an example :- [SUMO simulation in the left and AIMSUN which is more realistic is in the right]- enter image description here

We can employ RL in the simulator using the following techniques :-

  • Create a GYM environment using the simulator and then deploy the RL model
  • Using Simulator frames to process Junction images using computer vision and then supply action using our RL model

Some useful links for reference :-

  1. Using OpenAI gym and RL
  2. Create you own GYM environment
  3. Creating Scenarios in SUMO
  4. Using AIMSUN realistic maps


Following prerequisites are desired, but not strictly required:-

  1. Familiarity with Python
  2. Primitive Understanding of Reinforcement Learning
  3. Familarity with Keras or Chainer
  4. Understanding of Deep Network Architectures
  5. Some experience with OpenAI-gyms and environments
  6. Basics of Computer Vision

I have also created a video explaining breifly about reinforcement learning-

What exactly is Reinforcement Learning? - a video overview

Slides to the talk are here

Content URLs:

Github Repository Link

Speaker Info:

Ujwal Tewari is a Research Engineer at Siemens Pvt. Limited and a graduate from Indian Institute of Information Technology Vadodara. He mentors the students enrolled for the Deep Reinforcement Learning nanodegree at Udacity as their classroom mentor. He is also in the Intel community as an Intel Student Ambassador. He has worked on numerous deep learning projects and is currently working on various use cases of multi-agent Reinforcement Learning in Indian Traffic Scenarios. He is an active open source contributor and medical imaging using AI and Deep learning networks.

I have gained this invaluable knowledge during my work with -

  • Ishan Maheshwari (SDE at Microsoft) at IIIT-Delhi under the guidance of Dr Chetan Arora (Associate Professor with Department of Computer Science and Engineering at IIT Delhi)
  • Siemens R&D department at Bengaluru under the mentorship of Varsha Raveendran(Lead Research Engineer) and Vinay Sudhakaran(Senior Key Expert (Infield Analytics))

I thank them for their support and mentorship during my work.

Id: 1086
Section: Data Science, Machine Learning and AI
Type: Talks
Target Audience: Intermediate
Last Updated: