CarND-Capstone

Self Driving Car Nanodegree - Capstone Project: System Integration by Team Sakura

Udacity - Self-Driving Car NanoDegree

carla

Table of Contents

Results

3rd submission (Tag 1.2.0)

Carla was creeping, not full stop, when the traffic light was red.
When the traffic light turned green, Carla had a smooth acceleration.

Real Sefl-Driving Car

The DBW node outputs /vehicle/brake_cmd, but Carla does not stop at the stop position.
Maybe the brake torque is not enough.

current_velocity, throttle_cmd and brake_cmd /base_waypoints and /current_pose

2nd submission (Tag 1.1.0)

The car did not stop even though the traffic light was red.
The DBW node outputs /vehicle/brake_cmd, but Carla does not stop at the stop position.
The cause is that it accelerates too much at first, and the transition to deceleration is slow.

current_velocity, throttle_cmd and brake_cmd /base_waypoints and /current_pose

For 2nd submission

We limit the velocity toward the stop line to approximately 10 km/h and improve the detection performance for blurred images.

MAX_DECEL and decelerate_vel real_model.h5 (1.1.0)

1st submission (Tag 1.0.0)

The car did not stop even though the traffic light was red.
The cause is that the blurred traffic lights are can not be detected when the velocity exceeds about 10 km/h on the way to /traffic_waypoint.

/current_velocity and /traffic_waypoint real_model.h5 (1.0.0)

Team Sakura

Night Day
night day

Team Members

Submission checklist and requirements

System Architecture

The following is a system architecture diagram showing the ROS nodes and topics used in the project. The ROS nodes and topics shown in the diagram are described briefly in the Code Structure section below.

System Architecture

Code Structure

Below is a brief overview of the repo structure, along with descriptions of the ROS nodes.

(path_to_project_repo)/ros/src/tl_detector/

This package contains the traffic light detection node: tl_detector.py. This node takes in data from the /image_color, /current_pose, and /base_waypoints topics and publishes the locations to stop for red traffic lights to the /traffic_waypoint topic.

The /current_pose topic provides the vehicle’s current position, and /base_waypoints provides a complete list of waypoints the car will be following.

We build both a traffic light detection node and a traffic light classification node. Traffic light detection should take place within tl_detector.py, whereas traffic light classification should take place within ../tl_detector/light_classification_model/tl_classfier.py.

Traffic Light Detection Node

See code in /ros/src/tl_detector/.

(path_to_project_repo)/ros/src/waypoint_updater/

This package contains the waypoint updater node: waypoint_updater.py.

Waypoint Updater Node

See code in /ros/src/waypoint_updater/.

(path_to_project_repo)/ros/src/twist_controller/

Carla is equipped with a drive-by-wire (DBW) system, meaning the throttle, brake, and steering have electronic control. This package contains the files that are responsible for control of the vehicle: the node dbw_node.py and the file twist_controller.py.

DBW Node

See code in /ros/src/twist_controller/.

(path_to_project_repo)/ros/src/styx/

A package that contains a server for communicating with the simulator, and a bridge to translate and publish simulator messages to ROS topics.

(path_to_project_repo)/ros/src/styx_msgs/

A package which includes definitions of the custom ROS message types used in the project.

(path_to_project_repo)/ros/src/waypoint_loader/

A package which loads the static waypoint data and publishes to /base_waypoints.

(path_to_project_repo)/ros/src/waypoint_follower/

A package containing code from Autoware which subscribes to /final_waypoints and publishes target vehicle linear and angular velocities in the form of twist commands to the /twist_cmd topic.

Traffic Light Detection

Model for Simulator

We use Tiny YOLOv3 for traffic light detection and classification.

  1. Download Tiny YOLOv3 (also called YOLOv3-tiny) weight from YOLO website
  2. Convert the Darknet YOLO model to a Keras model using keras-yolo3.
  3. Transfer learning for traffic light detection.
Loss Val Loss

Graph of the Tiny YOLOv3 model:

Model for Site

We use YOLOv3 for traffic light detection and classification.

YOLOv3 (open youtube video on click):

YOLOv3 for site

Tiny YOLOv3 (open youtube video on click):

Tiny YOLOv3 for site

Original Instructions

This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.

Please use one of the two installation options, either native or docker installation.

Native Installation

Docker Installation

Install Docker

Build the docker container

docker build . -t capstone

Run the docker file

docker run -p 4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone

Port Forwarding

To set up port forwarding, please refer to the instructions from term 2

Usage

  1. Clone the project repository 
    git clone https://github.com/udacity/CarND-Capstone.git
  2. Install python dependencies 
    cd CarND-Capstone
pip install -r requirements.txt
  3. Make and run styx 
    cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch
  4. Run the simulator

Real world testing

  1. Download training bag that was recorded on the Udacity self-driving car.
  2. Unzip the file 
    unzip traffic_light_bag_file.zip
  3. Play the bag file 
    rosbag play -l traffic_light_bag_file/traffic_light_training.bag
  4. Launch your project in site mode 
    cd CarND-Capstone/ros
roslaunch launch/site.launch
  5. Confirm that traffic light detection works on real life images
This site is open source. Improve this page.