Object detection using Darknet and YOLO

Darknet is an open-source neural network written in C and CUDA. Darknet supports both CPU and GPU computation. Darknet source code can be cloned from its Github repository. We can clone the repository and use it as per needs.

Darknet can be used directly and doesn’t need any other library. It has 2 optional dependencies OpenCV if you want a wider variety of supported image types and CUDA for GPU computation.

Darknet is famous for its Real-time object detection architecture: YOLO but it has several other models and architecture also. We can use these as per our needs:

1. ImageNet classification: We can use the darknet19 model to classify images from the ImageNet challenge classes. Darkent19 is a pretrained model on the Imagenet dataset and can directly be used.
2. Nightmare: This is a darknet network that runs the VGG-16 pretrained model backwards. it takes the convolution layer of the VGG-16 model and runs it backwards and generates an image from it.
3. RNNs in Darknet: Recurrent neural networks(RNNs) are a class of neural networks that allow previous outputs to be used as inputs while having hidden states. The common application for the RNNs involves time series prediction and text generation where there is a dependence of next input to previous state or input. In the darknet repository, there are some pretrained models for George R.R. Martin, William Shakespeare, Leo Tolstoy etc. or you can train your own model using the darknet.
4. DarkGO: DarkGo is a neural network that predicts the most likely moves in the game of GO. You can play along with professional games and see what moves are likely to happen next, make it play itself, or try to play against it!
5. Tiny Darknet: it is a small network for Imagenet classification which is 4 MB in size. However, there is another version of darknet called Darknet reference network which is 28 MB in size and performs better than Alexnet and Squeezenet.
6. YOLO: You Only Look Once is a state of the art, real-time object detection system. That is extremely fast and accurate. The Scaled YOLO v4 is the best neural network for object detection with a 55.8% AP Microsoft COCO test-dev dataset.

Using YOLO and Darknet for building object detection model

YOLO can be used to train your own object detection model as per the needs. We can make some changes in the exiting YOLO model and train it again from scratch or using the pre-trained weights. I am using google Colab to train the model so you might need to adapt the code in case you are using anything else. Let's train a model for ourselves:

Clone darknet repository:

Clone the darknet GitHub repo to your own workspace:

!git clone https://github.com/AlexeyAB/darknet

Make changes in the make file.

Make file is mainly used to enable CPU, GPU, CUDNN and then build the darknet. Let's enable the GPU. Also, Whenever you make changes to Make file you need to rebuild the darknet.

%cd 'drive/MyDrive/LP/darknet'
!sed -i 's/OPENCV=0/OPENCV=1/' Makefile
!sed -i 's/GPU=0/GPU=1/' Makefile
!sed -i 's/CUDNN=0/CUDNN=1/' Makefile
!sed -i 's/CUDNN_HALF=0/CUDNN_HALF=1/' Makefile

Install Dependencies

Installing OpenCV and other dependencies. you can skip if you do not want to use OpenCV:

!sudo apt install libopencv-dev python3-opencv

Build Darknet:

Move to the darknet folder and run the make command to build the darknet. It will take a couple of minutes to build this:

!make

While it is built you can download and prepare your dataset. If you want to build your dataset using google open image dataset you can follow my blog post here.

Prepare Data

I have downloaded the images of ‘Car’ and ‘Bicycle’ and created 2 folders with annotated datasets. One naming ‘train’ for training the data and the other ‘test’ for testing data. copy or unzip the data to the ‘darknet/data’ folder:

!unzip ../train.zip -d data/
!unzip ../test.zip -d data/

Adjusting YOLO Config File:

In the ‘darknet/cfg’ folder you will find the config files for the YOLO. This file contains detail about the model hyperparameter and model architecture. Download the ‘./darknet/cfg/yolov4-custom.cfg' file and we need to make the following changes to the file:

1. For training set ‘batch’ = 64 and ‘subdivisions’ = 16 parameter.

2. Set image ‘width’ and ‘height’ parameters. The number should be in multiple for the 32. I have set it to 416.

3. Change ‘max_batch’ to 1000 (*Generally it is set to (number of classes x 2000) but in increase the time taken to train the model so I have reduced to 1000 for testing*)

4. Changed the ‘step’ to 80%, 90% of the ‘max_batch’ value i.e. 800, 900

5. Make changes in all the YOLO layers. Search for ‘YOLO’ in the file and set the classes to 2 and filters for the previous layer to 21. Filter size is calculated on the basis of the number of classes.

Number of filters = ( no. of classes +5) *3. here (2+5)*3 = 21

Save this file and upload it to the drive.

I have created another file for testing configuration and set the ‘batch’ = 1 and ‘subdivisions’ = 1 rest other things needs to be the same. you can either create a separate file for testing or training or you can you the same by making changes after training the model.

Copy to file form drive to ‘darknet /cfg/’ folder.

!cp /content/drive/MyDrive/yolov4/yolov4-custom.cfg ./cfg

Create and copy dataset files

We need to create 2 files now.

1. obj.names file which contained the name of the classes for which we are training as shown below:

2. obj.data file which contains the following detail for the dataset:

classes -> number of classes

train -> path and name of the file containing train images name with path

valid -> path and name of the file containing test images name with path

names -> path and name of obj.name file

backup -> backup folder path used to save model wights.

!cp /content/drive/MyDrive/yolov4/obj.names ./data
!cp /content/drive/MyDrive/yolov4/obj.data  ./data

Create two files for test.txt and train.txt for test and train to be used by darknet for training purposes. These files contain paths to all the images to be used for training and testing.

You can use these 2 scripts from this repo to generate these files:

!cp /content/drive/MyDrive/yolov4/train.py ./
!cp /content/drive/MyDrive/yolov4/test.py ./

Run scripts

!python train.py
!python test.py

These scripts simply take all images for the train folder and create a list in train.txt and all images for the test folder and create a list in test.txt

Now we have all the necessary setup to train the model. We can now start the model training from scratch or we can use a pre-train model weight and train over them. I am here using a pre-train yolov4 weight so let's download the weight:

!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.conv.137

Train the model:

The general format command for running the yolo is :

!./darknet detector <train/test/map> <obj.data file with path> <config file with path> <weight file with path>

!./darknet detector train data/obj.data cfg/yolov4-custom.cfg  yolov4.conv.137

Our model is trained and the weight is saved to the google drive backup folder. Let's see our model training process by seeing the loss vs iteration chart generated

imShow('chart.png')

We can also check the mean average precision for the model:

!./darknet detector map data/obj.data cfg/yolov4-custom.cfg /content/drive/MyDrive/yolov4/backup/yolov4-custom_best.weights

This is the output from the model:

Our model has an mAP value of 0.71. We can make the following changes to improve the accuracy further

1. The above chart indicated that our model has not converged yet we can increase the number of iteration
2. We can increase the dataset size as we are only using 400 images of each class.
3. We can increase the image size to train the model.

Let's make a prediction from our saved model:

copy test config file. (this is the same file with batch=1 and subdivisions=1)

!cp /content/drive/MyDrive/yolov4/yolov4-custom-test.cfg ./cfg

make predictions:

!./darknet detector test data/obj.data cfg/yolov4-custom-test.cfg /content/drive/MyDrive/yolov4/backup/yolov4-custom_best.weights /content/drive/MyDrive/yolov4/test_image/car.jpg -thresh 0.3

Darknet saves the prediction as in images as ‘predictions.jpg’ showing the image below:

imShow(‘predictions.jpg’)

We have trained the YOLOv4 model to classify the images of cars and bicycles. similarly, we can follow the above approach to train a model for the different use cases.

Endnotes:

During working on this project I have learned from some errors that I think might be useful for you so just sharing them here:

1. Try to follow the darknet framework to avoid errors. Like, keep the dataset in the ‘data’ folder and config files in the ‘cfg’ folder.
2. Always run the darknet command files from the darknet folder only as in some cases darknet uses a relative path and we can file not found errors.
3. Use GPU for training as it is very fast.
4. Always create the backup folder. While training YOLO saves the weight to the backup folder after every 1000 steps. In case if your training stop abruptly or your notebook gets disconnected you can start training from the last save step. Thus saving time and effort to retrain the model.

References:

YOLO (https://pjreddie.com/darknet/yolo/)

YOLO paper (https://arxiv.org/abs/2004.10934)

YOLO official tutorial (https://colab.research.google.com/drive/12QusaaRj_lUwCGDvQNfICpa7kA7_a2dE)

Github Repo (https://github.com/G-VJ/Yolo_object_detection)