Semantic Segmentation Using DeepLab V3

Introduction

DeepLab is a series of image semantic segmentation models, whose latest version, i.e. v3+, proves to be the state-of-art. Its major contribution is the use of atrous spatial pyramid pooling (ASPP) operation at the end of the encoder. While the model works extremely well, its open sourced code is hard to read. Here we re-implemented DeepLab v3, the earlier version of v3+, which only additionally employs the decoder architecture, in a much simpler and understandable way. This is a collaborative project developed by me and Shengjie Lin from Toyota Technological Institute at Chicago.

Dependencies

  • Python 3.5
  • TensorFlow 1.8
  • Tqdm 4.26.0
  • Numpy 1.14
  • OpenCV 3.4.3
  • Pillow 5.3.0

Files

1
2
3
4
5
6
7
8
9
10
11
12
.
├── archived
├── download.py
├── feature_extractor.py
├── LICENSE.md
├── model.py
├── modules.py
├── nets
├── README.md
├── test_demo.py
├── train.py
└── utils.py

The nets directory contains network definition files that are directly copied from tensorflow/models/research/slim/nets.

Usages

Download Dataset

Download and extract VOC2012 dataset, SBD dataset, and pretrained models to designated directories.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ python download.py --help
usage: download.py [-h] [--downloads_dir DOWNLOADS_DIR] [--data_dir DATA_DIR]
[--pretrained_models_dir PRETRAINED_MODELS_DIR]
[--pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]]

Download DeepLab semantic segmentation datasets and pretrained backbone
models.

optional arguments:
-h, --help show this help message and exit
--downloads_dir DOWNLOADS_DIR
Downloads directory
--data_dir DATA_DIR Data directory
--pretrained_models_dir PRETRAINED_MODELS_DIR
Pretrained models directory
--pretrained_models PRETRAINED_MODELS [PRETRAINED_MODELS ...]
Pretrained models to download: resnet_50, resnet_101,
mobilenet_1.0_224

For example, to download and extract datasets and models into directories specified:

1
$ python download.py --downloads_dir ./downloads --data_dir ./data --pretrained_models_dir ./models/pretrained --pretrained_models resnet_50 resnet_101 mobilenet_1.0_224

For simplicity, please just run the following command in terminal:

1
$ python download.py

Train Model

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
$ python train.py --help
usage: train.py [-h] [--network_backbone NETWORK_BACKBONE]
[--pre_trained_model PRE_TRAINED_MODEL]
[--trainset_filename TRAINSET_FILENAME]
[--valset_filename VALSET_FILENAME] [--images_dir IMAGES_DIR]
[--labels_dir LABELS_DIR]
[--trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME]
[--images_augmented_dir IMAGES_AUGMENTED_DIR]
[--labels_augmented_dir LABELS_AUGMENTED_DIR]
[--model_dir MODEL_DIR] [--log_dir LOG_DIR]
[--random_seed RANDOM_SEED]

Train DeepLab v3 for image semantic segmantation.

optional arguments:
-h, --help show this help message and exit
--network_backbone NETWORK_BACKBONE
Network backbones: resnet_50, resnet_101,
mobilenet_1.0_224. Default: resnet_101
--pre_trained_model PRE_TRAINED_MODEL
Pretrained model directory
--trainset_filename TRAINSET_FILENAME
Train dataset filename
--valset_filename VALSET_FILENAME
Validation dataset filename
--images_dir IMAGES_DIR
Images directory
--labels_dir LABELS_DIR
Labels directory
--trainset_augmented_filename TRAINSET_AUGMENTED_FILENAME
Train augmented dataset filename
--images_augmented_dir IMAGES_AUGMENTED_DIR
Images augmented directory
--labels_augmented_dir LABELS_AUGMENTED_DIR
Labels augmented directory
--model_dir MODEL_DIR
Trained model saving directory
--log_dir LOG_DIR TensorBoard log directory
--random_seed RANDOM_SEED
Random seed for model training.

For simplicity, please run the following command in terminal:

1
$ python train.py

With learning rate of 1e-5, the mIOU could be greater 0.7 after 20 epochs, which is comparable to the test statistics of DeepLab v3 in the publication.

Demos

To show some demos, please run the following command in terminal:

1
$ python test_demo.py
Image Label Prediction
Image Label Prediction
Image Label Prediction
Image Label Prediction

References

GitHub

Semantic Segmentation Using DeepLab V3

https://leimao.github.io/project/DeepLab-V3/

Author

Lei Mao

Posted on

12-11-2018

Updated on

12-11-2018

Licensed under


Comments