Unlike ordinary pixel-to-pixel translation models, cycle-consistent adversarial networks (CycleGAN) has been proved to be useful for image translations without using paired data. This is a reproduced implementation of CycleGAN for image translations, but it is more compact.
- Python 3.5
- Numpy 1.14
- TensorFlow 1.8
- ProgressBar2 3.37.1
- OpenCV 3.4
. ├── convert.py ├── download.py ├── figures ├── LICENSE.md ├── model.py ├── module.py ├── README.md ├── train.py └── utils.py
Download and unzip specified dataset to designated directories.
$ python download.py --help usage: download.py [-h] [--download_dir DOWNLOAD_DIR] [--data_dir DATA_DIR] [--datasets DATASETS [DATASETS ...]] Download CycleGAN datasets. optional arguments: -h, --help show this help message and exit --download_dir DOWNLOAD_DIR Download directory for zipped data --data_dir DATA_DIR Data directory for unzipped data --datasets DATASETS [DATASETS ...] Datasets to download: apple2orange, summer2winter_yosemite, horse2zebra, monet2photo, cezanne2photo, ukiyoe2photo, vangogh2photo, maps, cityscapes, facades, iphone2dslr_flower, ae_photos
For example, to download
horse2zebra datasets to
download directory and extract to
$ python download.py --download_dir ./download --data_dir ./data --datasets apple2orange horse2zebra
To have a good conversion capability, the training would take at least 100 epochs, which could take very long time even using a NVIDIA GTX TITAN X graphic card. The model also consumes a lot of graphic card memories. But this could be reduced by reducing the number of convolution filters
num_filters in the model.
$ python train.py --help usage: train.py [-h] [--img_A_dir IMG_A_DIR] [--img_B_dir IMG_B_DIR] [--model_dir MODEL_DIR] [--model_name MODEL_NAME] [--random_seed RANDOM_SEED] [--validation_A_dir VALIDATION_A_DIR] [--validation_B_dir VALIDATION_B_DIR] [--output_dir OUTPUT_DIR] Train CycleGAN model for datasets. optional arguments: -h, --help show this help message and exit --img_A_dir IMG_A_DIR Directory for A images. --img_B_dir IMG_B_DIR Directory for B images. --model_dir MODEL_DIR Directory for saving models. --model_name MODEL_NAME File name for saving model. --random_seed RANDOM_SEED Random seed for model training. --validation_A_dir VALIDATION_A_DIR Convert validation A images after each training epoch. If set none, no conversion would be done during the training. --validation_B_dir VALIDATION_B_DIR Convert validation B images after each training epoch. If set none, no conversion would be done during the training. --output_dir OUTPUT_DIR Output directory for converted validation images.
For example, to train CycleGAN model for
$ python train.py --img_A_dir ./data/horse2zebra/trainA --img_B_dir ./data/horse2zebra/trainB --model_dir ./model/horse_zebra --model_name horse_zebra.ckpt --random_seed 0 --validation_A_dir ./data/horse2zebra/testA --validation_B_dir ./data/horse2zebra/testB --output_dir ./validation_output
output_dir set, we could monitor the conversion of validation images after each epoch using our bare eye.
Convert images using pre-trained models.
$ python convert.py --help usage: convert.py [-h] [--model_filepath MODEL_FILEPATH] [--img_dir IMG_DIR] [--conversion_direction CONVERSION_DIRECTION] [--output_dir OUTPUT_DIR] Convert images using pre-trained CycleGAN model. optional arguments: -h, --help show this help message and exit --model_filepath MODEL_FILEPATH File path for the pre-trained model. --img_dir IMG_DIR Directory for the images for conversion. --conversion_direction CONVERSION_DIRECTION Conversion direction for CycleGAN. A2B or B2A. The first object in the model file name is A, and the second object in the model file name is B. --output_dir OUTPUT_DIR Directory for the converted images.
To convert images, put images into
img_dir and run the following commands in the terminal, the converted images would be saved in the
$ python convert.py --model_filepath ./model/horse_zebra/horse_zebra.ckpt --img_dir ./data/horse2zebra/testA --conversion_direction A2B --output_dir ./converted_images
The convention for
conversion_direction is first object in the model file name is A, and the second object in the model file name is B. In this case,
horse = A and
zebra = B.
Horse and Zebra Conversion
The horse and zebra conversion model was trained for more than 500 epochs, and some of the selected test images were presented below.
|Horse||Horse to Zebra||Zebra||Zebra to Horse|
It should be noted that the above presented conversions look extremely well. However, for most of the test images, the conversions always have defects which are visually pickable. Most horses or zebras that have “abnormal” poses does not convert well. The conversion of zebra to horse looks worse than the conversion of horse to zebra. I think one of the reasons that the conversions have defects is that the dataset is small and it does not contain all the poses for horse and zebra.
The test images after 200 epochs does not look too much different from the test images after 500 epochs, although the training loss kept decreasing. To further optimize the training, I think it might be worthy of trying learning rate decay and cycle loss weight decay.
Download the pre-trained horse-zebra conversion model from Google Drive.
Yosemite Summer and Winter Conversion
The Yosemite summer and winter conversion model was trained for more than 200 epochs, and some of the selected test images were presented below.
|Summer||Summer to Winter||Winter||Winter to Summer|
Most of the conversions works pretty well, except that the snows could not be converted probably because there is no objects that could be mapped to snow.
Download the pre-trained Yosemite summer-winter conversion model from Google Drive.
Cityscapes Vision and Semantic Conversion
The Cityscapes vision and semantic conversion model was trained for more than 150 epochs, and some of the selected test images were presented below.
|Vision||Vision to Semantic||Semantic||Semantic to Vision|
The conversion from vision to semantic would usually leads to unexist objects or the object position has been moved in the semantic views, suggesting that CycleGAN, probably even GAN, is not a good technique to do semantic labelings. The conversion from semantic to vision failed to provide details of the object.
Download the pre-trained Cityscapes vision-semantic conversion model from Google Drive.
- Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. 2017.
- Xiaowei Hu’s CycleGAN TensorFlow Implementation Repository
- Hardik Bansal’s CycleGAN TensorFlow Implementation Repository