Noise-Robust Traffic Density Estimator

Project page for Capstone Design 1(2020/03 ~ 2020/06) and Capstone Design 2(2020/09 ~ 2020/12).

Traffic density is estimated using the mask of the vehicle extracted from satellite image through Mask R-CNN.
By attaching the denosing network to the Mask R-CNN, the above process can be more robust to noise in image.

[Step 1] Noise Removal

[Step 2] Vehicle Mask Extraction

[Step 3] Density Map Generation

Environments

We have tested the code in the following environment.

OS	Python	Pytorch	CUDA	GPU	NVIDIA Driver
Ubuntu 18.04.5 LTS	3.7.13	1.9.1	11.1	NVIDIA RTX A6000	470.57.02

Preparations

# [Step 1]: Create new conda environment and activate.
#           Set [ENV_NAME] freely to any name you want. (Please exclude the brackets.)
conda create --name [ENV_NAME] python=3.7
conda activate [ENV_NAME]

# [Step 2]: Clone the repository.
git clone https://github.com/2gunsu/Traffic-Density-Estimator
cd Traffic-Density-Estimator

# [Step 3]: Install some packages using 'requirements.txt' in the repository.
pip install -r requirements.txt

# [Step 4]: Install Pytorch v1.9.1
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

# [Step 5]: Install Detectron2 v0.5
#           Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
python -m pip install detectron2==0.5 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html

Datasets

DOTA: A Large-scale Dataset for Object Detection in Aerial Images [Paper] [Site]

There are several classes in this dataset, but we only used the classes belonging to the vehicle among them.
You can download pre-processed DOTA dataset in this link directly. (7.3GB)
Some image samples and its corresponding annotations are shown below.

Usages

Pretrained Weights

Download the config files and pretrained weights from the table below.

Trained Dataset	With Denoiser	Noise Type	Backbone	Config File	Weight File
DOTA	X	-	ResNeXt-101-FPN	Download	Download
DOTA	X	-	ResNet-101-FPN	Download	Download
DOTA	X	-	ResNet-50-FPN	Download	Download

Training

Please check more detailed parameters in train.py and follow the script below for a quick start.
The data path(--train_path or --val_path) must contain an Image folder and a Label.json file.
Skip the training if you only want to use the pretrained model.

python train.py --train_path    [str]   # Directory of training data
                --val_path      [str]   # Directory of validation data
                --output_dir    [str]   # Output directory
                --backbone_arch [str]   # Select one in ['R50-FPN', 'R101-FPN', 'X101-FPN'].
                                        # 'R' denotes for ResNet, 'X' denotes for ResNeXt
                --gpu_id        [int]   # Index of the GPU to be used for training
                --epochs        [int]   
                --batch_size    [int]

Evaluation

If you want to proceed with quantitative evaluation, follow the script below.

python evaluation.py --eval_path      [str]   # Directory of evaluation data
                     --config_file    [str]   # Path of config file (.yaml)
                     --weight_file    [str]   # Path of weight file (.pth)
                     --gpu_id         [int]   # Index of the GPU to be used for evaluation

Inference on Single Image

Follow the script below to test general-sized image.

python test.py --config_file    [str]   # Path of config file (.yaml)
               --weight_file    [str]   # Path of weight file (.pth)
               --conf_score     [float] # Confidence threshold for inference
               --gpu_id         [int]   # Index of the GPU to be used for inference
               --image_file     [str]   # Path of single image file
               --save_dir       [str]

But if you want to process very high resolution images, follow the script below.
The script below splits the large image into smaller patches determined by --grid_size, inferences them individually, and merges them back together.

python test.py --config_file    [str]   # Path of config file (.yaml)
               --weight_file    [str]   # Path of weight file (.pth)
               --conf_score     [float] # Confidence threshold for inference
               --gpu_id         [int]   # Index of the GPU to be used for inference
               --image_file     [str]   # Path of large-sized image file
               --save_dir       [str]

               --grid_split             # [Optional]
               --grid_size      [int]   # [Optional] Determine the size of patches

Inference on Multiple Images

python test.py --config_file    [str]   # Path of config file (.yaml)
               --weight_file    [str]   # Path of weight file (.pth)
               --conf_score     [float] # Confidence threshold for inference
               --gpu_id         [int]   # Index of the GPU to be used for inference
               --image_dir      [str]   # Directory which contains multiple images
               --save_dir       [str]

               --grid_split             # [Optional]
               --grid_size      [int]   # [Optional] Determine the size of patches

Quantitative Results

Qualitative Results

Please click to enlarge the image.
The images below are very high resolution, so loading may take some time.

(1) Daejeon, South Korea / 7602 X 7602

(2) Incheon, South Korea / 7602 X 7602

(3) Seoul, South Korea / 7602 X 7602

(4) Busan, South Korea / 7602 X 7602

(5) New York, United States of America / 7602 X 7602

(6) Shanghai, China / 7602 X 7602

References

[1] Krull, Alexander, Tim-Oliver Buchholz, and Florian Jug. "Noise2Void - Learning Denoising from Single Noisy Images." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[2] K. He, G. Gkioxari, P. Dollár and R. Girshick, "Mask R-CNN," 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980-2988, doi: 10.1109/ICCV.2017.322.