Skip to content

Commit d2df4b3

Browse files
clibdevclibdev
committed
update requirements file
1 parent a92f209 commit d2df4b3

3 files changed

Lines changed: 25 additions & 177 deletions

File tree

‎README.md‎

Lines changed: 18 additions & 169 deletions
Original file line numberDiff line numberDiff line change
@@ -1,179 +1,28 @@
1-
# ESPCN-PyTorch
1+
# Fork of [Lornatang/ESPCN-PyTorch](https://github.com/Lornatang/ESPCN-PyTorch)
22

3-
## Overview
3+
Differences between original repository and fork:
44

5-
This repository contains an op-for-op PyTorch reimplementation of [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://arxiv.org/abs/1609.05158v2).
5+
* Compatibility with PyTorch >=2.0. (🔥)
6+
* Installation with updated [requirements.txt](requirements.txt) file.
67

7-
## Table of contents
8+
# Installation
89

9-
- [ESPCN-PyTorch](#espcn-pytorch)
10-
- [Overview](#overview)
11-
- [Table of contents](#table-of-contents)
12-
- [About Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](#about-real-time-single-image-and-video-super-resolution-using-an-efficient-sub-pixel-convolutional-neural-network)
13-
- [Download weights](#download-weights)
14-
- [Download datasets](#download-datasets)
15-
- [How Test and Train](#how-test-and-train)
16-
- [Test ESPCN_x4](#test-espcn_x4)
17-
- [Train ESPCN_x4](#train-espcn_x4)
18-
- [Resume ESPCN_x4](#resume-train-espcn_x4)
19-
- [Result](#result)
20-
- [Credit](#credit)
21-
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](#real-time-single-image-and-video-super-resolution-using-an-efficient-sub-pixel-convolutional-neural-network)
22-
23-
## About Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
24-
25-
If you're new to ESPCN, here's an abstract straight from the paper:
26-
27-
Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational
28-
performance for single image super-resolution. In these methods, the low resolution (LR)
29-
input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means
30-
that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this
31-
paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose
32-
a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer
33-
which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the
34-
handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the
35-
computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and
36-
show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based
37-
methods.
38-
39-
## Download weights
40-
41-
- [Google Driver](https://drive.google.com/drive/folders/17ju2HN7Y6pyPK2CC_AqnAfTOe9_3hCQ8?usp=sharing)
42-
- [Baidu Driver](https://pan.baidu.com/s/1yNs4rqIb004-NKEdKBJtYg?pwd=llot)
43-
44-
## Download datasets
45-
46-
Contains DIV2K, DIV8K, Flickr2K, OST, T91, Set5, Set14, BSDS100 and BSDS200, etc.
47-
48-
- [Google Driver](https://drive.google.com/drive/folders/1A6lzGeQrFMxPqJehK9s37ce-tPDj20mD?usp=sharing)
49-
- [Baidu Driver](https://pan.baidu.com/s/1o-8Ty_7q6DiS3ykLU09IVg?pwd=llot)
50-
51-
Please refer to `README.md` in the `data` directory for the method of making a dataset.
52-
53-
## How Test and Train
54-
55-
Both training and testing only need to modify the `config.py` file.
56-
57-
### Test ESPCN_x4
58-
59-
Modify the `config.py` file.
60-
61-
- line 31: `model_arch_name` change to `espcn_x4`.
62-
- line 36: `upscale_factor` change to `4`.
63-
- line 38: `mode` change to `test`.
64-
- line 40: `exp_name` change to `ESPCN_x4-Set5`.
65-
- line 84: `lr_dir` change to `f"./data/Set5/LRbicx{upscale_factor}"`.
66-
- line 86: `gt_dir` change to `f"./data/Set5/GTmod12"`.
67-
- line 88: `model_weights_path` change to `./results/pretrained_models/ESPCN_x4-T91-64bf5ee4.pth.tar`.
68-
69-
```bash
70-
python3 test.py
71-
```
72-
73-
### Train ESPCN_x4
74-
75-
Modify the `config.py` file.
76-
77-
- line 31: `model_arch_name` change to `espcn_x4`.
78-
- line 36: `upscale_factor` change to `4`.
79-
- line 38: `mode` change to `test`.
80-
- line 40: `exp_name` change to `ESPCN_x4-Set5`.
81-
- line 84: `lr_dir` change to `f"./data/Set5/LRbicx{upscale_factor}"`.
82-
- line 86: `gt_dir` change to `f"./data/Set5/GTmod12"`.
83-
84-
```bash
85-
python3 train.py
10+
```shell
11+
pip install -r requirements.txt
8612
```
8713

88-
### Resume train ESPCN_x4
14+
# Pretrained models
8915

90-
Modify the `config.py` file.
16+
| Name | Scale | Set5 (PSNR) | Set14 (PSNR) | Link |
17+
|----------|-------|-------------|--------------|------------------------------------------------------------------------------------------------------------|
18+
| ESPCN_x2 | 2 | 36.64 | 32.35 | [PyTorch](https://github.com/clibdev/ESPCN-PyTorch/releases/latest/download/ESPCN_x2-T91-da809cd7.pth.tar) |
19+
| ESPCN_x3 | 3 | 32.55 | 29.20 | [PyTorch](https://github.com/clibdev/ESPCN-PyTorch/releases/latest/download/ESPCN_x3-T91-647e91f3.pth.tar) |
20+
| ESPCN_x4 | 4 | 30.26 | 27.41 | [PyTorch](https://github.com/clibdev/ESPCN-PyTorch/releases/latest/download/ESPCN_x4-T91-64bf5ee4.pth.tar) |
9121

92-
- line 31: `model_arch_name` change to `espcn_x4`.
93-
- line 36: `upscale_factor` change to `4`.
94-
- line 38: `mode` change to `test`.
95-
- line 40: `exp_name` change to `ESPCN_x4-Set5`.
96-
- line 57: `resume_model_weights_path` change to `./samples/ESPCN_x4-Set5/epoch_xxx.pth.tar`.
97-
- line 84: `lr_dir` change to `f"./data/Set5/LRbicx{upscale_factor}"`.
98-
- line 86: `gt_dir` change to `f"./data/Set5/GTmod12"`.
22+
# Inference
9923

100-
```bash
101-
python3 train.py
102-
```
103-
104-
## Result
105-
106-
Source of original paper results: [https://arxiv.org/pdf/1609.05158v2.pdf](https://arxiv.org/pdf/1609.05158v2.pdf)
107-
108-
In the following table, the value in `()` indicates the result of the project, and `-` indicates no test.
109-
110-
| Method | Scale | Set5 (PSNR) | Set14 (PSNR) |
111-
|:--------:|:-----:|:----------------:|:----------------:|
112-
| ESPCN_x4 | 2 | -(**36.64**) | -(**32.35**) |
113-
| ESPCN_x3 | 3 | 32.55(**32.55**) | 29.08(**29.20**) |
114-
| ESPCN_x4 | 4 | 30.90(**30.26**) | 27.73(**27.41**) |
115-
116-
```bash
117-
# Download `ESPCN_x4-T91-64bf5ee4.pth.tar` weights to `./results/pretrained_models/ESPCN_x4-T91-64bf5ee4.pth.tar`
118-
# More detail see `README.md<Download weights>`
119-
python3 ./inference.py
120-
```
121-
122-
Input:
123-
124-
<span align="center"><img width="240" height="360" src="figure/comic.png"/></span>
125-
126-
Output:
127-
128-
<span align="center"><img width="240" height="360" src="figure/sr_comic.png"/></span>
129-
130-
```text
131-
Build `espcn_x4` model successfully.
132-
Load `espcn_x4` model weights `./results/pretrained_models/ESPCN_x4-T91-64bf5ee4.pth.tar` successfully.
133-
SR image save to `./figure/sr_comic.png`
134-
```
135-
136-
### Credit
137-
138-
#### Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
139-
140-
_Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, Zehan Wang_ <br>
141-
142-
**Abstract** <br>
143-
Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational
144-
performance for single image super-resolution. In these methods, the low resolution (LR)
145-
input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means
146-
that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this
147-
paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose
148-
a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer
149-
which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the
150-
handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the
151-
computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and
152-
show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based
153-
methods.
154-
155-
[[Paper]](https://arxiv.org/pdf/1609.05158)
156-
157-
```
158-
@article{DBLP:journals/corr/ShiCHTABRW16,
159-
author = {Wenzhe Shi and
160-
Jose Caballero and
161-
Ferenc Husz{\'{a}}r and
162-
Johannes Totz and
163-
Andrew P. Aitken and
164-
Rob Bishop and
165-
Daniel Rueckert and
166-
Zehan Wang},
167-
title = {Real-Time Single Image and Video Super-Resolution Using an Efficient
168-
Sub-Pixel Convolutional Neural Network},
169-
journal = {CoRR},
170-
volume = {abs/1609.05158},
171-
year = {2016},
172-
url = {http://arxiv.org/abs/1609.05158},
173-
archivePrefix = {arXiv},
174-
eprint = {1609.05158},
175-
timestamp = {Mon, 13 Aug 2018 16:47:09 +0200},
176-
biburl = {https://dblp.org/rec/journals/corr/ShiCHTABRW16.bib},
177-
bibsource = {dblp computer science bibliography, https://dblp.org}
178-
}
24+
```shell
25+
python inference.py --model_arch_name espcn_x2 --upscale_factor 2 --model_weights_path results/pretrained_models/ESPCN_x2-T91-da809cd7.pth.tar --inputs_path figure/comic.png --output_path figure/sr_comic_x2.png
26+
python inference.py --model_arch_name espcn_x3 --upscale_factor 3 --model_weights_path results/pretrained_models/ESPCN_x3-T91-647e91f3.pth.tar --inputs_path figure/comic.png --output_path figure/sr_comic_x3.png
27+
python inference.py --model_arch_name espcn_x4 --upscale_factor 4 --model_weights_path results/pretrained_models/ESPCN_x4-T91-64bf5ee4.pth.tar --inputs_path figure/comic.png --output_path figure/sr_comic_x4.png
17928
```

‎inference.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
import cv2
1818
import torch
19-
from flatbuffers.builder import np
19+
import numpy as np
2020
from torch import nn
2121

2222
import config

‎requirements.txt‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
1-
opencv-python
2-
numpy
3-
tqdm
4-
torch
5-
setuptools
6-
torchvision
7-
natsort
1+
torch>=2.0.0
2+
torchvision>=0.15.0
3+
opencv-python>=4.7.0
4+
tqdm>=4.65.0
5+
scipy>=1.10.0
6+
natsort>=8.4.0

0 commit comments

Comments
 (0)