Pytorch series | data loading and preprocessing tutorial

PyTorch series | Data loading and preprocessing tutorial _ data

author ：Damiano Baschiera

The original title is | DATA LOADING AND PROCESSING TUTORIAL

author | Sasank Chilamkurthy

brief introduction

This tutorial is mainly about how to load 、 Methods of preprocessing and enhancing data .

First, make sure to install the following python library ：

scikit-image ： Processing image data
pandas ： Handle csv file

The import module code is as follows ：

from __future__ import print_function, division
import os
import torch
import pandas as pd
from skimage import io, transform
import numpy as np
import matplotlib.pyplot as plt
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms, utils
# Ignore warnings
import warnings
warnings.filterwarnings("ignore")
plt.ion() # interactive mode

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.

This tutorial uses a face pose data set , The picture is as follows ：

PyTorch series | Data loading and preprocessing tutorial _ Data sets _02

Every face has 68 Personal face key points , It is from dlib Generated , Specific implementation can see its official website ：

https://blog.dlib.net/2014/08/real-time-face-pose-estimation.html

Dataset download address ：

https://download.pytorch.org/tutorial/faces.zip

In dataset csv The format of the file is as follows , Picture name and coordinates of each key point x, y

image_name,part_0_x,part_0_y,part_1_x,part_1_y,part_2_x, ... ,part_67_x,part_67_y
0805personali01.jpg,27,83,27,98, ... 84,134
1084239450_e76e00b7e7.jpg,70,236,71,257, ... ,128,312

1.
2.
3.

Download and decompress the dataset and put it in the folder data/faces in , Then we quickly open it face_landmarks.csv file , View file contents , I.e. marking information , The code is as follows ：

landmarks_frame = pd.read_csv('data/faces/face_landmarks.csv')
n = 65
img_name = landmarks_frame.iloc[n, 0]
landmarks = landmarks_frame.iloc[n, 1:].as_matrix()
landmarks = landmarks.astype('float').reshape(-1, 2)
print('Image name: {}'.format(img_name))
print('Landmarks shape: {}'.format(landmarks.shape))
print('First 4 Landmarks: {}'.format(landmarks[:4]))

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

The output is as follows ：

PyTorch series | Data loading and preprocessing tutorial _ load _03

Then write an auxiliary function to display the face image and its key points , The code is as follows ：

def show_landmarks(image, landmarks):
"""Show image with landmarks"""
plt.imshow(image)
plt.scatter(landmarks[:, 0], landmarks[:, 1], s=10, marker='.', c='r')
plt.pause(0.001) # pause a bit so that plots are updated
plt.figure()
show_landmarks(io.imread(os.path.join('data/faces/', img_name)),
landmarks)
plt.show()

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

The output is as follows ：

PyTorch series | Data loading and preprocessing tutorial _ Preprocessing _04

Dataset class

torch.utils.data.Dataset Is an abstract class that represents a dataset , When you customize your dataset, you need to inherit Dataset Category , And override the following methods ：

len ： call len(dataset) The number of data sets that can be returned when ;
getitem： get data , Index access can be realized , namely dataset[i] You can visit the i Sample data

Next, we will customize a category for our face key data set , stay __init__ Method will read the data set information , And in __getitem__ Data set obtained by method call , This is mainly a memory based consideration , This approach does not need to read all the data once and store it in memory , You can read and load data into memory when you need to read data .

Samples of the dataset will be represented in a dictionary ：{'image': image, 'landmarks': landmarks}, There is also an optional parameter transform Sample data for preprocessing read , This is covered in the next section transform Usefulness .

The code for the custom function is as follows ：

class FaceLandmarksDataset(Dataset):
"""Face Landmarks dataset."""
def __init__(self, csv_file, root_dir, transform=None):
"""
Args:
csv_file (string): With annotation information csv File path
root_dir (string): Folder of pictures
transform (callable, optional): Optional method for preprocessing pictures
"""
self.landmarks_frame = pd.read_csv(csv_file)
self.root_dir = root_dir
self.transform = transform
def __len__(self):
return len(self.landmarks_frame)
def __getitem__(self, idx):
# Read the picture
img_name = os.path.join(self.root_dir,
self.landmarks_frame.iloc[idx, 0])
image = io.imread(img_name)
# Read keys and convert to numpy Array
landmarks = self.landmarks_frame.iloc[idx, 1:]
landmarks = np.array([landmarks])
landmarks = landmarks.astype('float').reshape(-1, 2)
sample = {'image': image, 'landmarks': landmarks}
if self.transform:
sample = self.transform(sample)
return sample

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.

Next is a simple example to use our custom dataset class , In the example, before reading 4 Samples and show ：

face_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',
root_dir='data/faces/')
fig = plt.figure()
# Before reading 4 Picture and show
for i in range(len(face_dataset)):
sample = face_dataset[i]
print(i, sample['image'].shape, sample['landmarks'].shape)
ax = plt.subplot(1, 4, i + 1)
plt.tight_layout()
ax.set_title('Sample #{}'.format(i))
ax.axis('off')
show_landmarks(**sample)
if i == 3:
plt.show()
break

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.

The output is as follows ：

PyTorch series | Data loading and preprocessing tutorial _ Preprocessing _05

Transforms

From the output structure of the above example, we can see a problem , The size of the pictures is not the same , But most neural networks need to input a fixed size of the image . therefore , The next step is to give some preprocessing code , There are three main preprocessing methods ：

Rescale ： Resize the picture
RandomCrop： Crop pictures randomly , This is a data enhancement method
ToTensor： take numpy Format picture to pytorch Data format tensors, We need to exchange coordinates here .

These methods will be written as callable classes , Instead of simple functions , So you don 't have to pass parameters every time . therefore , We need to achieve __call__ Method , And if necessary ,__init__ Method is also to be realized , You can then call these methods as follows ：

Rescale The implementation code of the method is as follows ：

class Rescale(object):
""" Resize picture to given size .
Args:
output_size (tuple or int): Expected output picture size . If it is tuple type , The output image size is given output_size;
If it is int type , Then the shortest edge of the image will match the given size , Then adjust the maximum edge to maintain the same ratio .
"""
def __init__(self, output_size):
assert isinstance(output_size, (int, tuple))
self.output_size = output_size
def __call__(self, sample):
image, landmarks = sample['image'], sample['landmarks']
h, w = image.shape[:2]
# Determine the form of a given size ,tuple still int type
if isinstance(self.output_size, int):
# int type , Given size as shortest edge , The maximum side length shall be adjusted according to the original size scale
if h &gt; w:
new_h, new_w = self.output_size * h / w, self.output_size
else:
new_h, new_w = self.output_size, self.output_size * w / h
else:
new_h, new_w = self.output_size
new_h, new_w = int(new_h), int(new_w)
img = transform.resize(image, (new_h, new_w))
# According to the size scale before and after adjustment , Adjust the coordinate position of key points , also x Corresponding w,y Corresponding h
landmarks = landmarks * [new_w / w, new_h / h]
return {'image': img, 'landmarks': landmarks}

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.

RandomCrop Code implementation of ：

class RandomCrop(object):
""" Given picture , Randomly cut any area as large as the given size .
Args:
output_size (tuple or int): The size of the picture you want to crop . If it is int, You'll get a picture the size of a square .
"""
def __init__(self, output_size):
assert isinstance(output_size, (int, tuple))
if isinstance(output_size, int):
self.output_size = (output_size, output_size)
else:
assert len(output_size) == 2
self.output_size = output_size
def __call__(self, sample):
image, landmarks = sample['image'], sample['landmarks']
h, w = image.shape[:2]
new_h, new_w = self.output_size
#  Randomly select the upper left corner of the crop region , The starting point ,(left, top), Range is from original size - Output size
top = np.random.randint(0, h - new_h)
left = np.random.randint(0, w - new_w)
image = image[top: top + new_h,
left: left + new_w]
# Adjust key coordinates , Translate the selected crop start point
landmarks = landmarks - [left, top]
return {'image': image, 'landmarks': landmarks}

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.

ToTensor Method implementation of ：

class ToTensor(object):
""" take ndarrays Convert to tensors."""
def __call__(self, sample):
image, landmarks = sample['image'], sample['landmarks']
# Adjust coordinate dimensions ,numpy The dimension of is H x W x C, and torch The image dimension of is C X H X W
image = image.transpose((2, 0, 1))
return {'image': torch.from_numpy(image),
'landmarks': torch.from_numpy(landmarks)}

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

Combination of preprocessing methods

Next is an example of using the custom preprocessing method described above .

Suppose we want to adjust the shortest side length of the picture to 256, Then cut one randomly 224*224 Size picture area , That is, we need to combine calls Rescale and RandomCrop Pretreatment method .

torchvision.transforms.Compose It is a class that can implement the combination of methods to be processed , The implementation code is as follows ：

scale = Rescale(256)
crop = RandomCrop(128)
composed = transforms.Compose([Rescale(256),
RandomCrop(224)])
# Call the above for picture data 3 Different pretreatment methods , I.e. use alone Rescale,RandomCrop, Use a combination of Rescale and RandomCrop
fig = plt.figure()
sample = face_dataset[65]
for i, tsfrm in enumerate([scale, crop, composed]):
transformed_sample = tsfrm(sample)
ax = plt.subplot(1, 3, i + 1)
plt.tight_layout()
ax.set_title(type(tsfrm).__name__)
show_landmarks(**transformed_sample)
plt.show()

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.

Output structure ：

PyTorch series | Data loading and preprocessing tutorial _ Preprocessing _06

Iterate over the entire dataset

Now we've defined a class to process the dataset ,3 Classes of preprocessing data , So we can integrate them , The process of loading and preprocessing data , The process is as follows ：

First read the picture according to the picture path
Call preprocessing methods for all images
The preprocessing method can also realize data enhancement

The implementation code is as follows ：

transformed_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',
root_dir='data/faces/',
transform=transforms.Compose([
Rescale(256),
RandomCrop(224),
ToTensor()
]))
for i in range(len(transformed_dataset)):
sample = transformed_dataset[i]
print(i, sample['image'].size(), sample['landmarks'].size())
if i == 3:
break

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.

Output results ：

PyTorch series | Data loading and preprocessing tutorial _ Data sets _07

This is just a simple process , In fact, when processing and loading data , We usually do the following processing for the data ：

Divide the data into batches and batches according to the given size
Disarrange data order
use multiprocessing To load data in parallel

torch.utils.data.DataLoader Is an iterator that can implement the above operations . The required parameters are shown in the following code , One of the parameters collate_fn It is used to specify how to batch data operation , But you can also use the default function .

dataloader = DataLoader(transformed_dataset, batch_size=4,
shuffle=True, num_workers=4)
# Auxiliary function , Used to show a batch The data of
def show_landmarks_batch(sample_batched):
"""Show image with landmarks for a batch of samples."""
images_batch, landmarks_batch = \
sample_batched['image'], sample_batched['landmarks']
batch_size = len(images_batch)
im_size = images_batch.size(2)
grid_border_size = 2
grid = utils.make_grid(images_batch)
plt.imshow(grid.numpy().transpose((1, 2, 0)))
for i in range(batch_size):
plt.scatter(landmarks_batch[i, :, 0].numpy() + i * im_size + (i + 1) * grid_border_size,
landmarks_batch[i, :, 1].numpy() + grid_border_size,
s=10, marker='.', c='r')
plt.title('Batch from dataloader')
for i_batch, sample_batched in enumerate(dataloader):
print(i_batch, sample_batched['image'].size(),
sample_batched['landmarks'].size())
# observe 4th batch and stop.
if i_batch == 3:
plt.figure()
show_landmarks_batch(sample_batched)
plt.axis('off')
plt.ioff()
plt.show()
break

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.

Output results ：

PyTorch series | Data loading and preprocessing tutorial _ Preprocessing _08

torchvision

Finally, it introduces torchvision This library , It provides some common data sets and preprocessing methods , With this library, you don't need custom classes , The common method is ImageFolder , It assumes that the path to save the image is as follows ：

there ants,bees Wait, it's all category labels , Besides PIL.Image The pretreatment method of , Such as RandomHorizontalFlip 、Scale All contained in torchvision in , An example is as follows ：

import torch
from torchvision import transforms, datasets
data_transform = transforms.Compose([
transforms.RandomSizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
hymenoptera_dataset = datasets.ImageFolder(root='hymenoptera_data/train',
transform=data_transform)
dataset_loader = torch.utils.data.DataLoader(hymenoptera_dataset,
batch_size=4, shuffle=True,
num_workers=4)

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.

Summary

This tutorial mainly introduces how to customize a class to load your dataset , And pretreatment methods , At the same time, it also introduces PyTorch Medium torchvision ,torch.utils.data.DataLoader Method .

The code of this article is uploaded to Github：

https://github.com/ccc013/DeepLearning_Notes/blob/master/Pytorch/pytorch_dataloader_tutorial.ipynb

in addition , It's also useful dlib Generate face key code ：

https://github.com/ccc013/DeepLearning_Notes/blob/master/Pytorch/create_landmark_dataset.py

Besides , You can also reply to the official account. “PyTorch” Get the dataset and code for this tutorial .

Welcome to my WeChat official account. -- The growth of algorithmic apes , Or scan the QR code below , Let's talk , Learning and progress ！