Created
May 4, 2021 19:20
-
-
Save aicrowd-bot/b7df9aa7e1635b3dd183cbdee94deb0d to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Tutorial_GWC_2021.ipynb","provenance":[{"file_id":"1RPrqEr_WIyHbk0AOOkSJiZqW2g3NrbfN","timestamp":1620155867235},{"file_id":"1jvkLe5ocdLG8-RHmfx69JGiUK4BZ5tTu","timestamp":1620065109363},{"file_id":"1msPigLz7p7APKgtRyfBG0yqwXuZpD8IF","timestamp":1619680443846},{"file_id":"1B6n0YZMGNmOAVVYjnsRxL42M7MT3GxOO","timestamp":1619016741533}],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"VWkCOw-s55AE"},"source":["# Global Wheat Competition 2021 - Starting notebook\n","\n","- The goal of the notebook is to help you to train your first model and submit !\n","- We will use Pytorch / Torchvision / Pytorch Lightning to go through your first model !\n","- Before starting, Please check in Edit / Prefrences if GPU is selected\n","\n","\n","\n","---\n","\n","\n","\n","---\n","\n","\n",""]},{"cell_type":"markdown","metadata":{"id":"0H3J2MVK_5TW"},"source":["### Download Aicrowd-cli 📚\n","\n","It helps to download dataset and make submission directly via the notebook."]},{"cell_type":"code","metadata":{"id":"psxJC7EC_5TX"},"source":["!pip install aicrowd-cli"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"D5L6NxEe_5TZ"},"source":["## Download Data\n","The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.\n","\n","### Please enter your API Key from [here](https://www.aicrowd.com/participants/me)."]},{"cell_type":"code","metadata":{"id":"lyF9gbdx_5TZ"},"source":["API_KEY = \"\" \n","!aicrowd login --api-key $API_KEY"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"hx8pHTHZ_5Ta"},"source":["!aicrowd dataset download --challenge global-wheat-challenge-2021"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"ESHbptKy3pEQ"},"source":["!unzip train.zip\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"22Hst_AHrBAQ"},"source":["!unzip test.zip"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"gpqqMTDjf2L3"},"source":["# Download Necessary Packages"]},{"cell_type":"code","metadata":{"id":"dYTanbk7Agvi"},"source":["!pip install albumentations==0.4.6\n","!pip install pytorch_lightning\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"BrI4npQQ5rMV"},"source":["# Common imports\n","import math\n","import sys\n","import time\n","from tqdm.notebook import tqdm\n","import numpy as np\n","from pathlib import Path\n","import pandas as pd\n","import random\n","import cv2\n","import matplotlib.pyplot as plt\n","\n","# Torch imports \n","import torch\n","import torch.nn as nn\n","import torch.nn.functional as F\n","from torch.utils.data import Dataset, DataLoader\n","\n","import torchvision\n","from torchvision.models.detection.faster_rcnn import FastRCNNPredictor\n","import torchvision.transforms as transforms\n","from torchvision.ops.boxes import box_iou\n","from torchvision.models.detection._utils import Matcher\n","from torchvision.ops import nms, box_convert\n","import torch.nn.functional as F\n","\n","# Albumentations is used for the Data Augmentation\n","import albumentations as A\n","from albumentations.pytorch import ToTensorV2\n","\n","# Pytorch import\n","from pytorch_lightning.core.lightning import LightningModule\n","from pytorch_lightning import Trainer, seed_everything\n","\n","\n","\n","\n","\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"fPUM96K27GYf"},"source":["## Dataloader and Dataset\n","\n","- It is used by torch to load the images and the annotations\n","- Documentation: https://pytorch.org/tutorials/beginner/data_loading_tutorial.html"]},{"cell_type":"markdown","metadata":{"id":"fcIAcyVN-eNS"},"source":["We will write two extension of the Dataset class:\n","* One that will read the train.csv to get images, boxes and domain\n","* One that will read the submission.csv to retrieve images to predict and their associated domains.\n","\n","# 💻 Labels\n","\n","* All boxes are contained in a csv with three columns `image_name`, <span style=\"font-size: 14.08px; white-space: pre-wrap; background-color: rgb(254, 242, 242);\">BoxesString</span> and <span style=\"font-size: 14.08px; white-space: pre-wrap; background-color: rgb(254, 242, 242);\">domain</span>\n","* `image_name` is the name of the image, without the suffix. All images have a .png extension\n","* <span style=\"font-size:14.08px\"><span style=\"white-space:pre-wrap\"><span style=\"background-color:#fef2f2\">BoxesString</span></span></span> is a string containing all predicted boxes with the format \\[x\\_min,y\\_min, x\\_max,y\\_max\\]. To concatenate a list of boxes into a PredString, please concatenate all list of coordinates with one space (\" \") and all boxes with one semi-column \";\". If there is no box, BoxesString is equal to \"no\\_box\".\n","* <span style=\"font-size:14.08px\"><span style=\"white-space:pre-wrap\"><span style=\"background-color:#fef2f2\">domain</span></span></span> give the domain for each image \n","\n","\n"," "]},{"cell_type":"code","metadata":{"id":"5p4H0te66-Qs"},"source":["class WheatDataset(Dataset):\n"," \"\"\"A dataset example for GWC 2021 competition.\"\"\"\n","\n"," def __init__(self, csv_file,root_dir , transform=None):\n"," \"\"\"\n"," Args:\n"," csv_file (string): Path to the csv file with annotations.\n"," root_dir (string): Directory with all the images.\n"," transform (callable, optional): Optional data augmentation to be applied\n"," on a sample.\n"," \"\"\"\n","\n"," self.root_dir = Path(root_dir)\n"," annotations = pd.read_csv(csv_file)\n","\n"," self.image_list = annotations[\"image_name\"].values\n"," self.domain_list = annotations[\"domain\"].values\n"," self.boxes = [self.decodeString(item) for item in annotations[\"BoxesString\"]]\n"," \n"," self.transform = transform\n","\n"," def __len__(self):\n"," return len(self.image_list)\n","\n"," def __getitem__(self, idx):\n"," \n"," imgp = str(self.root_dir / (self.image_list[idx]+\".png\"))\n"," domain = self.domain_list[idx] # We don't use the domain information but you could !\n"," bboxes = self.boxes[idx]\n"," img = cv2.imread(imgp)\n"," image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # Opencv open images in BGR mode by default\n","\n"," if self.transform:\n"," transformed = self.transform(image=image,bboxes=bboxes,class_labels=[\"wheat_head\"]*len(bboxes)) #Albumentations can transform images and boxes\n"," image = transformed[\"image\"]\n"," bboxes = transformed[\"bboxes\"]\n","\n"," if len(bboxes) > 0:\n"," bboxes = torch.stack([torch.tensor(item) for item in bboxes])\n"," else:\n"," bboxes = torch.zeros((0,4))\n"," return image, bboxes, domain\n","\n","\n"," def decodeString(self,BoxesString):\n"," \"\"\"\n"," Small method to decode the BoxesString\n"," \"\"\"\n"," if BoxesString == \"no_box\":\n"," return np.zeros((0,4))\n"," else:\n"," try:\n"," boxes = np.array([np.array([int(i) for i in box.split(\" \")])\n"," for box in BoxesString.split(\";\")])\n"," return boxes\n"," except:\n"," print(BoxesString)\n"," print(\"Submission is not well formatted. empty boxes will be returned\")\n"," return np.zeros((0,4))"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"P_Y4m7NL_ns2"},"source":["transform = A.Compose([\n"," A.LongestMaxSize(1024,p=1),\n"," A.PadIfNeeded(min_height=1024,min_width=1024,p=1,border_mode=1,value=0),\n"," A.Normalize(\n"," mean=[0.485, 0.456, 0.406],\n"," std=[0.229, 0.224, 0.225],\n"," ),\n"," ToTensorV2(),\n","\n","],bbox_params=A.BboxParams(format='pascal_voc',label_fields=['class_labels'],min_area=20))"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"RLTOa7mCUVtI"},"source":["dataset = WheatDataset(\"train.csv\",\"train\",transform=None)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"61JSx4gKXoD5"},"source":["# Sanity check of the Dataset\n","\n","> Here we randomly load some images to check our dataloard"]},{"cell_type":"code","metadata":{"id":"-ZlcfmOoUoqd"},"source":["\n","hlines = []\n","for i in range(10):\n"," vlines = []\n"," for j in range(5):\n"," img , bboxes , metadata = dataset[random.randint(0,len(dataset))]\n","\n"," for (x,y,xx,yy) in bboxes:\n"," cv2.rectangle(img,(int(x.item()),int(y.item())),(int(xx.item()),int(yy.item())),(255,255,0),5)\n","\n"," vlines.append(img)\n"," hlines.append(cv2.vconcat(vlines))\n","\n","final_img = cv2.hconcat(hlines)\n","\n","fig ,ax = plt.subplots(1,1,figsize=(20,20))\n","plt.imshow(final_img)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"xn02UXE7A7f_"},"source":["## Set up the Dataloader\n","\n","The Dataloader is the utility that will load each images and form batch"]},{"cell_type":"code","metadata":{"id":"g8Uc9r4C6jc_"},"source":["def collate_fn(batch):\n"," \"\"\"\n"," Since each image may have a different number of objects, we need a collate function (to be passed to the DataLoader).\n","\n"," :param batch: an iterable of N sets from __getitem__()\n"," :return: a tensor of images, lists of varying-size tensors of bounding boxes, labels, and difficulties\n"," \"\"\"\n","\n"," images = list()\n"," targets=list()\n"," metadatas = list()\n","\n"," for i, t, m in batch:\n"," images.append(i)\n"," targets.append(t)\n"," metadatas.append(m)\n"," images = torch.stack(images, dim=0)\n","\n"," return images, targets,metadatas\n","\n","\n","dataset = WheatDataset(\"train.csv\",\"train\",transform=transform)\n","train_size = int(len(dataset)*0.9)\n","val_size = len(dataset)-train_size\n","train_set, val_set = torch.utils.data.random_split(dataset, [train_size, val_size]) # We sample 10% of the images as a validation dataset\n","train_dataloader = torch.utils.data.DataLoader(train_set,\n"," batch_size=4,\n"," shuffle=False,\n"," collate_fn=collate_fn)\n","val_dataloader = torch.utils.data.DataLoader(val_set,\n"," batch_size=4,\n"," shuffle=False,\n"," collate_fn=collate_fn)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"WQoFUNamZKs0"},"source":["batch = next(iter(train_dataloader)) # We test is the dataloader is working"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Yp_wQlBABIn9"},"source":["## Train Faster-RCNN with Pytorch_lightning and torchvision\n","\n","> We propose to finetune Faster-RCNN with a ResNet 50 FPN from torchvision thanks to Pytorch Lightning\n"]},{"cell_type":"code","metadata":{"id":"O8I5q1wfAzEQ"},"source":["seed_everything(25081992)\n","\n","class FasterRCNN(LightningModule):\n"," def __init__(self,n_classes):\n"," super().__init__()\n"," self.detector = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\n"," in_features = self.detector.roi_heads.box_predictor.cls_score.in_features\n"," self.detector.roi_heads.box_predictor = FastRCNNPredictor(in_features, n_classes)\n"," self.lr = 1e-4\n","\n"," def forward(self, imgs,targets=None):\n"," # Torchvision FasterRCNN returns the loss during training \n"," # and the boxes during eval\n"," self.detector.eval()\n"," return self.detector(imgs)\n","\n"," def configure_optimizers(self):\n"," optimizer = torch.optim.Adam(self.parameters(), lr=self.lr)\n"," return optimizer\n","\n"," def training_step(self, batch, batch_idx):\n","\n"," imgs = batch[0]\n","\n"," targets = []\n"," for boxes in batch[1]:\n"," target= {}\n"," target[\"boxes\"] = boxes.cuda()\n"," target[\"labels\"] = torch.ones(len(target[\"boxes\"])).long().cuda()\n"," targets.append(target)\n","\n"," # fasterrcnn takes both images and targets for training, returns\n"," loss_dict = self.detector(imgs, targets)\n"," loss = sum(loss for loss in loss_dict.values())\n"," return {\"loss\": loss, \"log\": loss_dict}\n","\n"," def validation_step(self, batch, batch_idx):\n"," img, boxes, metadata = batch\n"," pred_boxes =self.forward(img)\n","\n"," self.val_loss = torch.mean(torch.stack([self.accuracy(b,pb[\"boxes\"],iou_threshold=0.5) for b,pb in zip(boxes,pred_boxes)]))\n"," return self.val_loss\n","\n"," def test_step(self, batch, batch_idx):\n"," img, boxes, metadata = batch\n"," pred_boxes = self.forward(img) # in validation, faster rcnn return the boxes\n"," self.test_loss = torch.mean(torch.stack([self.accuracy(b,pb[\"boxes\"],iou_threshold=0.5) for b,pb in zip(boxes,pred_boxes)]))\n"," return self.test_loss\n","\n"," def accuracy(self, src_boxes,pred_boxes , iou_threshold = 1.):\n"," \"\"\"\n"," The accuracy method is not the one used in the evaluator but very similar\n"," \"\"\"\n"," total_gt = len(src_boxes)\n"," total_pred = len(pred_boxes)\n"," if total_gt > 0 and total_pred > 0:\n","\n","\n"," # Define the matcher and distance matrix based on iou\n"," matcher = Matcher(iou_threshold,iou_threshold,allow_low_quality_matches=False) \n"," match_quality_matrix = box_iou(src_boxes,pred_boxes)\n","\n"," results = matcher(match_quality_matrix)\n"," \n"," true_positive = torch.count_nonzero(results.unique() != -1)\n"," matched_elements = results[results > -1]\n"," \n"," #in Matcher, a pred element can be matched only twice \n"," false_positive = torch.count_nonzero(results == -1) + ( len(matched_elements) - len(matched_elements.unique()))\n"," false_negative = total_gt - true_positive\n","\n"," \n"," return true_positive / ( true_positive + false_positive + false_negative )\n","\n"," elif total_gt == 0:\n"," if total_pred > 0:\n"," return torch.tensor(0.).cuda()\n"," else:\n"," return torch.tensor(1.).cuda()\n"," elif total_gt > 0 and total_pred == 0:\n"," return torch.tensor(0.).cuda()\n"," \n","\n","\n","\n","\n","\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"pvN32Ma07Zt_"},"source":["detector = FasterRCNN(2)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"AjOAAu-qZLe7"},"source":["# Training time ! "]},{"cell_type":"code","metadata":{"id":"LoZJ_xWjkDTu"},"source":["from pytorch_lightning.callbacks.early_stopping import EarlyStopping\n","early_stop_callback = EarlyStopping(\n"," monitor='val_accuracy',\n"," min_delta=0.00,\n"," patience=3,\n"," verbose=False,\n"," mode='max'\n",")\n","\n","\n","# run learning rate finder, results override hparams.learning_rate\n","trainer = Trainer( gpus=1, progress_bar_refresh_rate=1, max_epochs=1,deterministic=False)\n","\n","# call tune to find the lr\n","# trainer.tune(classifier,train_dataloader,val_dataloader) # we already did it once = 1e-4\n","trainer.fit(detector,train_dataloader,val_dataloader)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"mP_WpTMvA0PC"},"source":["# You can save your model for ensembling with : torch.save(detector,\"path/to/model.pth\")"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"pa8Qq5SqEbEi"},"source":["## Sanity check before submission\n","\n","> We need another Dataset for prediction, that does not transform data nor retrieve labels"]},{"cell_type":"code","metadata":{"id":"BmiF0kR3w_3c"},"source":["class WheatDatasetPredict(Dataset):\n"," \"\"\"A dataset example for GWC 2021 competition.\"\"\"\n","\n"," def __init__(self, csv_file,root_dir):\n"," \"\"\"\n"," Args:\n"," csv_file (string): Path to the csv file with annotations.\n"," root_dir (string): Directory with all the images.\n"," transform (callable, optional): Optional transform to be applied\n"," on a sample.\n"," \"\"\"\n","\n"," self.root_dir = Path(root_dir)\n"," annotations = pd.read_csv(csv_file)\n","\n"," self.image_list = annotations[\"image_name\"].values\n"," self.domain_list = annotations[\"domain\"].values\n"," \n"," self.transform = A.Compose([\n"," A.Normalize(\n"," mean=[0.485, 0.456, 0.406],\n"," std=[0.229, 0.224, 0.225],\n"," ),\n"," ToTensorV2(),\n"," ])\n","\n"," def __len__(self):\n"," return len(self.image_list)\n","\n"," def __getitem__(self, idx):\n"," \n"," imgp = str(self.root_dir / (self.image_list[idx]+\".png\"))\n"," domain = self.domain_list[idx]\n"," img = cv2.imread(imgp)\n"," img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n","\n"," if self.transform:\n"," transformed = self.transform(image=img)\n"," image = transformed[\"image\"]\n"," return image, img, self.image_list[idx],domain\n","\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"LG_5N4WCA02U"},"source":["\n","detector.freeze()\n","test_dataset = WheatDatasetPredict(\"submission.csv\",\"test\") #the domain information is included in the submission file\n","\n","\n","hlines = []\n","for i in range(4):\n"," vlines = []\n"," for j in range(4):\n"," idx = random.randint(0, len(test_dataset))\n","\n"," norm_img, img , img_name, domain = test_dataset[idx] #norm_image is used for prediction and img for visualisation\n","\n","\n"," predictions = detector(norm_img.unsqueeze(dim=0))\n","\n"," pboxes = predictions[0][\"boxes\"]\n"," scores = predictions[0][\"scores\"]\n"," pboxes = pboxes[scores > 0.5]\n","\n","\n"," for (x,y,xx,yy) in pboxes:\n"," cv2.rectangle(img,(int(x.item()),int(y.item())),(int(xx.item()),int(yy.item())),(0,255,255),5)\n","\n"," vlines.append(img)\n"," hlines.append(cv2.vconcat(vlines))\n","\n","final_img = cv2.hconcat(hlines)\n","\n","fig ,ax = plt.subplots(1,1,figsize=(20,20))\n","plt.imshow(final_img)\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"-HtZX0YmEodV"},"source":["## Writing submission file"]},{"cell_type":"code","metadata":{"id":"MN_Q9-wMEiH7"},"source":["from tqdm import tqdm_notebook as tqdm\n","detector.detector.eval()\n","test_dataloader = torch.utils.data.DataLoader(test_dataset,\n"," batch_size=1,\n"," shuffle=False)\n","\n","\n","def encode_boxes(boxes):\n","\n"," if len(boxes) >0:\n"," boxes = [\" \".join([str(int(i)) for i in item]) for item in boxes]\n"," BoxesString = \";\".join(boxes)\n"," else:\n"," BoxesString = \"no_box\"\n"," return BoxesString\n","\n","results = []\n","for batch in tqdm(test_dataloader):\n"," norm_img, img , img_names , metadata = batch\n","\n"," predictions = detector.detector(norm_img)\n","\n"," for img_name, pred, domain in zip(img_names,predictions,metadata):\n"," boxes = pred[\"boxes\"]\n"," scores = pred[\"scores\"]\n"," boxes = boxes[scores > 0.5].cpu().numpy()\n"," PredString = encode_boxes(boxes)\n"," results.append([img_name,PredString,domain.item()])"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"hz2RxqXDZvxf"},"source":["results = pd.DataFrame(results,columns =[\"image_name\",\"PredString\",\"domain\"])\n","results.to_csv(\"submission_final.csv\")\n"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"4u5YCoCCKzqX"},"source":["results"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5IX79Ffa_5Te"},"source":["## Making Direct Submission thought Aicrowd CLI"]},{"cell_type":"code","metadata":{"id":"2Ewdrf4wYjQO"},"source":["!aicrowd submission create -c global-wheat-challenge-2021 -f submission_final.csv"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"yI4IjbL8IIxE"},"source":[""],"execution_count":null,"outputs":[]}]} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment