Different accuracies for the same code

vishak_bharadwaj · December 16, 2019, 4:59am

So I was going through this tutorial on Pytorch that builds a resnet for the CIFAR10 dataset.

The code is here

Paperspace/PyTorch-101-Tutorial-Series/blob/master/PyTorch 101 Part 2 - Building Neural Networks With PyTorch.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Building Neural Networks With PyTorch.ipynb","version":"0.3.2","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"Nc8sigOEA3Xf","colab_type":"text"},"source":["# PyTorch 101: Building Neural Networks \n","\n","In this tutorial, we are going to build a Resnet based Neural network to classify the CIFAR 10 Dataset. Before, we begin, let me say that the purpose of this tutorial is not to achieve the best possible accuracy on the task, but to show you how to use PyTorch."]},{"cell_type":"code","metadata":{"id":"3NmJoq9YA3Xi","colab_type":"code","colab":{}},"source":["import torch\n","import torch.nn as nn\n","import torch.utils.data\n","import torch.optim as optim\n","import numpy as np\n","import pickle\n","import os\n","from PIL import Image\n","import random\n","import time\n","import torchvision\n","\n","cuda_available = torch.cuda.is_available()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"aXBsHO_IA3X5","colab_type":"text"},"source":["### Building the Network\n","While PyTorch provided many layers out of the box with it's torch.nn module, we will have to implement the residual block ourselves. We first begin by defining the the resnet block.  \n","\n","\n","## A Simple Neural Network\n","In this tutorial, we will be implementing a very simple neural network.   \n","\n","### Building the Network\n","The `torch.nn.Module` is the cornerstone of designing neural networks in PyTorch. This class can be used to implement a layer like a fully connected layer, a convolutional layer, a pooling layer, an activation function, and also an entire neural network by instantiating a `torch.nn.Module` object. (From now on, I'll refer to it as merely nn.module) \n","\n","Multiple `nn.Module` objects can be strung together to form a bigger nn.Module object, which is how we can implement a neural network using many layers. In fact, nn.Module can be used to represent an arbitrary function f in PyTorch.\n","\n","The nn.Module class has two methods that you have to override.\n","1. `__init__` function. This function is invoked when you create an instance of the nn.Module. Here you will define the various parameters of a layer such as filters, kernel size for a convolutional layer, dropout probability for the dropout layer. \n","\n","2. `forward` function. This is where you define how your output is computed. This function doesn't need to be explicitly called, and can be run by just calling the nn.Module instance like a function with the input as it's argument. "]},{"cell_type":"code","metadata":{"id":"Up7wAgO6OGJC","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"d2942f23-a46b-4272-b70c-afa4f434a1c1","executionInfo":{"status":"ok","timestamp":1557348877214,"user_tz":-330,"elapsed":1081,"user":{"displayName":"Ayoosh Kathuria","photoUrl":"https://lh5.googleusercontent.com/-hC2hkjwNr9s/AAAAAAAAAAI/AAAAAAAACpo/DPqp1uUqR4E/s64/photo.jpg","userId":"11533138969683019189"}}},"source":["class MyLayer(nn.Module):\n","  def __init__(self, param):\n","    super().__init__()\n","    self.param = param \n","  \n","  def forward(self, x):\n","    return x * self.param\n","  \n","myLayerObject = MyLayer(5)\n","output = myLayerObject(torch.Tensor([5, 4, 3]) )    #calling forward inexplicitly \n","print(output)"],"execution_count":20,"outputs":[{"output_type":"stream","text":["tensor([25., 20., 15.])\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"1AKR9RyZOnBi","colab_type":"text"},"source":["Another widely used and important class is the `nn.Sequential` class.  When initiating this class we can pass a list of `nn.Module` objects in a particular sequence. The object returned by `nn.Sequential` is itself a `nn.Module` object. When this object is run with an input, it sequentially runs the input through all the nn.Module object we passed to it, in the very same order as we passed them. "]},{"cell_type":"code","metadata":{"id":"lbYYWtMLOu29","colab_type":"code","colab":{}},"source":["combinedNetwork = nn.Sequential(MyLayer(5), MyLayer(10))\n","\n","output = combinedNetwork([3,4])\n","\n","#equivalent to..\n","# out = MyLayer(5)([3,4])\n","# out = MyLayer(10)(out)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"SGoPAtVeO09a","colab_type":"text"},"source":["Let us now start implementing our classification network. We will make use of convolutional and pooling layers, as well as a custom implemented residual block.\n","\n","While PyTorch provided many layers out of the box with it's `torch.nn` module, we will have to implement the residual block ourselves. Before implementing the neural network, we implement the ResNet Block."]},{"cell_type":"code","metadata":{"id":"XE0FkyqQA3X7","colab_type":"code","colab":{}},"source":["class ResidualBlock(nn.Module):\n",

This file has been truncated. show original

My implementation of that code is here.

github.com

VishakBharadwaj94/Resnets_from_scratch/blob/master/CIFAR_Dataset_with_ResNet_from_Scratch.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "CIFAR_Dataset_with_ResNet_from_Scratch",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",

This file has been truncated. show original

For some reason(I’m too blind to recognize) the accuracy is consistently lower for me.

Can someone have a look and see what’s different?
thanks!