Couldn't quantize the model I used

vishak_bharadwaj · November 16, 2020, 5:35pm

I have a notebook here that has a image classifier skeleton

VishakBharadwaj94/CNN_starter/blob/main/CNN_Boilerplate.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "source": [
    "# CNN outline for DL with CIFAR10\n",
    "\n",
    "![cifar10](https://miro.medium.com/max/709/1*LyV7_xga4jUHdx4_jHk1PQ.png)\n",
    "\n",
    "\n",
    "In this tutorial, we'll use the following techniques to achieve over 90% accuracy in less than 5 minutes:\n",
    "\n",
    "- Data normalization\n",
    "- Data augmentation\n",
    "- Residual connections\n",
    "- Batch normalization\n",

This file has been truncated. show original

I tried to fuse layers and quantize the model as in the video but got an error.

Are there any best practices for writing your model etc that overcome this problem?
Thanks!

vishak_bharadwaj · November 17, 2020, 7:42pm

Hi guys any pointers would be helpful here!

ptrblck · November 21, 2020, 1:41am

Your notebook doesn’t show any error, so could you describe what issues you are seeing and what you’ve tried so far?

vishak_bharadwaj · November 26, 2020, 6:28am

Hi Patrick, the issue arose when I ended up using Torchserve. The API broke, and it worked when I tried a straight MNIST classifier with a clean architecture. What I mean here is that the model didn’t call a small conv function that has a ( conv_bn_relu) combo.

ptrblck · November 26, 2020, 8:16am

How did you check that this modules wasn’t called? Did you get any error message or do you think Torchserve just skipped this module somehow?

vishak_bharadwaj · November 26, 2020, 10:54am

Got an error messgae @ptrblck

Let me organize this into a full question with pics and error messages. It will help people. Please give me an hour.

vishak_bharadwaj · November 26, 2020, 5:03pm

LOL ok I got my question completely wrong. The Torchserve issue was completely different since I’d screwed up the ModelHandler messing up the preprocess method

The quantization is different, but I still haven’t figured it out yet. Here’s the model.
I’m not sure how I can quantize this, compared to this sweet simple tutorial.

github.com

VishakBharadwaj94/quant_wip/blob/master/quant.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "quant.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "Eay3SI8fa6P9"
      },

This file has been truncated. show original