Global structure pruning in pytorch

yichao_Liu · September 23, 2024, 7:20am

Hi, I have a question about structure pruning. I used pytorch structure pruning for global pruning. I could find 0 in weight_mask, before prune.remove(module, ‘weight’)
My code is like this:
`
for name, module in model.model.named_modules():

if isinstance(module, nn.Conv2d):
    if 0 in module.weight_mask:
        print(module.weight)
prune.remove(module, 'weight')

`
print(module.weight) can show 0 weights, but after prune.remove, it still shows that the weight as original. Do you know what problem is there? Thank you so much

Soumya_Kundu · September 23, 2024, 3:10pm

gist.github.com

https://gist.github.com/aymuos15/10c8348148d3de5954a6b4cc5cfebf2c

Pruning_nuances_in_pytorch.py

import torch
import torch.nn as nn
import torch.nn.utils.prune as prune
import numpy as np
import matplotlib.pyplot as plt

class SimpleCNN(nn.Module):
    def __init__(self, conv1_out=32, conv2_out=64):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, conv1_out, 3, 1)

This file has been truncated. show original

This gist should answer this question!

yichao_Liu · September 24, 2024, 12:19pm

Thank you. yes it doesn’t remove the 0 parameters, so I think in order to do it, a new model have to be created, but in yolov8n model, there’s no architecture code.

Soumya_Kundu · September 24, 2024, 2:09pm

github.com

ultralytics/ultralytics/blob/main/ultralytics/models/yolo/model.py

# Ultralytics YOLO 🚀, AGPL-3.0 license

from pathlib import Path

from ultralytics.engine.model import Model
from ultralytics.models import yolo
from ultralytics.nn.tasks import ClassificationModel, DetectionModel, OBBModel, PoseModel, SegmentationModel, WorldModel
from ultralytics.utils import ROOT, yaml_load


class YOLO(Model):
    """YOLO (You Only Look Once) object detection model."""

    def __init__(self, model="yolov8n.pt", task=None, verbose=False):
        """Initialize YOLO model, switching to YOLOWorld if model filename contains '-world'."""
        path = Path(model)
        if "-world" in path.stem and path.suffix in {".pt", ".yaml", ".yml"}:  # if YOLOWorld PyTorch model
            new_instance = YOLOWorld(path, verbose=verbose)
            self.__class__ = type(new_instance)
            self.__dict__ = new_instance.__dict__

This file has been truncated. show original

Is this not what you’re looking for?

yichao_Liu · September 28, 2024, 8:13am

Thank you so much. I think the whole structure is written by yaml file. What I want is a python code architecture. This allows me to do some changes like pruning, but yaml file doesn’t allow me to do changes.

Soumya_Kundu · September 30, 2024, 3:26pm

Sorry, should’ve made myself clear (and the late reply!) → You should look into the trainer.py

This is what is inherited overall.