# What is the best way to perform hyper parameter search in PyTorch?

**URL:** <https://discuss.pytorch.org/t/what-is-the-best-way-to-perform-hyper-parameter-search-in-pytorch/19943>\
**Category:** vision\
**Created:** [June 19, 2018, 7:27pm UTC](https://discuss.pytorch.org/t/what-is-the-best-way-to-perform-hyper-parameter-search-in-pytorch/19943 "2018-06-19T19:27:52Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![dedeco](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dedeco/32/7564_2.png) [@dedeco](https://discuss.pytorch.org/u/dedeco)\
**Post date:** [February 23, 2019, 2:58pm UTC](https://discuss.pytorch.org/t/what-is-the-best-way-to-perform-hyper-parameter-search-in-pytorch/19943/3 "2019-02-23T14:58:39Z")

</div>

An example:

```auto
class Net(torch.nn.Module):
    def __init__ (self):
        '''
        A feedForward neural network.
        Argurmets:
            n_feature: How many of features in your data
            n_hidden: How many of neurons in the hidden layer
            n_output: How many of neuros in the output leyar (defaut=1)
        '''
        super(Net, self). __init__ ()
        self.hidden = torch.nn.Linear(D_in, H, bias=True) # hidden layer
        self.predict = torch.nn.Linear(H, D_out, bias=True) # output layer
        self.n_feature, self.n_hidden, self.n_output = D_in, H, D_out
    def forward(self, x,**kwargs):
        '''
        Argurmets:
            x: Features to predict
        '''
        torch.nn.init.constant_(self.hidden.bias.data,1)
        torch.nn.init.constant_(self.predict.bias.data,1)
        x = torch.sigmoid(self.hidden(x)) # activation function for hidden layer
        x = torch.sigmoid(self.predict(x)) # linear output
        return x

```

```auto
from skorch import NeuralNetRegressor
net = NeuralNetRegressor(Net
                         , max_epochs=100
                         , lr=0.001
                         , verbose=1)

```

```auto
X_trf = X
y_trf = y.reshape(-1, 1)
print(X_trf.shape,y_trf.shape)

```

```auto
from sklearn.model_selection import GridSearchCV

params = {
    'lr': [0.001,0.005, 0.01, 0.05, 0.1, 0.2, 0.3],
    'max_epochs': list(range(500,5500, 500))
}

gs = GridSearchCV(net, params, refit=False, scoring='r2', verbose=1, cv=10)

gs.fit(X_trf, y_trf)

```

---

_[View the full topic](https://discuss.pytorch.org/t/what-is-the-best-way-to-perform-hyper-parameter-search-in-pytorch/19943)._