How to use pytorch to output the probability of binary classfication?

The data in my train.csv likes this:

[
[0, 0,……, 0],
[0, 1,……, 0],
[0, 2,……, 0],
[0, 3,……, 0],
[1, 0,……, 0],
[2, 0,……, 0],
[3, 0,……, 0],
[1, 1,……, 1],
[2, 1,……, 1],
[1, 2,……, 1],
[3, 1,……, 1],
]

The training set has 800,000 rows data and the test set has 20,000 rows data.
The value of y_train is only 0 and 1, and there are more than 150 columns of data in csv.

I want to use pytorch to predict the probability of 0 and 1 with x_test, how to do it?

Here is my code:

import pandas as pd

train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]

I want 2 columns of probability, the first column is the probability of 0 , and the second column is the probability of 1 , such as this:

[[0.23334,0.76267]
 ……
 [0.84984,0.15685]
 [0.16663,0.83291]]

When I use keras,I can do it like this:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from sklearn.preprocessing import LabelEncoder
from keras.utils import np_utils

data_train = np.array([
[0, 0, 0],
[0, 1, 0],
[0, 2, 0],
[0, 3, 0],
[1, 0, 0],
[2, 0, 0],
[3, 0, 0],
[1, 1, 1],
[2, 1, 1],
[1, 2, 1],
[3, 1, 1],
])

data_test = np.array([
[1, 3],
[0, 4],
[5, 0]
])

x_train = data_train[:, :-1]
y_train = data_train[:, -1]
x_test = data_test
encoder = LabelEncoder()
encoder.fit(y_train)
encoded_y = encoder.transform(y_train)
y_train = np_utils.to_categorical(encoded_y)

model = Sequential()
model.add(Dense(512, activation='relu', input_dim=2))
model.add(Dense(200, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['binary_accuracy'])

model.fit(x_train, y_train, epochs=5, batch_size=1, verbose=1)
predict = model.predict_proba(x_test, batch_size=1)
print(predict)

result:

[[0.46373594 0.53626406]
 [0.99079037 0.00920963]
 [0.72976    0.27024   ]]

But how to do it with pytorch?

You could create a model with two output neurons (e.g. via nn.Linear) and setup a multi-label classification use case using nn.BCEWithLogitsLoss.
Since the model output would be logits, you could apply torch.sigmoid on them to get the probabilities for each class.
Here is a very simple example:

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(128, 64)
        self.fc2 = nn.Linear(64, 40)
        
    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()

data = torch.randn(64, 128)
target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(100):
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output, target)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(data)
probs = torch.sigmoid(output)
print(probs)
print(target)
1 Like

@ptrblck
Hi,thanks!
I use this code:

import torch
import torch.nn as nn
import torch.nn.functional as F


train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(800000, 153)
        self.fc2 = nn.Linear(153, 2)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
x_train = torch.Tensor(x_train.values)
y_train = torch.Tensor(y_train.values)
x_test = torch.Tensor(x_test.values)
# data = torch.randn(64, 128)
# target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(5):
    optimizer.zero_grad()
    output = model(x_train)
    loss = criterion(output, y_train)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(x_test)
probs = torch.sigmoid(output)
print(probs)

But the error messgae is:

RuntimeError:
size mismatch, m1: [800000 x 153], m2: [800000 x 153] at …\aten\src\TH/generic/THTensorMath.cpp:41

How to set these 2 lines?

self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)

It seems the x_train tensor has the shape [153, 800000] and should be permuted before passing it to the model via x_train = x_train.permute(1, 0).

@ptrblck
Hi,when I do it as you said:

import torch
import torch.nn as nn
import torch.nn.functional as F


train = pd.read_csv(r'train.csv')
x_test = pd.read_csv(r'test.csv')

x_train = train[:, :-1]
y_train = train[:, -1]

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(800000, 153)
        self.fc2 = nn.Linear(153, 2)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
x_train = torch.Tensor(x_train.values)
y_train = torch.Tensor(y_train.values)
x_train = x_train.permute(1, 0)
x_test = torch.Tensor(x_test.values)
# data = torch.randn(64, 128)
# target = torch.randint(0, 2, (64, 40)).float()

for epoch in range(5):
    optimizer.zero_grad()
    output = model(x_train)
    loss = criterion(output, y_train)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

# check predictions
output = model(x_test)
probs = torch.sigmoid(output)
print(probs)

The error message is:

ValueError:
Target size (torch.Size([800000])) must be the same as input size (torch.Size([153, 2]))
So how should I set this 2 lines?

self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)

Ok, I might be wrong with the permute suggestion.
Could you remove the permute operation again and post the shape of x_train before passing it to the model?

Hi,my training set is 800,000 rows and 153 columns. The data I want to predict is 200,000 rows and 153 columns. How should I write these 2 lines?

self.fc1 = nn.Linear(800000, 153)
self.fc2 = nn.Linear(153, 2)

If I use this code,the error message is:

RuntimeError:
size mismatch, m1: [800000 x 153], m2: [800000 x 153] at …\aten\src\TH/generic/THTensorMath.cpp:41

800,000 would then be the batch size (or the total number of samples) while 153 would be the feature dimension.
In that case use

self.fc1 = nn.Linear(153, any_value)
self.fc2 = nn.Linear(any_value, 2)

The batch size is not part of the layer definition, as all PyTorch layers accept a variable batch size.
Your target should have the same batch size so you can calculate the loss using the model output.