Use pytorch cuda for ridge regression

1

Hi, I have big datasets and can anybody help with using pytorch cuda for ridge regression?

2

Could you describe what you’ve tried so far and where you are stuck at the moment?

3

ok. I used sklearn for ridge regression. For the datasets, the activation features are from various DNN layers, thus it takes a lot of time. I would like to map activation features to fMRI data, from example, activation feature (of a specific layer) of size 10000 * 7896, fMRI feature of size 10000 * 2345. I have 10000 samples here. When I use sklearn it takes a lot of time. Thus I wonder whether I can use torch.nn.linear to realize that, using cuda.

4

for example, can we convert this into pytorch code:

Keras approach

import keras
from keras.layers import Input, Dense
from keras.models import Model

#Define the model
def regressor_model(m_features, regularizer=None, learning_rate=0.01):

input_x = Input(shape = (m_features,))
lin_fn = Dense(1, activation = None, kernel_regularizer = regularizer)(input_x)
yx_model = Model(inputs = input_x, outputs = lin_fn)
sgd = keras.optimizers.SGD(lr=learning_rate)
yx_model.compile(loss = 'mean_squared_error', optimizer = sgd)

return yx_model

reg_par = [0, 0.001, 0.01, 0.1, 1]

for i_reg in reg_par:
start = time.time()

# Training
yx_model = regressor_model(m_features,learning_rate=0.1,regularizer=keras.regularizers.l2(i_reg))
log_train = yx_model.fit(X, y, epochs = 50, batch_size = N, verbose = 0)

weights = yx_model.get_weights()

b = weights[0]

print("%.2f sec."%(time.time() - start), end=' - ')
print("Coefficients for x_1, x_2, x_3 are %.3f, %.3f, %.3f, respectively" %
      (b[0], b[1], b[2]))

# Testing
y_hat = yx_model.predict(X)
print("R square: %.3f"%r2_score(y,y_hat))           # R^2 (coefficient of determination) regression score function.

In this case, no regularization is needed!

5

Yes, you can port the Keras model to PyTorch. E.g. Keras’ Dense layers would be replaced with PyTorch’s nn.Linear layers etc. If this is your first time creating a PyTorch model, this tutorial might be helpful.