Feed data which has multiple patterns into regression model?

1

I try to feed data into one deep regression model, but always can’t catch patterns.

When I use random forest, it works find and be able to catch patterns
But it failed when I want to use deep networks.

What I thought is using relu and linear to catch patterns, is there anything wrong?

import torch
import torch.nn.functional as FUNC
class Reg(torch.nn.Module):
	def __init__(self, n_feature, n_hidden, n_output):
		super(Reg, self).__init__()
		self.name = self.__class__.__name__
		self.hidden = torch.nn.Linear(n_feature, n_hidden) # hidden layer
		self.F = FUNC.relu
		self.out = torch.nn.Linear(n_hidden, n_output)

	def forward(self, features):
		x = self.hidden(x)
		x = self.F(x)
		x = self.out(x)
		return x

optimizer=ADAM, criterion=torch.nn.MSELoss()
n_feature=2, n_hidden=256, n_output=1.

2

Did you normalize your data? Normalization helps especially in training neural networks. If I’m not mistaken, RandomForest classifiers are not really sensitive to different value ranges.

Also, try to overfit a small data sample, e.g. just 10 samples. Once your model predicts these samples (almost) perfectly, try to scale out the training.

3

@ptrblck Yes, I normalize all data to 0~1.
Or maybe I need to adjust hidden layer? e.g. add more relu or dense layers?