Advice for machine learning task with chemical data

1

I am relatively new to CNNs, and I’m working on a machine learning classification problem with chemical data. I’m looking for advice on 1) how to structure the input data, and 2) architecture for the neural network classifier.

For this task I have raman spectra to use as input. I’ve already removed the baseline and normalized the spectra to lie between 0 and 1. Next I want to input them into the model. I have been operating under the assumption that the data should be input into the model in the shape [batch, 1, length_of_spectra], that is if each spectrum is length 1000, and I have a batch size of 32, then the input data shape would be [32, 1, 1000]. Is this the correct shape? I’m a little confused about the difference between the number of (input) channels and the signal length. Should I consider each wavenumber to be a channel or is the entire spectrum a single channel? Effectively I want the neural net operators to operate on each spectrum independently of the other spectra in a particular batch. That is, if I do a 1-D convolution layer, I want the kernel to operate only on a single spectrum at a time (not convolve multiple spectra together).

I’ve so far tried two different structures, all relatively simple and taken from literature on using CNNs with raman spectroscopy data. Structure 1 (taken from Deep learning-based component identification for the Raman spectra of mixtures - Analyst (RSC Publishing)):

layer1 = Dropout(p=0.5)(
   Maxpool1d(kernel_size=2, stride=2, padding=1)(
      leaky_relu(
         Conv1d(in_channels=1, out_channels=32, kernel_size=5, stride=2, padding=2)
      )
   )
)

layer2 = Dropout(p=0.5)(
   Maxpool1d(kernel_size=2, stride=2, padding=1)(
      leaky_relu(
         Conv1d(in_channels=32, out_channels=64, kernel_size=5, stride=2, padding=2)
      )
   )
)

layer3 = Linear(8128, 1024)(
   torch.flatten(start_dim=1)
)

layer4 = Linear(1024, 1)

When I’m training this, I do get nan loss after some time, and I’m not entirely sure why. If you have any suggestions on that, I’d very much appreciate it. Structure 2 (taken from Classifying breast cancer tissue by Raman spectroscopy with one-dimensional convolutional neural network - ScienceDirect):

layer1 =  maxpool1d(kernel_size=2, stride=2)(
   leaky_relu(
      batchnorm1d(num_features=5)(
         conv1d(in_channels=1, out_channels=5, kernel_size=10, stride=2)
      )
   )
)

layer2 = dropout(p=0.5)(
   leaky_relu(
      batchnorm1d(num_features=5)(
         Linear(in_features=2125, out_features=5)(torch.flatten(start_dim=1))
      )
   )
)

layer3 = Linear(in_features=5, out_features=1)

This model also reached nan loss after some time, and I’m not sure why. Anyhow, this is what I’ve done so far, and I appreciate any advice on either of these models, but especially on the data input format/shape. Thanks!

2

[batch, 1, length_of_spectra] is the correct shape for 1D convolution. Normalization and regularization is the proper way too.
Regarding your loss becoming NaN after a while, try applying this function to your model before you update weights

def sanitize_grads(module):
	for param in module.parameters():
		if param.grad is not None:
			torch.nan_to_num(param.grad, nan=0, posinf=1e5, neginf=-1e5, out=param.grad)
1 Like