Gpytorch: RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float?

1

Hello, all. This is my first time within the PyTorch ecosystem (having switched over because of GpyTorch’s seemingly excellent GP modelling capabilities), and I would appreciate help getting through my first model.

Following the first example in the docs, I want to run a very simple model, but I encounter an error at the loss = -mll(output, y_torch) step:

## Generating data for regression
# First, regular sine wave + normal noise
x = np.linspace(0,40, num=300)
noise1 = np.random.normal(0,0.3,300)
y = np.sin(x) + noise1

# Second, an upward trending starting halfway, with its own normal noise
temp = x[150:]
noise2 = 0.004*temp**2 + np.random.normal(0,0.1,150)
y[150:] = y[150:] + noise2

x_torch = torch.from_numpy(x)
y_torch = torch.from_numpy(y)

# initialize likelihood and model
class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())
        
    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)


likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(x_torch, y_torch, likelihood)

# Find optimal model hyperparameters
model.train()
likelihood.train()

# Use the adam optimizer
optimizer = torch.optim.Adam([
    {'params': model.parameters()},  # Includes GaussianLikelihood parameters
], lr=0.1)

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

training_iter = 50
for i in range(training_iter):
    
    # Zero gradients from previous iteration
    optimizer.zero_grad()
    
    # Output from model
    output = model(x_torch)
    
    # Calc loss and backprop gradients
    loss = -mll(output, y_torch)
    loss.backward()
    
    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (
        i + 1, training_iter, loss.item(),
        model.covar_module.base_kernel.lengthscale.item(),
        model.likelihood.noise.item()
    ))
    
    optimizer.step()

At that line, I get a RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float error. Having looked around, this seems like a common problem, but always has to do with CPU/CUDA errors, with the answers being vague, and I just want to run my model first on my CPU.

I would appreciate any help on the matter.

2

maybe you can use model.cpu():
mdel.cpu()
likelihood.cpu()

Find optimal model hyperparameters

model.train()
likelihood.train()

3

Not sure why this happens, but the solution for me was to call .double() on the pytorch Tensors:

x_torch = torch.from_numpy(x).double()
y_torch = torch.from_numpy(y).double()

and also on the model, before calling train:

model.double()
model.train()
3 Likes