Gpytorch: RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float?

Hello, all. This is my first time within the PyTorch ecosystem (having switched over because of GpyTorch’s seemingly excellent GP modelling capabilities), and I would appreciate help getting through my first model.

Following the first example in the docs, I want to run a very simple model, but I encounter an error at the loss = -mll(output, y_torch) step:

## Generating data for regression
# First, regular sine wave + normal noise
x = np.linspace(0,40, num=300)
noise1 = np.random.normal(0,0.3,300)
y = np.sin(x) + noise1

# Second, an upward trending starting halfway, with its own normal noise
temp = x[150:]
noise2 = 0.004*temp**2 + np.random.normal(0,0.1,150)
y[150:] = y[150:] + noise2

x_torch = torch.from_numpy(x)
y_torch = torch.from_numpy(y)

# initialize likelihood and model
class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel())
    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(x_torch, y_torch, likelihood)

# Find optimal model hyperparameters

# Use the adam optimizer
optimizer = torch.optim.Adam([
    {'params': model.parameters()},  # Includes GaussianLikelihood parameters
], lr=0.1)

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

training_iter = 50
for i in range(training_iter):
    # Zero gradients from previous iteration
    # Output from model
    output = model(x_torch)
    # Calc loss and backprop gradients
    loss = -mll(output, y_torch)
    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (
        i + 1, training_iter, loss.item(),

At that line, I get a RuntimeError: expected backend CPU and dtype Double but got backend CPU and dtype Float error. Having looked around, this seems like a common problem, but always has to do with CPU/CUDA errors, with the answers being vague, and I just want to run my model first on my CPU.

I would appreciate any help on the matter.

maybe you can use model.cpu():

Find optimal model hyperparameters


Not sure why this happens, but the solution for me was to call .double() on the pytorch Tensors:

x_torch = torch.from_numpy(x).double()
y_torch = torch.from_numpy(y).double()

and also on the model, before calling train: