How improve accuracy on this CNN image classifier?

I implemented the CNN here:

It creates a classifier for CIFAR-10 dataset.

The accuracy on training data is about 87%.
The accuracy on testing data is about 68%.

What can I do to try to increase the accuracy? Should I focus on
making the CNN more complex?

Here is the current CNN:

N_CLASSES  = 10
LEARN_RATE = 0.001
MOMENTUM   = 0.9
N_EPOCHS   = 20
BATCH_SIZE = 64
ERROR      = torch.nn.CrossEntropyLoss()

class CNN(torch.nn.Module):
        """
        Defines a convolutional neural network.
        """

        def __init__(self, N_CLASSES):
                super(CNN, self).__init__()
                self.conv_1 = torch.nn.Conv2d( 3, 32, 3)
                self.conv_2 = torch.nn.Conv2d(32, 32, 3)
                self.pool_1 = torch.nn.MaxPool2d(2, stride = 2)
                self.conv_3 = torch.nn.Conv2d(32, 64, 3)
                self.conv_4 = torch.nn.Conv2d(64, 64, 3)
                self.pool_2 = torch.nn.MaxPool2d(2, stride = 2)
                self.line_1 = torch.nn.Linear(1600, 128)
                self.relu_1 = torch.nn.ReLU()
                self.line_2 = torch.nn.Linear(128, N_CLASSES)

        def forward(self, inputs):
                results = inputs
                for f in [self.conv_1,
                          self.conv_2,
                          self.pool_1,
                          self.conv_3,
                          self.conv_4,
                          self.pool_2]:
                        results = f(results)
                results = results.reshape(results.size(0), -1)
                for f in [self.line_1, self.relu_1, self.line_2]:
                        results = f(results)

                return results