Hello, I’m trying to train Neural Networks using format datatype Posit8 in Pytorch.
To start, I’ve tried to train LeNet5 with MNIST dataset.
My problem is located when I try to get the train and valid datasets. Firstly, I get them in tensors of float32 datatype with the next code:
transforms = transforms.Compose([transforms.Resize((32, 32)),
transforms.ToTensor()]) #Variable que tendrá las especificaciones de los tensores de datos
# download and create datasets
train_dataset = datasets.MNIST(root='mnist_data',
train=True,
transform=transforms,
download=True) #Descargamos y creamos el dataset de entrenamiento
valid_dataset = datasets.MNIST(root='mnist_data',
train=False,
transform=transforms) #Descargamos y creamos el dataset de validación
# define the data loaders
train_loader = DataLoader(dataset=train_dataset,
batch_size=BATCH_SIZE,
shuffle=True) #Definimos el dataset de entrenamiento
valid_loader = DataLoader(dataset=valid_dataset,
batch_size=BATCH_SIZE,
shuffle=False) #Descargamos y creamos el dataset de validación
After getting the datasets, I use softposit in order to change the datatype. I didn’t find a way to change the entire tensor, because softposit only change the dataype of a number, not a tensor or array. So, to change the datasets I used 5 for loops to get each one of the numbers of the tensor and change the datatype. Here is my code:
def TensorToP8(X_array, Y_array, narray):
c1=0
c2=0
c3=0
c4=0
c5=0
k=0
for i1,y in narray:
for i2 in i1:
for i3 in i2:
for i4 in i3:
for i5 in i4:
X_array[c1][c2][c3][c4][c5] = sp.posit8(np.double(i5))
c5+=1
c4+=1
c5=0
c3+=1
c4=0
c2+=1
c3=0
for y2 in y:
Y_array[c1][k] = y2.numpy()
k+=1
c1+=1
c2=0
k=0
return X_array,Y_array
And this:
X_train= np.ndarray([1875,32,1,32,32])
Y_train = np.ndarray([1875,32])
X_validate = np.ndarray([313,32,1,32,32])
Y_validate = np.ndarray([313,32])
X_train, Y_train = TensorToP8(X_train,Y_train,train_loader)
X_validate, Y_validate = TensorToP8(X_validate,Y_validate, valid_loader)
My problem is the way I cast the dataset, because when I try to use it, the datatype of the arrays is float64, so it means that the np.ndarray I use doesn’t save the posit datatype.
Is there a better way of cast the dataset to Posit8 ?
Thank you