Batch optimization

I have a lot of data. At the entrance of the network, I submit a window the size of batch.
Batch I create in a loop.
I made the code, it works, but not very efficiently.
I have to repeat the same thing in every epoch. Inside the epoch, code also creates batch for a very long time. Are there any ideas how to optimize my code?

for epoch in range(epochs):
    for wn_start in range(0,len_batch-wn1,batch):
        wn_t = wn_t + wn1
        wn_all = []
        los_target = []
        for b_iter in range(batch):
            wn_all = wn_all + [st_s[wn_start+b_iter:wn_t+b_iter,:]]
            los_target = los_target + [st_target[wn_t-1+b_iter]]
        wn_all = torch.as_tensor(wn_all, dtype=torch.float32)
        los_target = torch.Tensor([los_l])

Hi Slavavs!

In short, you build (wrap arrays as) torch tensors inside your
loop. Instead, you should build one large tensor and then index
(slice) into it.

I haven’t tried to figure out your code or reproduce its exact
functionality. But here’s a sample that illustrates how to do
what I think you are trying to do:

import numpy as np
import torch

st_s = np.random.randn (100, 5).astype (np.float32)
st_target = np.random.randint (0, 5, (100,)).astype (np.int64)

st_s_as_tensor = torch.as_tensor (st_s)
st_target_as_tensor = torch.as_tensor (st_target)

type (st_s_as_tensor)
type (st_target_as_tensor)

for  i in range (5):
     # 5 batches of 20 inputs and targets
     st_s_batch = st_s_as_tensor[20 * i : 20 * (i + 1)]
     st_target_batch = st_target_as_tensor[20 * i : 20 * (i + 1)]
     st_s_batch.shape
     st_s_batch.dtype
     st_target_batch.shape
     st_target_batch.dtype

# and that's the tea!

Here is the relevant part of the output I get from running this script:

torch.Size([20, 5])
torch.float32
torch.Size([20])
torch.int64
torch.Size([20, 5])
torch.float32
torch.Size([20])
torch.int64
<etc.>

In my example, I assume that your input is a numpy array of 100
samples of 5 floating-point features each, and that your targets
are 100 integer class labels that match up to your 100 samples.

The script then wraps your numpy arrays (using the same
underlying memory) in torch tensors, and then indexes into
the torch tensors to get 5 batches of 20 samples each.

Best.

K. Frank

1 Like