Pick one random value from tensors of different sizes

1

Hello everyone,

i want to extract one random value from a number of different tensor, how to get the random number out of the tensor should not depend on its size.

Just to get an idea here are some examples of sizes.
So either 4, 2 or 1 indicies

|features.0.0.weight | torch.Size([16, 3, 3, 3])|
|features.0.1.weight | torch.Size([16])|
|features.16.1.weight | torch.Size([960])|
|classifier.0.weight | torch.Size([1280, 960])|
|classifier.3.weight | torch.Size([1000, 1280])|

I started quite generic and easy and thought just pick 4 random values that choose the index location of the value within the tensor:

size_tensor = sd[location].size() #The location basically the name of the weight tensor which I state in
the beginning, where the sd is the state_dict()

a = random.randint(0,size_tensor[0]-1)

b = random.randint(0,size_tensor[1]-1)

c = random.randint(0,size_tensor[2]-1)

d = random.randint(0,size_tensor[3]-1)

random_value_from_tensor = weight_tensor[a][b][c][d]

So in the example of taking features.0.0.weight as the location the size is 16, 3, 3, 3 so the code choses the random value as “weight_tensor[random value between 0 and 15][0 and 2]” and so on.

This works as intendet. But my problem is that I have to do that for every form or size for the different tensors, if I choose one with torch.Size([16])| for example, the code obviously does not work, since there is only one index for the values in the weight tensor, instead of 4.

So the question is, if there is a easier way that does not depent on the torch.size to extract one random value from the tensor? Basically “rand_value(tensor) = one random value from tensor”
And if not, does anyone have a better idea to pick random values and not having to code exceptions and random values for every from of torch.size between 1 to 4 indices?

2

Hello,

Does this function solve your problem?

import random

def get_rand_value(tensor):
    number_of_elements = tensor.numel()
    random_element_number = random.randrange(number_of_elements)
    value_of_random_element = tensor.flatten()[random_element_number]
    return value_of_random_element

Regards,
Thomas

1 Like
3

Yes, that solves my problem, thank you very much!