Angular Features

class_features_mean is a dict and contains the mean tensor for each class.
To print the shape of the mean tensor for class0, you could use:

print(class_features_mean[0].shape)
1 Like

Thanks a lot ptrblck

Cheers,
Angelina

Hi ptrblck.
Could you please help to write code about:
.Train class Counts… [5000 3871 2997 2320 1796 1391 1077 834 645 500]
In these I train classes I want to determine if class count is less than 1000 then it is tail class else head class. Then I want to extract features of tail and head classes individually.

You can get the indices for the head and tail classes using:

class_count = torch.tensor([5000, 3871, 2997, 2320, 1796, 1391, 1077, 834, 645, 500])
tail_idx = (class_count < 1000).nonzero()
head_idx = (class_count >= 1000).nonzero()
1 Like

Thank you very much ptrblck for you help.

Hi, ptrblck,
After finding the indices for tail and head classes, how to extract the features of classes corresponding to these indices.
.Tail_indexes…
tensor([[7],
[8],
[9]])
…Head_indexes… tensor([[0],
[1],
[2],
[3],
[4],
[5],
[6]])
Cheers,
Angelina

You can reuse the same code snippet from here and use the tail and head class indices to create different class_features dicts.

1 Like

Hi, ptrblck,
In the following code (provided by you) I want to multiply the Center (Mean) of each class with all its features, however, it is causing an error. Could you please have a look?

import torch
import torch.backends.cudnn as cudnn

nb_classes = 3
batch_size = 10
features = 4
output = torch.randn(batch_size, features)
targets = torch.randint(0, nb_classes, (batch_size,))

print(targets)
#> tensor([0, 0, 0, 2, 0, 1, 1, 2, 2, 2])

class_features = {idx: [] for idx in range(nb_classes)}
for class_index in range(nb_classes):
idx = targets == class_index
class_feat = output[idx]
class_features[class_index].extend(class_feat)

print(‘Class feature Shape :’,len(class_features))
#> {0: [tensor([-0.8338, 0.3141, -0.2840, 0.6104]), tensor([-1.5458, -0.3546, -0.3190, 0.6153]), tensor([-1.3919, -0.4112, 0.4425, 0.8475]), tensor([ 0.3294, -0.2577, 0.3397, 0.4239])], 1: [tensor([-1.4398, 0.2516, 1.6932, -0.0364]), tensor([-0.6315, -0.6568, 0.7358, 0.5755])], 2: [tensor([ 1.3726, -0.7952, 2.1696, 0.6634]), tensor([ 0.4024, -2.2702, 2.2658, 2.6325]), tensor([ 0.2029, -1.4608, -0.2938, 1.0877]), tensor([-1.2883, -0.5849, 0.2535, -0.0638])]}

class_features_mean = {idx: [] for idx in range(nb_classes)}
for class_index in range(nb_classes):
tmp = torch.stack(class_features[class_index])
class_features_mean[class_index] = tmp.mean(dim=0)

print(‘Class feature Mean Shape :’,class_features_mean[1].shape)
#> {0: tensor([-0.8606, -0.1773, 0.0448, 0.6243]), 1: tensor([-1.0357, -0.2026, 1.2145, 0.2696]), 2: tensor([ 0.1724, -1.2778, 1.0988, 1.0799])}

#Upto here working fine

Multiplying Class Features with corresponding class mean i.e Ci * fi,k (Ci is the Center(Mean) of ith class and fi,k is the kth feature sample of ith class)

class_multi = {idx: [] for idx in class_features_mean}
for class_index in class_features:
mul = torch.stack(class_features_mean[idx]*class_features[class_index])
class_multi[class_index] = mul
print (class_multi)

Cheers,
Angelina

Hi ptrblck.

Could u plz guide, how to ensure are target classes are selected in the following statement instead of randomly selecting the targets?
targets = torch.randint(0, nbclasses, (bsz,)).cuda(non_blocking=True)
Cheers,
Angelina

Sorry, I meant, how to ensure all target classes are selected in the following statement instead of randomly selecting the targets?
targets = torch.randint(0, nbclasses, (bsz,)).cuda(non_blocking=True)
Cheers,
Angelina

Do you want to make sure each class is sampled at least once?
If so, you could use torch.arange(nb_classes) and add random classes to fill the batch afterwards.

1 Like

Thanks a lot.

Cheers,
Angelina

Hi ptrblck,
Is this what you meant:

trget=torch.arange(nbclasses)

targets = torch.randint(0, trget, (bsz,)).cuda(non_blocking=True)
It generated the following error:

TypeError: randint() received an invalid combination of arguments - got (int, Tensor, tuple), but expected one of:

  • (int high, tuple of ints size, torch.Generator generator, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool requires_grad)
  • (int high, tuple of ints size, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool requires_grad)
  • (int low, int high, tuple of ints size, torch.Generator generator, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool requires_grad)
  • (int low, int high, tuple of ints size, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool requires_grad)

No, I was thinking about e.g.:

nb_classes = 10
batch_size = 64

target = torch.arange(0, nb_classes)
random_targets = torch.randint(0, nb_classes, (batch_size - nb_classes, ))
target = torch.cat((target, random_targets))

assuming the batch size is larger than the number of classes. Otherwise you won’t be able to sample each class.

1 Like

Thanks a lot ptrblck.

1 Like

Hi, ptrblck,
Could u plz clarify the intuition behind dim in the following statement:

tail_class_angle_mean_var[class_index]= torch.var_mean(angvar,dim=0)
for dim=0, it gnerates output with nan and numbers as shown below:

For dim=1, it generate output without nan i.e.
tail_class_angle_mean_var[class_index]= torch.var_mean(angvar,dim=1)

If we dont specify dim i.e.

tail_class_angle_mean_var[class_index]= torch.var_mean(angvar) output is:

Tail Class Angle Mean Variance 8 :
(tensor(3.4355e-05, device=‘cuda:0’, grad_fn=), tensor(1.5670, device=‘cuda:0’, grad_fn=)) size : 2
Tail Class Angle Mean Variance 9 :
(tensor(3.5429e-05, device=‘cuda:0’, grad_fn=), tensor(1.5671, device=‘cuda:0’, grad_fn=)) size : 2

Could u plz make me clear about this?

The NaNs in the first call look wrong. Could you check, if all values in angvar are valid via torch.isfinite(angvar).all() and if so upload the tensor somewhere, so that we could reproduce and debug this issue?
Are you also using the latest stable release?

1 Like

Hi ptrblck,
I have features of 7 head classes stored at head_idx (0,1…6) and 3 tail classes stored at tail_idx (7,8,9), could u please guide how to combine them in a single variable, so that they can be passed for training.

Cheers,
Angelina

Based on the description I assume they are already in a tensor at the mentioned indices.
If that’s not the case, could you give me a (pseudo) code example what you would like to achieve?

Thanks ptrblck for your reply,
Although they are stored at the indices, however, variables are different i.e. for tail_classes they are stored at tail[idx] and for head class they are stored at head[idx]. I want to combine them in a single variable say all_class[idx] and then pass all_class[idx] for training.

Cheers