I am reading one of notebook DAE for tabular data , i know AE but not DAE. In AE i did not see any embed_dim and hidden_size parameter. I just saw hidden layer (bottleneck) layer in AE and this hidden layer is a latent space which is normally less than in number than original input layer.
But here while seeing DAE from one of notebook i see hidden_size , not sure what is it ? If it is hidden layer as i learnt in AE then why it is bigger than original input layer?
DAE_CFG = {‘batch_size’ : 384,
‘init_lr’ : 3e-4,
‘lr_decay’ : .998,
‘noise_decay’ : 0.999,
‘max_epochs’ : 600,
‘save_freq’:50,
‘hidden_size’:1024,
‘num_subspaces’:8,
‘embed_dim’:128,
‘num_heads’:8,
‘dropout’:0,
‘feedforward_dim’:512,
‘emphasis’:.75,
‘task_weights’:[len(fts_categorical), len(fts_continuous)],
‘mask_loss_weight’:2,
‘prob_categorical’ : 0.5,
‘prob_continuous’ : 0.5,}
Any could please share some info about
What is hidden_size?
What is embed_dim?
What is num_subspace?
Whats is num_heads?
Whats is emphasis,mask_loss_weight,prob_categorical,prob_continuous?