I having trouble with making something like spatial separable convolution.

I have tensor parameter with [64, 16, 3, 1] and its transposed [64, 16, 1, 3]. What I want is to make [64, 16, 3, 3] tensor parameter by matrix multiplication of both tensors.

I know that it can be done with [1 x 3], [3 x 1] convolution if the input and output # of channel are same. But if not as in my case, it is impossible to use this method.

I assume your parameters are conv weights.
Based on the shape you provided, both weights have the same number of input and output channels.
Have a look at this example: