Merging different size vectors in fully connected layers

Hi everyone. I needed your help and advice on my problem. I want to do a survival prediction based on CT images and two clinical variables of patients(age and gender). For CT images feature representation I tried to do a classification and I extracted the features before fully connected layers the feature vector for every patient has a size of 2560. So for the second network, I have to give age, gender, and feature vector to do the survival estimation. I don’t know how to merge the feature vector and the two variables. One way can be the concatenation but I think because of the big size of the image vector the age and gender wouldn’t affect the prediction. Another way can be using some fully connected layers to decrease the image feature vector size and then add age and gender to the output of that FC layer and have other FC layers to predict the survival. But the problem is that when we increase the number of FC layers the accuracy decreases. Do you have any idea how it is possible to merge a feature vector and two variables in a way that gets a good result and accuracy?