Improve neural machine translation with word embeddings + additional word features

Hi everyone, i’m trying to create a pretrain word embedding with additional word features such as part of speech tag, named entity, lemma,etc. Are there any way to create word embedding with word features ? Should i train word embedding with corpus that each word concate with its pos tag ?