Multi attribute multi class classification

Say I have 2 attributes which is color and direction.

In color we have blue, red, green,
In direction we have left, right, front, back.

So How to design a classifier to make it work output 2 attributes and according class?

I know that we using 3x3=9 classes to classify is a solution, but that not very good seems. Does there any better way to do this?

You could use two classification “heads” and return two outputs.
These outputs would separately compute the losses, which you could sum and backward together.