You could create a color mapping as described here to transform the color values to the class indices.