Yes, but why do you refuse to believe it? What’s your evidence that your experience of color is ontologically primitive? It’s just baseless assumption.
Physicalists who aren’t thoroughgoing eliminativists or illusionists, are actually dualists.
Can you imagine believing in dualistic non-physical parts of your experience that you are not aware of?
Oh, ok, I misunderstood you.
Physics is monistic panpsychism—there are no just geometric-causal-numerical ingredients, there is also implicit statement that universe that equations describe has intrinsic property of existence.