It seems like the burden is on the physicalist to prove why phenomena that we only know via their appearance "in mind" are truly existent independent of it. I don't mean that in a solipsistic sense, or even in the sense of Berkeley's idealism. I think some form of objective idealism is probably true.
But anyways, there is definitely a kind of epistemic asymmetry that goes against the physicalist here, so I don't think you get to just handwave away the hard problem.
Never said we should
But we also can't assert a definitive answer for the "hard problem", as none exist