There is a simple intuition for why PSD testing cannot be hard for matrix multiplication or inversion: regardless of how you do it and what matrix you apply it to, it only gives you one bit of information. Getting even just one bit of information about each matrix element of the result requires n2 applications of PSD testing. The only way out would be if one only needed to apply PSD testing to tiny matrices.
There is a simple intuition for why PSD testing cannot be hard for matrix multiplication or inversion: regardless of how you do it and what matrix you apply it to, it only gives you one bit of information. Getting even just one bit of information about each matrix element of the result requires n2 applications of PSD testing. The only way out would be if one only needed to apply PSD testing to tiny matrices.