To me, this is a clear example of there being no such thing as an “objective” truth about the the validity of the parallel postulate—you are entirely free to assume either it or incompatible alternatives. You end up with equally valid theories, it’s just those theories are applicable to different models
This is true, but there’s an important caveat: Mathematicians accepted Euclidean geometry long before they accepted non-Euclidean geometry, because they took it to be intuitively evident that a model of Euclid’s axioms existed, whereas the existence of models of non-Euclidean geometry was AFAIK regarded as non-obvious until such models were constructed within a metatheory assuming Euclidean space.
From the perspective of modern foundations, it’s not so important to pick one kind of geometry as fundamental and use it to construct models of other geometries, because we now know how to construct models of all the classical geometries within more fundamental foundational theories such as arithmetic or set theory. But OP was asking about incompatible variants of the axioms of set theory. We don’t have a more fundamental theory than set theory in which to construct models of different set theories, so we instead assume a model of set theory and then construct models of other set theories within it.
For example, one can replace the axiom of foundation of ZFC with axioms of anti-foundation postulating the existence of all sorts of circular or infinitely regressing chains of membership relations between sets. One can construct models of non-well-founded set theories within well-founded set theories and vice versa, but I don’t know of anyone who claims that therefore both kinds of set theory are equally valid as foundations. The existence of models of well-founded set theories is natural to assume as a foundation, whereas the existence of models satisfying strong anti-foundation axioms is not intuitively obvious and is therefore treated as a theorem rather than an axiom, the same way non-Euclidean geometry was historically.
Set theories are particularly good and convenient for simulating other theories, but one can alsosimulate set theories within other seemingly more “primitive” theories (e.g. simulating it in theories of basic arithmetic via Godel numbering).
Yes, there are ways of interpreting ZFC in a theory of natural numbers or other finite objects. What there is not, however, is any known system of intuitively obvious axioms about natural numbers or other finite objects, which makes no appeal to intuitions about infinite objects, and which is strong enough to prove that such an interpretation of ZFC exists (and therefore that ZFC is consistent). I don’t think any way of reducing the consistency of ZFC to intuitively obvious axioms about finite objects will ever be found, and if I live to see a day when I’m proved wrong about that, I would regard it as the greatest discovery in the foundations of math since the incompleteness theorems.
This is an appealingly parsimonious account of mathematical knowledge, but I feel like it leaves an annoying hole in our understanding of the subject, because it doesn’t explain why practicing math as if Platonism were correct is so ridiculously reliable and so much easier and more intuitive than other ways of thinking about math.
For example, I have very high credence that no one will ever discover a deduction of 0=1 from the ZFC axioms, and I guess I could just treat that as an empirical hypothesis about what kinds of physical instantiations of ZFC proofs will ever exist. But the early set theorists weren’t just randomly sampling the space of all possible axioms and sticking with whatever ones they couldn’t find inconsistencies in. They had strong priors about what kinds of theories should be consistent. Their intuitions sometimes turned out to be wrong, as in the case of Russel’s paradox, but overall their work has held up remarkably well, after huge amounts of additional investigation by later generations of mathematicians.
So where did their intuitions come from? As I said in my answer, I have doubts about Platonism as an explanation, but none of the alternatives I’ve investigated seem to shed much light on the question.