Upvoted for discussion, but I disagree with the advice. I’ve spent way too much time trying to harmonize dependencies and track down weird versioning problems between dev and production systems. These things aren’t easy in independent repos, but they’re nigh-impossible in giant monorepos with almost-disjoint projects and very weak separation between service models and clients.
Was this specifically with protos? “very weak separation between service models and clients” doesn’t sound like something that’d happen with protos, since clients are generated from the service models directly.
Can you go into more detail on the specific failure modes you ran into that seemed to be downstream of everything living in a monorepo? I agree you need to be more careful about maintaining proper separation of concerns, but I’m not seeing how monorepos would be more likely to cause versioning issues across environments. I can imagine that if protos didn’t have a build step, you might run into problems similar to e.g. dynamic linking (or resolution at runtime like with protobufjs), and it might be easier to establish a versioning strategy other than “just pull latest main, it’s fine!” with separate repos, but that’s typically not how protos work. I guess there can be a similar failure mode if you tell your build system to resolve proto dependencies at build time by compiling from whatever protos are currently in the repo, instead of using a fixed published version for each target? I might not be understanding what you’re pointing at, though.
this was not specifically with protos, it’s possible that it makes sense to centralize those in some way (but it still worries me that you’d have to version a bunch of diverse service definitions together).
Protos have some relatively unique characteristics and pathologies that push me to recommend this pattern, where I wouldn’t necessarily do so for other “shared dependencies” (though I think I probably still favor a monorepo in most situations).
They’re fairly distinct as far as build targets go, since you often end up generating packages in multiple languages, and the relevant versioning strategy is per-build-target, not per-source-file. It is important to avoid the issues with inappropriate shared dependencies across protos that I mentioned in the article because that is one way you run into trouble with versioning, but imo the solution to that is “programmatically enforce proper separation of concerns within the monorepo”.
Upvoted for discussion, but I disagree with the advice. I’ve spent way too much time trying to harmonize dependencies and track down weird versioning problems between dev and production systems. These things aren’t easy in independent repos, but they’re nigh-impossible in giant monorepos with almost-disjoint projects and very weak separation between service models and clients.
Was this specifically with protos? “very weak separation between service models and clients” doesn’t sound like something that’d happen with protos, since clients are generated from the service models directly.
Can you go into more detail on the specific failure modes you ran into that seemed to be downstream of everything living in a monorepo? I agree you need to be more careful about maintaining proper separation of concerns, but I’m not seeing how monorepos would be more likely to cause versioning issues across environments. I can imagine that if protos didn’t have a build step, you might run into problems similar to e.g. dynamic linking (or resolution at runtime like with protobufjs), and it might be easier to establish a versioning strategy other than “just pull latest
main, it’s fine!” with separate repos, but that’s typically not how protos work. I guess there can be a similar failure mode if you tell your build system to resolve proto dependencies at build time by compiling from whatever protos are currently in the repo, instead of using a fixed published version for each target? I might not be understanding what you’re pointing at, though.this was not specifically with protos, it’s possible that it makes sense to centralize those in some way (but it still worries me that you’d have to version a bunch of diverse service definitions together).
Protos have some relatively unique characteristics and pathologies that push me to recommend this pattern, where I wouldn’t necessarily do so for other “shared dependencies” (though I think I probably still favor a monorepo in most situations).
They’re fairly distinct as far as build targets go, since you often end up generating packages in multiple languages, and the relevant versioning strategy is per-build-target, not per-source-file. It is important to avoid the issues with inappropriate shared dependencies across protos that I mentioned in the article because that is one way you run into trouble with versioning, but imo the solution to that is “programmatically enforce proper separation of concerns within the monorepo”.