How I operationalize crash-only code in my data generation code, given that Data Denormalization Is Broken [0]:
When operating on database data, I try to make functions whose default behavior on each invocation is to re-process large chunks of data and regenerate all the generated values, and make it idempotent. (I would regenerate the whole database on every invocation if I could, but there’s some tradeoff of how big a chunk is sufficiently fast to reprocess.)
How I operationalize crash-only code in my data generation code, given that Data Denormalization Is Broken [0]:
When operating on database data, I try to make functions whose default behavior on each invocation is to re-process large chunks of data and regenerate all the generated values, and make it idempotent. (I would regenerate the whole database on every invocation if I could, but there’s some tradeoff of how big a chunk is sufficiently fast to reprocess.)
[0] https://lironshapira.medium.com/data-denormalization-is-broken-7b697352f405