It’s tough to extract completely unbiased quasi-experimental data from the world.
Understatement.
I can’t go into more details here unfortunately, and some of the details are changed for various reasons, but one of the best examples I’ve seen of selection bias was roughly the following:
Chips occasionally failed. When chips failed, they were sent to FA[1] where they were decapped[2] and (destructively) examined[3] to see what went wrong. This is a fairly finicky process[4], and about half the chips that went through FA were destroyed in the FA process without really getting any useful info.
Chips that went through FA had a fairly flat failure profile of various different issues, none of which contributed more than a few percent to the overall failure rate.
Only… it turns out that ~40% of chip failures shared a single common cause. Turns out there was an issue that was causing the dies to crack[5], and said chips FA received then promptly discarded because they thought they had cracked the die during the FA process. (FA did occasionally crack the dies, but at about an order of magnitude lower rate than they had thought.)
Understatement.
I can’t go into more details here unfortunately, and some of the details are changed for various reasons, but one of the best examples I’ve seen of selection bias was roughly the following:
Chips occasionally failed. When chips failed, they were sent to FA[1] where they were decapped[2] and (destructively) examined[3] to see what went wrong. This is a fairly finicky process[4], and about half the chips that went through FA were destroyed in the FA process without really getting any useful info.
Chips that went through FA had a fairly flat failure profile of various different issues, none of which contributed more than a few percent to the overall failure rate.
Only… it turns out that ~40% of chip failures shared a single common cause. Turns out there was an issue that was causing the dies to crack[5], and said chips FA received then promptly discarded because they thought they had cracked the die during the FA process. (FA did occasionally crack the dies, but at about an order of magnitude lower rate than they had thought.)
failure analysis
Removed, very carefully, from their packaging.
e.g. by lapping and using an electron microscope.
To put it mildly.
Amusingly, a cracked die can actually sometimes kind of limp along due to e.g. capacitive coupling. And then you bump it or the temperature changes...