(I’m not including cases where it was a developer account—i.e. an account used to publish apps—that was banned.)
The core issue seems to have two sides:
Google’s automated systems are sometimes too eager about banning accounts, if you in any way happen to stand out as a (false) positive case
Google’s script-following customer support seems pretty bad at then dealing with these cases, giving no information, and worse, taking in no information provided to them, and in general the best customer support avenue seems to be Hacker News or some other news outlet.
This is the visible tip of the iceberg where the accounts do (generally) get reinstated, but we don’t know how large the invisible part of the iceberg is, and what the chances of getting your account back are if you aren’t lucky enough to be in the visible part.
Turns out it can also happen for sending your child’s medical pictures to a doctor (and also send the cops after you).
I think it’s pretty likely that there’s more to the story than we’re hearing: there was a ton of scrutiny on this and Google didn’t change their decision, which makes me think the most likely reason is that the human review on the case really did come up with CSAM concerns beyond those pictures.
(Still, I don’t take pictures of people without clothing, including my kids: CSAM misclassification and escalation is a small risk of something very painful.)
This is the visible tip of the iceberg where the accounts do (generally) get reinstated, but we don’t know how large the invisible part of the iceberg is, and what the chances of getting your account back are if you aren’t lucky enough to be in the visible part.
Yes, but we also don’t know how many of the not reinstated cases were ones were correct.
Companies generally say things like, “out of concern for our user’s privacy we can’t release more information”, but I’d like to see something where someone making a public complaint could waive this and challenge the company to actually release their full case either publicly or to an independent ombud. Tesla does this (pretty sure, can’t find the link now) though they release things publicly even without the customer’s permission which I don’t think makes sense.
Another example of this that comes to mind is when fans of a YouTube streamer got their entire Google accounts banned for posting too many emojis (as votes, at the streamer’s behest) in YouTube chat.
Turns out it can also happen for sending your child’s medical pictures to a doctor (and also send the cops after you).
Or for storing “educational content on Palestine”. Unverified story, and there’s a few more such stories and links at [1] and [2],
(I’m not including cases where it was a developer account—i.e. an account used to publish apps—that was banned.)
The core issue seems to have two sides:
Google’s automated systems are sometimes too eager about banning accounts, if you in any way happen to stand out as a (false) positive case
Google’s script-following customer support seems pretty bad at then dealing with these cases, giving no information, and worse, taking in no information provided to them, and in general the best customer support avenue seems to be Hacker News or some other news outlet.
This is the visible tip of the iceberg where the accounts do (generally) get reinstated, but we don’t know how large the invisible part of the iceberg is, and what the chances of getting your account back are if you aren’t lucky enough to be in the visible part.
I think it’s pretty likely that there’s more to the story than we’re hearing: there was a ton of scrutiny on this and Google didn’t change their decision, which makes me think the most likely reason is that the human review on the case really did come up with CSAM concerns beyond those pictures.
(Still, I don’t take pictures of people without clothing, including my kids: CSAM misclassification and escalation is a small risk of something very painful.)
Yes, but we also don’t know how many of the not reinstated cases were ones were correct.
Companies generally say things like, “out of concern for our user’s privacy we can’t release more information”, but I’d like to see something where someone making a public complaint could waive this and challenge the company to actually release their full case either publicly or to an independent ombud. Tesla does this (pretty sure, can’t find the link now) though they release things publicly even without the customer’s permission which I don’t think makes sense.