(Short version: Yes, you’ve basically got the intended meaning right, but this
figure of speech has its detractors who don’t like its use as an all-purpose
escape hatch. End of short version. For what it’s worth I’m not one of the
aforementioned detractors. My translation of the phrase is: “What I just said
was intended not as an insult or provocation, but as a factual observation,
and I’m letting you know that in a mildly humorous way by using a current
figure of speech.”)
“Just saying” / “I’m just sayin’”:
Urban Dictionary
Mark Liberman from Language Log
Scott Simon editorial from the radio show Weekend Edition Saturday
(Short version: Yes, you’ve basically got the intended meaning right, but this figure of speech has its detractors who don’t like its use as an all-purpose escape hatch. End of short version. For what it’s worth I’m not one of the aforementioned detractors. My translation of the phrase is: “What I just said was intended not as an insult or provocation, but as a factual observation, and I’m letting you know that in a mildly humorous way by using a current figure of speech.”)
Edit: Bonus fun link: Here’s the song “Punch Bowl” by Punch Brothers, which uses the “I’m not sayin’, I’m just sayin’” variant of this phrase.
Edit 2: My Mom says she only likes this phrase when it’s used by an animal.