That makes sense.. but immediately afterward both Harry and McGonnagal thought it was unusual how bright the remembrall shined; neither seemed to think it was solely due to the use of the Time Turner:
“The Remembrall was glowing bright red in his hand, blazing like a miniature sun that cast shadows on the ground in broad daylight.”
and
“”More importantly, why did the Remembrall go off like that?” Harry said. “Does it mean I’ve been Obliviated?”
“That puzzles me as well,” Professor McGonagall said slowly. “If it were that simple, I would think that the courts would use Remembralls, and they do not. I shall look into it, Mr. Potter.”″
And then of course she doesn’t. Perhaps the courts simply don’t use Remembralls because they would never definitively prove obliviation—only that something was forgotten. Harry’s remembrall “blazing like a miniature sun” may be due to an overwhelmingly large obliviation—like an entire life as a Dark Lord? Obliviating a day or week may just produce a normal glow...
Also, Fawkes did have some connection with Harry after he returned from Azkaban—he asked Harry to help him destroy the place, and it’s clear he asked the same of Dumbledore (perhaps many times).
So he’s not totally off the mark in thinking that Fawkes would come to him—in that case he’d be agreeing to do what Fawkes wants, instead of restraining him like Dumbledore usually does. I bet if Harry called to Fawkes with a wish to go to Azkaban and destroy it, Fawkes would be there to take him in a heartbeat.