The quality of roads is relevant, but not really the answer. Bicycles can be ridden on dirt roads or sidewalks (although the latter led to run-ins with pedestrians and made bicycles unpopular among the public at first). And historically, roads didn’t improve until after bicycles became common—indeed it seems that it was in part the cyclists who called for the improvement of roads.
From this post about why humanity waited so long for the bicycle. I particularly recommend the discussion of how long it took to invent pedals and gears.
A counterpoint to your first sentence:
From this post about why humanity waited so long for the bicycle. I particularly recommend the discussion of how long it took to invent pedals and gears.