Yes! And just speaking about Western or European culture, it was just elite women covering their breasts until more recently than that.
References about ancient Greek and Roman society are fairly limited with respect to ordinary women's clothing. We know much more about how elite women dressed, both because we have some written descriptions and because paintings and sculptures of elite women have come down to us.
But we do have some tantalizing clues — including women in the background of certain art, and a few textual references— that more ordinary women sometimes went out in public with their breasts uncovered.
Historians don't know exactly how it worked, because people who wrote were men, and they spent very little time writing about women.
It's possible that the few references we have are about slaves, who were largely seen as sexual objects anyway. But then, that was part of the problem even then.
In Greek and Roman high society, men viewed women largely as property, either as daughters or wives. It seems that men expected their wives and daughters to cover themselves because those women were their property. Caesar's wife must be beyond reproach — Julius Caesar didn't say that (if he actually said it) primarily because of political considerations, though that was the context of the quote.
He was echoing Roman patriarchal values, expressing the idea the respectable women stayed home, and on the limited occasions when they did venture out in public, they kept themselves mostly out of sight of other men, who did not own them. If they weren't out of sight from the general public, they were expected to be almost completely hidden by their clothing and accompanied by male slave-bodyguards who would physically keep them apart from other men.
Compare and contrast this to the fact that lower-class women are known to have (at least sometimes) gone topless in public.
People didn't complain about that, because their fathers or husbands weren't important enough to have their property interests considered or taken seriously.
It took awhile for attitudes about covering female bodies to trickle all the way down society's layers.
But women as property was the beginning of why it happened.