One doesn’t have to look back very far in history to find European societies in which sexual fidelity was not unexpected part of marriage. At least not for men.
Oh, sexual fidelity might have been a religious principle, but it was observed more in the breach than in reality.
Going further back, pre-christian societies — as in Celtic, Greek, Roman, Persian, Egyptian, etc — not only accepted sexual activity outside of marriage, they often regulated it in particular ways, either legally or by enforcing social norms.
The Greeks even had particular vocabulary for it. A man would describe his love for his wife as agape, an enduring family love that could include a sexual element but did not have to. People would also refer their love for their children and parents, for example, as agape.
When Greek men had temporary sexual relationships outside of marriage, as they apparently often did, they would refer to that kind of love as eros. Scholars tend to think that the difference between agape and eros is not sexuality, as it is usually popularly claimed, but rather permanence.
To illustrate, the homosexual relationships between older and younger men, idealized among the elite in archaic and classical Greece, lasted for a lifetime in theory. While the relationships had a strong sexual element for a few years, they presumably did not after a period of time.
Yet the love in the relationships, even when the relationships were clearly sexual, was usually described as agape and not as eros.
I mention all this only to say that the idea of sexual infidelity was so ingrained in some societies that they had sophisticated vocabulary to describe the different types of relationships and feelings.
I should mention of course that I’m not trying to idealize these societies or promote their values. Most classical Greek women were not sexually free in the way Greek men were. Women of the elite classes, except perhaps in Sparta, were punished severely for having sex outside of marriage. They were essentially property.
Nonetheless, idealized romantic, sexual fidelity was not a concept anyone would have recognized. As you say, that’s a very new notion in the human experience.