“And they (the two Hittite women that Esav married) were a source of bitterness to Isaac and to Rebecca.” (Genesis 26:35)
The Midrash notes that the source of aggravation was because these women were idol worshipers.
The Midrash also picks up on an interesting detail in the verse. It doesn’t say they were a source of bitterness to Isaac and Rebecca – but that they were a source of bitterness to Isaac and to Rebecca.
Based on this, the Midrash teaches that Isaac was provoked first and only then Rebecca was provoked. The Midrash explains that this is because Rebecca grew up in a home of idolaters, but Isaac grew up in Abraham’s home.
Rabbi Dr. Abraham Twerski notes a deep lesson here. Rebecca has been away from her father’s home for many years at this point. But because she was exposed to idolatry in her childhood, she was not as repulsed by it. The worst kinds of violence or lewdness can become less abhorrent if we are routinely exposed to them in our youth.