Long time no see my friend. Sorry to hear you haven't been off all too well.
What you are describing here
i alwaus try to explain things my PoV and he just refuses to see it. When i do so it just "turning things around" on him.
This self-victimization is definitely something abusers do a lot, often without even being aware themselfs.
On it's own that doesn't nessecarily mean that the relationship is abusive, but that it could be.
It all depends on other factors too, like how often does this happen, what you are fighting about etc.
Regarding the idea of a break, if he in any way loves you back he won't forget you over a short break.
If he does, then, as hard to swallow as the pill might be, he never has and at least then you know the truth.
I don't mean to intrude on private matters, but the fact that he's married is rather unusual, is it an open-relationhip kinda deal or more of an "they are technically divorced just not legally" kinda deal?
I totally understand how you are feeling buddy, i would love nothing more than to feel like that around someone, but you shouldn't allow your desire for imtimacy to devalue yourself if worst case scenario this actually is abusive.
You realistically need to consider, in this relationship to another person, how are you feeling most of the time? Joyful? Miserable? Anxious? Do you dread or fear something? And then act accordingly.
A relationship should make you a happier, better person. If it doesn't it's not worth having. But we here so far away from the situation can't judge that.
Anyways, i hope it'll work out for you regardless of what's the case