I don’t normally weigh in on things like this, but I recognise your predicament, been through something like it myself, more than once, and hopefully learned from my own mistakes.
The issue seems to be that you are opening up to the idea of bringing other people into your marriage. At least one person that is.
And there is the rub. What if the tables were turned and you found out your wife had been getting intimate with another man, a stranger, suggesting she bring him home for a threesome, whilst you are busy concentrated with your mind only on earning a crust for your family?
Could you live with that, still loving her?
And don’t forget, even a woman with moderate attributes will easily out-perform you when it comes down to who can pull the highest number of partners, no matter how much of a hunk you might be.
If the answer is yes, then you need to establish if she could be similarly minded, then maybe you can come to an arrangement that works.
If you couldn’t live with that, or if she couldn’t live with that, the only chance you have of saving your marriage is ditching the bit on the side, mate. Spectacularly.