This happened to me once too. We were at a local swingers party, I hooked up with another man, who, unknown to me at the time, my husband loathed. I never hooked up with him again; I mean to me the whole swingers scene was meant to be fun, why ruin it over something trivial.