Meh, let him join. And if he figures out its you two, enjoy the akward silences and such. You can use it as a form of *******. Be completely oblivious to the fact that you do it around him. Even make obvious gestures to your wife and such, but act like it's not so obvious. Would be totally worth...