I think you have a right to speak with him about it. If you have not set any ground rules, now may be the time to have a straightforward and equitable talk about your feelings and needs. Remember he will need to be able to do the same. It can be a hard thing to want to share your new relationships with your spouse but as long as you both respect the others feelings and don’t modify any rules or agreements that you have made with each other without consulting each other then things will be easier. Note I did not say easy, just easier.

The second there are secrets and lack of open communication you have every right to question why. The entire idea of being a swinging couple is there is no reason or need for hiding anyrhing. Especially anything involving sexual activities. If distrust and deceit and insecurity raise their ugly heads in a relationship, there are definitely issues to be addressed.