Oooh, your office sounds like mine LOL
Seriously though, I've found that the more about us we share, the more people will comment. The office staff who are open about what they do get a lot of joking and questions. However, those who keep their personal lives to themselves seem to miss out on those conversations.
I'd say if you're not comfortable with them joking, then try to avoid discussing your personal life.