If I add someone on Skype who may or may not want my IP, but don't call him, will my IP be safe?

Basically, I'm playing on a Minecraft server and this one guy who a lot of people know of is asking me to add him on Skype. He and I have been having a bit of a flame war between us, and I don't know if it is wise to add him on Skype, lest he try to DDoS me. So my question is: can I add him on Skype safely if we don't call, and what could he do if he had my IP?