Trouble connecting to my friends minecraft server?

It's stuck on loading terrain and I get time out after that. We live in the same city but I get like 200 ping. But if I try to connect to a public server originated from a different continent, I get 60 ping. Firewall is not a problem. How can we fix this?