Extreme amounts of lag on my modded Minecraft server?

My friends and I have decided to make a private server that we decided to mod. The mods work, but we have extreme amounts of lag.
The specs of my computer are as follows:
Core i7 3.9Ghz
16GB DDR3 RAM
NVidia GTX GeForce 660
2TB HDD
120GB SSD

I'm the one running the server. Normally, on vanilla minecraft there's a little lag but not as bad as it is currently.

Mods can be very laggy your pc may just not be able to handle the extra pc resources the mod requires to work lag free