How do I invite friends on Minecraft PE to my world?

I have a long distance relationship and a few internet friends that I often play games with online. Unfortunately, when I try to add friends onto my friends list it takes me to the Xbox live phone application. I have already added my friends to my Xbox live account. But I also can't send invites from inside my world and sometimes my friends don't see me in my friends list when I'm online.
I have no idea what I'm doing and I have no idea how to fix it.

What's up with Minecraft? And how do I fix it?
What needs to be changed.

P.S; Some friends can send me invites and it works, but sometimes other friends would send me invites and it would not load.

I don't know

Sounds like a bad internet connection or your world are private. Regardless if you send one(on ps4)if your world is private or not the invite is still sent.