Will minecraft run with this kind of computer?

I have recently bought a HP desktop yesterday that I bought for gaming(Minecraft) but I did not know what to look at(specs.) now, when I play minecraft I run terribly slow… 1-10 frames(FPS!) why?!
My Computer specs are:
Processor: AMD E1-2500 APU with Radeon(TM) HD Graphics
4gb of ram
64 bit windows 8 os
It's and HP desktop too
GPU: amd Radeon HD graphic 8240

I believe that this desktop it's self is a bad gaming computer for minecraft.

I don't want to build my own because it costs too much, what are my options here? Buy a new PC, get a new GPU?
Also, can I put in my old desktops GPU (acer aspire M5620) into my HP Desktop? Will that work

Lower the setting, update java. If it doesn't do the trick… I wouldnt bother upgrading that pc. Also, prebuilt computers are known for being overpriced and whatnot. See if you can return it to the store maybe? Building your own pc can be (and is) a lot cheaper than getting a prebuilt one. If i knew what was your budget i could probably be more helpful:/