OK, it is a fuel issue.
If fuel system pressure is not at specifications, the fuel injectors will not inject the correct amount of fuel when the computer system is in open loop.
If the pressure is not too far off(your case), the computer will adjust the air/fuel mixture after the system begins closed loop operation. The vehicle will have a cold driveability problem that will disappear when it is warm. (exactly whats happening with your vehicle)
I dont think the fuel pump is the culprit(but it is possible).
I would suggest that you change the fuel pressure regulator before changing the pump. Fuel pressure regulators usually fail by causing pressure that is too low rather then high, but in your case, you have kinda both. 44 psi is a bit high from the manufacturers specs. The fuel pressure should drop to 28-32 psi with the engine at idle. This is due to the lower manifold pressure.
hope this helps, if you need more help, let me know.
If my reply has helped, Please click the GREEN ACCEPT button on the right hand side of my reply.