well something is not right for sure. you can remove the new bulb,inspect it and if it looks fine and was not connected 90 degrees off(which would be hard to do) try starting it with the new bulb removed and see if the other bulb lights up if you click the high beam switch. the light should light up while the switch is held then go out when you let go. i'm assuming that this is all happening with the head lights off. if the lights appear to be normal if the new bulb is not installed, take the bulb back to the shop or parts store and ask for a new one. if you cannot do that you can try putting the suspected bad bulb in the other position and the older bulb in the place where the new bulb was. if the problem is the bulb it should have the same problem on the other side.
if the lights have the problem with the new bulb removed or the same problem with the bulbs on different sides and the older bulb coming on when you start and the new bulb coming on when you click the switch, the problem is likely the switch. good luck...