It's most likely a problem with fluid returning to the master cylinder. It could be debris blocking the return port to the reservoir. Or one of the pistons sticking or prevented from returning by the push rod.
To determine where the pressure is building up, get the brakes to start dragging. Then loosen the brake lines one at a time, starting at the master cylinder line closest to the booster(primary piston). You should get a bit of a spurt, when you get to the line in question, and the brakes will free up. If it's the line at the primary piston, make sure your brake pedal fully returns. Also the booster push rod can be out of adjustment. To determine if it is, create the brake drag, then loosen the master cylinder mounting nuts. Then see if the brakes free up. If pressure releases at the secondary piston, the problem is in the master.
If you don't find a problem at the master, then follow the lines to other components (hill holder etc.), and do the same tests.