Is it possible that the dilemma you're having with the Kool Stop Pads is specific to the Kool Stops we stock in the shop?
If you're using the ones I'm thinking of, they don't have the bowled washers that allow for angle adjustment. But Kool Stop does make those pads.
This:

Instead of this:
