Detector imperfections in photon-pair source characterization