Raffensperger claims ‘last minute nature’ of long-planned debate forced him to skip event