By too many numbers I refer to the people that schedule conference calls all the time
Basically I can get invited to a call without knowing the person before, which means i would first need to add the person to my phonebook while the # is already in the calendar invite.
So its not related to any restrictions on the phone itself.