If we assume there are a countable number of signs, and any sentence contains a finite number of signs, then there must be an uncountable number of truths of mathematics that cannot be represented, let alone be proved. This is because there are an uncountable number of real numbers. It is true that we can talk about many irrational real numbers such as pi and e, but these can be specified by a finite number of symbols. There must be many more irrationals which cannot be so specified.
One could take an uncountable number of signs as primitive, i.e. by using points on a circle, varying continuously. The points on such a circle can be mapped to the whole real line. If one did this, I think one would have also to have an uncountable number of axioms – how else could the primitive signs enter in?
It is a valid objection that we could not do this in practice. But it is also true that there is some finite limit to the size of sentences that humans (or machines we can build) can handle, so a countably infinite assumption is also only of theoretical interest, it would seem.