How do educated laymen single out numbers for reference? One suggestion has been that they do so on the basis of the ordinal properties of numbers, another that we individuate by their cardinal properties. I discuss arguments by Linnebo (2009, 2018) against the latter suggestion. While these fail to decide the basis for individuation, they do establish that we individuate numbers via numerals. So, I then turn to two kinds of empirical evidence: linguistic evidence regarding the semantics of number words and evidence regarding the acquisition of number concepts. Both support the slightly different claim that we individuate numbers by their cardinal properties, but still via numerals. This requires the formulation of a new formal criterion of identity to describe how we individuate numbers. It is slightly less elegant, but captures in detail how humans do so. By answering the question of semantic individuation, the door is then opened to an account of reference that closely fits how educated laymen refer to the natural numbers.