Well, depending on how long the migration is, many of the animals (mainly birds) could get sick and die, lowering the population. They could be shot by hunters, lowering the population. The children of the population could get left behind and lose their way, lowering the population. I'm not entirely sure how the population could grow other than them finding and staying with another group of their own kind. hope I was helpful! :)
