From what I’ve been learning about human history, your first paragraph does seem to be the case. However, we didn’t have near the same numbers then as we do now. It’s the scarily, or appearance thereof, that causes the selfishness in my opinion.