I think Diamond is pointing out that it may be a misconception to think of hunting/gathering as a "terrible" quality of life to indicate that population pressure might have forced us into ag. This suggests an order to the chicken/egg problem.
Basically the population grew to a large enough size where the game they were hunting and the berries they were gathering weren't enough (or went extinct) so they had to switch to a agrarian lifestyle, which needed more work and yielded less.
and i suppose what i'm trying to say is that if you have a group large enough to outgrow hunting/gathering, its likely to be large enough to generate at least some of the issues described in the article attributed to large groups of people.
Basically the population grew to a large enough size where the game they were hunting and the berries they were gathering weren't enough (or went extinct) so they had to switch to a agrarian lifestyle, which needed more work and yielded less.