Increased running costs (more energy required and more heat loss through the head).There is, obviously, even in societies/groups where high intelligence is a benefit, selection pressures in both directions because of the costs associated with increased brain size.