Life expectancy is a little misunderstood I think. I suppose it depends on the study, but historically this averages out including infant death. If you think about it, if 20% of babies die at birth, that's going to lower the average a lot.
This is why we have the impression that in medieval times, we'd be lucky to reach the age of 30 but in reality, once we get past the dangers of infancy, we tended to live a pretty long life of 60 years or more.
I'd have to look into it more but I think we've only managed to extend our lives maybe 10-15 years since those days. Which is still cool.
RE: Body Size Vs Lifespan: and the debate continues.