Until very recently lead and mercury were commonly considered to be swell things to ingest as medicine. With the exception of "these berries taste good and these berries will kill you"^, history of human consumption is a pretty bad thing to go by.
At least we have a long history of data on which to base conclusions. Unlike some of the stuff Monsanto is eager to spray on your food, lead wasn't invented in a lab last year.
And see how long it took for the harmful effects to become apparent? Longer than a FDA testing cycle, definitely!
As for tomatoes, there was a serious lack of information regarding many topics in Europe at that time. Did the people living in lands to which the tomato is indigenous suffer from that misperception?
Anyway, the point is to analyze these substances based on what we know now, not what someone thought 250 years ago.
Lead, mercury, and Tobaco where known to be unsafe before I was born. That's a fairly long time from my perspective. Also, in the short term they are reasonably safe. It's only because we can look at long term exposure that we understand how harmful these things are.
To ingest, use as cosmetics . . you name it they tried it. A long history of use is neither a sign of safety or efficacy, but try telling the non scientific public that.
The problem is that some non-natural chemicals are GRAS, and the people who make the most noise about 'all-natural' will never accept them as being benign.
^ and they even messed this up on occasion: http://en.wikipedia.org/wiki/Tomato#History