In 1748, what is now Lafayette & Astor Place, was New York City’s first botanical garden, established by a Swiss physician, Jacob Sperry, who farmed flowers & hothouse plants. Jacob Sperry, born in Zurich in 1728, came to New York at the age of 20, & although educated a physician, decided to become a florist. He had means at his command, with which he purchased this then uncultivated tract of pasture land, & established himself as a horticulturist. He built a house near by, where he resided, rearing a family of 4 sons & 5 daughters. In 1804, Jacob Sperry sold the much improved property to John Jacob Astor for $45,000.
An 1801 map of the Astor Place when it was the land of Jacob Sperry, a Swiss florist, physician, and gentleman.
In the British American colonies, just as in Europe, many early botanical gardens focused on the medicinal uses of plants being collected. In 1769, Dr Peter Middleton, professor of medicine at King's College, speaking at the opening of the Columbia Medical School in New York City stated, "By botany, we are instructed in the natural history and distinguishing characters of plants. This, pursued as a science, or branch of medical study, presents to us a fund of knowledge, both valuable and ornamental As this continent yields most of the medical plants now in use, and abounds also with a variety of others, whose qualities we are as yet but little acquainted with... a teacher of botany will soon be appointed, and a botanical garden laid out, and properly furnished? This would open an extensive field for further discoveries in, and for large acquisitions to the materia medicia." David Hosack, who would eventually establish the Elgin Botanic Garden, reported that in 1794, the New York Agricultural Society was endorsing that the botanical garden be connected with an endowed professorship in Botany. In the next 20 years, botanical gardens would pop up at Harvard, Princeton, and at the universities of Pennsylvania & South Carolina.
Botanic Garden at Elgin in the Vicinity of the City of New York. About 1806 William Satchwell Leney (American artist, b. England, 1769–1831) after Louis Simond (American artist, b. France, 1767–1831)

+after+Louis+Simond+(American+artist,+b.+France,+1767%E2%80%931831).jpg)
