John Q. Smith was born to Thomas Edward Smith (1783-1841) and Mary Kennedy Whitehill (1788-1849), natives of Virginia, on their Warren County, Ohio farm near Waynesville. A voracious reader, his early schooling was limited because of his duties on the family farm, but his father believed in the advantages of an education, so that John Quincy was able to spend a short time at Miami College.

In July 1852, John Q. Smith married Lydia Emeline Evans, a native of Warren county. They had six children, one of whom died in childhood. In 1854, he relocated his young family to Clinton County, Ohio.

He was elected to the Ohio Senate in 1859 as a Republican. In Columbus, during the legislative sessions, Smith's roommate was James A. Garfield, who was just starting out on his public career, and other intimate acquaintances were John Sherman and Ulysses S. Grant. In 1861, he was elected to the Ohio House of Representatives, and served two years. In 1870, he was elected as a member of the Ohio State Board of Equalization. He was again elected State Senator in 1871.

In 1872, John Q. Smith was elected to Congress from Ohio's Third Congressional District. In 1874, he was renominated for Congress, but defeated by John S. Savage.

He was Commissioner of Indian Affairs in the Grant Administration from 1875 to 1877. President Rutherford B. Hayes appointed Smith as United States consul general to Montreal, Canada, serving from 1878 until he resigned in 1882.

He remained an ardent Republican until President Grover Cleveland's first administration, when he allied himself with the Democratic party because of his views on tariff reform, and thereafter he remained a Democrat. His published articles on tariff in the New York Evening Post attracted wide attention throughout the country and were extensively quoted by the press and on the stump.

John Quincy Smith left public life and retired to his farm “Sycamores” in Oakland, Ohio where he died. He is buried in Miami Cemetery, Waynesville, Ohio.