Tyson was born in Pitt County, North Carolina and attended Greenville (N.C.) Academy. He then was appointed to the United States Military Academy at West Point, New York, graduating in 1883 and being commissioned as a second lieutenant. He took part in campaigns against the Apache Indians in the West. He was professor of military science at the University of Tennessee from 1891 to 1895. During that time, he also enrolled in the University's law school, graduating from it in 1894 and being admitted to the Tennessee bar that same year. Resigning his U.S. Army commission, he began the practice of law in Knoxville.

Upon the outbreak of the Spanish-American War in 1898, Tyson desired to return to active military duty. Receiving an appointment as a colonel, he organized and trained the 6th Regiment U.S. Volunteer Infantry. He and his unit were shipped to Puerto Rico, where they saw no direct combat. He returned to his law practice in Knoxville in 1899 and was also involved with a manufacturing concern. Simultaneously, he was serving as a brigadier general and inspector general of the Tennessee National Guard, which position he held from 1902 until 1908. From 1903 to 1905 he was Speaker of the Tennessee House of Representatives.

In 1913 he made an unsuccessful run for the U.S. Senate. Upon U.S. entry into World War I, he again applied to return to active military duty, and was appointed brigadier general over all Tennessee National Guard troops by governor of Tennessee Tom C. Rye. This commission was subsequently federalized by President Woodrow Wilson, and Tyson was assigned to the 59th Brigade, 30th Division, training troops at Camp Sevier near Greenville, South Carolina. He later was sent to Europe and saw combat in France and Belgium. Tyson's son McGhee, a Navy pilot, was lost over the North Sea while scouting for mines and Tyson left the front to help search for his son off the coast of Scotland. Tyson left active duty for the final time in 1919.

In 1920 he made an unsuccessful effort to receive the vice presidential nomination at the Democratic National Convention. In 1924 he mounted a successful campaign for the U.S. Senate, receiving the Democratic nomination and then being elected in November. He was sworn in as a Senator on March 4, 1925. He died in 1929 in a sanitarium in Strafford, Pennsylvania and is buried in Old Gray Cemetery in Knoxville, the burial place of several prominent East Tennessee political figures. Tyson Park and Tyson Junior High School in Knoxville are named in his honor, as was Camp Tyson, the World War II U.S. Army training post near Paris, Tennessee. Tyson donated land for Knoxville's airport which was named after his son, McGhee. McGhee Tyson Air National Guard Base is also named after Tyson's son.

This article incorporates facts obtained from the public domain Biographical Directory of the United States Congress.