He was born in Maynardville, Tennessee, third of five children. He played semi-professional baseball, but a sunstroke in 1929 and a nervous breakdown in 1930 ended his aspirations to play for the New York Yankees. He then turned his attention to his father's fiddle and began playing traveling medicine show. He toured the Southern United States. He eventually formed a band "The Crazy Tennesseans".

In 1936, he recorded a cover of the traditional song "The Great Speckled Bird". His performance of it in his Grand Ole Opry debut was not well received. Acuff became a regular on the Grand Ole Opry in 1938, forming a backing band called the Smoky Mountain Boys, led by friend and Dobro player Bashful Brother Oswald.

Acuff released several singles in the 1940s such as The Wreck on the Highway, Beneath That Lonely Mound of Clay and The Precious Jewel. He later formed a music publishing venture with Chicago songwriter Fred Rose. Hank Williams, the Everly Brothers and Roy Orbison, among others, all initially signed with Acuff-Rose Music.

Acuff had a brief affair with politics, losing a run for the office of Governor of Tennessee as a Republican in 1948. Acuff later campaigned in 1970 for his friend Tex Ritter in his campaign for GOP nomination for U.S. Senate in Tennessee.

Acuff spent most of the 1950s and 1960s touring constantly, becoming one of the hottest tickets in country music. By the 1970s Acuff performed almost exclusively with the Grand Ole Opry, at Opryland USA, greatly legitimizing it as the top institution in country music.

A popular legend is that Japanese troops during World War II would enter battle yelling, "To hell with Roy Acuff."

Roy Acuff is distantly related to Amy Acuff.

In 1962, Roy Acuff was inducted into the Country Music Hall of Fame. For his contribution to the recording industry, he has a star on the Hollywood Walk of Fame located on 1541 Vine St.