Wheeler was born in Hudson, Massachusetts. He grew up in Massachusetts, attending the public schools and working as a stenographer in Boston, Massachusetts. He graduated from the University of Michigan law school in 1905. He initially headed for Seattle, Washington, but after getting off the train in Butte, Montana and losing his belongings in a poker game, he settled there and began practicing law.

He became a Montana state legislator in 1910 where he gained a reputation as a champion of labor against the Anaconda Copper Mining Company which dominated the state. He then served as a United States Attorney where he most famously refused to hand down a single sedition indictment during World War I, especially significant as Montana was a large stronghold of the IWW. In 1920 he was chosen to be the Democratic candidate for governor of Montana, but was defeated by Republican Joseph M. Dixon.

Wheeler won election to the United States Senate from Montana in 1922 and served four terms, being reelected in 1928, 1934 and 1940. He broke with the Democratic Party in 1924 to run for vice-president of the United States on the Progressive Party ticket led by Robert La Follette, Sr. He returned to the Democratic Party after the election, which was not successful for the Progressives. Wheeler supported President Franklin Delano Roosevelt's election, and many of his New Deal policies, but broke with him over his opposition to Roosevelt's court-packing schemes.

In 1940, there was a large movement to "Draft Wheeler" into the presidential race, possibly as a third party candidate, led primarily by John L. Lewis.

As tensions mounted in Europe, Wheeler became known for his isolationist beliefs. After the beginning of World War II in Europe, he continued to oppose any aid to Britain or any other countries involved in the war. He became a supporter of the America First Committee. Wheeler did not, however, vote against America's participation in World War II after the Japanese attack on Pearl Harbor. Wheeler sought renomination in 1946 but was defeated by Leif Erickson in the Democratic primary, who was then defeated by Republican Zales Ecton. Wheeler did not return to politics and returned to his law practice. He died in Washington, D.C.

In the novel The Plot Against America (2004) by Philip Roth, Wheeler serves as vice president in the administration of President Charles Lindbergh. Roth depicts Wheeler imposing martial law in Lindbergh's absence, whereas the real Wheeler had been a leading opponent of the martial law imposed in Montana during World War I. Author Bill Kauffman describes Wheeler as being, in fact an "anti-draft, antiwar, anti-big business defender of civil liberties".

The Plot Against America: Senator Wheeler and the Forces Behind Him is also the name of a pamphlet by David George Kin published against Wheeler during the 1946 campaign by supporters of the Communist Party USA, which accused both Wheeler and Harry Truman of a fascist conspiracy.