He was born in Pasadena, California, possibly of part Native American descent and showed a mechanical aptitude early on; while working as a car mechanic he met the director Allan Dwan, who took him on as a camera assistant. Fleming soon rose to the rank of cinematographer, working with both Dwan and D. W. Griffith, and directed his first film in 1919.

Many of Fleming's silent films were action movies, often starring Douglas Fairbanks, or Westerns, and with his robust attitude and love of outdoor sports he became known as a "man's director". But he also proved an effective director of women. Under his direction, Vivien Leigh won the Best Actress Oscar, and Ingrid Bergman was nominated.

In 1932 Fleming joined MGM and directed some of the studio's most prestigious films. Red Dust (1932), Bombshell (1933), and Reckless (1935) showcased Jean Harlow, while Treasure Island (1934) and Captains Courageous (1937) brought a touch of literary distinction to boy's-own adventure stories. His two most famous films came in 1939, when The Wizard of Oz was closely followed by Gone with the Wind. Their fame has outstripped that of their credited director - not unfairly, since both were essentially producer-led projects, and in each case he replaced the original directors after filming had begun. His version of Dr. Jekyll and Mr. Hyde (1941), with Spencer Tracy, was generally rated below Rouben Mamoulian's 1931 version with Fredric March. Fleming's few remaining films were disappointing, and he died from a heart attack soon after completing Joan of Arc (1948) with Ingrid Bergman.