He was born in New York City, New York, son of a German-born Jewish father and an Irish Catholic mother, in whose faith he was raised. He graduated from Williams College in Williamstown, Massachusetts in 1951. While serving as an Air Force Lieutenant during the Korean War, Frankenheimer became interested in directing. He directed service films for the Air Force and after being discharged, went onto an interview for a TV station. According to Frankenheimer on a profile from The Directors Series, the interviewer said that they weren't hiring at the moment, but if needed, Frankenheimer would be the first one they called. Frankenheimer said that he spent two weeks locked up inside his hotel room, only going out for food and unable to rely on the hotel's messaging service that he received a call. It was then that Frankenheimer began directing live television productions. He directed over 140 films including one that starred Ronald and Nancy Reagan.

Frankenheimer directed his first theatrical film, The Young Stranger, in 1957, but did not direct another until 1961. 1962 was a pivotal year for the young director, he directed three films All Fall Down, Birdman Of Alcatraz and The Manchurian Candidate that were all box office hits. The Manchurian Candidate (based on the 1959 novel of the same title by Richard Condon) is the director's best known work and was recently named as one of the top 100 films of all time. It was pulled from circulation but re-released to great acclaim in 1988.

Frankenheimer directed Seven Days in May in 1964 and The Train in 1965 that were also well received along with Grand Prix, and Seconds, both in 1966 and The Fixer. in 1968. However, his career took a downward spiral shortly thereafter. Frankenheimer had been a close friend of Senator Robert Kennedy and in fact drove him to the Ambassador Hotel in Los Angeles the night he was assassinated in June 1968. It was this traumatic incident and a disillusionment with mainstream filmaking that led to the director developing a serious problem with alcohol that he was eventually able to overcome. At that time, Frankenheimer was also experiencing a messy divorce with his first wife that caused the actor to clean up and move to France where he immersed himself in the language and enrolled in cooking school. This later lead to Frankenheimer directing French Connection II, which was set in Frankenheimer's adopted home country at the time.

Frankenheimer was able to make a comeback in the 1990s by returning to the medium of television. He directed two films for HBO in 1994: Against The Wall and The Burning Season that won him a slew of awards and renewed acclaim. The director also helmed two films for Turner Network Television in 1996 and 1997, Andersonville and George Wallace that were highly praised. He even appeared for the first time onscreen as a desperate U.S. General in The General's Daughter (1999) in a crucial cameo appearance. Frankenheimer's last film was Path To War for HBO in 2002. He was scheduled to direct a prequel to The Exorcist but died suddenly in Los Angeles, California, from a stroke due to complications following spinal surgery at the age of 72, shortly before filming started.