Born in Orange, New Jersey, Scheider was an athlete as a child, participating in organized baseball and boxing competitions. He traded his boxing gloves for the stage, studying drama at both Rutgers University and Franklin and Marshall College, where he was a member of Phi Kappa Psi Fraternity. After three years in the United States Air Force, he appeared with the New York Shakespeare Festival, and won an Obie Award in 1968.

Scheider's first film role was in a forgettable horror film in 1963 called Curse of the Living Corpse (he was billed as "Roy R. Sheider"). In 1971 he appeared in two highly popular movies, Klute and The French Connection, the latter garnering him an Oscar nomination for Best Supporting Actor.

In 1975, Scheider starred in Jaws, and in 1978, its first sequel. In 1979, he broke the tough-guy stereotype he had found himself in by starring as musical theater director Joe Gideon in Bob Fosse's autobiographical movie All That Jazz, for which he was nominated for an Academy Award for Best Actor.

He was originally cast as Michael in The Deer Hunter, as the second movie of a three movie deal with Universal Studios. Because he did not believe that the character would travel around the world to find his friend, he quit the picture. Universal executives were furious, but they agreed to let him out of his Universal contract if he made Jaws 2, which he did.

In 1993, Scheider signed on to star in the Steven Spielberg-produced television series seaQuest DSV. After the first season, Scheider voiced disdain for the direction in which the series was heading. His comments were highly publicized and the media criticized him for critizicing his own show. NBC made additional changes in casting and writing in the third season, and Scheider decided to exit the show. His contract however, required that he make several guest appearances in season three.

Scheider went on to star in films such as The Myth of Fingerprints (1997) and Silver Wolf (1998). He has also repeatedly guest starred on the NBC television series Third Watch. Among his most recent films is the crusty father of hero Frank Castle in The Punisher (2004).

In 2004, Scheider was diagnosed with multiple myeloma, a cancer of the plasma cells. In June 2005, he underwent a bone marrow transplant to successfully treat the cancer.