He was born in Edinburgh into a family of tailors. He was an elocution lecturer at New College, Edinburgh University from 1925 until 1930, and would later be rector from 1948 until 1951. He married Naomi in 1932 and they remained together until his death in 1976. Sim was always ambivalent about fame and never signed autographs.

Preferring the stage, Sim made his London début in Othello in 1930. He also appeared for a season at the Old Vic. He made his film debut in The Case of Gabriel Perry (1935). He spent the remainder of the decade playing supporting roles in films, and was often credited as "stealing the scene" from the leading actors. By the 1940s he had progressed to leading roles, and in 1950 he was voted the most popular film actor in Britain in a national cinema poll.

His films include Green for Danger (1946), Waterloo Road (1947), Alfred Hitchcock's Stage Fright (1950), Laughter in Paradise (1951), Folly to be Wise (1953), The Belles of St. Trinian's (1954) and An Inspector Calls (also 1954). He played the headmistress Miss Fritton (and her brother Clarence) in the St. Trinian's series. He portrayed Captain Hook in six different productions of Peter Pan between 1941 and 1968. Probably his best-remembered performance, however, was as the title character in Scrooge, the 1951 adaptation of Charles Dickens' A Christmas Carol. (In 1971 Sim revisited the Scrooge character by lending his voice to an Academy Award-winning animated version of Dickens' story.)

Sim's performance as Mr. Squales in London Belongs to Me (1948) so impressed Alec Guinness that he based his own performance in The Ladykillers (1955) on it.

Sim was made a Commander of the British Empire in 1953. Prior to his death, he played the judge in the popular television series Misleading Cases by A. P. Herbert. He died in London, England from cancer.