He won an Academy Award for Best Supporting Actor in 1972 for his performance as the Master of Ceremonies in Cabaret. HIs victory was part of a "Cabaret" sweep, which saw Liza Minelli win Best Actress and Bob Fosse win Best Director. Grey's and Fosse's victories were surprising considered the juggernaut of that year, "The Godfather." Grey beat front-runner Al Pacino for Best Supporting Actor while Fosse shocked Francis Ford Coppola for Best Director, a loss Coppola would make up for two years later.

He is one of only seven people who have won both a Tony Award and an Academy Award (Oscar) for the same role. He is the father of actress Jennifer Grey, the star of the huge box office hit "Dirty Dancing," and the son of entertainer Mickey Katz.

In the 1996 revival of Chicago, Joel played Amos Hart. Joel recently appeared in the Broadway musical Wicked as the Wizard himself, opposite Idina Menzel and Kristin Chenoweth. He was also one of the first people to guest star on the Muppet Show in its first season singing Razzle Dazzle and Cabaret.

He has had recurring television roles on Buffy the Vampire Slayer (as Doc, 2001) and Alias (as Arvin Clone, 2005). Grey played the role of a demon in the final episode of Dallas.