Born Rose Marie Mazetta in New York City to Italian-American Frank Mazzetta and Polish-American Stella Gluszcak, she became a performer at the age of three. At five she became a radio star on NBC and made a series of films. She was such a strong and convincing blues singer that there was a controversy over her actual age. Some people thought that NBC was trying to pass a midget off as a young girl. But indeed she was only a child. As a teenager she became a nightclub performer before becoming a comedian. She was known then as The Darling of the Airwaves.

After appearing for many years on The Dick Van Dyke Show (in the role created by Sylvia Miles in the pilot episode), Rose Marie co-starred on The Doris Day Show, and later had a semi-regular seat on the 1970s version of Hollywood Squares, along with her friend and The Dick Van Dyke Show co-star Morey Amsterdam.

Rose Marie and Morey Amsterdam guest-starred together in a February 1996 episode of the NBC sitcom Caroline in the City, shortly before Amsterdam's death in October of that same year. She appeared with the surviving Dick Van Dyke Show cast members in a 2004 reunion special.

As of 2005, she continues to perform.