Having played drums in his high school band, Vallee played clarinet and saxophone in various bands around New England in his youth. In 1917, he felt that Uncle Sam needed one more brave young man in the Great War, but was discharged when the Navy authorities found out that he was only 15. In 1924-5, he played with the "Savoy Havana Band" in London, England. He then returned to the States to obtain a degree in Philosophy from Yale and to form his own band, "Rudy Vallee and the Connecticut Yankees." With this band, which featured two violins, two saxophones, a piano, a banjo and drums, he started taking vocals (supposedly reluctantly at first). He had a rather thin, wavering tenor voice and seemed more at home singing sweet ballads than attempting vocals on jazz numbers. However his singing, together with his suave manner and handsome boyish looks attracted great attention, especially from young women. Vallee was given a recording contract, and in 1928 started performing on the radio.

Vallee became the most prominent and arguably the first of a new style of popular singer, the "crooner". Previously, popular singers needed strong projecting voices to fill theaters in the days before the electric microphone. Crooners had soft voices that were well suited to the intimacy of the new medium of radio. Vallee's trombone-like vocal phrasing on "Deep Night" would inspire later crooners such as Frank Sinatra to model their voice on jazz instruments.

Vallee also became what was perhaps the first complete example of the 20th century mass media pop star. Flappers (the predecessors of "bobby-soxers") mobbed him wherever he went. His live appearances were usually sold out, and even if his singing could hardly be heard in those venues not yet equipped with the new electronic microphones, his screaming female fans went home happy if they had caught sight of his lips through the opening of the trademark megaphone he sang through.

In 1929 Vallee did his first film, "Vagabond Lover". His first films were made to cash in on his singing popularity, but Hollywood was pleasantly surprised to find that Vallee could act as well. Also in 1929 Vallee started hosting The Fleischmann's Yeast Musical Variety Hour. Performers first introduced to the American public on that program included Jack Benny and Kate Smith.

Vallee would continue hosting popular radio variety shows through the 1940s, including: "The Royal Gelatin Hour, directed by Rudy Vallee", which featured various film performers of the era, such as Fay Wray and Richard Cromwell, in dramatic skits.

Along with his group, "The Connecticut Yankees," Vallee's best known popular recordings included: "The Stein Song" (aka University of Maine fighting song) in the early part of the decade and "Vieni, Vieni" in the latter '30s. A note of trivia: "Vieni, Vieni" can be heard in the background as Jimmy Stewart enters the restaurant in Frank Capra's holiday classic, It's a Wonderful Life. Remarkably for an American, Vallee sang fluently in three Mediterranean languages, and always varied the keys, thus paving the way for later pop crooners such as Dean Martin and Gene Pitney. Another memorable rendition of his is "Life Is Just A Bowl of Cherries", in which he imitates Willie Howard's voice in the final chorus. Vallee was also entertaining in Gershwin's witty "Kitty from Kansas City" and "Betty Co-ed", not to mention the first recording of tipsy laughter in "There is a Tavern in the Town", decades before Elvis Presley was to use that gimmick on stage in "Are You Lonesome Tonight?". His last significant hit song was the reissue of the melancholic ballad "As Time Goes By" from the soundtrack of "Casablanca" in 1946, which he had recorded fifteen years before it was used in this classic movie. During WWII, Vallee performed with the Coast Guard Band, entertaining US troops with this forty-piece orchestra until 1944.

When Vallee took his contractual vacations from his national radio show in 1936, he insisted his sponsor hire Louis Armstrong as his substitute (this was the first instance of an African-American fronting a national radio program). That same year Vallee also wrote the introduction for Armstrong's book "Swing That Music".

Vallee acted in a number of Hollywood films in the 1930s and 1940s. One of his best acting roles is as the millionaire playboy on whom Claudette Colbert relies in the 1942 screwball comedy directed by Preston Sturges, "The Palm Beach Story".

In 1955, Vallee displayed his comedic abilities in Gentlemen Marry Brunettes, co-starring Jane Russell, Alan Young, and Jeanne Crain. The production was filmed on location in Paris. The film was based on the Anita Loos novel that was a sequel to her acclaimed Gentlemen Prefer Blondes. Gentlemen Marry Brunettes was popular throughout Europe at the time and was released in France as A Paris Pour les Quatre ("Paris For The Four"), and in Belgium as Tevieren Te Parijs.

In middle age Vallee's voice matured into a robust baritone. (In his later years he told a collector of his early records that "Everything I did before 1950 you can shit on.") He performed on Broadway in the show "How to Succeed in Business Without Really Trying" and appeared in the film of the same name. He appeared in the campy 1960s Batman television show as the character "Lord Marmaduke Fogg". He toured with a one-man theater show into the 1980s.

His reputation in Hollywood was that of a tightwad, but he wasn't the only one. He was widely suspected of being bisexual, although not much hard evidence exists, and his blatant affair with Clark Gable's wife Carole Lombard stifled the malicious whispers. He was married briefly to the much-younger and sexy actress Jane Greer, but that ended in divorce in 1944. His previous marriage to Leonie Cuachois was annulled and the one to Fay Webb ended in divorce. After divorcing Jane Greer he married Eleanor Norris in 1946, who wrote a memoir, My Vagabond Lover. Their marriage lasted until his death in 1986.

Rudy Vallee died on July 3, 1986 at the age of 84, and he was interred in St. Hyacinth's Cemetery, Westbrook, Maine.