He was born in St Louis, making him the only main Stooge member not born on the East coast. He was the ninth child of Morris and Fanny Besser (Jewish immigrants from Eastern Europe). He had seven older sisters, and an older brother Manny who was in show business. From a young age, Joe was also fascinated with show business, especially the magic act of Howard Thurston that came annually. For years, he asked Thurston if he could join the act, and Thurston kept telling him to wait until he was older. When he was twelve, Thurston allowed Besser to play a minor audience plant (as in, "stooge"). Besser was so excited by this, he snuck into Thurston's train after the St. Louis run of the show was over, and was discovered the next day sleeping on top of the lion's cage in Detroit.

Thurston gave in, informed Besser's parents of the situation, and trained him as an assistant. The first act involved pulling a rabbit out of a hat. The trick involved two rabbits, one hidden in a pocket of Thurston's cape. But young Besser was so nervous that he botched badly, pulling out the rabbit from the cape at the same time as the other rabbit was on display, before the trick had been performed. The audience roared with laughter, and Besser from then on was assigned comic mishap roles only.

Besser ended up in vaudeville, doing comedy in general, and was a headliner. He specialized in playing a bratty, whiny child, and his catchphrases "You crazy you!" and "Not so fast!" became well known.

In 1932, he married the dancer Erna Kay, real name Erna Dora Kretschmer, known as "Ernie". They were neighbors and friends of Lou Costello, of Abbott and Costello fame. He was in demand in movies, radio, and eventually television. One of his most famous roles was "Stinky", the bratty child dressed in an oversized Little Lord Fauntleroy outfit, shorts, and a flat top hat with overhang. He played that for the first season of The Abbott and Costello Show.

When Shemp Howard died of a sudden heart attack in November 1955, Moe Howard recruited Besser to join the Three Stooges. He agreed, but had a clause in his contract prohibiting being hit excessively, insisting that his humor was more about comedic revenge for being bullied. In 1958, Columbia Pictures ended their "shorts" film making, and the Three Stooges were without a contract. In addition, Ernie had suffered heart attack. Besser was unwilling to leave his wife, and quit the Stooges.

Besser returned to films and television, most notably as the superintendent "Jillson" for four seasons (1961 - 1965) of The Joey Bishop Show, and the voice of Babu the genie in the animated I Dream of Jeannie series. He also made occasional appearances on the ABC late-night series, also called The Joey Bishop Show between 1967 and 1969.

The majority of Stooge Fans tend to turn a blind eye to the Besser-era shorts. This coldness cannot be lain at Besser's feet alone. Though his type of childish comedy did not fit the Stooge mold comfortably, the truth was that Moe and Larry were older, and the constant profusion of recycled scripts and stock footage hurt the quality of Besser's Stooge output. A minor contingent of Stooge fans feel Besser brought his own (quite amusing) character to the role and feel he did right in not trying to replicate Curly or Shemp, who he met with the other Stooges in 1932 and had especially formed a friendship with.

Considered a kind man, Besser enjoyed meeting fans of all ages as well as giving his time to the children of his community. He died of heart failure on March 1, 1988.