Shelley earned a law degree from the University of San Francisco in 1932. He served in the United States Coast Guard during World War II and was a member of the California State Senate from 1938 to 1946. He ran an unsuccessful race for the Lieutenant Governor's office against Goodwin Knight in 1946. Shelley was entered the House of Representatives in 1949 and served until 1964, when he ran for mayor of San Francisco and won by a 12-point margin against his opponent, Harold Dobbs.

Shelley's term as mayor was filled with challenges, including strikes over discrimatory hiring practices against African-Americans at the Palace Hotel, a public nurse strike in 1966, and a threatened San Francisco Symphony Orchestra strike in 1967. Shelley was mayor during the Summer of Love, a time of radicalism in the Haight-Ashbury and turmoil throughout the city. Shelley was faced with riots in Bayview-Hunters Point on September 27, 1966, after a white police officer fatally shot a black youth accused of auto theft. Shelley declared a state of emergency in the city for 6 days. After the riots ended, Shelley took several public steps to improve relations between city government and the African-American community. He appointed the city and county's first African-American supervisor, Terry Francois. Shelley took an aggressive stance against several prominent anti-development mobilizations during his tenure, including movements in opposition to development at the Yerba Buena Center and in the Western Addition. Shelley bowed out of running for a second term in office; his stated reasons were health-related, but it was thought that prominent political forces in the city's establishment wanted a more stringently pro-development mayor in office.

Shelley's son, Kevin Shelley, was a member of the California State Assembly from 1997-2003 and served as California's Secretary of State from 2003-2005.