Nelson was born in Clear Lake, Wisconsin. In 1939 he received a BA degree from San Jose State College in California and graduated from the University of Wisconsin Law School in 1942. In the same year he enlisted in the U.S. Army and fought in World War II for nearly four years, serving as a First Lieutenant during the Battle of Okinawa. After returning to Madison, Wisconsin, Nelson stood for office in 1946 but was not elected. He married Carrie Lee Dotson in 1947 and practiced law from 1946 to 1958. In 1948 he was elected to the Wisconsin State Senate and ten years later, in 1958, was elected as the 35th Governor of Wisconsin. Four years later, in 1962, he was elected to the US Senate, where he served from January 8, 1963 until January 3, 1981.

Nelson was always passionate about the environment. In 1963 he travelled on the Conservation Tour with President John F. Kennedy and was the founder of Earth Day, the first of which was held in 1970. Nelson left the Senate in 1980 and became counselor for The Wilderness Society in January 1981. He received the Presidential Medal of Freedom Award in September 1995 in recognition of his environmental work.

Nelson also viewed the stabilization of the nation's population as an important aspect of environmentalism. In his words: "The bigger the population gets, the more serious the problems become... We have to address the population issue. The United Nations, with the U.S. supporting it, took the position in Cairo in 1994 that every country was responsible for stabilizing its own population. It can be done. But in this country, it's phony to say 'I'm for the environment but not for limiting immigration.'"

Nelson died on July 3, 2005 of cardiovascular failure at age 89.

The Gaylord Nelson Insititute for Environmental Studies at the University of Wisconsin-Madison is named after Gaylord Nelson in recognition of his love for nature.

Governor Nelson State Park near Waunakee, Wisconsin is also named after him.