Conrad was born in Bismarck, North Dakota to German-American parents and lived much of his early life in Bismarck. Orphaned at a young age, he was raised by his grandparents. He attended Roosevelt Elementary and Hughes Junior High, and several years of high school in Tripoli, Libya. He graduated in 1966 from Phillips Exeter Academy, like his opposite on the Senate Budget Committee, Chairman Judd Gregg. He went to college at Stanford and received an MBA from George Washington University.

After graduating college, he became an assistant to the North Dakota tax commissioner, who later became his colleague in the Senate, Byron Dorgan. In 1980, Conrad succeeded Dorgan as tax commissioner. He was tax commissioner until 1986, when he ran for Senate.

In the 1986 election, Conrad defeated the Republican incumbent, Mark Andrews. During the campaign Andrews gave a pledge that he would not run for re-election if the Federal Budget deficit had not fallen by the end of his term. By 1992 it became obvious that this would not be the case, and although polls showed that the electors would have welcomed him going back on his pledge, Conrad considered it binding and stood down. Dorgan won the primary election to succeed him as Democratic candidate.

Conrad got an unusual opportunity to retain his senate position when the other North Dakota senator, long-serving Dem-NPLer Quentin N. Burdick, died on September 8, 1992. Burdick's widow, Jocelyn Birch Burdick, was appointed to that seat temporarily, but a special election was needed to fill the rest of the term. As this was not running for re-election, Conrad ran for and secured the Democratic-NPL nomination. He won the election and was sworn in December 5, 1992, resigning his other seat the same day. (The first seat was then filled by Dorgan, who was appointed by the governor to fill the seat for the brief interim until he would have been sworn in under normal circumstances.) Conrad was elected to another full term in 1994 and reelected in 2000. He was chairman of the Senate Budget Committee from 2001 until 2003, and still serves on that committee as the ranking Democrat.

Conrad has been very vocal in his opposition to the spending policies of the Bush Administration. He contends that they have worsened the problems of national debt. Conrad is more moderate than many in his own party on the issues of abortion, gun control, and gay rights. He is one of the few members of the Democratic Party to have voted consistently in favor of banning the procedure commonly referred to as "partial-birth" abortion. He also opposes public funding of abortion. On January 31, 2006, Conrad was one of only four Democrats to vote in favor of confirming Judge Samuel Alito to the Supreme Court. He is opposed to most free-trade measures and is a strong supporter of farming subsidies. Conrad voted against approving use of military force in Iraq in 1991 and was one of only 23 senators to vote against the war resolution of 2002.

In April 2006, he was selected by Time as one of "America's 10 Best Senators."