In 1953, Caro was graduated by the Horace Mann School and in 1957 with a degree in English by Princeton, where he was managing editor of The Daily Princetonian. He also earned a master's degree from Columbia. He began his professional career as a reporter with the New Brunswick Daily Home News in New Jersey. He first achieved some degree of success in his subsequent job with the Long Island, New York tabloid Newsday.

After spending a year as a Nieman Fellow at Harvard University, Caro began work on his first book, The Power Broker, which is both a biography of New York urban planner Robert Moses and a study of Caro's favorite theme, the acquisition and use of power. Not finished until 1974, the work was based on extensive research and numerous interviews, including those with Moses himself. His wife, Ina Caro, functioned as his research assistant. In fact, her master's thesis on the Triborough Bridge stemmed from this work. The Power Broker was a commercial and critical success, winning the Pulitzer Prize in biography and the Francis Parkman Prize, among others.

Following this success, Caro turned his attention to Lyndon B. Johnson. Caro retraced Johnson's life by temporarily moving to rural Texas and Washington, D.C., in order to better understand Johnson's upbringing and to interview anyone who had known Johnson. The work, entitled The Years of Lyndon Johnson, will eventually comprise four volumes. The Path to Power (1982) covers Johnson's life through his failed 1941 campaign for the U.S. Senate. The second volume, Means of Ascent (1989), commences in the aftermath of that defeat and continues through his election to that office in 1948. The third and last published volume, Master of the Senate (2002) chronicles Johnson's rapid ascent and rule as Majority Leader in the Senate and garnered Caro a second Pulitzer Prize. Caro is currently at work on the last volume, tentatively entitled The Presidency.

Caro's books portray Johnson as alternating between scheming opportunist and visionary progressive. Caro argued, for example, that Johnson's victory in the 1948 runoff for the Democratic nomination for the U.S. Senate was achieved through extensive fraud and ballot box stuffing. Caro also highlighted some of Johnson's campaign contributions, such as those from the Texas construction firm Brown and Root; in 1962 the company was acquired by another Texas firm, Halliburton, which became a major contractor in the Vietnam War. In addition, Caro argued that Johnson had falsified reports of winning a medal during his World War II naval service. Despite these criticisms, Caro's portrayal of Johnson also notes his struggles on behalf of progressive causes such as the Voting Rights Act.