McElroy grew up in Brooklyn Heights, NY, a neighborhood that features prominently in much of his fiction. He received his B.A. from Williams College in 1951, and his M.A. from Columbia University in 1952. He served in the Coast Guard from 1952-4 and then returned to Columbia to complete his Ph.D. in 1961. He retired from teaching in 1995 after thirty-one years in the English department at Queens College, New York, City University of New York.

McElroy's writing is often grouped with that of William Gaddis and Thomas Pynchon because of the encyclopedic quality of his novels, particularly the 1191 pages of Women and Men (1987). Echoes of McElroy's work can be found in that of Don DeLillo and David Foster Wallace. McElroy's work often reflects a preoccupation with how science functions in American society; Exponential, a collection of essays published in Italy in 2003, collects science and technology journalism written primarily in the 1970s and 1980s for the New York Review of Books.

He has received the Award in Literature from the American Academy of Arts and Letters and fellowships from the Guggenheim, Rockefeller, and Ingram Merrill Foundations, and the National Endowment for the Arts.

Novels A Smuggler’s Bible (1966) Hind’s Kidnap (1969), Ancient History: A Paraphase (1971) Lookout Cartridge (1974) PLUS (1977) Women and Men (1987) The Letter Left to Me (1989) Actress in the House (2003)