Born in Watertown, Wisconsin, he played infield for sixteen seasons in the major leagues with the New York Giants, Brooklyn Robins, and Chicago Cubs of the National League, and after playing in the International League from 1921 to 1925 he appeared in eight games with the New York Yankees of the American League before retiring in 1926.

He is best remembered for a baserunning error, which led to one of baseball's greatest controversies, while playing for the Giants in a game against the Chicago Cubs at New York's Polo Grounds on September 23, 1908 that cost his team the pennant. It would forever be known as the "Merkle Boner", and Merkle himself would be dogged by the unflattering (and somewhat unfair) appellation of "Bonehead".

With the score tied in the bottom of the 9th inning, with two outs, and Moose McCormick on first base, Merkle singled, advancing McCormick to third base. The next batter, Al Bridwell, also singled, apparently allowing McCormick to score and ending the game as a victory for the Giants. The fans in attendance subsequently stormed the field--in celebration, but also because the exit was located at the center field wall.

However, Cubs second baseman Johnny Evers noticed that Merkle, thinking the game was over, walked to the Giants' clubhouse without touching second base. Evers shouted for the ball (it was relayed to him from the Cub dugout), touched second, and appealed to umpire Hank O'Day to call Merkle out. By the strict letter of baseball rules, Merkle's not touching second meant that when Evers did, Merkle was out on a force play, and that McCormick's run did not count.

What Merkle had done was actually common practice at the time for players in games ending in that fashion. In previous similar situations, there had been other appeals to umpires from the losing side to apply the rules strictly against the winning team, declare the negligent baserunner out, and thus nullify the would-be winning run. Those previous appeals were denied, including, ironically, an occasion earlier in that same season between Evers and O'Day. On that previous occasion, O'Day called the runner safe. But this time, perhaps being more prepared after the previous incident and with the magnitude of the situation, O'Day enforced the letter of the law upon Merkle and the Giants.

For his part, Merkle maintained until his death that he had only started to walk off the field without touching second base, but that he had realized his error, turned around, and touched second after all. The umpires did not see it that way, however, and ruled him out.

With the run nullified, the Giants' victory was erased and the score of the game remained tied. However, the game could not be continued because by the time that ruling was made, there were thousands of fans on the field. Consequently, the umpire ruled the game a tie. When the Cubs and Giants finished the season tied, the game was replayed. The Cubs won this makeup game and thus the National League pennant.

Nevertheless, Merkle, who was only 19 years of age at the time, was able to deal with the publicity and pressure and went on to a successful baseball career.

Fred Merkle died in Daytona Beach, Florida at age 67 and was interred there in Bellevue Cedar Hill Memory Gardens.