Anderson was born in Long Beach, California, to a Mormon family of Swedish-Danish descent. He grew up in Salt Lake City, Utah. He served two years as a missionary in the church's Southern States Mission. His writing career began at his local newspaper, The Murray Eagle. He joined The Salt Lake Tribune in 1940, where his muckraking exploits included infiltrating polygamous Mormon fundamentalist sects. He served in the U.S. armed forces during World War II in China, where he reportedly fought the Japanese alongside Chinese guerrillas and worked on the Shanghai edition of Stars and Stripes.

After a stint as a war correspondent during 1945, he was hired by Drew Pearson for the staff of his column, the "Merry-Go-Round," which Anderson took over after Pearson's death in 1969. In its heyday, Anderson's column was the most influential and widely read in the U.S.; published in nearly a thousand newspapers, he reached an audience of 40 million.

He co-founded Citizens Against Government Waste with J. Peter Grace in 1984.

Anderson feuded with FBI chief J. Edgar Hoover in the 1950s, when he exposed the scope of the Mafia, a threat that Hoover had long downplayed. Hoover's retaliation and continual harassment lasted into the 1970s. Hoover once described Anderson as "lower than the regurgitated filth of vultures."

Anderson told his staff, "Let's do to Hoover what he does to others." Anderson had his people go through Hoover's garbage, which is the kind of thing FBI people used to do. Anderson's investigations were a tipping point in the attitude of the public and the press towards Hoover. Prior to Anderson's exposés few people of stature had dared to publicly criticize Hoover. After Anderson, many followed suit, and the man who had been the public persona of exemplary law enforcement became exposed for his failures and questionable activities in the areas of organized crime and civil rights, many of which were of questionable legality.

Anderson grew close to Joseph McCarthy, and the two exchanged information from sources, but when Pearson went after McCarthy, Anderson reluctantly followed at first, then actively assisted with the eventual downfall of his one-time friend.

In the mid-1960s, Anderson exposed the corruption of Senator Thomas J. Dodd, unearthed a memo by an ITT executive admitting the company paid off Richard Nixon's campaign to stymie anti-trust prosecution. His reporting on Nixon-ITT corruption earned him a place on the master list of Nixon political opponents.

Anderson collaborated with Pearson on "The Case Against Congress," published in 1969.

According to the family jewels documents, in 1971, during the Indo-Pakistani War, the director of the CIA, Richard Helms, put Anderson under tap. Jack Anderson had written two articles on assassination attempts on Castro through John Roselli.

In addition to his notable discoveries and breaking coverage, Anderson's reporting has included several mistakes and compelled retractions. Notably, during the 1972 presidential race, he accused Democratic Vice-Presidential nominee Thomas Eagleton of multiple drunk driving arrests. Anderson subsequently retracted the accusations.

In 1972, Anderson was the target of an assassination plot in the White House. Two Nixon administration conspirators admitted under oath they plotted to poison Anderson on orders from a senior White House aide. White House "plumbers" G. Gordon Liddy and E. Howard Hunt met with a CIA operative to discuss the possibilities, including drugging Anderson with LSD, poisoning his aspirin bottle, or staging a fatal mugging. The plot was aborted when the plotters were arrested for the Watergate break-in. Nixon had long been angry with Anderson, blaming Anderson's election eve story about a secret loan from Howard Hughes to Nixon's brother "Getting the Scoop" by Mark Feldstein for Nixon's loss of the 1960 presidential election. Anderson remained a target of FBI investigation after his death; in February 2006, the FBI contacted Anderson's family to obtain his files and search for classified documents. The FBI agents claimed to be looking for documents pertaining to American Israel Public Affairs Committee (AIPAC) as part of an espionage investigation. In November 2006, the FBI quietly gave up its pursuit of the archive. The archive, as revealed in The Chronicle of Higher Education, contains Anderson's CIA file, along with information about prominent public figures such as Richard Nixon, Ronald Reagan, Thomas Dodd, and J. Edgar Hoover.

To demonstrate the threat of terrorism within the US Capitol, Anderson once brought a gun to an interview in the office of then minority leader Bob Dole. This led to a reprimand and a change of rules for reporters.

Anderson had a staff of legmen on his payroll, who earned little but gained valuable experience. Among Anderson's legmen — reporters who actually went out into the field and gathered the information, forwarding it on to writers such as Anderson — was Brit Hume, later a reporter for ABC News and Washington managing editor for Fox News Channel.