Hersh's music is known for its creative chord chemistry, hypnotic sonic treatments, and a vocal style ranging from softly melodic singing to impassioned screaming. Some of her signature contributions to popular music include addressing the complexities of marriage and motherhood through impressionistic, sometimes hallucinatory lyrics about everyday feelings and varying mental states. A few of her numerous songwriting subjects have included childbirth ("Hysterical Bending"), love ("Tar Kissers", "Lavender"), surreal vignettes ("Delicate Cutters", "Fish"), death ("Limbo"), emotional anguish ("The Letter"), loss of custody of her first son ("Candyland"), and the shedding of a relationship's angst or anxiety ("Snake Oil").

Hersh has also sprinkled her lyrical canvas with concrete images from nature and the ordinary physical realm, such as apples, water, diamonds, eyes, the sea, snow, ice, rain, fire, the sun, sand, and cowboys. On occasion she has used historical figures like anorexic suicide Ellen West as metaphors in depicting a state of mind ("Ellen West"). Eccentric characters encountered in her family's travels have made occasional appearances in songs such as "Ruthie's Knocking"; a 2005 live solo set list included a then-untitled song about a "parrot lady" character she happened upon while visiting Lake Michigan.

In the past, Hersh has listed among her early musical influences The Raincoats, Talking Heads, Violent Femmes, Meat Puppets, Dead Kennedys, Hüsker Dü, Velvet Underground, R.E.M., and X. She has said her parents' albums by Patti Smith, the Carter Family, Stevie Wonder, Robert Johnson, Talking Heads, The Clash, Steve Miller, The Beatles, Philip Glass, and traditional music influenced her when she was growing up.