A former staff writer for PolyGram Publishing in Nashville, Pettis' musical career was started in 1979 when Joan Baez covered one of his songs, "Song at the End of the Movie", on her album Honest Lullaby. Following that release, Pettis became heavily involved in the "Fast Folk movement" in New York in the 1980s alongside artists such as Shawn Colvin and Suzanne Vega.
In 1984, Pettis released his first independent solo album, Moments, followed by a string of releases for High Street Records; While the Serpent Lies Sleeping, Tinseltown and Chase the Buffalo. None of these releases have made Pettis a household name, but his music has become extremely popular with other artists. His songs have been covered by artists like Dar Williams ("Family" on Mortal City), Garth Brooks ("You Move Me" on Sevens), Dion & the Belmonts, Sara Groves, Randy Stonehill and others.
In the early 1990s, Pettis developed a working relationship with songwriter and producer Mark Heard. Pettis was performing with Heard at the Cornerstone Festival in July of 1992, when Heard had a heart attack on stage. Heard finished the show, but collapsed off-stage afterwards and died the following August. That loss led to the start of a tradition for Pettis, who has started off each of his albums with one of Heard's songs since 1993's Chase the Buffalo, where he recorded Heard's "Nod Over Coffee". Other Heard covers include "Satellite Sky" (1996's Making Light of It), "Tip of My Tongue" (1998's Everything Matters), "Rise from the Ruins" (2001's State of Grace) and "Another Day in Limbo" (2004's Great Big World.) Pettis also appeared on a 1994 tribute to Heard entitled Strong Hand of Love.
In 1996, Pettis signed on with Compass Records and released Making Light of It, produced by David Miner (T-Bone Burnett, Elvis Costello), and featuring Derri Daugherty and Steve Hindalong of The Choir.
Two years later Everything Matters was released, produced by Grammy award winning Gordon Kennedy (who is best known for co-writing Eric Clapton's "Change the World.") Pettis followed that release with State of Grace in 2001, which featured cover art by the late southern folk artist Howard Finster. .