Pretty Girls Make Graves formed in Seattle in 2001, named after The Smiths song of the same name. Andrea and Derek had played together previously in the Hookers, as well as the Death Wish Kids and Area 51 along with Dann Gallucci, with whom Derek had formed Murder City Devils. Not long before the Murder City Devils disbanded, Derek and Andrea formed Pretty Girls Make Graves along with Jay, Nick and Nathan.
The phrase "pretty make graves" was originally coined by Jack Kerouac in his 1958 book The Dharma Bums in referrence to his then Buddhist lifestyle of celibacy.