I always loved Faith No More's cover of "Let's Lynch the Landlord" by the Dead Kennedys, and the entirety of Fantomas' album "The Director's Cut" - especially "Night of the Hunter".
EDIT: ****, I completely forgot about "Why Does the Sun Shine? (The Sun Is a Mass of Incandescent Gas)" and "Istanbul (Not Constantinople)" by They Might Be Giants, too.