Duffy was born and grew up in Manchester, England. He has Irish and Jewish heritage and ancestry. He began playing the guitar at the age of fourteen, being influenced by the music of Queen, Thin Lizzy, the Who, Aerosmith, Blue Öyster Cult, and the early work of Led Zeppelin. In the late 1970s he became involved in the punk movement, influenced by the New York Dolls, the Stooges, Buzzcocks, the Sex Pistols, and AC/DC. He started playing lead guitar with a number of different punk rock acts whilst still in school in the late 1970s, including the Studio Sweethearts.