The physics term comes ultimately from the cheese. When quarks were hypothesized it was known they come in threes. This made a physicist who happened to have read some of Finnegan's Wake recall a phrase from there, "three quarks for Muster Mark", and he suggested they be called quarks. James Joyce included "quark" in Finnegan's Wake because, I imagine, he thought it was an amusing word and confusable with "quart" -- Finnegan's Wake is all about wordplay and alcohol, which can come in quarts. Joyce was familiar with quark as the dairy product.
I may not have all these details precisely right, but at least this rhymes with the truth.