Presuming they are struvite crystals, they tend to form in urine that is too alkaline - the best way to make the urine more acidic as it should be - is to feed wet food (with no cereals, grains, veggies - or other unnecessary fillers) with a high meat percentage if poss, or even a raw diet (properly balanced) will keep the urine at the correct pH.
My Ben had crystals and ultimately went on to block several years ago no, I didnt know so much about diet then and went along with the presciption diet (dry biscuits) recommended by vet - well he wouldnt eat it - so I did my research and immediately changed all cats onto raw diet (well not immediately, but did the change over a week or so) no more dry (except as treats) and we never had any more prob with him. He now eats raw, various tinned (none with cereals etc) and probably a small handful of dry as a bedtime treat.
If you can do timed meals, rather than leaving dry out, that would help, then your others could have their dry at a later meal, while JO Ripley is having his plate of wet food perhaps.
By the way, stress can also be a factor in crystals forming, so I'd get some Feliway on the go if you havent already
