Reading between the lines, I think the VN's are telling you only half a story. If the owner didn't care, he wouldn't have took the cat in and he would just sign her over. I really think this needs to checked out properly because in my eyes, people are always complaining about a certain rescue barging in and removing animals without any proper evidence, and I'm sorry but I think this seems the same sort of thing. Okay, he's not been to visit but we don't know why and as mentioned by Gill, he could be scared if they've been threatening all sorts, we just don't know what's going on and are assuming.
The fostering idea is a good one and you could work it that once he's paid, the cat is returned but if he can't afford the bill, what happens then? I would mention to them about him contacting his local RSPCA to help, I know I've said this before but chances are they will help in this situation, our branch has done this numerous times for people on Benefits who have been stung by a large bill.