It isn't only feral cats, or ones with baggage who are difficult to socialize. I had a Russian Blue kitten before Trigger, who never came to me, or showed much affection. He spent all day outside with a neighbour's 5 cats, and at night, when I closed the cat flap, he would sit for hours perched up high on a partially open window. before eventually retreating to the top of the wardrobe.
He wasn't timid or nervous - he just had no interest in being indoors with a person. He came to me at 13 weeks direct from his breeder, so it was not as if he had learned not to distrust humans. I was just unlucky with him, I suppose.
These two are reacting to you, even if not in the way you want at the moment. I can see how frustrating it is, but the rewards, when they do start to come round, are going to be that much greater. The important thing, I think, is to keep to a routine and avoid any major traumas with them (or what they would see as a major trauma). If they got startled outside they might run off. But I guess this is something you will have to risk eventually. Can you make part of your garden cat proof, temporarily?