What kind of condition is this cat in? Does it look well-fed? If so, it's probably on the scrounge. On the other hand, if it seems thin and neglected, it may need the food you are giving it. It may be best to have a word with your neighbour. Maybe it's coming to you when your neighbours are out. Do they shut it outside?
Some cats are experts at the 'I haven't been fed for a month' look. Mine used to do it with one of my neighbours. When they were home, they worried me by not eating. When I realised what was happening, a chat with my neighbour sorted the problem out.
A friendly chat would be recommended. If the cat is being well-fed and well cared for; then shooing it away may be the best move.