It is tricky, because I can imagine what you feel. I personally believe cats should have free outdoors access, they love to wander, hunt and explore, all those things that are natural to them. I do not believe in keeping cats indoors , but I totally understand why people do, and I know that cats can be just as happy being indoors/cat proofed garden cats.
At the end of the day it is such a personal choice, and you just got to do what you feel in your heart is the best for your puddy paw. All our circumstances are different, and so are our cats!