Buffy I wouldn't worry abut your babies when you are at work. As the slave to 3 kitties and having lived with cats all my life and can guarantee that as soon as you go out in the morning your little ones will probably follow ALL OR AT LEAST MOST OF this routine:
1. Wander round the house and check everything is in order.
2. Have a bit of food and a drink
3. Have a bit of play.
4. Find somewhere cosy, have a bit of wash, curl up and go to sleep
5. Sleep
6. Stretch ... sleep
7. Twitch .... stretch .... SLEEP (are you seeing the way this is going?)
8. Approximately 2 mins before you get home ... wake up, stretch and practise the "Where have you been all day" look.
Cats are VERY good at giving you a look that makes you think they have been waiting all day for you to come home ... however, in reality they are quite happy that you haven't been at home so it doesn't inturrpt the important work of SLEEPING!!!
Don't feel guilty about leaving them. They are fine
On several occasions I have come home at lunchtime, or finished work a bit early and got home an hour before my usual time and I don't hide or hair of my gang until 5:20 when they all seem to wake up and come downstairs. I usually walk through the door at about 5:25 / 5:30'ish.