This is such a cool idea! Maybe you could control the seasons, like with a command via chat/ command block. (Like changing the weather)
I have an idea for this too. Maybe, instead of them only being able to walk around 5×5 block spaces, they could go wherever, and if you wanted them to come back you could use some sort of a whistle...