Non-overpowered renewable allays

Allays are way too rare to be very useful on sorting systems
On the other hand, they make going to woodland mansions, for example, worthwhile
So how about mixing those up?
My idea would be to be able to convert a vex to an allay using a totem of undying (or golden apple, but it's not rare enough). That would add more questions to the lore of Minecraft as well as keeping allays rare, but renewable if you put in the effort.
The only downside I see is with raid farms, but since they're really late-game it should be OK
Making allays more common will make them usable, so they won't become a forgotten useless mob (like pandas: they are very cute and created a lot of excitement once introduced but nothing on the long run)
What are thoughts about that?