Already in game we have raids with pillagers. I'd like to expand on this.  At pillager towers there could be an item that spawns in the tower.  If you take this item and bring it back to a village and display it somewhere, raids will randomly spawn to try to "capture" this item back.  The Ravager or perhaps a new mob could be given the ability to break through blocks to create a real challenge.  They would only spawn in these types of raids so you don't have to worry about the game griefing your builds and you'd have to prepare for that.

New weapons and other items could be created to help defend such as a Cheval de frise. 

Each time the raid fails/you win your village will gain a level.  The higher the level of your village the higher the level of the next raid.  If you lose the raid your village is set back to level 0.

New mobs could be added to give more of a challenge to combat common defense set ups. Perhaps better mob AI to avoid easy traps.

See how high you can get your village level.