Add downsides to enchantments (swords especially)

Swords are neat, but boring. Since all good enchantments are solely good and have no downsides there is no finess to enchanting. It's just to add everything and then you have a sword which is best in pretty much all situation. Of course you have to choose between the different attack damage enhancers (sharpness, smite and arthropods) and some might not want fire and knockback, but this choice seems to be more down to the individual and not the situation. It would be more fun if there were downsides to enchantments, so that no sword was best at everything, but rather different swords would be good in different situations. 

My suggestion is to add downsides to enchantments. So for instance sharpness and looting might be some sort of opposites by having looting actually decrease the attack damage and sharpness decrease the drops. This would mean that you could choose to have a sword for maximum damage or maximum drops, but you couldn't have it all. I realise this would be a majorly disliked change, but perhaps it could be incorporated in potential future enchantments. 

This would also remove the need for having mutually exclusive enchantments in most situations, which I personally dislike as I find it invasive and obviously done to balance the game (which is good) without it making much sense logically (which is not so good). With downsides, enchantments could instead be implemented so that for example smite enhanced the damage dealt to undead mobs but reduced the damage dealt to all other mobs. This way there would be no need to make these kind of enchantments mutually exclusive, since adding all of them to a sword wouldn't necessarily make that sword overpowered. Depending on the ratio between the upside and the downside of the enchantments a sword with all attack damage enhancers could be made suitably better at slaying every mob compared to a sword without any enhancers; but worse at slaying any specific mob, for instance zombies compared to a sword with just smite.