Unable to Swap Artifacts Occasionally in Quick Inventory Menu

As the title says.

e.g.

First artifact holds Witch Brew, but the game won't allow me to select the icon in the quick inventory menu. It just seemingly ignores the input, even when theres no cooldown.

Small detail I noticed:
Going to the tailsman under the bugged artifact and pressing "up" results in the header showing the name of the artifact that WOULD'VE been selected, while not actually moving the cursor.