All in on MatMul? Don’t Put All Your Tensors in One Basket!