It shouldn't be too difficult to dispute as it is up to the merchant to prove that they have authorization to charge you. However, if you refuse to assist the bank in its investigation / rectification of the issue, they can adversely effect your credit rating and likely sue you.
we had something similar happen to us - we went to the credit card company, asked what was going on? Who in particular gave the authority to use the new number - this is traceable and they were held accountable. It was reversed and they haven't done it since.