If we have Custom Errors turned off however, it redirects to the Customers 'My Account' page as it is supposed to.
Is it supposed to? Do you mean you modified it to do that? Because it does not as standard. It should show a text summary of the order on either callback or checkoutcomplete, depending on the payment system.