Images creating issue

I uploaded images with a specific file name, but had to delete it to make some changes but then I had to change something and so I had to delete those images. Now, when I am uploading the new images with the same file name as that of old image, the old image is still being displayed instead of the updated one. How can I replace the old image with the new one while keeping the same file name? How can I prevent the old image from showing under the same name?

Let me know the method which doesn’t impact the website performance.