Image Captions

If I add a small image at the side of text I can successfully right or left align the image to the paragraph of text. If I now add a caption to the image the left and right alignment is ignored. It appears that a paragraph tag is added to the image and the image appears on a separate line. Anyone else seen this or have a resolution?