Code Errors when viewing archive of catagory

Hi, My wordpress site is up and doing just fine besides one thing. I have only one post on it so far, and it shows up as intended on the home page and when I click on the post link.

However when I click in the archive of the post I get brought to a page with an excerpt of the post but preceding the post’s text I see the following code:

[caption id="attachment_24" align="alignleft" width="270" caption="sitename"][/caption]

Anyone know what causes this and how to fix it? By doing a google search this code comes up on alot of peoples wordpress sites.

Any help appreciated it.
Thanks!