Archive thumbnails?

Hi,
I’ve searched like crazy for this, but I’m not even sure I’m searching for the right terms. Is there a way to have a thumbnail show next to a post when it’s listed by category? I thought this was considered archives, but maybe not? Here’s a link to a category list on my page.
http://www.mycrappyhouse.com/category/chronological/

I’d like to have a thumbnail picture show up for each post, but I can’t figure out how. I was hoping it would be as simple as adding a plugin, but I haven’t found one to do what I want.