How to edit widgets html

Fairly new to WordPress and I’ve come to a fork in the road when trying to edit the sidebar widgets html.

My sidebar.php in the theme has;

<div id="Sidebar">
<?php 	/* Widgetized sidebar, if you have the plugin installed. */
		if ( !function_exists('dynamic_sidebar') || !dynamic_sidebar() ) : ?>
<?php endif; ?>
</div>
<!-- end Sidebar -->

I have added widgets: search, categories, archives links and recent posts.

By default, the widgets are setup in a ul / li and I wanted to edit each individual widget to display something like this;

<div id="search">Search...</div>

      <div id="recentPostHds">Recent Posts</div>
      <div id="recentPostDwn">Content for  id "recentPostDwn" Goes Here</div>
      <div id="recentPostBtm"></div>

      <div id="categoriesHds">Categories</div>
      <div id="categoriesDwn">Content for  id "categoriesDwn" Goes Here</div>
      <div id="categoriesBtm"></div>

      <div id="archivesHds">Archives</div>
      <div id="archivesDwn">Content for  id "archivesDwn" Goes Here</div>
      <div id="archivesBtm"></div>

      <div id="linksHds">Links</div>
      <div id="linksDwn">Content for  id "linksDwn" Goes Here</div>
      <div id="linksBtm"></div>

Where can I edit each individual widget to output the above html? I don’t mind hard coding the core.

Thanks in advance!