Change Icon?

Once you have selected an icon, how do you change it if you want a different one? There are no options to change the icon I can see in either the contextual strip or the block sidebar. Do I really have to delete the block and place a new one just to change the icon??