To delete your podcast, go to the All Podcasts section of the Content Management System, choose the podcast and click the yellow EDIT button. From the left-hand-side menu, click on Delete Podcast. Then follow the prompts that appear.
If your podcast contains episodes, you will need to delete the episodes first.