In the world of Git, tags are very useful for keeping track of your project's version history. A lot of folks will argue that you shouldn't delete tags, but there are real-world examples in which tags need to be deleted. That said, it's both a good and a bad thing that GitHub hasn't built the ability to delete tags into it's web app.
It can be frustrating, however, if you ever come across that need. If you're one of those individuals, open up a terminal window and navigate to your local Git repository. Once there, simply run these commands, replacing [tag] with the tag name:
git tag -d [tag]; git push origin :[tag]
And if your tag has the same name as one of your branches, use this instead:
git tag -d [tag] git push origin :refs/tags/[tag]
Of course, these steps assume that you have Git running on your local machine.
Update August 8, 2016: GitHub now lets you delete releases from its website, but this will not delete the tag:
As of right now, to delete the actual tag, you need to use the command line.