You can delete a public or private key whenever required.
To delete a key:
Click on the Manage OpenPGP keys link in the Tools area.
Click on the button next to the required key.