The selected message(s) can be deleted from the current folder by clicking the Delete icon in the toolbar or by hitting the <Del> (or Backspace on Mac) key on your keybaord. This will by default move the message to the Trash folder and not delete it right away unless you do this for messages in the Trash folder. 

In order to immediately delete the message in first place, hold down the Shift key on your keyboard while clicking the Delete button or when pressing <Del>.

There’s a preference in Settings > Preferences > Server Settings where the default behavior of message deletion can be changed.

This section provides more advanced settings that control how messages are treated by the email server.

Mark the message as read on delete

With this option enabled, unread messages are also flagged as read when deleting them right away.

Flag the message for deletion instead of delete

Email messages can be flagged as deleted first before they’re finally removed from a folder. That also allows to “undelete” them later on. In order to finally delete them, use the Compact command from the Mailbox folders operations menu.

Do not show deleted messages

This option suppresses messages flagged as deleted from being listed.

If moving messages to Trash fails, delete them

Moving to Trash can fail if the Trash folder isn’t selected or over quota. With this option enabled, messages are deleted from the current folder when you attempt to move them to Trash.

Directly delete messages in Junk

Messages in the Junk folder are also move to the Trash first when deleting them. Skip that step by enabling this option.

Clear Trash on logout

As the option name says, this will empty the Trash folder when you terminate the session.

Compact Inbox on logout

This will remove all messages flagged for deletion from the Inbox when you log-off.