Once attachments are activated on the server and Outlook on the client is restarted you should see an extra pane selector in the Activity window:
In this pane you can see all attachments currently associated with the Activity, as well as add a new attachment ("Browse"), view an attachment ("Display") or delete an attachment. Due to limitations in the SAP API only attachments that have not yet been committed (by saving the Activity) can be deleted through Outlook.