A SharePoint Designer custom list form was created and a form toolbar added to enable attachments.  This was then tested.  On adding an attachment and clicking OK - nothing happened.

A script error was produced: "L_delete_text is undefined". This post says adding the below script code to the page resolves the issue.

<script>var L_Delete_Text = "";</script>

That works reasonably well but with a minor side effect. Attachments added this way lack the default "Delete" link. The way to resolve this is pretty obvious when you look at that script again and realize that the empty string there is the problem. Thus, the slightly altered script:

<script>var L_Delete_Text = "Delete";</script>

When you provide that 'Delete' and it has something to show, your delete links appear.

Note: At the risk of stating the obvious 'Delete' is not interpreted it is just presented.  You could substitute any text you prefer such as the equivalent in your user's language and that would be displayed in the link and perform deletion.