You should be able to delete the file if the preview handler is loaded with the metadata edit panel closed. The shell sends an unload command, when deleting or the file properties dialog is invoked. Sometimes there is the need to click the "file in use" dialog retry button, because the PDF is still unloading.
And I have already coded a solution that fixes this issue, and related for when trying to rename the file, for when the metadata panel is open too. Should be a no issue anymore, with the next release.