All of a sudden today it worked. I RDP'ed in to the server, opened the Hyper-V manager, and deleted the VM without issue. I have no idea why it worked today and not yesterday. No other changes have been done to the system in the meantime. Maybe some file was locked by some thread and/or process, and this suddenly ended and allowed access to the file. Well, either way, problem solved, I guess...