This is indeed an issue ever since the Media Library modal was introduced in the customizer. The Media Library JS is not Customizer-aware. Beyond uploading new attachments, there is also the issue whereby if you edit an attachment's fields (e.g. title or caption) then these changes get saved immediately as soon as you blur the field. All of these changes need to be added to the Customizer state to then be persisted if the user hits Save & Publish.
As for deleting attachments that are never published, what could done is to upload them with an auto-draft status (as opposed to inherit) and then only transition them to auto-draft once the customizer changes are saved. This would be taking the same approach as Customize Posts and the newly-committed #34923 which uses auto-draft posts as placeholders. Posts with auto-draft status are automatically garbage-collected after 7 days, though I'm not sure if an attachment post with an auto-draft status will get its associated uploaded file also deleted.
Custom fields in the media modal seem to be the most challenging to me. Plugins will have to take care to support the customizer usage. In particular, custom fields added to the media modal normally get saved by plugins via the attachment_fields_to_save action. Plugins would need to not do that in a customizer context, but instead add it to the customizer state and defer the save until the customizer state is published.
Queries for attachments in the media library would also need to include all existing attachment posts that have inherit status OR which are auto-draft and which have an ID that was created in the current customizer session. So that attachment SQL query would need to be updated to account for that.
All in all, I think it's doable, but it will take some serious R&D.