1) Use a function with range keyword (`:h func-range`) for processing your visual selection. Moreover, that also allows for `[count]` in Normal mode.

2) Prefer buffer-local mappings whenever it makes sense.

    " using range-aware function
    function! QFdelete() range
        " get current qflist
        let l:qfl = getqflist()
        " no need for filter() and such; just drop the items in range
        call remove(l:qfl, a:firstline - 1, a:lastline - 1)
        " replace items in the current list, do not make a new copy of it;
        " this also preserves the list title
        call setqflist([], 'r', {'items': l:qfl})
    endfunction

    " using buffer-local mappings
    " note: still have to check &bt value to filter out `:e quickfix` and such
    augroup QFList | au!
        autocmd BufWinEnter quickfix if &bt ==# 'quickfix'
        autocmd BufWinEnter quickfix    nnoremap <silent><buffer>dd :call QFdelete()<CR>
        autocmd BufWinEnter quickfix    vnoremap <silent><buffer>d  :call QFdelete()<CR>
        autocmd BufWinEnter quickfix endif
    augroup end