yanking a certain amount of bytes

Question

I'm working on large files for wikisource, along with another guy. I generate a big text file (a few Mb) witch I format a bit using scripts, then I upload the text to a temporary page, where the other guy does some more formatting.

My friends computer and internet connection are not the best in the market, and he asked me not to upload more than 400Kb of data, so his computer can handle all the text.

the file is formatted like this:

==header 1==
.
.
.
==header 2==
.
.
.

and has a few hundreds of headers.

What I need to do:

I need to yank at most 400Kb (as close as I can to this number), but I also need to yank all the text between headers - I can't have the text between headers to split over a few files.

I can find the next header using \^={2}[^=]+={2}$, but I don't know how to carry on yanking until I have ~400Kb of data.

Any ideas?

Answer 1

Ok, this is somewhat of a quick hack, let's hope it works:

function! <SID>Chunk()
    " find first header
    let l_start = search('\m^==[^=]\+==$', 'cW')
    if !l_start
        " beep
        execute "normal \<Esc>"
        return
    endif

    " translate to byte position
    let b_start = line2byte(l_start)
    if b_start < 0
        " beep
        execute "normal \<Esc>"
        return
    endif

    " start marking
    execute 'normal V'

    " mode 400 KB down
    execute 'goto ' . (b_start + 400 * 1024)
    if line('.') == line('$')
        return
    endif

    " find previous header
    let l_end  = search('\m^==[^=]\+==$', 'bcsW', l_start)

    if l_end <= l_start || l_end <= 0
        " not found
        " go to beginning of line
        execute 'normal 0'

        " search next header
        let l_end  = search('\m^==[^=]\+==$', 'sW')
        if l_end <= 0
            " not found
            " go to end of file
            execute 'normal G$'
            return
        endif
    endif

    " move up
    exec 'normal k$'
    return
endfunction

nnoremap <silent> <leader>H :call <SID>Chunk()<CR>

It marks the next ~400 KB region of full headers. After yanking it you can go to the end of the marked region with '>.

Answer 2

You can try something like this: place you cursor on the next line after a header, and hit:

y/\_.\{0,400000}==$/e(Enter)

or (more precise):

y/\_.\{0,400000}\n^==[^=]\+==$/e(Enter)

(translation: yank from cursor at most 400000 characters (not exactly bytes, though), including newlines, ending with two equals at the end of the line (the next header)). The e is needed to move the cursor (here: the yank range) to the end of the match. As the search is greedy, it can contain more than one section, but will end with a new heading and fit into the specified limit.

To repeat, move cursor on next line after the new header, and hit y/(Enter). To view the selection before copying it, use v/(Enter) and, if all is ok, y to yank it.

Answer 3

I'd do this in a shell script, which has the advantage that not only can you do this from Vim, with :!./dosplit %, but you can also do it from the command line or as part of a bigger script that also uploads the files.

Here's a dosplit script written in awk. It's usable out of the box on Unix-like systems, and requires an awk program to be installed on Windows (it's available ).

#!/usr/bin/awk -f
BEGIN {max = 409600}
/^==/ {    # It's a header line
    # Decide whether to start a new file
    if (length(chunk) + size > max) {++n; size=0}
    # Write the previous chunk
    size += length(chunk);
    printf "%s", chunk >sprintf("output-%04d.txt", n);
    chunk = "";
}
END {printf "%s", chunk >sprintf("output-%04d.txt", n)} # Write the last chunk
{chunk = chunk $0 "\n"} # Accumulate the current chunk