Job continuously outputting to Vim buffer

Question

I have a job that produces output continually. I want to write this output to a Vim buffer. I start it with:

let job = job_start(cmd, { 'out_io': 'buffer', 'out_name': 'joboutput' })

But I can't see the job's output as it's produced-- I only see the first line "Reading from channel output...". Only when I call:

call job_stop(job)

Will I see the rest of the output, which is all written to the buffer. So it appears that the job's output is buffered by the OS before I can see it, and stopping the job causes it to be flushed. Is there a way to have Vim continually update the output buffer, the way it would appear if I ran the same job (i.e. process) in a terminal?

Answer 1

This is almost surely happening because the output from cmd is getting buffered when being sent to a pipe, so until you get enough of it (a few kilobytes), Vim won't really receive any.

A very simple fix for this, that will work in most cases, is to have Vim use a pty instead of a pipe to communicate with the external job. The standard library in Linux/Unix will typically set up output to be unbuffered when writing to a terminal, so using a pty is a good fix.

You can do that by including 'pty': 1 as part of the job options.

let job = job_start(cmd, { 'out_io': 'buffer', 'out_name': 'joboutput', 'pty': 1 })

See documentation for job options, which includes:

"pty": 1                Use a pty (pseudo-tty) instead of a pipe when
                        possible.  This is most useful in combination with a
                        terminal window, see terminal.

---

Alternatively, you can make sure your external cmd will use unbuffered output, for which there are some options depending on what cmd is. If it's a C program, then stdbuf is usually an option, if it's a Python program, then using python -u or setting the PYTHONUNBUFFERED=1 environment variable works. Some of the solutions (such as unbuffer) actually use a pty wrapper. (Search Unix&Linux StackExchange and you'll find tons of answers about buffered output!)