Where are buffer level variables stored in Vim? And how to get their RAM / storage usage?

Question

Where does Vim store b: buffer level variables / dictionaries? RAM or some kind of file on a disk?

If it stores it in RAM, how to get an estimate of RAM / storage used by a particular variable / dictionary?

Asking for both Vim & Neovim.

Answer 1

In Vim, if you look at the source code for buflist_new() which adds a filename to the buffer list you'll find the part where the b: dictionary is initialized

// init b: variables
buf->b_vars = dict_alloc();
if (buf->b_vars == NULL)
{
    vim_free(ffname);
    vim_free(buf);
    return NULL;
}
init_var_dict(buf->b_vars, &buf->b_bufvar, VAR_SCOPE);

And in dict.c you'll see that this dictionary is allocated in RAM, you'll also see that there is no code to store these dictionaries on disk.

Now to determine the size of a variable you'll need to do as you'd usually do with any other C program, I don't believe there is a particular interface to let the use to that easily from Vim itself.

To get an idea of how the data is stored in a dictionary you can see that in the signature of dict_add(dict_T *d, dictitem_T *item) that the dictionary items are typed dictitem_T, have a look at structs.h to see:

/*
 * Structure to hold an item of a Dictionary.
 * Also used for a variable.
 * The key is copied into "di_key" to avoid an extra alloc/free for it.
 */
struct dictitem_S
{
typval_T    di_tv;      // type and value of the variable
char_u  di_flags;   // DI_FLAGS_ flags (only used for variable)
char_u  di_key[1];  // key (actually longer!)
};
typedef struct dictitem_S dictitem_T;

which leads us to the typval_T definition:

/*
 * Structure to hold an internal variable without a name.
 */
typedef struct
{
vartype_T   v_type;
char    v_lock;     // see below: VAR_LOCKED, VAR_FIXED
union
{
    varnumber_T v_number;   // number value
#ifdef FEAT_FLOAT
    float_T     v_float;    // floating number value
#endif
    char_u      *v_string;  // string value (can be NULL!)
    list_T      *v_list;    // list value (can be NULL!)
    dict_T      *v_dict;    // dict value (can be NULL!)
    partial_T   *v_partial; // closure: function with args
#ifdef FEAT_JOB_CHANNEL
    job_T       *v_job;     // job value (can be NULL!)
    channel_T   *v_channel; // channel value (can be NULL!)
#endif
    blob_T      *v_blob;    // blob value (can be NULL!)
}       vval;
} typval_T;

And from there you'll find out that these variables are simple types, for example the char_u type used for the string is defined here for a string:

/*
 * Shorthand for unsigned variables. Many systems, but not all, have u_char
 * already defined, so we use char_u to avoid trouble.
 */
typedef unsigned char   char_u;

As for neovim I didn't check but I think they do the same thing.

Answer 2

In addition to the answer given by @statox, I did some testing & found out the RAM used by dictionary variables in (Neo)Vim. I don't have any experience with C language, so I did testing in a following way.

Process

1. I created 100 dummy .txt files with random data around a few KB. Totaling 1.5 MB size. With bash script.

2. I added below code in my .vimrc. This opens those 100 random files when Vim starts & defines the below long dictionary with 640 properties & lots of nestings to b:BUG variable (sorry for irrelevant name).

3. I checked the RAM usage of Vim in bashtop, with & without defining those 100 dictionaries. Both scenarios multiple times to get accurate RAM usage.

Random file creation code (Bash)

for n in {0..100}; do
    dd if=/dev/urandom of=$( printf %01d "$n" ).txt bs=1 count=$(( RANDOM + 1024 ))
done

(Renamed)

Vimrc code

"autocmd BufEnter * call Add_BUG()

function! Add_BUG()
if !exists('b:BUG')
let l:bnum = bufnr()
let b:bnum = l:bnum
let b:BUG = {'aaa': l:bnum, 'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{'alpha':'the-alpha-man', 'beta':'the-beta-man', 'gamma':'the-gamma-man', 23:23, 'rada':{}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
endif
endfunction

let g:BUGINT = 0

while g:BUGINT < 100
tabnew
execute 'edit ' . g:BUGINT . '.txt'
let g:BUGINT = g:BUGINT + 1
endwhile

Result

With 100 variables: 33 MB RAM.

Without 100 variables: 25.5 MB RAM.

Total RAM used by 100 variables: 7.5 MB.

RAM used per variable: ~75 KB.

Conclusion

I needed this info (in my statusline Vim plugin) to decide whether to re-calculate info on buffer enter or store it in a variable. There is a lot of data I need to store.

Since the RAM usage is very very low, it's a good idea to store it rather than re-calculate it.