# Wrong encoding while calling shell

I was experimenting with a DOT diagram and tried to do the following:

:! dot -Tpng -oFab.png %


I got an error because my filename has a special character ("ó" in "Fabricación"):

C:\windows\system32\cmd.exe /c ( dot -Tpng -oFab.png Fabricaci├│n.gv)
Error: dot: can't open Fabricaci├│n.gv
shell returned 2
Hit any key to close this window...


As you can see, the special character is being changed for "├│". This is with vim and gVim 7.4 under Win7 and NTFS, so I assume the filename is in UTF16. I also assume that when invoking the shell/cmd the filename is being interpreted as some other encoding (thanks to Carpetsmoker for pointing out it defaults to code page 850).

How can I fix this?

Sure, I can simply rename the file, but I'd like to know why this happens and how to correct it.

Update: I just found this question in superuser.SE (thanks to feedback by @ChristianBrabandt), but it doesn't seem to help either.

• I'm curious if you would get the same error using Vim on the command line under Cygwin or MobaXterm (portable Unix-like environments for Windows). I suspect not. There may be a way to actually fix this so Windows cmd accepts the filename, but getting a Unix-like environment installed would be my own preferred handling. – Wildcard Mar 31 '16 at 19:38
• From what I read, the default for cmd.exe isn't unicode, but code page 850. Also see this answer. – Martin Tournoij Apr 3 '16 at 5:02
• Thanks @Carpetsmoker. I took the liberty of updating my question with the information you provided. – Roflo Apr 4 '16 at 18:42
• I am not completely sure, but you might want to tweak the 'termencoding' option. – Christian Brabandt Jul 22 '16 at 18:49
• @ChristianBrabandt Unless I'm doing something wrong, that doesn't seem to help. I tried setting tenc to latin1, utf8, and cp850. None seems to do the trick. – Roflo Jul 22 '16 at 20:17

The problem lies in dot.exe. GraphViz can open files with Unicode paths in Linux but not Windows, unless (maybe) if compiled with Visual Studio 2005.

# Research

The code page is set to 850, Vim encoding to UTF-8.

It doesn't give the exact same error, but the dot.exe seems to receive a wrong argument. I tried passing the same filename to the other program.

And it worked just right. Executing both dot.exe and typedirectly from cmd.exe gives the same result, so neither the Windows Console nor Vim are the problem. The next thing that could cause that error was dot.exe itself. My suspicion was that it just doesn't know how to handle the Unicode coded arguments properly, as not even all console commands do:

https://ss64.com/nt/chcp.html

If you need full Unicode support use PowerShell. There is still VERY limited support for Unicode in the CMD shell, piping, redirection and most commands are still ANSI only. The only commands that work are DIR, FOR /F and TYPE, this allows reading and writing (UTF-16LE / BOM) files and filenames but not much else.

I searched on the web if there is support for Unicode in GraphViz and found that it does support Unicode files but nothing about Unicode support for the filenames. Neither I found any reports on the GraphViz bug tracker nor posts on the forum about anyone else being interested in reading a Unicode named file. So I looked it up in the source. Here what is dot.exe entry point looks like:

graphviz-2.40.1\cmd\dot\dot.c

int main(int argc, char **argv)
{
. . .

/* --------------------> ARGS ARE BEING PASSED HERE */
gvParseArgs(Gvc, argc, argv);

. . .


Following the argv down the rabbit hole: graphviz-2.40.1\lib\common\args.c

int gvParseArgs(GVC_t *gvc, int argc, char** argv)
{
int rv;
if ((argc = neato_extra_args(gvc, argc, argv)) < 0)    return (1-argc);
if ((argc = fdp_extra_args(gvc, argc, argv)) < 0)      return (1-argc);
if ((argc = memtest_extra_args(gvc, argc, argv)) < 0)  return (1-argc);
if ((argc = config_extra_args(gvc, argc, argv)) < 0)   return (1-argc);

/* -------------------->  HERE GO ALL NON-FLAG ARTUMENTS */
if ((rv = dotneato_args_initialize(gvc, argc, argv)))  return rv;

if (Verbose) gvplugin_write_status(gvc);
return 0;
}


graphviz-2.40.1\lib\common\input.c

int dotneato_args_initialize(GVC_t * gvc, int argc, char **argv)
{
for (i = 1; i < argc; i++) {
if (argv[i] && argv[i][0] == '-') {

. . .

/* -------------------->  JUST CASUALLY COPYING CHAR POINTERS */
} else if (argv[i])
gvc->input_filenames[nfiles++] = argv[i];
}


And finaly graphviz-2.40.1\lib\common\input.c

graph_t *gvNextInputGraph(GVC_t *gvc)
{
. . . .

/* -------------------->  OPENING THE FILES FOR READ WITH FOPEN */
while ((fn = gvc->input_filenames[fidx++]) && !(fp = fopen(fn, "r")))  {

. . .

}


As the MDSN states:

The fopen function opens the file specified by filename. _wfopen is a wide-character version of fopen; the arguments to _wfopen are wide-character strings. _wfopen and fopen behave identically otherwise. Simply using _wfopen has no effect on the coded character set used in the file stream.

In Visual C++ 2005, fopen supports Unicode file streams.

Sadly, the only option there is to rename the file.