2

I have a large file which have some corrupt lines like this

0.6 8   379699  730 0.999769    0.802075    0.912155
0.6 8   379699  731 0.999698    0.8181070.8 8   377999  0   0.674844
0.765493    0.900248
0.8 8   377999  1   0.835223    0.913123    0.883744
0.8 8   377999  1   0.816537    1   0.852433

The file ideally should look like

0.6 8   379699  730 0.999769    0.802075    0.912155
0.6 8   379699  731 0.999698    0.818107    nan
0.8 8   377999  0   0.674844    0.765493    0.900248
0.8 8   377999  1   0.835223    0.913123    0.883744
0.8 8   377999  1   0.816537    1   0.852433

I thought that a simple (lazy) way to fix this problem might be to just delete any lines which don't have 7 columns. Can this be done? I was looking into using awk -F' ' ' to obtain the number of columns in each line. is there a better way?

1
  • Wouldn't it be better to "find" the start of a new line and add a newline? :%s/\d\zs\d\.\d/\r&/ Aug 5 at 13:59
4

Delete lines without seven columns? Sure...

:v/\v^\S+(\s+\S+){6}$/d

:vglobal, the "inverse" :global command, looks for all lines that don't match the search criteria and then deletes them.

The search criteria is

  • \v : Use "very magic mode"
  • ^ : Beginning of line.
  • \S+ : Some non-whitespace.
  • (\s+\S+){6} : Six occurrences of whitespace followed by non-whitespace.
  • $ : End of line.

Now this is a little loose in that you wouldn't want to run it against mixed text. I deliberately made it simple and tuned to the example text provided. If the columns, for example, are separated by a single tab the pattern could reflect that more strict criteria. Let me know if you need help with that kind of refinement.

2
  • It works perfectly! Thank you :)
    – Thomas
    Aug 4 at 15:40
  • @Thomas Good to hear! Cheers.
    – B Layer
    Aug 4 at 16:20
3

Here's the awk solution: :%!awk 'NF == 7'. The condition says to only run the action (default when omitted print) when the number of fields is 7. By default awk's fields are just whitespace (not including newlines) separated.

It can also be nice to format the file a bit to glance at the columns: :%!column -t.

Of course, these both require some kind of *nix-like system or utilities.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.