I have a sparse CSV file with data shaped like in this simplified example. I am omitting the header,
Age,Fruit,Color,Count, but I can include it in my file if needed. You can see that the sparse rows without a
Color are sums of all the
Colors of the fruit, but I don't need to treat those any differently from the other rows.
Old,Apple,,22 New,Apple,,51 Old,Apple,Green,8 New,Apple,Green,8 Old,Apple,Red,14 New,Apple,Red,43 Old,Banana,,3 New,Banana,,3 Old,Banana,Yellow,3 New,Banana,Yellow,3 Old,Cherry,,10 New,Cherry,,10 Old,Cherry,Black,2 New,Cherry,Black,10 Old,Cherry,Red,8 New,Cherry,Red,0
I know that there are exactly two lines for each fruit sum, and for each fruit-color pairing in the file, one "Old" and one "New."
How can I remove all instances of any lines that were repeated, ignoring the first column but preserving it? That is, I want to keep only these lines:
Old,Apple,,22 New,Apple,,51 Old,Apple,Red,14 New,Apple,Red,43 Old,Cherry,Black,2 New,Cherry,Black,10 Old,Cherry,Red,8 New,Cherry,Red,0
I don't need to preserve any sort order, and I can also separate the data however I want. For example, I could place all the rows with no color in one file, and all the rows with color in another.
It doesn't matter if this is a one-line sort or regex command, a sequence of separate commands, a macro, etc., and it doesn't matter if it's slow to run. Sadly, I'm using GVIM 8.2 on Windows Server 2019, so I don't have access to POSIX commands. (Vim probably isn't the ideal tool for this job, but I'm sure there's a way to pull this off and it seems like a fun puzzle.)