5

I have a multiline text file. Some lines have URL's. What would be the most efficient way to extract those URL's?

I wrote macros and did a bunch of unnecessary things. Now I'm thinking about regex groups or global command...

text example:

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, https://en.wikipedia.org/wiki/Lorem_ipsum#1  quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit https://en.wikipedia.org/wiki/Lorem_ipsum#2 in voluptate velit esse cillum https://en.wikipedia.org/wiki/Lorem_ipsum#3 dolore eu fugiat nulla pariatur.
Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

expectation:

https://en.wikipedia.org/wiki/Lorem_ipsum#1
https://en.wikipedia.org/wiki/Lorem_ipsum#2
https://en.wikipedia.org/wiki/Lorem_ipsum#3
8

The tricky part is getting a valid URL. Take some reasonable regex (say /http[s]\?:\/\/[[:alnum:]%\/_#.-]*/ - or if you want to be pedantic, this one). First get the URLs on individual lines, and then delete lines which aren't URLs:

:%s/.\{-}\(http[s]\?:\/\/[[:alnum:]%\/_#.-]*\)/\1\r/g
:g!//d
  • The substitution uses a non-greedy regex to match characters before a URL. Otherwise, if multiple URLs were present in the same line, all but the last would be covered by .*.
  • The second command specifies an empty regex, reusing the last regex used. The initial .\{-} will match the empty string, so this regex can be reused without problems.

If you have grep with -o:

:%!grep -o 'http[s]\?:\/\/[[:alnum:]\%\/_\#.-]*'
  • The % and # have to escaped to prevent vim from interpreting them.
  • 1
    For a quicker and dirtier regex use: http.\?:\/\/\S\+'. Works with grep as well. – Peter Rincker Feb 20 '15 at 0:48
  • 1
    so the shortest way would be %s/.\{-}\(http[s]\?:\/\/[[:alnum:]%\/_#.-]*\)/\1\r/g|g!//d – A B Feb 20 '15 at 9:05
  • 2
    The problem with parsing URL's is that it's very very tricky to get all URL's; this will for example fail on http://en.wikipedia.org/wiki/Vi_(Buffy_the_Vampire_Slayer), since it doesn't recognize the ( ... There is also a disparity between what the standard says, and what browser accept... Not saying this a bad answer (+1 from me), just saying it's very difficult (maybe even impossible?) to get this 100% correct (and most, if not all, sites don't get it 100% correct). – Martin Tournoij Feb 20 '15 at 10:31
  • 3
    @Carpetsmoker I expected someone to point that out, hence the link to that classic SO post. :D – muru Feb 20 '15 at 10:36
  • Expanding on @PeterRincker's quicker-and-dirtier regex, how about something like (in PCRE, because I'm unfamiliar with Vim's flavor of regex) /[a-z]+:\/\/\S+/? – alxndr Feb 21 '15 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.