Excelling

• Substring for Excel
• Filter for duplicates
• Then trimming (excel can't handle, it crashes): http://stackoverflow.com/questions/11265381/how-to-delete-everything-in-a-string-after-a-specific-character

  while read -r; do echo "${REPLY%%HTTP*}"; done < 1456449636_2156704-3.txt >> -4.txt
  

• notepad++ duplicates