Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save GwynethLlewelyn/acfab579afddf1d853be6802a22894d1 to your computer and use it in GitHub Desktop.

Select an option

Save GwynethLlewelyn/acfab579afddf1d853be6802a22894d1 to your computer and use it in GitHub Desktop.
Retrieve list of AS numbers via whois from Nginx web logs
This assumes [`ugrep`](https://github.com/Genivia/ugrep) is installed. `ugrep` may not be the absolutely fastest `grep` replacement out there, but it has a substantially faster regular expression engine (invented by Genivia) than most alternatives which usually rely upon Boost (which *is* fast already!).
It also assumes that `whois` is installed and listed in `$PATH`. Adapt as appropriate.
The final `sort | uniq` may be replaced with `sort --unique` (if that option is available) and should improve things a little.
```bash
LOGFILE=/var/path/something
ug -e '(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})' -P --format="%1%~" $LOGFILE | uniq | while read -r line; do whois -H $line | ug -e '(AS\d{1,6})' -P --format="%1%~"; done | sort | uniq
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment