Hello everyone.
I was doing some volunteer work recently and there was a situation which involved performing a rather large number of Google searches for cross-referencing purposes and potential leak identification. The concept of performing >20k queries manually didn’t appeal.
I’ve created a small tool (Lamprey) to do the heavy lifting for me. Takes in N lists of queries, performs a combinatorial expansion and outputs a summary and the top 10 results for any matching hits. The outputs are in .csv so it can be analysed further in other programs.
However, one thing I wanted to point out the community towards is that Google offers US$300 in credit with their platform. I’ve written a document on how to set up a Custom Search Engine in the project’s wiki, which you can leverage for other projects. It’s just a JSON API, so once it is set up it is quite straightforward to use.
GPL as usual, source’s here: https://gitlab.com/hostile.node/lamprey
Hopefully of use to someone.
- hostile.node