Watcher - mapping addressable security cameras - a devlog

please keep us up to date you are doing great work

Linking Google queries with Watcher

In the previous post I’ve covered the initial stages of getting query results from Google and in the last week I’ve been working on the required plumbing to get these results into a format which is usable. This has taken a little longer than expected and more work is still definitely needed before I’m comfortable with people using this tool, but the vertical slice is certainly coming along.

Here you can see the map being populated as queries are processed and the associated geolocation request completes:

b75f4ba13347a1a1313b83829cd2773f

One of the unexpected issues I’ve had is that although in the previous post I have mentioned that Google’s Custom Search Engine allows for 10000 queries a day, this is actually further limited to 100 queries a day if no billing information is linked with the CSE. Now, Google does give ~US$300 worth of query credits and if just having those linked (without requiring an actual card linked) is enough to get the 10000 queries/day limit, then that will save me a great deal of hassle: otherwise I’ll have to keep track of which queries have been performed to allow people to get through the entire ruleset over multiple days, even if they restart the program.

Once I wrap up the issue above and add support for gracefully handling the “quota reached” error (rather than crashing in a ball of fire) I’ll move on to being able to select the invidual cameras, provide information (such as linked URL, geolocation data, camera type) and the part I have been most looking towards: actually streaming invidual programs from within Watcher.

1 Like

i can lend you my shodan member account , if required, has a few scan and export credits…

Thanks! I’ll let you know once I add the shodan plugin :smiley:

1 Like

i can text you my api key any time just let me know

1 Like

What would be a similar alternative written in Python? :grin:
Great topic by the way!

-Archangel

I’m afraid I’m not sufficiently familiar in Python to provide you a truly educated answer, so take this with a grain of salt. Technically nothing would prevent you from building it in Python: the community has provided Python bindings for imgui and SDL2. Sqlite and JSON have their own Python libraries.

It might become more complicated once you start processing the camera feeds themselves though, but from a cursory look you can still process e.g. MJPG without it being a hassle.

2 Likes

This topic was automatically closed after 30 days. New replies are no longer allowed.

@hostile.node if you have any more updates, I have reopened the topic :slight_smile:

1 Like

Streaming MJPG cameras

Knowing where the cameras are is one thing, but accessing them and streaming their content requires a fair amount of work. In theory, the easiest cameras to stream are the ones which use Motion JPG: these are, for the most part, old cameras which were set up years ago and often with no credentials.

Motion JPG itself is a naive format: this stream is just a series of base 64 encoded JPGs, separated by a text boundary (traditionally “–myboundary”). From an implementation point of view, writing a codec for this in Watcher isn’t particularly complicated as it already has support for loading JPGs anyway.

The flow of the program for this is fairly straightforward: the user clicks on a camera and Watcher internally creates a stream request, followed by a broadcast to any codec plugins to see if any of them can interact with that specific camera. In this case, we look for a “video.mjpg” in the URL and optionally do some additional manipulation on the URL to find the streaming endpoint. Once we start receiving data, we buffer all of it until we have a single JPG frame, at which point the JPG gets decoded, copied into a format OpenGL can deal with and uploaded to the GPU to present to the user.

Overall it is a good step forward as we can now start seeing what’s out there:

There are some performance issues with this though, as having to decode the JPG like this isn’t a fast operation. This isn’t much of a problem on most streams, but every now and then Watcher trips on a more modern camera which supports MJPG as a legacy format, and will happily stream >1080p as MJPG, causing some considerable stuttering. I’ll be looking into moving this into a separate thread so the interaction with the user interface doesn’t suffer. Additionally, there are other libraries such as libjpg-turbo which could be used further down the line as that could provide a 2x - 6x speedup.

Streaming other formats

The same pipeline which has been created for streaming MJPG cameras can be used with any other formats. This will likely lead to integrating ffmpeg into Watcher as most cameras these days will be capable of streaming something more reasonable than MJPG, such as H264. This would allow streaming video at a proper frame rate, rather than the 2 frames per second most MJPG cameras stream at.

Displaying camera statuses

For ease of use, camera pins in the atlas have been colour-coded: they currently start as as grey, but turn green if the user has opened it and successfuly started a stream. This will be further extended to identify cameras which require authentication, or which have had the authentication bypassed.

Next: leveraging Shodan

Now that the program can stream cameras and is essentially proven as a project, the next step is to feed it more data. The Google search plugin has been useful, but that API will only ever return 100 results per query. And, well, with Google’s history of deprecating things, entirely relying on their API might not be a great idea. So the next step will be to leverage Shodan to perform additional queries and to optionally use an API key.

The first part of this will be to create a “custom query” and to allow the user to pick and choose which results they want to have added to the database. After that is done, adding an automated mode based on stored queries will be straightforward. The user interface might require some attention afterwards so a high density of pins can be displayed in a way which can be interacted with, rather than just overlapping.

Compiling the project

Please note that this project remains in flux and has no actual release. If you want to have a play you can build it from source for Windows (Win64, Debug) using Visual Studio 2017 but be aware that this is a pre-alpha piece of software and your results database will likely get eaten further down the line.

Thank you for reading. :slight_smile:

5 Likes

Hey! Love the idea on your project.

Very interesting is there anything we could do to help with this project?.