ICS/OT OSINT: Using Gemini AI for PLC and HMI Image Analysis

Welcome to the first installment of our series “ICS/OT OSINT” exploring the role of Open Source Intelligence (OSINT) in enhancing cybersecurity within Industrial Control Systems (ICS) and Operational Technology (OT). This series will provide insights into practical strategies, highlight the unique challenges, and share breakthroughs that are shaping the way OSINT is used in these critical sectors.” As a personal and experimental endeavor, I acknowledge that my analyses may not be perfect. There might be areas where I am mistaken or information that I have overlooked.

What many don’t realize is that images related to our critical infrastructures, such as those from PLCs and HMIs, can inadvertently reveal more than intended. From sensitive operational data to unintentional privacy disclosures, the details within can provide a wealth of information to those who know how to look.

Gemini AI, a generative AI application powered by Google. I recently decided to put this tool to the test by uploading images of industrial control systems to see how well it could extract and analyze data.

One of my first experiments involved an image of the web interface for a Siemens Simatic S7–300 PLC. I tasked Gemini AI with detailing everything it could about the PLC as displayed on the interface. The application performed remarkably well, providing a comprehensive breakdown of the displayed details.

Curious to push the limits further, I uploaded an image of a Human Machine Interface (HMI) that did not include the vendor’s name. Despite this challenge, Gemini AI not only successfully identified the HMI but also its vendor and model name. I Googled the suggested answer and found it to be correct. The model name indeed matched the image. However, when asked to list specific vulnerabilities and exploits related to the model, the AI returned only general issues rather than specific vulnerabilities. This suggests there is still room for improvement in its ability to deep-dive into more technical aspects.

One of the standout features of Gemini AI is its ability to identify information that is not explicitly listed in the images. This capability is likely supported by Google’s extensive database and its robust image data handling. Such a feature is incredibly useful for Open Source Intelligence (OSINT) as it allows analysts to gather data on industrial control systems (ICS) and operational technology (OT) through images of devices, network diagrams, documents, and more.

In comparison, I tested ChatGPT to see if it could identify the vendor from the same set of images. Interestingly, it was unable to do so, highlighting the specialized strengths of Gemini AI in image analysis.

This exploration into Gemini AI’s capabilities has been quite enlightening. The ability to extract unseen information from images could significantly enhance how we analyze ICS/OT information and generate OSINT.

Stay tuned for more insights and discussions on similar topics as we continue to delve into the possibilities offered by AI in technology analysis and cybersecurity.

Explore ICSrank.com , my specialized OSINT tool crafted for uncovering ICS/OT equipment and evaluating their cybersecurity status. It’s currently undergoing development and is in beta phase