Almost a year ago, reporter Kurt Eichenwald was the victim of a potentially deadly attack. Knowing that the journalist suffered from epilepsy, alt-right Twitter users tweeted animated strobe GIFs at him, in an effort to trigger an epileptic seizure. Prosecutors believe that Eichenwald was chosen due his Jewish descent and his critical view of Donald Trump. While the Internet has revolutionized communication and various other functions of our society, the relative freedom and anonymity on the web often inspires questionable and occasionally dangerous content. Rather than relying on social media companies and other tech giants, to police content and cater to the various needs of users, we decided to build pro.tech.

pro.tech is a browser extension that utilizes deep learning to identify user-defined visually malicious images and GIFs. We allow users to toggle filters, and use sophisticated deep learning technologies to identify:

• GIFs that may potentially trigger an epileptic seizures
• images/GIFs that may have content that you’d like to avoid (e.g., spiders, rats, Donald Trump…etc.)
• images/GIFs that may contain nudity

We also use Microsoft’s Cognitive Services API to caption blocked images and ensure that your browser experience is hindered as little as possible. The extension itself was built in vanilla Javascript with the web extensions API, allowing it to run in Chrome, Edge, Firefox and Opera. We’re identifying GIFs that may trigger epileptic seizures by calculating the L2-norm between subsequent frames in the GIF and using medically-inspired heuristics to determine whether a GIF can be potentially dangerous.

The server-side functionality is built with Python and the Tornado framework. For image tagging, we are using a pre-trained VGG-16 model in PyTorch as well as the Microsoft Cognitive Services API. We also use Microsoft’s Cognitive Services API to caption blocked images and ensure that your browser experience is hindered as little as possible. To keep track of analytics, we utilize stdlib to calculate and record metrics pertaining to the effectiveness of our extension.

It is our hope that pro.tech will enable users to browse the web with a restored sense of safety. When the content gets questionable, we will pro.tech.