Building a threat intelligence feed using the Twitter API and a bit of code | Hacker News Reader

Building a threat intelligence feed using the Twitter API and a bit of code

grimminck.medium.com

95 pointsRobbedoes24 comments

I found several phishing crypto exchange sites that sent wallet recovery codes to a Telegram channel, currently having some fun spamming it :). Wonder if I should create a script that generates plausibly real wallet codes because it seems these scammers are getting these responses in a chat and are checking them by hand!

"Put a shoe on your head"

A bot that scrapes the internet and automatically "falls" for scams/phishing to waste their time with plausible info would be brilliant.

With your jq piped to jq, you could just pipe it within jq

curl https://twitter.threatintel.rocks/ --silent | jq -r '.malicious_urls | .[]'

Thanks!

It appears that some URLs are cut off by Twitter. You might have to add some code to expand those URLs.

For example: https://xunmaus.xn--rvg

In the meantime, I'm using a modified command like so: `curl https://twitter.threatintel.rocks/ --silent | jq -r '.malicious_urls | .[]' | sort -u | grep -v …`

It seems this is only the case if it's a tweet that contains a retweet. These are now removed from the stream as they're not of relevance.

It could be interesting to further enrich the data using info from urlscan.io or something similar.

Interesting! Will have a look on what they can freely offer.

I'd be hesitant to consume this in a firewall or DNS RPZ. What if someone reported google.com as malicious?

Happens all the time and those threat feeds aggregation platform that have a black box ML confidence generator for IOCs aren’t trustworthy either.

In this day and age, if people don’t put emphasis on TTPs and still only rely on old style threat intel, they will become obsolete.

TTPs?

http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pa...

Are use of the api and the website two separate things?

I got banned for posting too hard or something, I'm not clear on the specifics, I'm probably just gonna stand up a domain or join mastadon rather than try to crowd into some musky space.

This is interesting. I worked on a threat intelligence platform at HP that shared the same same as a previous threat intelligence platform that ingested twitter a lot but was found to be useless and just a lot of noise.

Maybe for threat intel it wasn't very useful, but when I worked at McAfee 6-7 years ago, I've heavily used Twitter as a source of new sites for URL categorization (content-wise) - it was very useful to get trending stuff, etc., and proactively put classification into URL database

I built something similar 7 years ago. Once I started by crawling for hashtags, I learned soon enough that #backdoor was causing lots of FPs :) that I couldn’t verify at work

What's an FP?

False Positive

Could be cool to have a honeypot that automatically visits those links via puppeteer or some other browser automation.

Plan in the future is to visit the links, download malware and upload it to VirusTotal, Malwarebazaar, etc.

Check out urlscan.io which lets you scan URLs very conveniently and returns a report, including phishing and brand impersonation verdicts.

Disclaimer: I’m the founder ;)

Am a fan of your work! Would it be useful if I throw these domains directly to URLScan? For retrieving the results I'll probably have to setup a proper DB and a workflow around parsing the Twitter content. Now, it's just a simple script.

Sure, I don't see why not, the worst thing that could happen is that the domain doesn't resolve or the webserver doesn't respond, but at least there will be a record of the domain having been scanned! Thanks for the kind words ;)

Reply on news.ycombinator.com