Mining RedditWithin the Reddit API we can mine for messages. In this case we will take a key search word and then find the Top 10 posts on a subreddit [List with keyword search][Twitter]. |
Theory
Within the Reddit API was can search for keywords or for the Top posts on a subreddit. In the following we search for the last 10 tweets for the key word of "cybersecurity":
import praw import pandas as pd import datetime as dt import sys reddit = praw.Reddit(client_id='xxxxx', \ client_secret='xxxxx', \ user_agent='xxxx', \ username='xxxxx', \ password='xxxx') search_term='cybersecurity' option="1" if (len(sys.argv)>1): search_term=(sys.argv[1]) if (len(sys.argv)>2): option=(sys.argv[2]) print "Search term: ",search_term print "Option: ",option subreddit = reddit.subreddit(search_term) resp = subreddit.top(limit=10) if (option=="2"): resp = subreddit.hot(limit=10) if (option=="3"): resp = subreddit.new(limit=10) if (option=="4"): resp = subreddit.controversial(limit=10) for submission in resp: print "=ID: ",submission.id print " Title: ",submission.title.encode('ascii', 'ignore') print " Score: ",submission.score print " URL: ",submission.url print " Text: ",submission.selftext[:100]
A sample run is:
Search term: cryptocurrency =ID: 7r0ftz Title: CryptoNick is deleting all of his BitConnect videos, and so are his buddies. Please never forget what he and his cohorts did to so many people, and how much money those people lost in the process thanks to CryptoNick, Trevon James, and Craig Grant! Score: 26503 URL: https://www.reddit.com/r/CryptoCurrency/comments/7r0ftz/cryptonick_is_deleting_all_of_his_bitconnect/ Text: We can't let these legendary affiliate scammers get away with what they did, and we have to show the =ID: 7vga1y Title: I will tell you exactly what is going on here, this is critical information to understand if you are going to make money in this space. How prices work, and what moves them - and it's not money invested/withdrawn. Score: 20145 URL: https://www.reddit.com/r/CryptoCurrency/comments/7vga1y/i_will_tell_you_exactly_what_is_going_on_here/ Text: /edit: Hi /r/all. While I have your attention, I want to take 5 seconds of your time and bring some =ID: 7sx5ze Title: Robinhood is launching a Crypto Trading app to compete with Coinbase Score: 19969 URL: http://blog.robinhood.com/news/2018/1/24/dont-sleep Text: =ID: 80xb4n Title: Checkmate, Bill. Score: 19626 URL: https://i.redd.it/vmcf9d93dzi01.jpg Text: =ID: 7qr6ky Title: Delta's app store description seems appropriate today. Score: 18473 URL: https://i.imgur.com/qcDbWMz.png Text: =ID: 7raztw Title: Listen up folks, if you "did", or still do promote cryptocurrency related scams, you will be called out on it via this sub-Reddit. We don't care about you, or your ill-gotten gains, we care about the general well-being of our community first and foremost. Score: 17890 URL: https://www.reddit.com/r/CryptoCurrency/comments/7raztw/listen_up_folks_if_you_did_or_still_do_promote/ Text: So apparently, some of you known scammers are getting a little butt-hurt about being added to our kn =ID: 8eto2e Title: Nasdaq is open to becoming cryptocurrency exchange, CEO says Score: 17161 URL: https://www.cnbc.com/2018/04/25/nasdaq-is-open-to-becoming-cryptocurrency-exchange-ceo-says.html?__source=sharebar|twitter&par=sharebar Text: =ID: 7r4vlc Title: Why we won't have a long term bear market, and how to systematically pick your future investments in crypto Score: 14596 URL: https://www.reddit.com/r/CryptoCurrency/comments/7r4vlc/why_we_wont_have_a_long_term_bear_market_and_how/ Text: With so much uncertainty right now it would be a good time to take some time to go over what happene =ID: c1zlny Title: The true power of Bitcoin Score: 14548 URL: https://i.redd.it/h4n3169pq2531.jpg Text: =ID: 7s9zmc Title: Great news from Korea! Banks will allow cryptocurrency trading again from today and next week account registration is opened again. Score: 13412 URL: http://m.news.naver.com/read.nhn?mode=LSD&mid=sec&sid1=101&oid=001&aid=0009829657 Text: URL: https://i.redd.it/vmcf9d93dzi01.jpg Text: