Dataset: Raw Twitter Timelines W/ No Retweets

About this Dataset

Raw Twitter Timelines W/ No Retweets

This is a dataset of tweets from various active scientists and personalities ranging from Donald Trump and Hillary Clinton to Neil deGrasse Tyson. More are forthcoming.

They were obtained through javascript scraping of the browser twitter timeline rather than a Tweepy python API or the twitter timeline API.

The inspiration for this twitter dataset is comparing tweets in my own twitter analysis to find who tweets like whom, e.g. does Trump or Hillary tweet more like Kim Kardashian than one another?

Thus, this goes further back in time than anything directly available from Twitter.

The data is in JSON format rather than CSV, which will be forthcoming as well.

Kim Kardashian, Adam Savage, BillNye, Neil deGrasse Tyson, Donald Trump, and Hillary Clinton have been collected up to 2016-10-14
Richard Dawkins, Commander Scott Kelly, Barack Obama, NASA, and The Onion, tweets up to 2016-10-15.

For your own pleasure, with special thanks to the Trump Twitter Archive for providing some of the code, here is the JavaScript used to scrape tweets off of a timeline and output the results to the clipboard in JSON format:

Construct the query with from:TWITTERHANDLE since:DATE until:DATE
In the browser console set up automatic scrolling with:
setInterval(function(){ scrollTo(0, document.body.scrollHeight) }, 2500)
Scrape the resulting timeline with:
var allTweets = []; var tweetElements = document.querySelectorAll('li.stream-item');

for (var i = 0; i < tweetElements.length; i++) { try {
var el = tweetElements[i]; var text = el.querySelector('.tweet-text').textContent; allTweets.push({ id: el.getAttribute('data-item-id'), date: el.querySelector('.time a').textContent, text: text, link: el.querySelector('div.tweet').getAttribute('data-permalink-path'), retweet: text.indexOf('"@') == 0 && text.includes(':') ? true : false }); } catch(err) {}
};
copy(allTweets);

Have fun!

Tables

Donaldtrumptweets

@kaggle.speckledpingu_rawtwitterfeeds.donaldtrumptweets

1.79 MB
17216 rows
7 columns


CREATE TABLE donaldtrumptweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Neildegrassetysontweets

@kaggle.speckledpingu_rawtwitterfeeds.neildegrassetysontweets

286.04 KB
2428 rows
7 columns


CREATE TABLE neildegrassetysontweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Hillaryclintontweets

@kaggle.speckledpingu_rawtwitterfeeds.hillaryclintontweets

375.91 KB
3356 rows
7 columns


CREATE TABLE hillaryclintontweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Kimkardashiantweets

@kaggle.speckledpingu_rawtwitterfeeds.kimkardashiantweets

968.39 KB
10688 rows
7 columns


CREATE TABLE kimkardashiantweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

N 10460kdnuggetstweets

@kaggle.speckledpingu_rawtwitterfeeds.n_10460kdnuggetstweets

980.19 KB
10440 rows
7 columns


CREATE TABLE n_10460kdnuggetstweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Scottkelly

@kaggle.speckledpingu_rawtwitterfeeds.scottkelly

135.8 KB
1219 rows
7 columns


CREATE TABLE scottkelly (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Fivethirtyeighttweets

@kaggle.speckledpingu_rawtwitterfeeds.fivethirtyeighttweets

906.41 KB
9761 rows
7 columns


CREATE TABLE fivethirtyeighttweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Barackobama

@kaggle.speckledpingu_rawtwitterfeeds.barackobama

669.48 KB
6896 rows
7 columns


CREATE TABLE barackobama (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Alltweets

@kaggle.speckledpingu_rawtwitterfeeds.alltweets

8.49 MB
88625 rows
8 columns


CREATE TABLE alltweets (
  "unnamed_0_1" BIGINT,
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Donaldtrump2014–01–01to2016–10–14tweets

@kaggle.speckledpingu_rawtwitterfeeds.donaldtrump2014_01_01to2016_10_14tweets

959.86 KB
8439 rows
7 columns


CREATE TABLE donaldtrump2014_01_01to2016_10_14tweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Richarddawkins

@kaggle.speckledpingu_rawtwitterfeeds.richarddawkins

739.47 KB
5839 rows
7 columns


CREATE TABLE richarddawkins (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Adamsavagetweets

@kaggle.speckledpingu_rawtwitterfeeds.adamsavagetweets

548.23 KB
4872 rows
7 columns


CREATE TABLE adamsavagetweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);

Hillaryclinton2014–01–01to2016–10–14tweets

@kaggle.speckledpingu_rawtwitterfeeds.hillaryclinton2014_01_01to2016_10_14tweets

366.88 KB
3256 rows
7 columns


CREATE TABLE hillaryclinton2014_01_01to2016_10_14tweets (
  "unnamed_0" BIGINT,
  "date" VARCHAR,
  "id" BIGINT,
  "link" VARCHAR,
  "retweet" BOOLEAN,
  "text" VARCHAR,
  "author" VARCHAR
);