Dataset: Another Ecommerce Scrape (Power Drill Prices)

About this Dataset

Another Ecommerce Scrape (Power Drill Prices)

This repository analyzes scraped data about Amazon, Home Depot, Lowe's, and Walmart's price competition in digital markets. The data involves power drills. For more information about the scrapping process see this repository. The scraper was active from January 2020 to June 2020.

prices, whether or not there was a sale (trade promotion)
shipping prices, shipping times, delivery methods
its rank in the search results (sorted by best seller). Also information about ads. Ads 1 means that it was sponsored, 0 means it wasn't promoted
inventory - There are up to 3 flexible columns that are used to record information available about quantity on the website. These variables have different meanings across the various retailers. They are cleaned and comparable in calc_inventory.
store locations.
whether or not the product is in stock.
The date variable is in epoch time. This is the date that the page was visited and the data was scraped.
the actual weight of the product as a proxy for size.

If you're curious about the photo for the dataset, it is the actual hardware the scrape was performed on.

Tables

Clean Prices

@kaggle.erichschulman_another_ecommerce_scrape.clean_prices

584.65 KB
20529 rows
38 columns


CREATE TABLE clean_prices (
  "platform" VARCHAR,
  "website" VARCHAR,
  "date" BIGINT,
  "zipcode" BIGINT,
  "rank" DOUBLE,
  "page" DOUBLE,
  "query" VARCHAR,
  "prod_id" VARCHAR,
  "upc" DOUBLE,
  "product" VARCHAR,
  "manufacturer" VARCHAR,
  "model" VARCHAR,
  "price" DOUBLE,
  "list_price" DOUBLE,
  "in_stock" DOUBLE,
  "max_qty" DOUBLE,
  "seller" VARCHAR,
  "arrives" DOUBLE,
  "shipping" VARCHAR,
  "shipping_price" DOUBLE,
  "shipping_options" DOUBLE,
  "store_stock" VARCHAR,
  "store_address" VARCHAR,
  "store_zip" DOUBLE,
  "store_price" DOUBLE,
  "weight" DOUBLE,
  "reviews" DOUBLE,
  "rating" DOUBLE,
  "quantity1" DOUBLE,
  "quantity2" DOUBLE,
  "quantity3" DOUBLE,
  "quantity4" VARCHAR,
  "quantity5" VARCHAR,
  "ads" DOUBLE,
  "calc_rank" DOUBLE,
  "calc_inven" DOUBLE,
  "calc_promo" DOUBLE,
  "calc_ship" BIGINT
);