Ethereum Effect Impact On PC Parts Prices
Data Warehouse for analysis of Ethereum Effect from 2013 to april 2018
@kaggle.raczeq_ethereum_effect_pc_parts
Data Warehouse for analysis of Ethereum Effect from 2013 to april 2018
@kaggle.raczeq_ethereum_effect_pc_parts
Project is not maintained anymore and probably will never be again. Whole python code used for data scrapping is pasted into a notebook attached to this project. I'm sorry for the inconvenience.
Data warehouse has been created as a University project throughout 3 months (march - may 2018).
I haven't find any useful databases containing historical prices of many computer parts (CPUs, GPUs and RAMs), so I had to web scrap it from web prices comparison engines:
For webscrapping I've used Python with BeautifulSoup
library as well as PCPartPicker-API
.
Stuff that I'd like to add but I have no time to do it:
DIM_PROD_GPU
-> Series
. I've made it too shallow and my analysis wasn't so great, additional step between Manufacturer
and Product Name
would make it easier for analytics.Database contains data about:
Few redditers who helped me to find some libraries and datasets to get inspired with.
As a gamer I've seen a huge prices spikes on a GPU market and as I haven't find too many analysis about this phenomenon and because it's very fresh I thought that might be a good topic to make a Uni project for a Data Warehouse course.
I release this data online so it won't waste. Maybe you will find some more interesting results that I did.
Uploaded csv files have been dumped from my data warehouse. Below you can see whole ERD diagram
Only difference: DIM_REGION
contains one additional column that I didn't include in my data warehouse
CREATE TABLE dim_cpu_prod (
"id" BIGINT,
"manufacturer" VARCHAR,
"series" VARCHAR,
"cpu_name" VARCHAR,
"cores" BIGINT,
"socket" VARCHAR
);
CREATE TABLE dim_crypto_data (
"id" BIGINT,
"code" VARCHAR,
"currency_name" VARCHAR,
"is_mineable" BIGINT
);
CREATE TABLE dim_gpu_prod (
"id" BIGINT,
"processor_manufacturer" VARCHAR,
"processor" VARCHAR,
"gpu_manufacturer" VARCHAR,
"memory_capacity" DOUBLE,
"memory_type" VARCHAR
);
CREATE TABLE dim_merchant (
"id" BIGINT,
"merchant" VARCHAR
);
CREATE TABLE dim_region (
"id" BIGINT,
"code" VARCHAR,
"currency" VARCHAR
);
CREATE TABLE dim_time (
"id" BIGINT,
"year" BIGINT,
"month" BIGINT,
"day" BIGINT,
"week" BIGINT
);
CREATE TABLE fact_cpu_price (
"prodid" BIGINT,
"timeid" BIGINT,
"regionid" BIGINT,
"merchantid" BIGINT,
"price_usd" DOUBLE,
"price_original" DOUBLE
);
CREATE TABLE fact_crypto_rate (
"codeid" BIGINT,
"timeid" BIGINT,
"open" DOUBLE,
"close" DOUBLE,
"high" DOUBLE,
"low" DOUBLE
);
CREATE TABLE fact_gpu_price (
"prodid" BIGINT,
"timeid" BIGINT,
"regionid" BIGINT,
"merchantid" BIGINT,
"price_usd" DOUBLE,
"price_original" DOUBLE
);
CREATE TABLE fact_ram_price (
"prodid" BIGINT,
"timeid" BIGINT,
"regionid" BIGINT,
"merchantid" BIGINT,
"price_usd" DOUBLE,
"price_original" DOUBLE
);
Anyone who has the link will be able to view this.