Name: Hourly Energy Demand Generation And Weather
Creator: Kaggle
License: https://creativecommons.org/publicdomain/zero/1.0/

About this Dataset

Hourly Energy Demand Generation And Weather

Context

In a paper released early 2019, forecasting in energy markets is identified as one of the highest leverage contribution areas of Machine/Deep Learning toward transitioning to a renewable based electrical infrastructure.

Content

This dataset contains 4 years of electrical consumption, generation, pricing, and weather data for Spain. Consumption and generation data was retrieved from ENTSOE a public portal for Transmission Service Operator (TSO) data. Settlement prices were obtained from the Spanish TSO Red Electric España. Weather data was purchased as part of a personal project from the Open Weather API for the 5 largest cities in Spain and made public here.

Acknowledgements

This data is publicly available via ENTSOE and REE and may be found in the links above.

Inspiration

The dataset is unique because it contains hourly data for electrical consumption and the respective forecasts by the TSO for consumption and pricing. This allows prospective forecasts to be benchmarked against the current state of the art forecasts being used in industry.

Visualize the load and marginal supply curves.
What weather measurements, and cities influence most the electrical demand, prices, generation capacity?
Can we forecast 24 hours in advance better than the TSO?
Can we predict electrical price by time of day better than TSO?
Forecast intraday price or electrical demand hour-by-hour.
What is the next generation source to be activated on the load curve?

Tables

Energy Dataset

@kaggle.nicholasjhana_energy_consumption_generation_prices_and_weather.energy_dataset

1.71 MB
35,064 rows
29 columns

CREATE TABLE energy_dataset (
  "time" VARCHAR,
  "generation_biomass" DOUBLE,
  "generation_fossil_brown_coal_lignite" DOUBLE,
  "generation_fossil_coal_derived_gas" DOUBLE,
  "generation_fossil_gas" DOUBLE,
  "generation_fossil_hard_coal" DOUBLE,
  "generation_fossil_oil" DOUBLE,
  "generation_fossil_oil_shale" DOUBLE,
  "generation_fossil_peat" DOUBLE,
  "generation_geothermal" DOUBLE,
  "generation_hydro_pumped_storage_aggregated" VARCHAR,
  "generation_hydro_pumped_storage_consumption" DOUBLE,
  "generation_hydro_run_of_river_and_poundage" DOUBLE,
  "generation_hydro_water_reservoir" DOUBLE,
  "generation_marine" DOUBLE,
  "generation_nuclear" DOUBLE,
  "generation_other" DOUBLE,
  "generation_other_renewable" DOUBLE,
  "generation_solar" DOUBLE,
  "generation_waste" DOUBLE,
  "generation_wind_offshore" DOUBLE,
  "generation_wind_onshore" DOUBLE,
  "forecast_solar_day_ahead" DOUBLE,
  "forecast_wind_offshore_eday_ahead" VARCHAR,
  "forecast_wind_onshore_day_ahead" DOUBLE,
  "total_load_forecast" DOUBLE,
  "total_load_actual" DOUBLE,
  "price_day_ahead" DOUBLE,
  "price_actual" DOUBLE
);

Weather Features

@kaggle.nicholasjhana_energy_consumption_generation_prices_and_weather.weather_features

2.56 MB
178,396 rows
17 columns

CREATE TABLE weather_features (
  "dt_iso" VARCHAR,
  "city_name" VARCHAR,
  "temp" DOUBLE,
  "temp_min" DOUBLE,
  "temp_max" DOUBLE,
  "pressure" BIGINT,
  "humidity" BIGINT,
  "wind_speed" BIGINT,
  "wind_deg" BIGINT,
  "rain_1h" DOUBLE,
  "rain_3h" DOUBLE,
  "snow_3h" DOUBLE,
  "clouds_all" BIGINT,
  "weather_id" BIGINT,
  "weather_main" VARCHAR,
  "weather_description" VARCHAR,
  "weather_icon" VARCHAR
);