NYC FloodSense Sensor Data by Kaggle | Environmental and Climate Sciences

About this Dataset

NYC FloodSense Sensor Data

High-Resolution 5-Minute Flood Depth Monitoring

About this dataset

This dataset provides an invaluable peek into the precarious state of New York City's flood events. With unparalleled real-time, high-resolution data from FloodSense street sign sensors, this dataset sheds light on any location where flooding can or has occurred in the five boroughs. The half-inch resolution and 5-minute updates makes granular analysis of NYC floods possible. Connecting to a unique deployment id within the metadata allows users to track and reference each event with ease. By delving into this rich source of data, patterns in flood risk areas become evident, promoting better decision making for cities and households to conserve life and property in the future

More Datasets

For more datasets, click here.

Featured Notebooks

🚨 Your notebook can be here! 🚨!

How to use the dataset

Introduction

Setting Up The Data

The first step to using the FloodSense Sensor Data is to download the dataset from Kaggle. Once you have downloaded and unzipped the folder, you will need to move both of the CSV files (FloodNet-data-export.csv and metadata.csv) into a separate folder that you can easily access in your coding environment or software of choice, such as Excel or Python's Pandas library.

Exploring The Data

Once you have acquired both datasets, it is time to begin exploring! Each CSV file contains distinct data with various columns that describe different characteristics of each sensor location:

metadata.csv includes all metadata related to each sensor including sensor name, deployment ID, type of deployment (fixed vs mobile), latitude/longitude coordinates and address information;

FloodNet-data-export.csv includes all flood depth measurements associated with each sensor including water depths over time as well as other characteristics such as airtime and measure type (predicted/unpredicted).

In order for one to truly make sense of this data set one must understand what values are stored within each column for exploration purposes:

Time: Timestamp of when measurement was taken; * Measurement: Type of measurement taken; * Night median mm: Median depth during night.; * App Name : Name of application used for collection; * Bandwidth Hz : Bandwidth in MHz.; * Date Deployed : Date deployed.; * Deploy Type : Type od deployment (Fixed vs Mobile); * *Error Flag : Flag indicating if an error occurred; * ***Frequency port : Frequency port number; + Latitude ; Longitude ; Network ;Sensor Name ; SF(Spreading Factor) Batt Voltage ,depth filt mm ,depth proc mm ,Depth raw mm , Dist MM F cnt gw1 rssi dbm gw1snr db l

Research Ideas

Developing algorithms to predict when and where flooding will occur in New York City using historical data from the dataset.

Developing applications that provide users with real-time flood depth maps for areas near them based on the sensor data.

Creating an IoT monitoring system based off of this dataset that allows cities to monitor their infrastructure easier and cheaper with precise information regarding potential flooding events anywhere in their area

Acknowledgements

If you use this dataset in your research, please credit the original authors.
Data Source

License

License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication
No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Columns

File: FloodNet-data-export.csv

Column name	Description
time	The time of the measurement. (DateTime)
night_median_mm	The median depth of the night. (Float)
app_name	The name of the application used. (String)
bw_hz	The bandwidth used. (Integer)
date_deployed	The date the sensor was deployed. (DateTime)
deploy_type	The type of deployment. (String)
error_flag	A flag indicating the presence of an error. (Boolean)
f_port	The frequency port number. (Integer)
lat	The latitude position. (Float)
lng	The longitude position. (Float)
network	The network associated with the sensor deployment. (String)
sensor_name	The name of the sensor. (String)
sf	The spreading factor applied to the individual unit. (Integer)
batt_v	The static battery voltage reading. (Float)
depth_filt_mm	The filtered depth in millimeters. (Float)
depth_proc_mm	The processed depth in millimeters. (Float)
depth_raw_mm	The raw depth in millimeters. (Float)
dist_mm	The distance incurred by the processed values. (Float)
f_cnt	The frequency count. (Integer)
gw_1_rssi_dbm	The RSSI (Received Signal Strength Indicator) of the first gateway in decibels per milliwatt. (Float)
gw_1_snr_db	The SNR (Signal-to-Noise Ratio) of the first gateway in decibels. (Float)
lora_freq_hz	The frequency of the LoRa signal in hertz. (Integer)
dev_addr	The device address. (String)
dev_eui	The device EUI (Extended Unique Identifier). (String)
gw_2_rssi_dbm	The RSSI (Received Signal Strength Indicator) of

File: metadata.csv

Column name	Description
Sensor Name	Unique identifier for each sensor deployment. (String)
Latitude	Latitude coordinate of the sensor deployment. (Float)
Longitude	Longitude coordinate of the sensor deployment. (Float)

Acknowledgements

If you use this dataset in your research, please credit the original authors.
If you use this dataset in your research, please credit .

Tables

Floodnet Data Export

@kaggle.thedevastator_nyc_floodsense_sensor_data.floodnet_data_export

149.25 MB
6383119 rows
68 columns


CREATE TABLE floodnet_data_export (
  "time" VARCHAR,
  "measurement" VARCHAR,
  "sensor_id" VARCHAR,
  "night_median_mm" DOUBLE,
  "app_name" VARCHAR,
  "bw_hz" DOUBLE,
  "date_deployed" VARCHAR,
  "deploy_type" VARCHAR,
  "deployment_id" VARCHAR,
  "dev_id" VARCHAR,
  "error_flag" DOUBLE,
  "f_port" DOUBLE,
  "lat" DOUBLE,
  "lng" DOUBLE,
  "network" VARCHAR,
  "sensor_name" VARCHAR,
  "sf" DOUBLE,
  "batt_v" DOUBLE,
  "depth_filt_mm" DOUBLE,
  "depth_proc_mm" DOUBLE,
  "depth_raw_mm" DOUBLE,
  "dist_mm" DOUBLE,
  "f_cnt" DOUBLE,
  "gw_1_rssi_dbm" DOUBLE,
  "gw_1_snr_db" DOUBLE,
  "lora_freq_hz" DOUBLE,
  "night_median_dist_mm" DOUBLE,
  "data_rate" VARCHAR,
  "airtime_s" DOUBLE,
  "sensor_address_borough" VARCHAR,
  "sensor_address_neighborhood" VARCHAR,
  "sensor_address_state" VARCHAR,
  "sensor_address_street" VARCHAR,
  "sensor_address_street_number" DOUBLE,
  "sensor_address_zip" DOUBLE,
  "dev_addr" VARCHAR,
  "dev_eui" VARCHAR,
  "gw_1_id" VARCHAR,
  "depth_proc_dist_mm" DOUBLE,
  "gw_2_id" VARCHAR,
  "gw_2_rssi_dbm" DOUBLE,
  "gw_2_snr_db" DOUBLE,
  "gw_3_id" VARCHAR,
  "gw_3_rssi_dbm" DOUBLE,
  "gw_3_snr_db" DOUBLE,
  "gw_4_id" VARCHAR,
  "gw_4_rssi_dbm" DOUBLE,
  "gw_4_snr_db" DOUBLE,
  "coding_rate" VARCHAR,
  "fw_ver" VARCHAR,
  "sensor_agg" DOUBLE,
  "sensor_meas_delta_ms" DOUBLE,
  "sensor_reading_count" DOUBLE,
  "sensor_sleep_s" DOUBLE,
  "sensor_state" VARCHAR,
  "gw_5_id" VARCHAR,
  "gw_5_rssi_dbm" DOUBLE,
  "gw_5_snr_db" DOUBLE,
  "gw_6_id" VARCHAR,
  "gw_6_rssi_dbm" DOUBLE,
  "gw_6_snr_db" DOUBLE,
  "gw_7_id" VARCHAR,
  "gw_7_rssi_dbm" DOUBLE,
  "gw_7_snr_db" DOUBLE,
  "filter_reason" VARCHAR,
  "depth_mm" VARCHAR,
  "gateway_1_id" VARCHAR,
  "measurement_1" VARCHAR
);

Metadata

@kaggle.thedevastator_nyc_floodsense_sensor_data.metadata

6.98 KB
34 rows
7 columns


CREATE TABLE metadata (
  "sensor_name" VARCHAR,
  "latitude" DOUBLE,
  "longitude" DOUBLE,
  "deployment_id" VARCHAR,
  "unnamed_4" VARCHAR,
  "unnamed_5" VARCHAR,
  "unnamed_6" VARCHAR
);