Dataset: Parameter, Compute And Data Trends In Machine Learning (Epoch, 2023)

About this Dataset

Parameter, Compute And Data Trends In Machine Learning (Epoch, 2023)

The affiliation of the research team building a particular notable AI system was classified according to the following:
— Academia: 100% of researchers affiliated with academia
— Collaboration, Academia-majority: 71–99% affiliated with academia
— Collaboration: 30–70% affiliated with academia
— Collaboration, Industry-majority: 71–99% affiliated with industry
— Industry: 100% of researchers affiliated with industry

This data corresponds to the "Organization Categorization" and "Domain" columns in the primary source data spreadsheet.

The authors selected the AI systems for inclusion based on the following necessary criteria:
— Have an explicit learning component
— Showcase experimental results
— Advance the state of the art

In addition, the systems had to meet at least one of the following notability criteria:
— Paper has more than 1000 citations
— Historical importance
— Important state-of-the-art advance
— Deployed in a notable context

The authors note that: "For new models (from 2020 onward) it is harder to assess these criteria, so we fall back to a subjective selection. We refer to models meeting our selection criteria as 'milestone models.'"
In cases where the model name was unavailable, OWID uses the institution name that published the model for identification.

Tables

Epoch 1

@owid.epoch.owid_epoch_1

32.3 KB
393 rows
11 columns


CREATE TABLE owid_epoch_1 (
  "index" BIGINT,
  "system" VARCHAR,
  "days_since_1949" INTEGER,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "publication_date" TIMESTAMP,
  "parameters" BIGINT,
  "training_compute_flop" FLOAT,
  "training_dataset_size_datapoints" VARCHAR,
  "training_time_hours" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_2

35.61 KB
837 rows
8 columns


CREATE TABLE owid_epoch_2 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_3

36.77 KB
865 rows
8 columns


CREATE TABLE owid_epoch_3 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_4

35.52 KB
825 rows
8 columns


CREATE TABLE owid_epoch_4 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_5

35.65 KB
828 rows
8 columns


CREATE TABLE owid_epoch_5 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_6

36.95 KB
867 rows
8 columns


CREATE TABLE owid_epoch_6 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_7

38.28 KB
862 rows
8 columns


CREATE TABLE owid_epoch_7 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" BIGINT,
  "training_dataset_size_datapoints" BIGINT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_8

38.55 KB
869 rows
8 columns


CREATE TABLE owid_epoch_8 (
  "days_since_1949" INTEGER,
  "model" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" BIGINT,
  "training_dataset_size_datapoints" BIGINT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_9

39.14 KB
889 rows
8 columns


CREATE TABLE owid_epoch_9 (
  "days_since_1949" INTEGER,
  "model" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" BIGINT,
  "training_dataset_size_datapoints" BIGINT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_10

39.23 KB
890 rows
8 columns


CREATE TABLE owid_epoch_10 (
  "days_since_1949" INTEGER,
  "model" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" BIGINT,
  "training_dataset_size_datapoints" BIGINT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_11

41.47 KB
956 rows
8 columns


CREATE TABLE owid_epoch_11 (
  "days_since_1949" INTEGER,
  "model" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" BIGINT,
  "training_dataset_size_datapoints" BIGINT,
  "training_computation_petaflop" FLOAT
);

Parameter, Compute And Data Trends In Machine Learning

@owid.epoch.owid_epoch_12

35.08 KB
805 rows
9 columns


CREATE TABLE owid_epoch_12 (
  "days_since_1949" INTEGER,
  "system" VARCHAR,
  "publication_date" TIMESTAMP,
  "domain" VARCHAR,
  "organization_categorization" VARCHAR,
  "parameters" FLOAT,
  "training_dataset_size_datapoints" FLOAT,
  "approach" VARCHAR,
  "training_computation_petaflop" FLOAT
);