Introduction

Protein secondary structure can be calculated based on its atoms' 3D coordinates once the protein's 3D structure is solved using X-ray crystallography or NMR. Commonly, DSSP is the tool used for calculating the secondary structure and assigns one of the following secondary structure types (https://swift.cmbi.umcn.nl/gv/dssp/index.html) to every amino acid in a protein:

1. C: Loops and irregular elements (corresponding to the blank characters output by DSSP)
2. E: β-strand
3. H: α-helix
4. B: β-bridge
5. G: 3-helix
6. I: π-helix
7. T: Turn
8. S: Bend

However, X-ray or NMR is expensive. Ideally, we would like to predict the secondary structure of a protein based on its primary sequence directly, which has had a long history. A review on this topic is published recently, Sixty-five years of the long march in protein secondary structure prediction: the final stretch?.

For the purpose of secondary structure prediction, it is common to simplify the aforementioned eight states (Q8) into three (Q3) by merging (E, B) into E, (H, G, I) into E, and (C, S, T) into C. The current accuracy for three-state (Q3) secondary structure prediction is about ~85% while that for eight-state (Q8) prediction is <70%. The exact number depends on the particular test dataset used.

Dataset

The main dataset lists peptide sequences and their corresponding secondary structures. It is a transformation of https://cdn.rcsb.org/etl/kabschSander/ss.txt.gz downloaded at 2018-06-06 from RSCB PDB into a tabular structure. If you download the file at a later time, the number of sequences in it will probably increase.

Description of columns:

1. pdb_id: the id used to locate its entry on https://www.rcsb.org/
2. chain_code: when a protein consists of multiple peptides (chains), the chain code is needed to locate a particular one.
3. seq: the sequence of the peptide
4. sst8: the eight-state (Q8) secondary structure
5. sst3: the three-state (Q3) secondary structure
6. len: the length of the peptide
7. has_nonstd_aa: whether the peptide contains nonstandard amino acids (B, O, U, X, or Z).

Key steps in the transformation:

• Both Q3 and Q8 secondary structure sequences are listed.
• All nonstandard amino acids, which includes B, O, U, X, and Z, (see here for their meanings) are masked with "*" character.
• An additional column (has_nonstd_aa) is added to indicate whether the protein sequence contains nonstandard amino acids.
• A subset of the sequences with low sequence identity and high resolution, ready for training, is also provided

For details of curation, please see https://github.com/zyxue/pdb-secondary-structure.

A subset (9079 sequences) based on sequences culled by PISCES with more strict quality control is also provided. This dataset is considered ready for training models.

The culled subset generated on 2018-05-31 with cutoffs of 25%, 2Å, and 0.25 for sequence identity, resolution and R-factor respectively, is used. The URL to the original culled list is http://dunbrack.fccc.edu/Guoli/culledpdb_hh/cullpdb_pc25_res2.0_R0.25_d180531_chains9099.gz, but it may not be permanently available. This dataset contains more columns from cullpdb_pc25_res2.0_R0.25_d180531_chains9099.gz with self-explanatory names.

For more about PISCES, please see https://academic.oup.com/bioinformatics/article/19/12/1589/258419.

Acknowledgements

The peptide sequence and secondary structure are downloaded from https://cdn.rcsb.org/etl/kabschSander/ss.txt.gz.
The culled subset is downloaded from http://dunbrack.fccc.edu/PISCES.php.

Inspiration

Kaggle provides a great platform for sharing ideas and solving data science problem. Sharing a cleaned dataset help prevent others from duplicated work and also provides a common dataset for more comparable benchmark among different methods.

Early attempts on this (or related) problem:

1. Baldi, Pierre, Søren Brunak, Paolo Frasconi, Gianluca Pollastri and Giovanni Soda. “Bidirectional Dynamics for Protein Secondary Structure Prediction.” Sequence Learning (2001). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.7092&rep=rep1&type=pdf
2. Chen, J. and Chaudhari, N. S.. "Protein Secondary Structure Prediction with bidirectional LSTM networks." Paper presented at the meeting of the Post-Conference Workshop on Computational Intelligence Approaches for the Analysis of Bio-data (CI-BIO), Montreal, Canada, 2005. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.7092&rep=rep1&type=pdf (Couldn't find a pdf)
3. Sepp Hochreiter, Martin Heusel, Klaus Obermayer; Fast model-based protein homology detection without alignment, Bioinformatics, Volume 23, Issue 14, 15 July 2007, Pages 1728–1736, https://doi.org/10.1093/bioinformatics/btm247