AWS r5-2xlarge (64 GB RAM)
Mapping RNA-seq Reads with STAR. Curr Protoc Bioinformatics. 2015; 51: 11.14.1-11.14.19.
$wget http://labshare.cshl.edu/shares/gingeraslab/www-data/dobin/STAR/STARgenomes/ENSEMBL/homo_sapiens/ENSEMBL.homo_sapiens.release-83/
-rw-rw-r– 1 ubuntu ubuntu 3208460789 Mar 7 2017 Genome
-rw-rw-r– 1 ubuntu ubuntu 1402373249 Mar 6 2017 Homo_sapiens.GRCh38.83.gtf
-rw-rw-r– 1 ubuntu ubuntu 3151425857 Mar 6 2017 Homo_sapiens.GRCh38.dna.primary_assembly.fa
-rw-rw-r– 1 ubuntu ubuntu 36852 Mar 7 2017 Log.out
-rw-rw-r– 1 ubuntu ubuntu 24878479456 Mar 7 2017 SA
-rw-rw-r– 1 ubuntu ubuntu 1565873619 Mar 7 2017 SAindex
-rw-rw-r– 1 ubuntu ubuntu 1200 Mar 7 2017 chrLength.txt
-rw-rw-r– 1 ubuntu ubuntu 1923 Mar 7 2017 chrName.txt
-rw-rw-r– 1 ubuntu ubuntu 3123 Mar 7 2017 chrNameLength.txt
-rw-rw-r– 1 ubuntu ubuntu 2129 Mar 7 2017 chrStart.txt
-rw-rw-r– 1 ubuntu ubuntu 41683445 Mar 7 2017 exonGeTrInfo.tab
-rw-rw-r– 1 ubuntu ubuntu 16888430 Mar 7 2017 exonInfo.tab
-rw-rw-r– 1 ubuntu ubuntu 970806 Mar 7 2017 geneInfo.tab
-rw-rw-r– 1 ubuntu ubuntu 644 Mar 7 2017 genomeParameters.txt
-rw-rw-r– 1 ubuntu ubuntu 763 Mar 7 2017 log
-rw-rw-r– 1 ubuntu ubuntu 10176231 Mar 7 2017 sjdbInfo.txt
-rw-rw-r– 1 ubuntu ubuntu 7974201 Mar 7 2017 sjdbList.fromGTF.out.tab
-rw-rw-r– 1 ubuntu ubuntu 7972476 Mar 7 2017 sjdbList.out.tab
-rw-rw-r– 1 ubuntu ubuntu 11757123 Mar 7 2017 transcriptInfo.tab
$ cat Homo_sapiens.GRCh38.dna.primary_assembly.fa | grep ">" | head -n 30
>1 dna:chromosome chromosome:GRCh38:1:1:248956422:1 REF
>10 dna:chromosome chromosome:GRCh38:10:1:133797422:1 REF
>11 dna:chromosome chromosome:GRCh38:11:1:135086622:1 REF
>12 dna:chromosome chromosome:GRCh38:12:1:133275309:1 REF
>13 dna:chromosome chromosome:GRCh38:13:1:114364328:1 REF
>14 dna:chromosome chromosome:GRCh38:14:1:107043718:1 REF
>15 dna:chromosome chromosome:GRCh38:15:1:101991189:1 REF
>16 dna:chromosome chromosome:GRCh38:16:1:90338345:1 REF
>17 dna:chromosome chromosome:GRCh38:17:1:83257441:1 REF
>18 dna:chromosome chromosome:GRCh38:18:1:80373285:1 REF
>19 dna:chromosome chromosome:GRCh38:19:1:58617616:1 REF
>2 dna:chromosome chromosome:GRCh38:2:1:242193529:1 REF
>20 dna:chromosome chromosome:GRCh38:20:1:64444167:1 REF
>21 dna:chromosome chromosome:GRCh38:21:1:46709983:1 REF
>22 dna:chromosome chromosome:GRCh38:22:1:50818468:1 REF
>3 dna:chromosome chromosome:GRCh38:3:1:198295559:1 REF
>4 dna:chromosome chromosome:GRCh38:4:1:190214555:1 REF
>5 dna:chromosome chromosome:GRCh38:5:1:181538259:1 REF
>6 dna:chromosome chromosome:GRCh38:6:1:170805979:1 REF
>7 dna:chromosome chromosome:GRCh38:7:1:159345973:1 REF
>8 dna:chromosome chromosome:GRCh38:8:1:145138636:1 REF
>9 dna:chromosome chromosome:GRCh38:9:1:138394717:1 REF
>MT dna:chromosome chromosome:GRCh38:MT:1:16569:1 REF
>X dna:chromosome chromosome:GRCh38:X:1:156040895:1 REF
>Y dna:chromosome chromosome:GRCh38:Y:2781480:56887902:1 REF
>KI270728.1 dna:scaffold scaffold:GRCh38:KI270728.1:1:1872759:1 REF
>KI270727.1 dna:scaffold scaffold:GRCh38:KI270727.1:1:448248:1 REF
>KI270442.1 dna:scaffold scaffold:GRCh38:KI270442.1:1:392061:1 REF
>KI270729.1 dna:scaffold scaffold:GRCh38:KI270729.1:1:280839:1 REF
>GL000225.1 dna:scaffold scaffold:GRCh38:GL000225.1:1:211173:1 REF
$ head -n 6 Homo_sapiens.GRCh38.83.gtf #!genome-build GRCh38.p5
#!genome-version GRCh38
#!genome-date 2013-12
#!genome-build-accession NCBI:GCA_000001405.20
#!genebuild-last-updated 2015-10
1 havana gene 11869 14409 . + . gene_id "ENSG00000223972"; gene_version "5"; gene_name "DDX11L1"; gene_source "havana"; gene_biotype "transcribed_unprocessed_pseudogene"; havana_gene "OTTHUMG00000000961"; havana_gene_version "2";
ENSEMBL.homo_sapiens.release-83$ head genomeParameters.txt
### /sonas-hs/gingeras/nlsas_norepl/user/dobin/STAR/Releases/FromGitHub/STAR-2.5.2b/bin/Linux_x86_64/STAR --runMode genomeGenerate --runThreadN 12 --genomeDir ./ --genomeFastaFiles Homo_sapiens.GRCh38.dna.primary_assembly.fa --sjdbGTFfile Homo_sapiens.GRCh38.83.gtf
versionGenome 20201
genomeFastaFiles Homo_sapiens.GRCh38.dna.primary_assembly.fa
genomeSAindexNbases 14
genomeChrBinNbits 18
genomeSAsparseD 1
sjdbOverhang 100
sjdbFileChrStartEnd
-
sjdbGTFfile Homo_sapiens.GRCh38.83.gtf
sjdbGTFchrPrefix
~/cpb_PMID_26334920$ wget https://www.encodeproject.org/files/ENCFF001RFG/@@download/ENCFF001RFG.fastq.gz
ENCFF001RFG.fastq.gz 100%[==================================================================>] 7.96G
~/cpb_PMID_26334920$ wget https://www.encodeproject.org/files/ENCFF001RFH/@@download/ENCFF001RFH.fastq.gz
ENCFF001RFH.fastq.gz 100%[====================================================================>] 7.81G
~/cpb_PMID_26334920$ wc -l ENCFF001RFG.fastq.gz 32937626 ENCFF001RFG.fastq.gz
~/cpb_PMID_26334920$ wc -l ENCFF001RFH.fastq.gz
32147739 ENCFF001RFH.fastq.gz
-rw-rw-r– 1 ubuntu ubuntu 8543476007 Feb 9 01:31 ENCFF001RFG.fastq.gz
-rw-rw-r– 1 ubuntu ubuntu 8385864242 Feb 9 01:32 ENCFF001RFH.fastq.gz
~/cpb_PMID_26334920/basic$ STAR --runThreadN 12 --genomeDir ../ENSEMBL.homo_sapiens.release-83/ --sjdbGTFfile ../ENSEMBL.homo_sapiens.release-83/Homo_sapiens.GRCh38.83.gtf --sjdbOverhang 100 --readFilesIn ../ENCFF001RFH.fastq.gz ../ENCFF001RFG.fastq.gz --readFilesCommand zcat
Apr 09 16:12:40 ….. started STAR run
Apr 09 16:12:40 ….. loading genome
Apr 09 16:14:17 ….. processing annotations GTF
Apr 09 16:14:29 ….. inserting junctions into the genome indices
Apr 09 16:15:52 ….. started mapping
Apr 09 16:44:15 ….. finished successfully
-rw-rw-r– 1 ubuntu ubuntu 75865701604 Apr 9 16:44 Aligned.out.sam
-rw-rw-r– 1 ubuntu ubuntu 1870 Apr 9 16:44 Log.final.out
-rw-rw-r– 1 ubuntu ubuntu 26046 Apr 9 16:44 Log.out
-rw-rw-r– 1 ubuntu ubuntu 3432 Apr 9 16:44 Log.progress.out
-rw-rw-r– 1 ubuntu ubuntu 8888522 Apr 9 16:44 SJ.out.tab
drwx—— 2 ubuntu ubuntu 4096 Apr 9 16:14 _STARgenome
Started job on | Apr 09 16:12:40
Started mapping on | Apr 09 16:15:52
Finished on | Apr 09 16:44:15
Mapping speed, Million of reads per hour | 222.15
Number of input reads | 105089150
Average input read length | 202
UNIQUE READS:
Uniquely mapped reads number | 96824749
Uniquely mapped reads % | 92.14%
Average mapped length | 200.92
Number of splices: Total | 41730816
Number of splices: Annotated (sjdb) | 41099145
Number of splices: GT/AG | 41211200
Number of splices: GC/AG | 342564
Number of splices: AT/AC | 42231
Number of splices: Non-canonical | 134821
Mismatch rate per base, % | 0.29%
Deletion rate per base | 0.02%
Deletion average length | 1.57
Insertion rate per base | 0.01%
Insertion average length | 1.48
MULTI-MAPPING READS:
Number of reads mapped to multiple loci | 6263113
% of reads mapped to multiple loci | 5.96%
Number of reads mapped to too many loci | 62633
% of reads mapped to too many loci | 0.06%
UNMAPPED READS:
% of reads unmapped: too many mismatches | 0.00%
% of reads unmapped: too short | 1.82%
% of reads unmapped: other | 0.02%
CHIMERIC READS:
Number of chimeric reads | 0
% of chimeric reads | 0.00%~/cpb_PMID_26334920$ samtools view -Su basic/Aligned.out.sam |head
##IVN:1.4
@SQ SN:1 LN:248956422
@SQ SN:10 LN:133797422
@SQ SN:11 LN:135086622
@SQ SN:12 LN:133275309
@SQ SN:13 LN:114364328
@SQ SN:14 LN:107043718
@SQ SN:15 LN:101991189
@SQ SN:16 LN:90338345
@SQ SN:17 LN:83257441
~/cpb_PMID_26334920$ samtools view -Su basic/Aligned.out.sam | samtools sort -o basic/Aligned.out.bam
[bam_sort_core] merging from 98 files…
-rw-rw-r– 1 ubuntu ubuntu 15276811925 Apr 9 19:18 Aligned.out.bam
239994382 + 0 in total (QC-passed reads + QC-failed reads)
33818658 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
239994382 + 0 mapped (100.00% : N/A)
206175724 + 0 paired in sequencing
103087862 + 0 read1
103087862 + 0 read2
206175724 + 0 properly paired (100.00% : N/A)
206175724 + 0 with itself and mate mapped
0 + 0 singletons (0.00% : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
~/cpb_PMID_26334920/BasicBamToCufflinks$ cufflinks -p 12 --library-type fr-firststrand ../basic/Aligned.out.bam
Warning: Could not connect to update server to verify current version. Please check at the Cufflinks website (http://cufflinks.cbcb.umd.edu).
[02:57:40] Inspecting reads and determining fragment length distribution.
> Processing Locus 14:49247828-49247929 [ ] 2%
~/cpb_PMID_26334920/alignedOutBam$ STAR --runThreadN 12 --genomeDir ../ENSEMBL.homo_sapiens.release-83/ --sjdbGTFfile ../ENSEMBL.homo_sapiens.release-83/Homo_sapiens.GRCh38.83.gtf --sjdbOverhang 100 --readFilesIn ../ENCFF001RFG.fastq.gz ../ENCFF001RFH.fastq.gz --readFilesCommand zcat --outSAMtype BAM SortedByCoordinate
Apr 10 03:45:50 ….. started STAR run
Apr 10 03:45:51 ….. loading genome
Apr 10 03:47:43 ….. processing annotations GTF
Apr 10 03:47:59 ….. inserting junctions into the genome indices
Apr 10 03:49:19 ….. started mapping
Apr 10 04:12:47 ….. started sorting BAM
Apr 10 04:22:12 ….. finished successfully
-rw-rw-r– 1 ubuntu ubuntu 17819538713 Apr 10 04:22 Aligned.sortedByCoord.out.bam
-rw-rw-r– 1 ubuntu ubuntu 1870 Apr 10 04:22 Log.final.out
-rw-rw-r– 1 ubuntu ubuntu 27191 Apr 10 04:22 Log.out
-rw-rw-r– 1 ubuntu ubuntu 3078 Apr 10 04:22 Log.progress.out
-rw-rw-r– 1 ubuntu ubuntu 8887960 Apr 10 04:22 SJ.out.tab
drwx—— 2 ubuntu ubuntu 4096 Apr 10 03:47 _STARgenome
Started job on | Apr 10 03:45:50
Started mapping on | Apr 10 03:49:19
Finished on | Apr 10 04:22:12
Mapping speed, Million of reads per hour | 191.75
Number of input reads | 105089150
Average input read length | 202
UNIQUE READS:
Uniquely mapped reads number | 96824009
Uniquely mapped reads % | 92.14%
Average mapped length | 200.92
Number of splices: Total | 41731310
Number of splices: Annotated (sjdb) | 41099477
Number of splices: GT/AG | 41211630
Number of splices: GC/AG | 342569
Number of splices: AT/AC | 42225
Number of splices: Non-canonical | 134886
Mismatch rate per base, % | 0.29%
Deletion rate per base | 0.02%
Deletion average length | 1.57
Insertion rate per base | 0.01%
Insertion average length | 1.48
MULTI-MAPPING READS:
Number of reads mapped to multiple loci | 6262912
% of reads mapped to multiple loci | 5.96%
Number of reads mapped to too many loci | 62642
% of reads mapped to too many loci | 0.06%
UNMAPPED READS:
% of reads unmapped: too many mismatches | 0.00%
% of reads unmapped: too short | 1.83%
% of reads unmapped: other | 0.02%
CHIMERIC READS:
Number of chimeric reads | 0
% of chimeric reads | 0.00%
239991992 + 0 in total (QC-passed reads + QC-failed reads)
33818150 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
239991992 + 0 mapped (100.00% : N/A)
206173842 + 0 paired in sequencing
103086921 + 0 read1
103086921 + 0 read2
206173842 + 0 properly paired (100.00% : N/A)
206173842 + 0 with itself and mate mapped
0 + 0 singletons (0.00% : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)
~/cpb_PMID_26334920/rnaseq$ STAR --runThreadN 12 --genomeDir ../ENSEMBL.homo_sapiens.release-83/ --sjdbGTFfile ../ENSEMBL.homo_sapiens.release-83/Homo_sapiens.GRCh38.83.gtf --sjdbOverhang 100 --readFilesIn ../ENCFF001RFG.fastq.gz ../ENCFF001RFH.fastq.gz --readFilesCommand zcat --quantMode TranscriptomeSAM --outFileNamePrefix 1align/
Apr 09 22:39:15 ….. started STAR run
Apr 09 22:39:15 ….. loading genome
Apr 09 22:41:07 ….. processing annotations GTF
Apr 09 22:41:24 ….. inserting junctions into the genome indices
Apr 09 22:42:43 ….. started mapping
Apr 09 23:13:35 ….. finished successfully
-rw-rw-r– 1 ubuntu ubuntu 75864947151 Apr 9 23:13 Aligned.out.sam
-rw-rw-r– 1 ubuntu ubuntu 19449197988 Apr 9 23:13 Aligned.toTranscriptome.out.bam
-rw-rw-r– 1 ubuntu ubuntu 1870 Apr 9 23:13 Log.final.out
-rw-rw-r– 1 ubuntu ubuntu 26491 Apr 9 23:13 Log.out
-rw-rw-r– 1 ubuntu ubuntu 3786 Apr 9 23:13 Log.progress.out
-rw-rw-r– 1 ubuntu ubuntu 8887960 Apr 9 23:13 SJ.out.tab
drwx—— 2 ubuntu ubuntu 4096 Apr 9 22:41 _STARgenome
total 89G
71G Aligned.out.sam
19G Aligned.toTranscriptome.out.bam
4.0K Log.final.out
28K Log.out
4.0K Log.progress.out
8.5M SJ.out.tab
4.0K _STARgenome
-rw-rw-r– 1 ubuntu ubuntu 75864947151 Apr 9 23:13 Aligned.out.sam
-rw-rw-r– 1 ubuntu ubuntu 19449197988 Apr 9 23:13 Aligned.toTranscriptome.out.bam
-rw-rw-r– 1 ubuntu ubuntu 1870 Apr 9 23:13 Log.final.out
-rw-rw-r– 1 ubuntu ubuntu 26491 Apr 9 23:13 Log.out
-rw-rw-r– 1 ubuntu ubuntu 3786 Apr 9 23:13 Log.progress.out
-rw-rw-r– 1 ubuntu ubuntu 8887960 Apr 9 23:13 SJ.out.tab
drwx—— 2 ubuntu ubuntu 4096 Apr 9 22:41 _STARgenome
Started job on | Apr 09 22:39:15
Started mapping on | Apr 09 22:42:43
Finished on | Apr 09 23:13:35
Mapping speed, Million of reads per hour | 204.28
Number of input reads | 105089150
Average input read length | 202
UNIQUE READS:
Uniquely mapped reads number | 96824009
Uniquely mapped reads % | 92.14%
Average mapped length | 200.92
Number of splices: Total | 41731310
Number of splices: Annotated (sjdb) | 41099477
Number of splices: GT/AG | 41211630
Number of splices: GC/AG | 342569
Number of splices: AT/AC | 42225
Number of splices: Non-canonical | 134886
Mismatch rate per base, % | 0.29%
Deletion rate per base | 0.02%
Deletion average length | 1.57
Insertion rate per base | 0.01%
Insertion average length | 1.48
MULTI-MAPPING READS:
Number of reads mapped to multiple loci | 6262912
% of reads mapped to multiple loci | 5.96%
Number of reads mapped to too many loci | 62642
% of reads mapped to too many loci | 0.06%
UNMAPPED READS:
% of reads unmapped: too many mismatches | 0.00%
% of reads unmapped: too short | 1.83%
% of reads unmapped: other | 0.02%
CHIMERIC READS:
Number of chimeric reads | 0
% of chimeric reads | 0.00%
399348542 + 0 in total (QC-passed reads + QC-failed reads)
260569470 + 0 secondary
0 + 0 supplementary
0 + 0 duplicates
399348542 + 0 mapped (100.00% : N/A)
138779072 + 0 paired in sequencing
69389536 + 0 read1
69389536 + 0 read2
138779072 + 0 properly paired (100.00% : N/A)
138779072 + 0 with itself and mate mapped
0 + 0 singletons (0.00% : N/A)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)