During sequencing, the nucleotide bases in a DNA or RNA sample (library) are determined by the sequencer. For each fragment in the library, a sequence is generated, besides called a read, which is but a succession of nucleotides.
Modern sequencing technologies tin generate a massive number of sequence reads in a single experiment. Nevertheless, no sequencing technology is perfect, and each instrument will generate different types and amount of errors, such every bit incorrect nucleotides existence called. These wrongly chosen bases are due to the technical limitations of each sequencing platform.
Therefore, it is necessary to sympathise, place and exclude fault-types that may impact the interpretation of downstream assay. Sequence quality control is therefore an essential outset pace in your analysis. Communicable errors early saves fourth dimension later on on.
Agenda
In this tutorial, nosotros will deal with:
Inspect a raw sequence file
Assess quality with FASTQE 🧬😎 - short reads only
Assess quality with FastQC - curt & long reads
Per base sequence quality
Per sequence quality scores
Per base sequence content
Per sequence GC content
Sequence Duplication Levels
Over-represented sequences
Trim and filter - short reads
Processing multiple datasets
Process paired-cease data
Assess quality with Nanoplot - Long reads only
Histogram of read lengths
Read lengths vs Average read quality plot using dots
Appraise quality with PycoQC - Nanopore but
Basecalled reads length
Basecalled reads length vs reads PHRED quality
Output over experiment time
Channel action over time
Inspect a raw sequence file
hands_on Hands-on: Data upload
Create a new history for this tutorial and give it a proper name
Tip: Creating a new history
Click the new-history icon at the superlative of the history panel.
If the new-history is missing:
Click on the galaxy-gear icon (History options) on the top of the history panel
Select the selection Create New from the menu
Tip: Renaming a history
Click on Unnamed history (or the current name of the history) (Click to rename history) at the top of your history panel
Type the new proper name
Press Enter
Import the file female_oral2.fastq-4143.gz from Zenodo or from the data library (ask your instructor) This is a microbiome sample from a serpent Jacques et al. 2021.
Open up the Galaxy Upload Managing director (galaxy-upload on the summit-right of the tool panel)
Select Paste/Fetch Information
Paste the link into the text field
Press Start
Close the window
Tip: Importing data from a data library
Every bit an alternative to uploading the information from a URL or your computer, the files may as well have been fabricated available from a shared data library:
Go into Shared information (top panel) then Data libraries
Navigate to the correct folder as indicated by your instructor
Select the desired files
Click on the To History push button near the top and select every bit Datasets from the dropdown carte
In the pop-up window, select the history y'all desire to import the files to (or create a new ane)
Click on Import
Rename the imported dataset to Reads.
We just imported a file into Galaxy. This file is similar to the data we could get directly from a sequencing facility: a FASTQ file.
hands_on Hands-on: Audit the FASTQ file
Inspect the file by clicking on the galaxy-centre (eye) icon
Although it looks complicated (and mayhap it is), the FASTQ format is like shooting fish in a barrel to understand with a niggling decoding.
Each read, representing a fragment of the library, is encoded by iv lines:
Line
Description
1
Always begins with @ followed past the data about the read
2
The actual nucleic sequence
3
Always begins with a + and contains sometimes the same info in line ane
4
Has a string of characters which represent the quality scores associated with each base of the nucleic sequence; must have the same number of characters as line 2
And so for case, the commencement sequence in our file is:
It means that the fragment named @M00970 corresponds to the Dna sequence GTGCCAGCCGCCGCGGTAGTCCGACGTGGCTGTCTCTTATACACATCTCCGAGCCCACGAGACCGAAGAACATCTCGTATGCCGTCTTCTGCTTGAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAGAAGCAAATGACGATTCAAGAAAGAAAAAAACACAGAATACTAACAATAAGTCATAAACATCATCAACATAAAAAAGGAAATACACTTACAACACATATCAATATCTAAAATAAATGATCAGCACACAACATGACGATTACCACACATGTGTACTACAAGTCAACTA and this sequence has been sequenced with a quality GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGFGGGFGGGGGGAFFGGFGGGGGGGGFGGGGGGGGGGGGGGFGGG+38+35*311*half-dozen,,31=******441+++0+0++0+*i*2++2++0*+*2*02*/***ane*+++0+0++38++00++++++++++0+0+two++*+*+*+*+*****+0**+0**+***+)*.***1**//*)***)/)*)))*)))*),)0(((-((((-.(four(,,))).,(())))))).)))))))-))-(.
But what does this quality score mean?
The quality score for each sequence is a string of characters, i for each base of operations of the nucleic sequence, used to characterize the probability of mis-identification of each base. The score is encoded using the ASCII character table (with some historical differences):
So there is an ASCII character associated with each nucleotide, representing its Phred quality score, the probability of an incorrect base of operations call:
Phred Quality Score
Probability of incorrect base call
Base of operations call accurateness
10
1 in 10
90%
20
1 in 100
99%
xxx
1 in one thousand
99.ix%
twoscore
i in 10,000
99.99%
50
ane in 100,000
99.999%
60
1 in 1,000,000
99.9999%
question Questions
Which ASCII character corresponds to the worst Phred score for Illumina 1.8+?
What is the Phred quality score of the 3rd nucleotide of the 1st sequence?
What is the accuracy of this 3rd nucleotide?
solution Solution
The worst Phred score is the smallest one, and then 0. For Illumina 1.8+, it corresponds to the ! grapheme.
The 3rd nucleotide of the 1st sequence has a ASCII grapheme G, which correspond to a score of 38.
The corresponding nucleotide Thousand has an accurateness of almost 99.99%
When looking at the file in Milky way, it looks similar near the nucleotides have a high score (G corresponding to a score 38). Is it true for all sequences? And along the full sequence length?
Assess quality with FASTQE 🧬😎 - short reads only
To accept a expect at sequence quality along all sequences, nosotros tin use FASTQE. It is an open-source tool that provides a simple and fun way to quality command raw sequence data and print them as emoji. You can utilize it to give a quick impression of whether your data has any problems of which y'all should exist aware before doing any farther analysis.
hands_on Hands-on: Quality check
FASTQETool: toolshed.g2.bx.psu.edu/repos/iuc/fastqe/fastqe/0.2.six+galaxytwo with the following parameters
param-files"FastQ data": Reads
param-select"Score types to show": Mean
Inspect the generated HTML file
Rather than looking at quality scores for each individual read, FASTQE looks at quality collectively across all reads within a sample and can calculate the mean for each nucleotide position along the length of the reads. Below shows the hateful values for this dataset.
Figure one: FASTQE hateful scores
You can see the score for each emoji here. The emojis below, with Phred scores less than 20, are the ones we hope nosotros don't see much.
Phred Quality Score
ASCII code
Emoji
0
!
🚫
i
"
❌
2
#
👺
3
$
💔
4
%
🙅
5
&
👾
six
'
👿
7
(
💀
8
)
👻
nine
*
🙈
10
+
🙉
eleven
,
🙊
12
-
🐵
xiii
.
😿
xiv
/
😾
15
0
🙀
16
1
💣
17
2
🔥
18
3
😡
xix
4
💩
question Questions
What is the lowest mean score in this dataset?
solution Solution
The lowest score in this dataset is 😿 13.
Appraise quality with FastQC - brusque & long reads
An additional or alternative way nosotros can check sequence quality is with FastQC. Information technology provides a modular ready of analyses which you can use to check whether your data has any issues of which yous should be aware before doing whatsoever further analysis. We can apply information technology, for example, to appraise whether there are known adapters nowadays in the data. We'll run information technology on the FASTQ file.
hands_on Easily-on: Quality check
FASTQCTool: toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0 with the post-obit parameters
param-files"Raw read information from your electric current history": Reads
Inspect the generated HTML file
question Questions
Which Phred encoding is used in the FASTQ file for these sequences?
solution Solution
The Phred scores are encoded using Sanger / Illumina 1.9 (Encoding in the top tabular array).
Per base sequence quality
With FastQC nosotros tin can use the per base of operations sequence quality plot to check the base of operations quality of the reads, similar to what we did with FASTQE.
Figure two: Per base of operations sequence quality
On the ten-centrality are the base position in the read. In this example, the sample contains reads that are up to 296 bp long.
details Non uniform x-centrality
The x-axis is not always compatible. When you have long reads, some binning is applied to keep things compact. Nosotros can see that in our sample. Information technology starts out with individual i-10 bases. Afterward that, bases are binned beyond a window a sure number of bases wide. Information binning means grouping and is a data pre-processing technique used to reduce the effects of minor ascertainment errors. The number of base positions binned together depends on the length of the read. With reads >50bp, the latter part of the plot will written report aggregate statistics for 5bp windows. Shorter reads will accept smaller windows and longer reads larger windows. Binning can be removed when running FastQC by setting the paramter "Disable grouping of bases for reads >50bp" to Yes.
For each position, a boxplot is fatigued with:
the median value, represented by the fundamental red line
the inter-quartile range (25-75%), represented by the xanthous box
the 10% and 90% values in the upper and lower whiskers
the hateful quality, represented by the bluish line
The y-centrality shows the quality scores. The college the score, the meliorate the base of operations call. The groundwork of the graph divides the y-centrality into very proficient quality scores (light-green), scores of reasonable quality (orange), and reads of poor quality (red).
It is normal with all Illumina sequencers for the median quality score to beginning out lower over the outset 5-7 bases and to so rise. The quality of reads on most platforms will drop at the end of the read. This is often due to betoken disuse or phasing during the sequencing run. The recent developments in chemistry applied to sequencing has improved this somewhat, just reads are now longer than e'er.
details Betoken decay and phasing
Point decay
The fluorescent signal intensity decays with each cycle of the sequencing procedure. Due to the degrading fluorophores, a proportion of the strands in the cluster are not beingness elongated. The proportion of the betoken being emitted continues to decrease with each cycle, yielding to a decrease of quality scores at the 3' finish of the read.
Phasing
The bespeak starts to blur with the increase of number of cycles because the cluster looses synchronicity. Every bit the cycles progress, some strands get random failures of nucleotides to comprise due to:
Incomplete removal of the 3' terminators and fluorophores
Incorporation of nucleotides without constructive 3' terminators
This leads to a decrease in quality scores at the 3' terminate of the read.
details Other sequence quality profiles
These are some per base of operations sequence quality profiles that can indicate issues with the sequencing.
Overclustering
Sequencing facilities can overcluster the catamenia cells. Information technology results in small-scale distances betwixt clusters and an overlap in the signals. Two clusters can be interpreted equally a single cluster with mixed fluorescent signals beingness detected, decreasing signal purity. Information technology generates lower quality scores beyond the entire read.
Instrumentation breakdown
Some issues tin occasionally happen with the sequencing instruments during a run. Any sudden drop in quality or a large percentage of low quality reads across the read could indicate a problem at the facility. Some examples of such issues:
Manifold flare-up
Cycles loss
Read 2 failure
With such information, the sequencing facility should exist contacted for word. Often, a resequencing then is needed (and from our feel too offered by the company).
question Questions
How does the hateful quality score change along the sequence?
Is this tendency seen in all sequences?
solution Solution
The mean quality score (blueish line) drops well-nigh midway though these sequences. It is common for the hateful quality to drop towards the end of the sequences, as the sequencers are incorporating more wrong nucleotides at the cease. However, in this sample there is a very large drop in quality from the centre onwards.
The box plots are getting wider from position ~100. Information technology means a lot of sequences have their score dropping from the eye of the sequence. Afterward 100 nucleotides, more than than 10% of the sequences have scores below twenty.
When the median quality is beneath a Phred score of ~20, we should consider trimming away bad quality bases from the sequence. Nosotros will explain that process in the Trim and filter department.
Adapter Content
Effigy 3: Adapter Content
The plot shows the cumulative pct of reads with the different adapter sequences at each position. One time an adapter sequence is seen in a read it is counted equally being present right through to the stop of the read then the percentage increases with the read length. FastQC can detect some adapters by default (eastward.thousand. Illumina, Nextera), for others nosotros could provide a contaminants file every bit an input to the FastQC tool.
Ideally Illumina sequence data should not have any adapter sequence present. But with long reads, some of the library inserts are shorter than the read length resulting in read-through to the adapter at the 3' end of the read. This microbiome sample has relatively long reads and we can see Nextera dapater has been detected.
details Other adapter content profiles
Adapter content may also exist detected with RNA-Seq libraries where the distribution of library insert sizes is varied and likely to include some short inserts.
We tin run an trimming tool such as Cutadapt to remove this adapter. We volition explain that process in the filter and trim section.
Per tile sequence quality
This plot enables you to wait at the quality scores from each tile across all of your bases to see if there was a loss in quality associated with only one part of the flowcell. The plot shows the deviation from the boilerplate quality for each flowcell tile. The hotter colours signal that reads in the given tile take worse qualities for that position than reads in other tiles. With this sample, you can see that certain tiles bear witness consistently poor quality, particularly from ~100bp onwards. A good plot should be blue all over.
Figure iv: Per tile sequence quality
This plot will only appear for Illumina library which retains its original sequence identifiers. Encoded in these is the flowcell tile from which each read came.
details Other tile quality profiles
In some cases, the chemicals used during sequencing becoming a bit exhausted over the time and the last tiles got worst chemicals which makes the sequencing reactions a bit error-prone. The "Per tile sequence quality" graph will and so take some horizontal lines like this:
Per sequence quality scores
It plots the average quality score over the full length of all reads on the x-axis and gives the total number of reads with this score on the y-axis:
Figure v: Per sequence quality scores
The distribution of average read quality should be tight acme in the upper range of the plot. It can also report if a subset of the sequences have universally low quality values: it can happen because some sequences are poorly imaged (on the border of the field of view etc), however these should represent only a minor percentage of the total sequences.
Per base sequence content
Figure vi: Per base of operations sequence content for a Dna library
"Per Base Sequence Content" plots the percentage of each of the four nucleotides (T, C, A, G) at each position across all reads in the input sequence file. Equally for the per base sequence quality, the x-axis is non-compatible.
In a random library we would look that there would be petty to no difference between the 4 bases. The proportion of each of the four bases should remain relatively constant over the length of the read with %A=%T and %Grand=%C, and the lines in this plot should run parallel with each other. This is amplicon information, where 16S DNA is PCR amplified and sequenced, so we'd expect this plot to have some bias and not show a random distribution.
details Biases past library type
It's worth noting that some library types will always produce biased sequence composition, normally at the beginning of the read. Libraries produced past priming using random hexamers (including well-nigh all RNA-Seq libraries), and those which were fragmented using transposases, volition contain an intrinsic bias in the positions at which reads get-go (the first x-12 bases). This bias does not involve a specific sequence, but instead provides enrichment of a number of different K-mers at the v' end of the reads. Whilst this is a truthful technical bias, it isn't something which tin can be corrected by trimming and in near cases doesn't seem to adversely affect the downstream analysis. It will, however, produce a warning or fault in this module.
ChIP-seq data can also encounter read commencement sequence biases in this plot if fragmenting with transposases. With bisulphite converted data, e.g. HiC data, a separation of Thousand from C and A from T is expected:
At the end, there is an overall shift in the sequence limerick. If the shift correlates with a loss of sequencing quality, it can be suspected that miscalls are made with a more even sequence bias than bisulphite converted libraries. Trimming the sequences fixed this problem, but if this hadn't been done it would have had a dramatic issue on the methylation calls which were fabricated.
question Questions
Why is there a warning for the per-base of operations sequence content graphs?
solution Solution
In the commencement of sequences, the sequence content per base of operations is non really proficient and the percentages are not equal, as expected for 16S amplicon information.
Per sequence GC content
Figure seven: Per sequence GC content
This plot displays the number of reads vs. pct of bases G and C per read. It is compared to a theoretical distribution assuming an uniform GC content for all reads, expected for whole genome shotgun sequencing, where the central peak corresponds to the overall GC content of the underlying genome. Since the GC content of the genome is not known, the modal GC content is calculated from the observed data and used to build a reference distribution.
An unusually-shaped distribution could indicate a contaminated library or some other kind of biased subset. A shifted normal distribution indicates some systematic bias, which is independent of base position. If there is a systematic bias which creates a shifted normal distribution then this won't be flagged as an error by the module since it doesn't know what your genome'due south GC content should be.
But in that location are besides other situations in which an unusually-shaped distribution may occur. For example, with RNA sequencing there may be a greater or lesser distribution of mean GC content amongst transcripts causing the observed plot to be wider or narrower than an ideal normal distribution.
question Questions
Why is there a fail for the per sequence GC content graphs?
solution Solution
There are multiple peaks. This tin be indicative of unexpected contagion, such as adapter, rRNA or overrepresented sequences. Or information technology may be normal if it is amplicon information or you have highly abundant RNA-seq transcripts.
Sequence length distribution
This plot shows the distribution of fragment sizes in the file which was analysed. In many cases this will produce a simple plot showing a peak but at i size, just for variable length FASTQ files this will testify the relative amounts of each different size of sequence fragment. Our plot shows variable length as we trimmed the data. The biggest peak is at 296bp only in that location is a second big top at ~100bp. And so even though our sequences range up to 296bp in length, a lot of the skillful-quality sequences are shorter. This corresponds with the drop nosotros saw in the sequence quality at ~100bp and the cerise stripes starting at this position in the per tile sequence quality plot.
Figure 8: Sequence length distribution
Some loftier-throughput sequencers generate sequence fragments of compatible length, but others can contain reads of widely varying lengths. Even within uniform length libraries some pipelines will trim sequences to remove poor quality base calls from the end or the first \(n\) bases if they lucifer the kickoff \(n\) bases of the adapter upward to ninety% (by default), with sometimes \(n = 1\).
Sequence Duplication Levels
The graph shows in bluish the percentage of reads of a given sequence in the file which are nowadays a given number of times in the file:
Figure 9: Sequence Duplication Levels
In a various library virtually sequences will occur only once in the last ready. A low level of duplication may indicate a very high level of coverage of the target sequence, simply a high level of duplication is more probable to indicate some kind of enrichment bias.
Two sources of indistinguishable reads tin be constitute:
PCR duplication in which library fragments accept been over-represented due to biased PCR enrichment
It is a business organization because PCR duplicates misrepresent the true proportion of sequences in the input.
Truly over-represented sequences such as very arable transcripts in an RNA-Seq library or in amplicon data (like this sample)
Information technology is an expected case and not of business concern because it does faithfully represent the input.
details More than details about duplication
FastQC counts the degree of duplication for every sequence in a library and creates a plot showing the relative number of sequences with unlike degrees of duplication. In that location are ii lines on the plot:
Blue line: distribution of the duplication levels for the full sequence set
Blood-red line: distribution for the de-duplicated sequences with the proportions of the deduplicated set which come from unlike duplication levels in the original data.
For whole genome shotgun information information technology is expected that nearly 100% of your reads will be unique (appearing only 1 time in the sequence data). Almost sequences should autumn into the far left of the plot in both the ruby-red and blueish lines. This indicates a highly diverse library that was not over sequenced. If the sequencing depth is extremely high (e.k. > 100x the size of the genome) some inevitable sequence duplication tin appear: in that location are in theory simply a finite number of completely unique sequence reads which can exist obtained from any given input DNA sample.
More specific enrichments of subsets, or the presence of depression complexity contaminants will tend to produce spikes towards the right of the plot. These high duplication peaks will most often appear in the blue trace as they make up a high proportion of the original library, but usually disappear in the red trace as they make upwards an insignificant proportion of the deduplicated gear up. If peaks persist in the cherry-red trace and so this suggests that there are a big number of different highly duplicated sequences which might indicate either a contaminant set or a very astringent technical duplication.
It is ordinarily the instance for RNA sequencing where there is some very highly abundant transcripts and some lowly abundant. It is expected that indistinguishable reads will be observed for high abundance transcripts:
Over-represented sequences
A normal high-throughput library will contain a various set of sequences, with no individual sequence making upwards a tiny fraction of the whole. Finding that a unmarried sequence is very over-represented in the set either means that it is highly biologically meaning, or indicates that the library is contaminated, or non every bit diverse as expected.
FastQC lists all of the sequence which make upward more than 0.1% of the total. For each over-represented sequence FastQC will look for matches in a database of common contaminants and volition written report the best striking it finds. Hits must be at least 20bp in length and accept no more than one mismatch. Finding a hit doesn't necessarily hateful that this is the source of the contamination, but may point y'all in the right direction. Information technology'due south as well worth pointing out that many adapter sequences are very similar to each other so you may go a hit reported which isn't technically right, simply which has a very similar sequence to the actual match.
RNA sequencing data may have some transcripts that are and so abundant that they annals as over-represented sequence. With Dna sequencing data no single sequence should be present at a loftier enough frequency to be listed, but we can sometimes see a small percentage of adapter reads.
question Questions
How could we find out what the overrepreseented sequences are?
solution Solution
We tin can Smash overrepresented sequences to run into what they are. In this case, if we accept the top overrepresented sequence
and use blastn against the default Nucleotide (nr/nt) database we don't get whatever hits. But if nosotros utilize VecScreen nosotros encounter it is the Nextera adapter.
Figure x: Nextera adapter
details More than details about other FastQC plots
Per base of operations N content
Figure eleven: Per base North content
If a sequencer is unable to brand a base call with sufficient confidence, it will write an "N" instead of a conventional base call. This plot displays the pct of base calls at each position or bin for which an N was chosen.
It'south not unusual to see a very high proportion of Ns appearing in a sequence, especially near the end of a sequence. Only this curve should never rises noticeably above zero. If it does this indicates a problem occurred during the sequencing run. In the example below, an fault acquired the instrument to exist unable to call a base for approximately 20% of the reads at position 29:
Kmer Content
This plot not output past default. As stated in the tool class, if y'all want this module information technology needs to be enabled using a custom Submodule and limits file. With this module, FastQC does a generic assay of all of the short nucleotide sequences of length k (kmer, with k = 7 past default) starting at each position along the read in the library to observe those which exercise not have an even coverage through the length of your reads. Any given kmer should be evenly represented beyond the length of the read.
FastQC will report the list of kmers which appear at specific positions with a greater frequency than expected. This can exist due to different sources of bias in the library, including the presence of read-through adapter sequences edifice up on the end of the sequences. The presence of any overrepresented sequences in the library (such every bit adapter dimers) causes the kmer plot to be dominated past the kmer from these sequences. Any biased kmer due to other interesting biases may be then diluted and not easy to run across.
The post-obit example is from a high-quality DNA-Seq library. The biased kmers nearby the commencement of the read likely are due to slight sequence dependent efficiency of DNA shearing or a result of random priming:
Figure 12: Kmer content
This module can be very hard to interpret. The adapter content plot and overrepesented sequences table are easier to interpret and may give you enough information without needing this plot. RNA-seq libraries may take highly represented kmers that are derived from highly expressed sequences. To learn more than nearly this plot, please check the FastQC Kmer Content documentation.
We tried to explain here there different FastQC reports and some apply cases. More virtually this and besides some mutual adjacent-generation sequencing problems can be found on QCFAIL.com
details Specific problem for alternate library types
Small/micro RNA
In small RNA libraries, we typically have a relatively small gear up of unique, curt sequences. Small RNA libraries are not randomly sheared before adding sequencing adapters to their ends: all the reads for specific classes of microRNAsouthward volition be identical. Information technology will result in:
Extremely biased per base of operations sequence content
Extremely narrow distribution of GC content
Very high sequence duplication levels
Abundance of overrepresented sequences
Read-through into adapters
Amplicon
Amplicon libraries are prepared by PCR amplification of a specific target. For case, the V4 hypervariable region of the bacterial 16S rRNA gene. All reads from this type of library are expected to be almost identical. It volition effect in:
Extremely biased per base of operations sequence content
Extremely narrow distribution of GC content
Very high sequence duplication levels
Abundance of overrepresented sequences
Bisulfite or Methylation sequencing
With Bisulfite or methylation sequencing, the majority of the cytosine (C) bases are converted to thymine (T). It will result in:
Biased per base sequence content
Biased per sequence GC content
Adapter dimer contagion
Whatsoever library blazon may comprise a very small percentage of adapter dimer (i.e. no insert) fragments. They are more probable to be found in amplicon libraries constructed entirely by PCR (by formation of PCR primer-dimers) than in Deoxyribonucleic acid-Seq or RNA-Seq libraries synthetic by adapter ligation. If a sufficient fraction of the library is adapter dimer information technology volition become noticeable in the FastQC report:
Driblet in per base of operations sequence quality subsequently base of operations 60
Possible bi-modal distribution of per sequence quality scores
Distinct pattern observed in per bases sequence content up to base of operations 60
Spike in per sequence GC content
Overrepresented sequence matching adapter
Adapter content > 0% starting at base ane
Trim and filter - brusk reads
The quality drops in the middle of these sequences. This could crusade bias in downstream analyses with these potentially incorrectly called nucleotides. Sequences must exist treated to reduce bias in downstream assay. Trimming can help to increase the number of reads the aligner or assembler are able to succesfully utilize, reducing the number of reads that are unmapped or unassembled. In general, quality treatments include:
Trimming/cutting/masking sequences
from low quality score regions
beginning/end of sequence
removing adapters
Filtering of sequences
with low mean quality score
likewise brusque
with too many ambiguous (N) bases
To achieve this task we volition use Cutadapt Marcel 2011, a tool that enhances sequence quality by automating adapter trimming too every bit quality control. Nosotros will:
Trim low-quality bases from the ends. Quality trimming is done earlier whatever adapter trimming. We volition set the quality threshold as xx, a commonly used threshold, see more here.
Trim adapter with Cutadapt. For that we need to supply the sequence of the adapter. In this sample, Nextera is the adapter that was detected. We can find the sequence of the Nextera adapter on the Illumina website hither CTGTCTCTTATACACATCT. We volition trim that sequence from the 3' end of the reads.
Filter out sequences with length < 20 afterwards trimming
hands_on Hands-on: Improvement of sequence quality
CutadaptTool: toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/3.iv+galaxy2 with the post-obit parameters
"Unmarried-end or Paired-end reads?": Single-end
param-file"Reads in FASTQ format": Reads (Input dataset)
tip Tip: Files not selectable?
If your FASTQ file cannot exist selected, you lot might check whether the format is FASTQ with Sanger-scaled quality values (fastqsanger.gz). You tin can edit the data type by clicking on the pencil symbol.
What % reads take been trimmed considering of bad quality?
What % reads have been removed considering they were too brusk?
solution Solution
58.6% reads incorporate adapter (Reads with adapters:)
35.1% reads take been trimmed because of bad quality (Quality-trimmed:)
0 % reads were removed considering they were too brusque
details Trimming with Cutadapt
One of the biggest reward of Cutadapt compared to other trimming tools (e.g. TrimGalore!) is that it has a good documentation explaining how the tool works in item.
Cutadapt quality trimming algorithm consists of 3 simple steps:
Decrease the chosen threshold value from the quality value of each position
Compute a fractional sum of these differences from the finish of the sequence to each position (every bit long equally the fractional sum is negative)
Cut at the minimum value of the fractional sum
In the post-obit example, we presume that the iii' stop is to be quality-trimmed with a threshold of ten and we have the post-obit quality values
Subtract the threshold
32 thirty 16 17 -2 -3 one -6 -8 -seven
Add up the numbers, starting from the 3' terminate (fractional sums) and stop early if the sum is greater than zero
(seventy) (38) 8 -8 -25 -23 -20, -21 -15 -seven
The numbers in parentheses are not computed (because 8 is greater than cipher), just shown hither for completeness.
Choose the position of the minimum (-25) as the trimming position
Therefore, the read is trimmed to the outset four bases, which have quality values
Note that therefore, positions with a quality value larger than the chosen threshold are as well removed if they are embedded in regions with lower quality (the partial sum is decreasing if the quality values are smaller than the threshold). The advantage of this procedure is that it is robust against a small number of positions with a quality higher than the threshold.
Alternatives to this procedure would exist:
Cutting afterward the first position with a quality smaller than the threshold
Sliding window arroyo
The sliding window approach checks that the average quality of each sequence window of specified length is larger than the threshold. Note that in contrast to cutadapt's approach, this approach has ane more parameter and the robustness depends of the length of the window (in combination with the quality threshold). Both approaches are implemented in Trimmomatic.
Nosotros tin can examine our trimmed information with FASTQE and/or FastQC.
hands_on Hands-on: Checking quality after trimming
FASTQETool: toolshed.g2.bx.psu.edu/repos/iuc/fastqe/fastqe/0.2.6+galaxy2 : Re-run FASTQE with the post-obit parameters
param-files"FastQ data": Cutadapt Read one Output
param-select"Score types to testify": Mean
Inspect the new FASTQE written report
question Questions
Compare the FASTQE output to the previous one earlier trimming above. Has sequence quality been improved?
Tip: Using the Scratchbook to view multiple datasets
If you would like to view 2 or more than datasets at once, you can utilize the Scratchbook feature in Galaxy:
Click on the Scratchbook icon galaxy-scratchbook on the top menu bar.
You lot should see a little checkmark on the icon now
Viewmilky way-center a dataset by clicking on the eye icon galaxy-center to view the output
You should see the output in a window overlayed over Milky way
You tin resize this window by dragging the lesser-right corner
Click outside the file to exit the Scratchbook
Viewgalaxy-eye a second dataset from your history
Yous should now see a second window with the new dataset
This makes it easier to compare the two outputs
Echo this for as many files as you would like to compare
You tin can turn off the Scratchbookgalaxy-scratchbook by clicking on the icon once more
solution Solution
Yes, the quality score emojis await meliorate (happier) now.
Effigy 13: Before trimmingFigure fourteen: After trimming
With FASTQE nosotros tin can run into we improved the quality of the bases in the dataset.
Nosotros tin can also, or instead, check the quality-controlled data with FastQC.
FASTQCTool: toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0 with the following parameters
param-files"Curt read data from your electric current history": Cutadapt Read ane Output
Audit the generated HTML file
question Questions
Does the per base sequence quality look ameliorate?
Is the adapter gone?
solution Solution
Yes. The vast bulk of the bases take a quality score higher up 20 now.
Figure 15: Per base sequence quality
Yes. No adapter is detected now.
With FastQC nosotros can see we improved the quality of the bases in the dataset and removed the adapter.
details Other FastQC plots after trimming
We have some red stripes every bit we've trimmed those regions from the reads.
We now have one tiptop of high quality instead of i high and one lower quality that we had previously.
Nosotros don't have equal representation of the bases as before equally this is amplicon data.
We now have a single main GC peak due to removing the adapter.
This is the same as before as we don't accept whatever Ns in these reads.
We at present have multiple peaks and a range of lengths, instead of the unmarried meridian with had before trimming when all sequences were the same length.
question Questions
What does the height overrepresented sequence GTGTCAGCCGCCGCGGTAGTCCGACGTGG correspond to?
solution Solution
If we take the top overrepresented sequence
>overrep_seq1_after GTGTCAGCCGCCGCGGTAGTCCGACGTGG
and use blastn confronting the default Nucleotide (nr/nt) database we see the top hits are to 16S rRNA genes. This makes sense as this is 16S amplicon data, where the 16S gene is PCR amplified.
Processing multiple datasets
Process paired-cease data
With paired-end sequencing, the fragments are sequenced from both sides. This approach results in two reads per fragment, with the start read in forward orientation and the 2nd read in reverse-complement orientation. With this technique, nosotros have the advantage to go more information well-nigh each DNA fragment compared to reads sequenced by merely single-end sequencing:
The distance between both reads is known and therefore is additional information that can improve read mapping.
Paired-end sequencing generates 2 FASTQ files:
One file with the sequences respective to frontward orientation of all the fragments
One file with the sequences corresponding to contrary orientation of all the fragments
Usually nosotros recognize these two files which belong to 1 sample by the proper noun which has the same identifier for the reads only a unlike extension, e.g. sampleA_R1.fastq for the forrard reads and sampleA_R2.fastq for the reverse reads. It tin as well be _f or _1 for the frontward reads and _r or _2 for the reverse reads.
The data we analyzed in the previous step was single-end data and so we volition import a paired-end RNA-seq dataset to use. We will run FastQC and amass the ii reports with MultiQC Ewels et al. 2016.
hands_on Hands-on: Assessing the quality of paired-cease reads
Import the paired-end readsGSM461178_untreat_paired_subset_1.fastq and GSM461178_untreat_paired_subset_2.fastq from Zenodo or from the data library (ask your instructor)
FASTQCTool: toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc/0.73+galaxy0 with both datasets:
param-files"Raw read data from your electric current history": both the uploaded datasets.
Tip: Select multiple datasets
Click on param-filesMultiple datasets
Select several files past keeping the Ctrl (orCOMMAND) key pressed and clicking on the files of interest
MultiQCTool: toolshed.g2.bx.psu.edu/repos/iuc/multiqc/multiqc/1.9+milky wayone with the following parameters to amass the FastQC reports of both forward and contrary reads
In "Results"
"Which tool was used generate logs?": FastQC
In "FastQC output"
"Type of FastQC output?": Raw data
param-files"FastQC output": Raw data files (output of both FastQCtool)
Inspect the webpage output from MultiQC.
question Questions
What do you retrieve about the quality of the sequences?
What should nosotros exercise?
solution Solution
The quality of the sequences seems worse for the reverse reads than for the forwards reads:
Per Sequence Quality Scores: distribution more on the left, i.due east. a lower mean quality of the sequences
Per base sequence quality: less smoothen curve and stronger decrease at the terminate with a mean value beneath 28
Per Base Sequence Content: stronger bias at the beginning and no clear stardom between C-G and A-T groups
The other indicators (adapters, duplication levels, etc) are similar.
We should trim the end of the sequences and filter them with Cutadapttool
With paired-terminate reads the boilerplate quality scores for forwards reads will almost ever be higher than for opposite reads.
Subsequently trimming, opposite reads will exist shorter because of their quality and so will exist eliminated during the filtering pace. If ane of the reverse reads is removed, its corresponding frontwards read should exist removed too. Otherwise nosotros will get different number of reads in both files and in different order, and guild is important for the side by side steps. Therefore it is important to treat the frontwards and opposite reads together for trimming and filtering.
hands_on Hands-on: Improving the quality of paired-stop information
CutadaptTool: toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/3.iv+galaxy2 with the following parameters
No adapters were found in these datasets. When you process your own data and y'all know which adapter sequences were used during library preparation, y'all should provide their sequences here.
In "Filter Options"
"Minimum length": 20
In "Read Modification Options"
"Quality cutoff": 20
In "Output Options"
"Report": Yes
Inspect the generated txt file (Report)
question Questions
How many basepairs has been removed from the reads because of bad quality?
How many sequence pairs have been removed because they were as well brusk?
solution Solution
44,164 bp (Quality-trimmed:) for the frontwards reads and 138,638 bp for the reverse reads.
1,376 sequences accept been removed considering at least ane read was shorter than the length cutoff (322 when only the forward reads were analyzed).
In improver to the report, Cutadapt generates two files:
Read 1 with the trimmed and filtered forwards reads
Read 2 with the trimmed and filtered reverse reads
These datasets can be used for the downstream analysis, e.g. mapping.
question Questions
What kind of alignment is used for finding adapters in reads?
What is the criterion to choose the all-time adapter alignment?
solution Solution
Semi-global alignment, i.due east., but the overlapping part of the read and the adapter sequence is used for scoring.
An alignment with maximum overlap is computed that has the smallest number of mismatches and indels.
Appraise quality with Nanoplot - Long reads just
In case of long reads, we can check sequence quality with Nanoplot (De Coster et al. 2018). It provides basic statistics with nice plots for a fast quality control overview.
hands_on Hands-on: Quality cheque of long reads
Create a new history for this part and requite it a proper proper name
Import the PacBio HiFi readsm64011_190830_220126.Q20.subsample.fastq.gz from Zenodo
param-select"Specify the bivariate format of the plots.": dot, kde
param-select"Evidence the N50 mark in the read length histogram.": Yep
Inspect the generated HTML file
question Questions
What is the hateful Qscore ?
solution Solution
The Qscore is around Q32. In case of PacBio CLR and Nanopore, it's around Q12 and shut to Q31 for Illumina (NovaSeq 6000).
Figure 16: Comparison of Qscore between Illumina, PacBio and Nanopore
Definition: Qscores is the average per-base fault probability, expressed on the log (Phred) calibration
What is the median, mean and N50?
solution Solution
The median, the hateful read length and the N50 as well are shut to xviii,000bp. For PacBio HiFi reads, the majority of the reads are by and large near this value as the library preparation includes a size selection pace. For other technologies like PacBio CLR and Nanopore, it is larger and mostly depends on the quality of your DNA extraction.
Histogram of read lengths
This plot shows the distribution of fragment sizes in the file that was analyzed. Unlike most of Illumina runs, long reads have a variable length and this will prove the relative amounts of each different size of sequence fragment. In this case, the distribution of read length is centered nigh 15kbp just the results can be very different depending of your experiment.
Figure 17: Histogram of read length
Read lengths vs Average read quality plot using dots
This plot shows the distribution of fragment sizes according to the Qscore in the file which was analysed. In general, there is no link betwixt read length and read quality but this representation allows to visualize both information into a single plot and detect possible aberrations. In runs with a lot of short reads the shorter reads are sometimes of lower quality than the rest.
Figure 18: Histogram of read length
question Questions
Looking at "Read lengths vs Boilerplate read quality plot using dots plot". Did yous detect something unusual with the Qscore? Can you lot explicate it?
solution Solution
There is no reads under Q20. The qualification for HiFi reads is:
A minimal number of three subreads
A read Qscore >=20
Effigy 19: PacBio HiFi sequencing
Appraise quality with PycoQC - Nanopore simply
PycoQC (Leger and Leonardi 2019) is a data visualisation and quality control tool for nanopore information. In contrast to FastQC/Nanoplot it needs a specific sequencing_summary.txt file generated past Oxford nanopore basecallers such as Guppy or the older albacore basecaller.
I of the strengths of PycoQC is that it is interactive and highly customizable, e.g., plots can be cropped, yous can zoom in and out, sub-select areas and export figures.
hands_on Hands-on: Quality cheque of Nanopore reads
Create a new history for this part and give it a proper noun
Import the nanopore readsnanopore_basecalled-guppy.fastq.gz and sequencing_summary.txt from Zenodo
~270k reads in full (see the Basecall summary table, "All reads") For most of basecalling profiles, Guppy will assign reads as "Pass" if the read Qscore is at least equal to 7.
What is the median, minimum and maximum read length, what is the N50?
solution Solution
The median read length and the N50 can exist found for all every bit well every bit for all passed reads, i.e., reads that passed Guppy quality settings (Qscore >= 7), in the basecall summary table. For the minimum (195bp) and maximum (256kbp) read lengths, it tin can be found with the read lengths plot.
Basecalled reads length
As for FastQC and Nanoplot, this plot shows the distribution of fragment sizes in the file that was analyzed. Equally for PacBio CLR/HiFi, long reads have a variable length and this volition show the relative amounts of each unlike size of sequence fragment. In this example, the distribution of read length is quite dispersed with a minimum read length for the passed reads effectually 200bp and a maximum length ~150,000bp.
Figure 20: Basecalled reads length
Basecalled reads PHRED quality
This plot shows the distribution of the Qscores (Q) for each read. This score aims to requite a global quality score for each read. The exact definition of Qscores is: the boilerplate per-base of operations error probability, expressed on the log (Phred) calibration. In instance of Nanopore data, the distribution is generally centered around 10 or 12. For old runs, the distribution can exist lower, as basecalling models are less precise than recent models.
Figure 21: Basecalled reads PHRED quality
Basecalled reads length vs reads PHRED quality
question Questions
What do the mean quality and the quality distribution of the run look similar?
solution Solution
The majority of the reads have a Qscore between 8 and 11 which is standard for Nanopore data. Beware that for the aforementioned data, the basecaller used (Albacor, Guppy, Bonito), the model (fast, hac, sup) and the tool version can give unlike results.
Equally for NanoPlot, this representation give a 2D visualisation of read Qscore according to the length.
Figure 22: Basecalled reads length vs reads PHRED quality
Output over experiment time
This representation gives data about sequenced reads over the time for a single run:
Each motion picture indicates a new loading of the flow cell (3 + the first load).
The contribution in full reads for each "refuel".
The product of reads is decreasing over time:
About of the material (Dna/RNA) is sequenced
Saturation of pores
Material/pores degradation
…
In this example, the contribution of each refueling is very low, and it can exist considered as a bad run. The "Cummulative" plot area (light blue) indicates that fifty% of all reads and almost fifty% of all bases were produced in the first 5h of the 25h experiment. Although it is normal that yield decreases over fourth dimension a decrease similar this is not a practiced sign.
Figure 23: Output over experiment time
details Other "Output over experiment time" profile
In this example, the data production over the time only slightly decreased over the 12h with a continuous increasing of cumulative data. This absence of a decreasing bend at the terminate of the run point that at that place is withal biological material on the flow cell. The run was ended earlier all was sequenced. Information technology's an fantabulous run, even can be considered as exceptional.
Read length over experiment time
question Questions
Did the read length change over fourth dimension? What could the reason exist?
solution Solution
In the current case the read length increases over the time of the sequencing run. 1 caption is that the adapter density is college for lots of short fragments and therefore the chance of a shorter fragment to attach to a pore is higher. Also, shorter molecules may move faster over the chip. Over time, however, the shorter fragments are becoming rarer and thus more long fragments attach to pores and are sequenced.
The read length over experiment time should be stable. Information technology tin slightly increment over the time as short fragments tend to be over-sequenced at the outset and are less present over the fourth dimension.
Figure 24: Read length over experiment time
Channel activity over fourth dimension
It gives an overview of available pores, pore usage during the experiment, inactive pores and shows if the loading of the menstruation cell is good (well-nigh all pores are used). In this example, the vast majority of channels/pores are inactive (white) throughout the sequencing run, and so the run can be considered as bad.
You would promise for a plot that it is dark well-nigh the X-axis, and with higher Y-values (increasing time) doesn't get besides low-cal/white. Depending if yous chose "Reads" or "Bases" on the left the colour indicates either number of bases or reads per fourth dimension interval
Figure 25: Aqueduct activeness over time
details Other "Channel action over fourth dimension" contour
In this example, almost all pores are active all forth the run (yellow/ruddy profile) which indicate an excellent run.
Determination
In this tutorial we checked the quality of FASTQ files to ensure that their data looks good before inferring any further information. This pace is the usual first step for analyses such as RNA-Seq, Scrap-Seq, or any other OMIC analysis relying on NGS information. Quality control steps are similar for any type of sequencing data:
Quality assessment with tools similar:
Short Reads: FASTQETool: toolshed.g2.bx.psu.edu/repos/iuc/fastqe/fastqe/0.2.6+galaxy2
Trimming and filtering for short reads with a tool like Cutadapttool
Key points
Perform quality command on every dataset before running whatsoever other bioinformatics assay
Assess the quality metrics and better quality if necessary
Bank check the touch of the quality control
Different tools are available to provide additional quality metrics
For paired-end reads analyze the forrad and opposite reads together
Frequently Asked Questions
Take questions about this tutorial? Check out the tutorial FAQ page or the FAQ folio for the Sequence analysis topic to meet if your question is listed there. If not, please ask your question on the GTN Gitter Aqueduct or the Milky way Help Forum
Useful literature
Further data, including links to documentation and original publications, regarding the tools, analysis techniques and the interpretation of results described in this tutorial can be institute here.
References
Marcel, M., 2011 Cutadapt removes adapter sequences from high-throughput sequencing reads. EMBnet.periodical 17: 10–12. http://journal.embnet.org/index.php/embnetjournal/article/view/200
Ewels, P., Yard. Magnusson, S. Lundin, and M. Käller, 2016 MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics 32: 3047–3048. https://academic.oup.com/bioinformatics/article/32/nineteen/3047/2196507
De Coster, W., Southward. D'Hert, D. T. Schultz, M. Cruts, and C. Van Broeckhoven, 2018 NanoPack: visualizing and processing long-read sequencing data. Bioinformatics 34: 2666–2669. 10.1093/bioinformatics/bty149
Leger, A., and T. Leonardi, 2019 pycoQC, interactive quality control for Oxford Nanopore Sequencing . Journal of Open up Source Software 4: 1236. 10.21105/joss.01236
Jacques, R. Chiliad. S., Due west. Grand. Maza, South. D. Robertson, A. Lonsdale, C. S. Murray et al., 2021 A Fun Introductory Command Line Lesson: Next Generation Sequencing Quality Analysis with Emoji! CourseSource viii: x.24918/cs.2021.17
Feedback
Did you utilize this textile equally an teacher? Experience gratis to requite the states feedback on how it went. Did you use this material as a learner or student? Click the class below to leave feedback.
Citing this Tutorial
Bérénice Batut, Maria Doyle, Alexandre Cormier, Anthony Bretaudeau, Laura Leroi, Erwan Corre, Stéphanie Robin, Erasmus+ Programme, 2021 Quality Control (Galaxy Training Materials). https://preparation.galaxyproject.org/training-cloth/topics/sequence-analysis/tutorials/quality-control/tutorial.html Online; accessed TODAY
Batut et al., 2018 Customs-Driven Information Assay Preparation for Biology Cell Systems x.1016/j.cels.2018.05.012
details BibTeX
@misc{sequence-analysis-quality-command, author = "Bérénice Batut and Maria Doyle and Alexandre Cormier and Anthony Bretaudeau and Laura Leroi and Erwan Corre and Stéphanie Robin and Erasmus+ Programme", title = "Quality Control (Milky way Grooming Materials)", year = "2021", month = "12", day = "14" url = "\url{https://training.galaxyproject.org/preparation-material/topics/sequence-analysis/tutorials/quality-command/tutorial.html}", note = "[Online; accessed TODAY]" } @article{Batut_2018, doi = {ten.1016/j.cels.2018.05.012}, url = {https://doi.org/10.1016%2Fj.cels.2018.05.012}, yr = 2018, month = {jun}, publisher = {Elsevier {BV}}, volume = {6}, number = {half dozen}, pages = {752--758.e1}, author = {B{\'{due east}}r{\'{e}}prissy Batut and Saskia Hiltemann and Andrea Bagnacani and Dannon Baker and Vivek Bhardwaj and Clemens Blank and Anthony Bretaudeau and Loraine Brillet-Gu{\'{eastward}}guen and Martin {\five{C}}ech and John Chilton and Dave Clements and Olivia Doppelt-Azeroual and Anika Erxleben and Mallory Ann Freeberg and Simon Gladman and Youri Hoogstrate and Hans-Rudolf Hotz and Torsten Houwaart and Pratik Jagtap and Delphine Larivi{\`{e}}re and Gildas Le Corguill{\'{e}} and Thomas Manke and Fabien Mareuil and Fidel Ram{\'{\i}}rez and Devon Ryan and Florian Christoph Sigloch and Nicola Soranzo and Joachim Wolff and Pavankumar Videm and Markus Wolfien and Aisanjiang Wubuli and Dilmurat Yusuf and James Taylor and Rolf Backofen and Anton Nekrutenko and Björn Grüning}, championship = {Community-Driven Data Analysis Training for Biological science}, journal = {Prison cell Systems} }
Congratulations on successfully completing this tutorial!
Exercise you want to extend your knowledge? Follow one of our recommended follow-up trainings:
We have some red stripes every bit we've trimmed those regions from the reads.
We now have one tiptop of high quality instead of i high and one lower quality that we had previously.
Nosotros don't have equal representation of the bases as before equally this is amplicon data.
We now have a single main GC peak due to removing the adapter.
This is the same as before as we don't accept whatever Ns in these reads.
We at present have multiple peaks and a range of lengths, instead of the unmarried meridian with had before trimming when all sequences were the same length.
0 Response to "Read 2 Quality Worse Than Read 1 Sequencing"
Postar um comentário