Last updated: 2020-07-14
Checks: 7 0
Knit directory: QGT-Columbia-HKI-repo/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20200603)
was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 24a96e2. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish
or wflow_git_commit
). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rproj.user/
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/analysis_plan.Rmd
) and HTML (docs/analysis_plan.html
) files. If you’ve configured a remote Git repository (see ?wflow_git_remote
), click on the hyperlinks in the table below to view the files as they were in that past version.
File | Version | Author | Date | Message |
---|---|---|---|---|
Rmd | 24a96e2 | meliao | 2020-06-12 | Changed google drive link |
Rmd | e575612 | meliao | 2020-06-12 | Two small edits |
html | 599cadd | Hae Kyung Im | 2020-06-12 | knitted |
Rmd | ae445af | HKI test 2 | 2020-06-12 | clean up |
Rmd | 5ca942b | HKI test 2 | 2020-06-12 | some edits |
Rmd | ecded2b | HKI test 2 | 2020-06-11 | moved download folder instr down |
Rmd | c598730 | meliao | 2020-06-11 | New google sheets link |
Rmd | 9765b82 | meliao | 2020-06-11 | Lots of work generating TWMR input |
html | 68910e2 | Hae Kyung Im | 2020-06-10 | todo list |
Rmd | 8c1225c | meliao | 2020-06-10 | Scripted running all of TWMR genes in R |
Rmd | af68200 | meliao | 2020-06-10 | Merge branch ‘master’ of https://github.com/hakyimlab/QGT-Columbia-HKI |
Rmd | 54f6caa | meliao | 2020-06-10 | Added TWMR R script |
Rmd | a108ccf | Hae Kyung Im | 2020-06-10 | link fix |
html | a108ccf | Hae Kyung Im | 2020-06-10 | link fix |
Rmd | 6c2c7ff | meliao | 2020-06-10 | Changes after merge; updating code/README.md |
html | 18fb5ae | Hae Kyung Im | 2020-06-10 | knit |
Rmd | 04ae09a | meliao | 2020-06-10 | Merge branch ‘master’ of https://github.com/hakyimlab/QGT-Columbia-HKI |
Rmd | c50ef0d | meliao | 2020-06-10 | Changes to loading data into R |
Rmd | 1ec682b | Hae Kyung Im | 2020-06-10 | edits |
html | 1ec682b | Hae Kyung Im | 2020-06-10 | edits |
Rmd | 9b3c7eb | Hae Kyung Im | 2020-06-10 | edits |
html | 9b3c7eb | Hae Kyung Im | 2020-06-10 | edits |
html | 45531d9 | Hae Kyung Im | 2020-06-10 | knitted |
Rmd | f5ab5b9 | HKI qgt test | 2020-06-10 | edits |
Rmd | b2b8f8b | HKI qgt test | 2020-06-10 | torus |
Rmd | 13520bc | HKI qgt test | 2020-06-09 | edits |
Rmd | 7df918e | Hae Kyung Im | 2020-06-09 | edits |
html | 7df918e | Hae Kyung Im | 2020-06-09 | edits |
Rmd | 64a96f7 | Hae Kyung Im | 2020-06-09 | edits |
html | 64a96f7 | Hae Kyung Im | 2020-06-09 | edits |
Rmd | e65d451 | Hae Kyung Im | 2020-06-09 | moved def up |
html | e65d451 | Hae Kyung Im | 2020-06-09 | moved def up |
html | e8079a7 | Hae Kyung Im | 2020-06-09 | figure moved |
Rmd | 09f235e | Hae Kyung Im | 2020-06-09 | added figure assoc |
Rmd | 7fc9808 | Hae Kyung Im | 2020-06-09 | editing preliminary notices |
html | 7fc9808 | Hae Kyung Im | 2020-06-09 | editing preliminary notices |
Rmd | b1c4d69 | Hae Kyung Im | 2020-06-09 | heads up and questionnaire |
html | b1c4d69 | Hae Kyung Im | 2020-06-09 | heads up and questionnaire |
Rmd | bc78582 | meliao | 2020-06-09 | Improved method of reading predicted expression (Thanks Tyson) |
Rmd | e5b04c3 | HKI qgt test | 2020-06-09 | height to cad |
Rmd | ad4f80b | HKI qgt test | 2020-06-09 | height to cad |
Rmd | db5a1ae | Hae Kyung Im | 2020-06-09 | edits |
html | db5a1ae | Hae Kyung Im | 2020-06-09 | edits |
Rmd | 83e040c | Hae Kyung Im | 2020-06-09 | edits |
Rmd | 3fc9ab8 | Hae Kyung Im | 2020-06-08 | edits |
Rmd | d4f7c78 | meliao | 2020-06-08 | Fixed merge |
Rmd | ca165eb | meliao | 2020-06-08 | Added to code directory and updated analysis plan |
Rmd | b7047c7 | Hae Kyung Im | 2020-06-08 | more comments |
html | b7047c7 | Hae Kyung Im | 2020-06-08 | more comments |
Rmd | c9cedba | Hae Kyung Im | 2020-06-08 | added brief explanation to chunks |
Rmd | f754512 | Hae Kyung Im | 2020-06-08 | removed - in spredixcan folder name |
html | f754512 | Hae Kyung Im | 2020-06-08 | removed - in spredixcan folder name |
Rmd | 0ae4586 | meliao | 2020-06-08 | Committing before merging in master |
html | 0ae4586 | meliao | 2020-06-08 | Committing before merging in master |
Rmd | 3489f1b | meliao | 2020-06-08 | Unfinished plotting changes. Committing before merge |
Rmd | b52e06a | Hae Kyung Im | 2020-06-08 | edits |
html | b52e06a | Hae Kyung Im | 2020-06-08 | edits |
Rmd | a2b9918 | Hae Kyung Im | 2020-06-07 | testing, reducing size of genotype file |
html | a2b9918 | Hae Kyung Im | 2020-06-07 | testing, reducing size of genotype file |
Rmd | 2257a0f | Hae Kyung Im | 2020-06-06 | raw figure links |
html | 2257a0f | Hae Kyung Im | 2020-06-06 | raw figure links |
Rmd | efbfceb | Hae Kyung Im | 2020-06-06 | raw urls for figures |
Rmd | 973fc2b | Hae Kyung Im | 2020-06-05 | prelim notes |
Rmd | 517e8f7 | meliao | 2020-06-05 | Merge branch ‘master’ of https://github.com/hakyimlab/QGT-Columbia-HKI |
Rmd | 409558f | Yanyu Liang | 2020-06-05 | updated some paths |
html | 409558f | Yanyu Liang | 2020-06-05 | updated some paths |
Rmd | 0deee3c | Hae Kyung Im | 2020-06-05 | added TWMR |
html | 0deee3c | Hae Kyung Im | 2020-06-05 | added TWMR |
Rmd | 064f6ee | Hae Kyung Im | 2020-06-05 | added figures and slides under extras |
html | 064f6ee | Hae Kyung Im | 2020-06-05 | added figures and slides under extras |
Rmd | 1b70eb7 | Hae Kyung Im | 2020-06-05 | updated prerequisites |
html | 1b70eb7 | Hae Kyung Im | 2020-06-05 | updated prerequisites |
Rmd | 339f40c | Hae Kyung Im | 2020-06-05 | added plan |
html | 339f40c | Hae Kyung Im | 2020-06-05 | added plan |
Rmd | a3aa03e | Hae Kyung Im | 2020-06-05 | prerequisites added |
Rmd | 09f5dae | Hae Kyung Im | 2020-06-05 | fastenloc |
Rmd | ccb1167 | Hae Kyung Im | 2020-06-05 | knit |
html | ccb1167 | Hae Kyung Im | 2020-06-05 | knit |
Rmd | d427aee | Hae Kyung Im | 2020-06-05 | minor comment sort1 |
Rmd | 4ca65b9 | Hae Kyung Im | 2020-06-05 | edits 2 |
Rmd | f59cd02 | Hae Kyung Im | 2020-06-04 | edits |
html | f59cd02 | Hae Kyung Im | 2020-06-04 | edits |
Rmd | 1ad820c | meliao | 2020-06-04 | Added shell source command |
Rmd | d65c555 | Hae Kyung Im | 2020-06-04 | twmr |
html | d65c555 | Hae Kyung Im | 2020-06-04 | twmr |
html | 682b6e2 | Hae Kyung Im | 2020-06-04 | Build site. |
Rmd | d3502e8 | Hae Kyung Im | 2020-06-04 | wflow_publish(“analysis/analysis_plan.Rmd”) |
Rmd | 4420fc7 | Hae Kyung Im | 2020-06-04 | wflow_rename(“analysis/predixcan_analysis.Rmd”, “analysis/analysis_plan.Rmd”) |
html | 4420fc7 | Hae Kyung Im | 2020-06-04 | wflow_rename(“analysis/predixcan_analysis.Rmd”, “analysis/analysis_plan.Rmd”) |
Linux is the operating system of choice to run bioinformatics software. We are offering two options
The latest version of the analysis plan markdown document that generated this page is on github here rendered here as an html page
We ask you to actively participate in today’s hands on activities. Notice that we may ask you to share your screen for pedagogic purposes.
As you run the analysis and programs, we ask you to respond the questions in this document. Find the tab with your name and fill out the questions as you go along.
You are welcome to check other people’s answers as guidelines but please make sure you write down your own answers.
If you have any concerns about this, please ask me or one of the TAs for assistance. We are here to help you learn.
PRE="/home/student/"
cd $PRE/lab/
git pull
conda activate imlabtools
Reminder: the bash chunks need to be copy-pasted to the terminal, not performed within the chunk.
suppressPackageStartupMessages(library(tidyverse))
print(getwd())
lab="/home/student/lab"
CODE=glue::glue("{lab}/code")
source(glue::glue("{CODE}/load_data_functions.R"))
source(glue::glue("{CODE}/plotting_utils_functions.R"))
PRE="/home/student/QGT-Columbia-HKI"
MODEL=glue::glue("{PRE}/models")
DATA=glue::glue("{PRE}/data")
RESULTS=glue::glue("{PRE}/results")
METAXCAN=glue::glue("{PRE}/repos/MetaXcan-master/software")
FASTENLOC=glue::glue("{PRE}/repos/fastenloc-master")
TORUS=glue::glue("{PRE}/repos/torus-master")
TWMR=glue::glue("{PRE}/repos/TWMR-master")
# This is a reference table we'll use a lot throughout the lab. It contains information about the genes.
gencode_df = load_gencode_df()
export PRE="/home/student/QGT-Columbia-HKI"
export LAB="/home/student/lab"
export CODE=$LAB/code
export DATA=$PRE/data
export MODEL=$PRE/models
export RESULTS=$PRE/results
export METAXCAN=$PRE/repos/MetaXcan-master/software
export TWMR=$PRE/repos/TWMR-master
Transcriptome-wide association methods
Visual summary of predixcan runs
We will predict expression of genes in whole blood using the Predict.py code in the METAXCAN folder.
Prediction models are located in the MODEL folder. Additional models for different tissues and transcriptome studies can be downloaded from predictdb.org
Remember you need to copy and paste this code chunk into the terminal to run it. Also make sure you activated the imlabtools environment which has all the necessary python modules.
Make sure all the paths and file names are correct
This run should take about one minute.
copy and paste the following block to the terminal and hit enter
printf "Predict expression\n\n"
python3 $METAXCAN/Predict.py \
--model_db_path $PRE/models/gtex_v8_en/en_Whole_Blood.db \
--vcf_genotypes $DATA/predixcan/genotype/filtered.vcf.gz \
--vcf_mode genotyped \
--variant_mapping $DATA/predixcan/gtex_v8_eur_filtered_maf0.01_monoallelic_variants.txt.gz id rsid \
--on_the_fly_mapping METADATA "chr{}_{}_{}_{}_b38" \
--prediction_output $RESULTS/predixcan/Whole_Blood__predict.txt \
--prediction_summary_output $RESULTS/predixcan/Whole_Blood__summary.txt \
--verbosity 9 \
--throw
prediction_fp = glue::glue("{RESULTS}/predixcan/Whole_Blood__predict.txt")
## Read the Predict.py output into a dataframe. This function reorganizes the data and adds gene names.
predicted_expression = load_predicted_expression(prediction_fp, gencode_df)
head(predicted_expression)
## read summary of prediction, number of SNPs per gene, cross validated prediction performance
prediction_summary = load_prediction_summary(glue::glue("{RESULTS}/predixcan/Whole_Blood__summary.txt"), gencode_df)
## number of genes with a prediction model
dim(prediction_summary)
head(prediction_summary)
print("distribution of prediction performance r2")
summary(prediction_summary$pred_perf_r2)
## download and read observed expression data from GEUVADIS
## from https://uchicago.box.com/s/4y7xle5l0pnq9d1fwmthe2ewhogrnlrv
## Remove the version number from the gene_id's (ENSG000XXX.ver)
head(predicted_expression)
## merge predicted expression with observed expression data (by IID and gene)
## plot observes vs predicted expressioni for
## ERAP1 (ENSG00000164307)
## PEX6 (ENSG00000124587)
## calculate spearman correlation for all genes
## what's the best performing gene?
\(Y = \sum_k T_k \beta_k + \epsilon\)
with random effects \(\beta_k \sim (1-\pi)\cdot \delta_0 + \pi\cdot N(0,1)\)
export PHENO="sim.spike_n_slab_0.01_pve0.1"
printf "association\n\n"
python3 $METAXCAN/PrediXcanAssociation.py \
--expression_file $RESULTS/predixcan/Whole_Blood__predict.txt \
--input_phenos_file $DATA/predixcan/phenotype/$PHENO.txt \
--input_phenos_column pheno \
--output $RESULTS/predixcan/$PHENO/Whole_Blood__association.txt \
--verbosity 9 \
--throw
More predicted phenotypes can be found in $DATA/predixcan/phenotype/
. The naming of the phenotypes provides information about the genic architecture: the number after pve is the proportion of variance of Y explained by the genetic component of expression. The number after spike_n_slab represents the probability that a gene is causal \(\pi\)(i.e. prob \(\beta \ne 0\))
## read association results
PHENO="sim.spike_n_slab_0.01_pve0.1"
predixcan_association = load_predixcan_association(glue::glue("{RESULTS}/predixcan/{PHENO}/Whole_Blood__association.txt"),
gencode_df)
## take a look at the results
dim(predixcan_association)
predixcan_association %>% arrange(pvalue) %>% head
predixcan_association %>% arrange(pvalue) %>% ggplot(aes(pvalue)) + geom_histogram(bins=20)
## compare distribution against the null (uniform)
gg_qqplot(predixcan_association$pvalue, max_yval = 40)
truebetas = load_truebetas(glue::glue("{DATA}/predixcan/phenotype/gene-effects/{PHENO}.txt"), gencode_df)
betas = (predixcan_association %>%
inner_join(truebetas,by=c("gene"="gene_id")) %>%
select(c('estimated_beta'='effect',
'true_beta'='effect_size',
'pvalue',
'gene_id'='gene',
'gene_name'='gene_name.x',
'region_id'='region_id.x')))
betas %>% arrange(pvalue) %>% head
## do you see examples of potential LD contamination?
betas %>% ggplot(aes(estimated_beta, true_beta))+geom_point()+geom_abline()
Now we will use the summary results from a GWAS of coronary artery disease to calculate the association between the genetic component of the expression of genes and coronary artery disease risk. We will use the SPrediXcan.py.
Visual summary of s-predixcan
python $METAXCAN/SPrediXcan.py \
--gwas_file $DATA/spredixcan/imputed_CARDIoGRAM_C4D_CAD_ADDITIVE.txt.gz \
--snp_column panel_variant_id --effect_allele_column effect_allele --non_effect_allele_column non_effect_allele --zscore_column zscore \
--model_db_path $MODEL/gtex_v8_mashr/mashr_Whole_Blood.db \
--covariance $MODEL/gtex_v8_mashr/mashr_Whole_Blood.txt.gz \
--keep_non_rsid --additional_output --model_db_snp_key varID \
--throw \
--output_file $RESULTS/spredixcan/eqtl/CARDIoGRAM_C4D_CAD_ADDITIVE__PM__Whole_Blood.csv
spredixcan_association = load_spredixcan_association(glue::glue("{RESULTS}/spredixcan/eqtl/CARDIoGRAM_C4D_CAD_ADDITIVE__PM__Whole_Blood.csv"), gencode_df)
dim(spredixcan_association)
spredixcan_association %>% arrange(pvalue) %>% head
spredixcan_association %>% arrange(pvalue) %>% ggplot(aes(pvalue)) + geom_histogram(bins=20)
gg_qqplot(spredixcan_association$pvalue)
run s-predixcan with liver model, do you find SORT1? Is it significant?
compare zscores in liver and whole blood.
python $METAXCAN/SMulTiXcan.py \
--models_folder $MODEL/gtex_v8_mashr \
--models_name_pattern "mashr_(.*).db" \
--snp_covariance $MODEL/gtex_v8_expression_mashr_snp_smultixcan_covariance.txt.gz \
--metaxcan_folder $RESULTS/spredixcan/eqtl/ \
--metaxcan_filter "CARDIoGRAM_C4D_CAD_ADDITIVE__PM__(.*).csv" \
--metaxcan_file_name_parse_pattern "(.*)__PM__(.*).csv" \
--gwas_file $DATA/spredixcan/imputed_CARDIoGRAM_C4D_CAD_ADDITIVE.txt.gz \
--snp_column panel_variant_id --effect_allele_column effect_allele --non_effect_allele_column non_effect_allele --zscore_column zscore --keep_non_rsid --model_db_snp_key varID \
--cutoff_condition_number 30 \
--verbosity 7 \
--throw \
--output $RESULTS/smultixcan/eqtl/CARDIoGRAM_C4D_CAD_ADDITIVE_smultixcan.txt
Visual summary of colocalization
the following code will format GWAS summary statistics into a format that the fine-mapping method torus can understand.
we precalculated this for you so there is no need to recalculate
## We ran this formatting for you because it takes over 10 minutes.
python $CODE/gwas_to_torus_zscore.py \
-input_gwas $DATA/spredixcan/imputed_CARDIoGRAM_C4D_CAD_ADDITIVE.txt.gz \
-input_ld_regions $DATA/spredixcan/eur_ld_hg38.txt.gz \
-output_fp $DATA/fastenloc/CARDIoGRAM_C4D_CAD_ADDITIVE.zval.gz
We will run torus due to time limitation but ideally we would like to run a method that allows multiple causal variants per locus, such as DAP-G or SusieR.
torus has been precompiled and placed within the PATH
export TORUSOFT=torus
$TORUSOFT -d $PRE/data/fastenloc/CARDIoGRAM_C4D_CAD_ADDITIVE.zval.gz --load_zval -dump_pip $PRE/data/fastenloc/CARDIoGRAM_C4D_CAD_ADDITIVE.gwas.pip
cd $PRE/data/fastenloc
gzip CARDIoGRAM_C4D_CAD_ADDITIVE.gwas.pip
cd $PRE
We can take a quick look at the z-values and finemapping PIPs:
cd $PRE/data/fastenloc
zless CARDIoGRAM_C4D_CAD_ADDITIVE.zval.gz
zless CARDIoGRAM_C4D_CAD_ADDITIVE.gwas.pip.gz
## check out tutorial https://github.com/xqwen/fastenloc/tree/master/tutorial
export eqtl_annotation_gzipped=$PRE/data/fastenloc/FASTENLOC-gtex_v8.eqtl_annot.vcf.gz
export gwas_data_gzipped=$PRE/data/fastenloc/CARDIoGRAM_C4D_CAD_ADDITIVE.gwas.pip.gz
export TISSUE=Whole_Blood
export FASTENLOCSOFT=fastenloc
##export FASTENLOCSOFT=/Users/owenmelia/projects/finemapping_bin/src/fastenloc/src/fastenloc
mkdir $RESULTS/fastenloc/
cd $RESULTS/fastenloc/
$FASTENLOCSOFT -eqtl $eqtl_annotation_gzipped -gwas $gwas_data_gzipped -t $TISSUE
#[-total_variants total_snp] [-thread n] [-prefix prefix_name] [-s shrinkage]
## optional - compare with s-predixcan results
fastenloc_results = load_fastenloc_coloc_result(glue::glue("{RESULTS}/fastenloc/enloc.sig.out"))
spredixcan_and_fastenloc = inner_join(spredixcan_association, fastenloc_results, by=c('gene'='Signal'))
ggplot(spredixcan_and_fastenloc, aes(RCP, -log10(pvalue))) + geom_point()
## which genes are both colocalized (rcp>0.10) and significantly associated (pvalue<0.05/number of tests)
TWMR
# Load the 'analyseMR' function
source(glue::glue("{CODE}/TWMR_script.R"))
# Collect the list of genes available to run
gene_lst <- list.files(TWMR)
gene_lst <- gene_lst[str_detect(gene_lst, "ENS.*")]
gene_lst <- (gsub("\\..*", "", gene_lst) %>% unique)
# Set the gene and run. The function writes output to a file.
for (gene in gene_lst) {
analyseMR(gene, TWMR)
}
twmr_results <- load_twmr_results(TWMR, gencode_df)
sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.6
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] Rcpp_1.0.5 rstudioapi_0.11 whisker_0.4 knitr_1.29
[5] magrittr_1.5 workflowr_1.6.2 R6_2.4.1 rlang_0.4.7
[9] stringr_1.4.0 tools_3.6.0 xfun_0.15 git2r_0.27.1
[13] htmltools_0.5.0 ellipsis_0.3.1 yaml_2.2.1 digest_0.6.25
[17] rprojroot_1.3-2 tibble_3.0.3 lifecycle_0.2.0 crayon_1.3.4
[21] later_1.1.0.1 vctrs_0.3.1 promises_1.1.1 fs_1.4.2
[25] glue_1.4.1 evaluate_0.14 rmarkdown_2.3 stringi_1.4.6
[29] compiler_3.6.0 pillar_1.4.6 backports_1.1.8 httpuv_1.5.4
[33] pkgconfig_2.0.3