{"data":{"site":{"siteMetadata":{"courseId":"metaanalysis"}},"markdownRemark":{"htmlAst":{"type":"root","children":[{"type":"element","tagName":"exercise","properties":{"id":"1","title":"Getting the data from the literature"},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"So what happens after you have collected all the papers, and you would like to get started with data collection?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"What general information do you need to extract?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Think about all the information you would like to transfer into your spreadsheet. Don’t just go for basic paper identifiers, the more information you extract now, the easier it will be to actually to back if you need to double check outliers at a later stage. Have a look for example at "},{"type":"element","tagName":"a","properties":{"href":"https://osf.io/kgm7z/"},"children":[{"type":"text","value":"Macartneys’ excel spreadsheet"}]},{"type":"text","value":". "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"What data do you need to extract?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"In the ideal case, means and SD or SE as well as sample sizes for your traits of interest are provided in the paper that you are examining, and can simply be transcribed into your data collection sheet. No problem there - if those data ARE provided! More often, however, these results are represented in figures. At the most basic, you can enlarge the figure on your screen and measure means and SE’s with a ruler (don’t forget to convert your measurements according to the Figure’s axes!), and add the crude results into your spreadsheet. This may be sufficient for the exercise here, but it is not likely to be very accurate, more error-prone than semi-automated methods, and is very time consuming."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Extracting means and variances from Figures"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Assuming that you will have many Figures from many studies, for your own meta analysis we recommend to use the R package "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metaDigitise"}]},{"type":"text","value":". This package allows to accurately measure the means and variance from  screenshots of plots.\n"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metaGear"}]},{"type":"text","value":" also looks very promising, as it automatically saves relevant figures. However, we have had issues with extracting information from bar plots and tweaking did not seem possible - so careful visual inspection and doublechecking of the extracted information with the initial figures (and their axes) will likely be a relatively time-consuming part of the process."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"However, possible online alternatives are also available, for example "},{"type":"element","tagName":"a","properties":{"href":"https://automeris.io/WebPlotDigitizer/"},"children":[{"type":"text","value":"WebPlotDigitizer"}]},{"type":"text","value":", that may be either downloaded and used locally or as a web-based resource (perhaps this may be the way to go if you only have very few Figures to measure?). Here, you upload your figure file, specify the chart type & define your axes, and may manually add datapoints on an enlarged subsection. This gives you very accurate estimates for your selected datapoints."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"For more information and an overview and pros and cons of the most common image-digitising tools have a look at Table 1 in the original "},{"type":"element","tagName":"a","properties":{"href":"https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13118"},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metaDigitise"}]},{"type":"text","value":" publication"}]},{"type":"text","value":": Pick, J. L., Nakagawa, S., & Noble, D. W. (2019). Reproducible, flexible and high‐throughput data extraction from primary literature: The metaDigitise r package. Methods in Ecology and Evolution, 10(3), 426-431."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"exercise","properties":{"id":"2","title":"Preparing your data for Meta-analysis"},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"If you have a few data points extracted (either form Figures or Tables), you can now calculate effect sizes. What are they, and how do I get them?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"What is an “effect size”?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"‘Effect size’ is a way to quantify the size of the difference between two groups. It is particularly valuable for quantifying the effectiveness of a particular intervention or treatment. It looks at the relative outcome of the treatment, and allows us to use this (and the associated measures of uncertainty) as the basis to compare multiple studies. This can make generalisations even across different methodologies and taxa possible. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"A common effect size measure is the "},{"type":"element","tagName":"u","properties":{},"children":[{"type":"text","value":"s"}]},{"type":"text","value":"tandardized "},{"type":"element","tagName":"u","properties":{},"children":[{"type":"text","value":"m"}]},{"type":"text","value":"ean "},{"type":"element","tagName":"u","properties":{},"children":[{"type":"text","value":"d"}]},{"type":"text","value":"ifference (SMD), calculated as the difference in means, standardized (divided) most commonly by the pooled SD of the two compared sets of data values (treatment and control). SMD comes in two versions: "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Cohen’s d"}]},{"type":"text","value":" and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Hedges’ g"}]},{"type":"text","value":". "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Hedges’ g"}]},{"type":"text","value":" is an unbiased version of "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Cohen’s d"}]},{"type":"text","value":", but gives very similar results. If you have small sample sizes, use of  "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Hedges’ g"}]},{"type":"text","value":" might be recommended."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/computertaskicon.svg?raw=true"},"children":[]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nWe can calculate different types of effect sizes in the R package "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"metafor"}]},{"type":"text","value":". The function "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"escalc"}]},{"type":"text","value":" allows us to call “SMD” (Standardised Mean Difference, Hedges’ "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"g"}]},{"type":"text","value":" in "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metafor"}]},{"type":"text","value":") or “ROM” (log transformed ratio of means). Which one you use is up to your preference; overall they should give consistent answers. Read more about their respective differences in a clinical context "},{"type":"element","tagName":"a","properties":{"href":"https://www.cebm.net/2020/04/tip-for-data-extraction-for-meta-analysis-29/"},"children":[{"type":"text","value":"here"}]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"library(metafor)\n#calculate standardized mean difference\nSMD <- escalc(measure = \"SMD\", data=YourData, m1i=TreatMean, m2i = ControlMean, sd1i=TreatSD, sd2i= ControlSD, n1i = NTreat, n2i=NControl, append = FALSE) \n\n#calculate ratio of means\nROM <- metafor::escalc(measure = \"ROM\", data=YourData, m1i=TreatMean, m2i = ControlMean, sd1i=TreatSD, sd2i= ControlSD, n1i = NTreat, n2i=NControl, append = FALSE"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Let’s give this a go using Macartney’s data."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_2a"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Note that there will be quite a few missing data in this list. This may be due to the fact that perhaps no standard deviation is provided, only standard errors. In this case you have different options - calculate yourself SD from SE’s for example, or use a different package - see Chapter 4 for how to use "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"compute.es"}]},{"type":"text","value":". Another reason why no effect sizes are calculated in this step might be perhaps no information on means and variances is provided, only inferential statistics."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Calculating effect sizes from inferential statistics"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This is necessary when means & variances are not reported, and not available in a figure, but overall statistics indicate outcomes (and are fully disclosed, i.e. type of test, degrees of freedom, test statistic are available). "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/pushpin.svg?raw=true"},"children":[]},{"type":"text","value":"  "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Note:"}]}]},{"type":"text","value":" If this is incomplete, but the study highly relevant for inclusion, your last resort is to write to the authors and ask for the missing information. Ideally they can then provide the means and variances that you’re after…"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/pushpin.svg?raw=true"},"children":[]},{"type":"text","value":"  "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Online Calculators"}]}]},{"type":"text","value":" In case you have only very few instances of inferential statistics to calculate your effect size from, there’s the option to use an online calculator to do this (or you could do that to conform your R results!). A good resource is provided by the "},{"type":"element","tagName":"a","properties":{"href":"https://campbellcollaboration.org/research-resources/effect-size-calculator.html"},"children":[{"type":"text","value":"Campbell Collaboration"}]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This example calculates the effect size for an Anova (using F statistics). To calculate this based on a T-test, specify "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"a.tes"}]},{"type":"text","value":" instead of "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"a.fes"}]},{"type":"text","value":", and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"t = t"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Try this by rewriting the relevant bits in the codeblock below! You might need to check out the hint to get it to work."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_es"},"children":[{"type":"text","value":"\nThe exclamation mark \"!\" in R code specifies \"not\" - so you will have to switch it around in the second line of code, to apply it to it to the right subset of data. It's a little twisted here, we give a double negative command: !is.na(t) means: do not use (!) the rows that show na (is.na) in column \"t\". For more info check section 5.2.2. [here](https://r4ds.had.co.nz/transform.html) \n"}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"frontmatter":{"id":3,"title":"Chapter 3: Data extraction & preparation","description":"We have the papers- what now?","next":"chapter4","prev":"chapter2"}}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"/chapter3"}}