Workflow for statistical analysis and report writing

by

I generally break my projects into 4 pieces:

  1. load.R
  2. clean.R
  3. func.R
  4. do.R

load.R: Takes care of loading in all the data required. Typically this is a short file, reading in data from files, URLs and/or ODBC. Depending on the project at this point I’ll either write out the workspace using save() or just keep things in memory for the next step.

clean.R: This is where all the ugly stuff lives – taking care of missing values, merging data frames, handling outliers.

func.R: Contains all of the functions needed to perform the actual analysis. source()‘ing this file should have no side effects other than loading up the function definitions. This means that you can modify this file and reload it without having to go back an repeat steps 1 & 2 which can take a long time to run for large data sets.

do.R: Calls the functions defined in func.R to perform the analysis and produce charts and tables.

The main motivation for this set up is for working with large data whereby you don’t want to have to reload the data each time you make a change to a subsequent step. Also, keeping my code compartmentalized like this means I can come back to a long forgotten project and quickly read load.R and work out what data I need to update, and then look at do.R to work out what analysis was performed.

More Related Contents:

  1. In R, how to turn characters (grades) into a number and put in a separate column
  2. How to find the statistical mode?
  3. Construct a manual legend for a complicated plot
  4. How to make execution pause, sleep, wait for X seconds in R?
  5. Why am I getting “algorithm did not converge” and “fitted prob numerically 0 or 1” warnings with glm?
  6. Linear Regression loop for each independent variable individually against dependent
  7. Function to calculate R2 (R-squared) in R
  8. Changing whisker definition in geom_boxplot
  9. Animated sorted bar chart with bars overtaking each other
  10. Geometric Mean: is there a built-in?
  11. Plotting a 3D surface plot with contour map overlay, using R
  12. Exact number of bins in Histogram in R
  13. Standard Deviation in R Seems to be Returning the Wrong Answer – Am I Doing Something Wrong?
  14. Stepwise regression using p-values to drop variables with nonsignificant p-values
  15. predict.lm() in a loop. warning: prediction from a rank-deficient fit may be misleading
  16. ggplot2 multiple scales/legends per aesthetic, revisited [duplicate]
  17. How to plot with a png as background? [duplicate]
  18. Add error bars to show standard deviation on a plot in R
  19. ggplot2 plot area margins?
  20. Add legend to geom_line() graph in r
  21. Calculating percentile of dataset column
  22. Fixing Cluttered Titles on Graphs
  23. For the same code, labels (q1, median) appear on one computer but don’t appear on another computer
  24. Align multiple ggplot2 plots with grid
  25. Efficient calculation of matrix cumulative standard deviation in r
  26. Identifying points by color
  27. How to use the ‘sweep’ function
  28. R optimization with equality and inequality constraints
  29. What does the period mean when used with ~ (in a formula)?
  30. ggplot scale color gradient to range outside of data range

Leave a Comment