How `poly()` generates orthogonal polynomials? How to understand the “coefs” returned?

I have just realized that there was a closely related question Extracting orthogonal polynomial coefficients from R’s poly() function? 2 years ago. The answer there is merely explaining what predict.poly does, but my answer gives a complete picture. Section 1: How does poly represent orthogonal polynomials My understanding of orthogonal polynomials is that they take … Read more

predict.lm() in a loop. warning: prediction from a rank-deficient fit may be misleading

You can inspect the predict function with body(predict.lm). There you will see this line: if (p < ncol(X) && !(missing(newdata) || is.null(newdata))) warning(“prediction from a rank-deficient fit may be misleading”) This warning checks if the rank of your data matrix is at least equal to the number of parameters you want to fit. One way … Read more

Plot polynomial regression curve in R

I like to use ggplot2 for this because it’s usually very intuitive to add layers of data. library(ggplot2) fit <- lm(mpg ~ hp + I(hp^2), data = mtcars) prd <- data.frame(hp = seq(from = range(mtcars$hp)[1], to = range(mtcars$hp)[2], length.out = 100)) err <- predict(fit, newdata = prd, se.fit = TRUE) prd$lci <- err$fit – 1.96 … Read more

Why is using update on a lm inside a grouped data.table losing its model data?

This is not an answer, but is too long for a comment The .Environment for the terms component is identical for each resulting model e1 <- attr(fit[[‘V1’]][[1]]$terms, ‘.Environment’) e2 <- attr(fit[[‘V1’]][[2]]$terms, ‘.Environment’) e3 <- attr(fit[[‘V1’]][[3]]$terms, ‘.Environment’) identical(e1,e2) ## TRUE identical(e2, e3) ## TRUE It appears that data.table is using the same bit of memory (my … Read more

Fast pairwise simple linear regression between variables in a data frame

Some statistical result / background (Link in the picture: Function to calculate R2 (R-squared) in R) Computational details Computations involved here is basically the computation of the variance-covariance matrix. Once we have it, results for all pairwise regression is just element-wise matrix arithmetic. The variance-covariance matrix can be obtained by R function cov, but functions … Read more