Deleting columns from a data.frame where NA is more than 15% of the column length [duplicate]

by

First, it’s always good to share some sample data. It doesn’t need to be your actual data–something made up is fine.

set.seed(1)
x <- rnorm(1000)
x[sample(1000, 150)] <- NA
mydf <- data.frame(matrix(x, ncol = 10))

Second, you can just use inbuilt functions to get what you need. Here, is.na(mydf) does a logical check and returns a data.frame of TRUE and FALSE. Since TRUE and FALSE equate to 1 and 0, we can just use colMeans to get the mean of the number of TRUE (is NA) values. That, in turn, can be checked according to your stipulations, in this case which columns have more than 15% NA values?

colMeans(is.na(mydf)) > .15
#    X1    X2    X3    X4    X5    X6    X7    X8    X9   X10 
#  TRUE  TRUE FALSE FALSE FALSE  TRUE FALSE  TRUE  TRUE FALSE

As we can see, we should drop X1, X2, X6, X8, and X9. Again, taking advantage of logical vectors, here’s how:

> final <- mydf[, colMeans(is.na(mydf)) <= .15]
> dim(final)
[1] 100   5
> names(final)
[1] "X3"  "X4"  "X5"  "X7"  "X10"

More Related Contents:

  1. Deleting specific rows with a pattern[start and end indicators] from a dataframe [closed]
  2. Select the row with the maximum value in each group
  3. Create a sequential number (counter) for rows within each group of a dataframe [duplicate]
  4. Extracting specific columns from a data frame
  5. Find indices of duplicated rows [duplicate]
  6. How to drop columns by name in a data frame
  7. Reshape multiple values at once
  8. Remove columns from dataframe where ALL values are NA
  9. Cumulatively paste (concatenate) values grouped by another variable
  10. How to move cells with a value row-wise to the left in a dataframe [duplicate]
  11. combining two data frames of different lengths [closed]
  12. Match values in data frame with values in another data frame and replace former with a corresponding pattern from the other data frame
  13. Detect at least one match between each data frame row and values in vector
  14. Converting nested list to dataframe
  15. Cartesian product data frame
  16. Merge Two Lists in R
  17. Find which interval row in a data frame that each element of a vector belongs in
  18. How to add a factor column to dataframe based on a conditional statement from another column?
  19. mutate_each / summarise_each in dplyr: how do I select certain columns and give new names to mutated columns?
  20. Remove all rows where length of string is more than n
  21. Split dataframe using two columns of data and apply common transformation on list of resulting dataframes
  22. Replace values in data frame based on other data frame in R
  23. Identify records in data frame A not contained in data frame B [closed]
  24. Add an image to a table-like output in R
  25. Replace value with the name of its respective column
  26. Count unique values for every column
  27. Numbers as column names of data frames
  28. Divide each value by the sum of values by group
  29. Convert all data frame character columns to factors
  30. Override column types when importing data using readr::read_csv() when there are many columns

Leave a Comment