VBA Excel large data manipulation taking forever

by

An array of 20,000 elements containing 4 Variants each will take up less than 2 MB of RAM. I don’t think memory has anything to do with your problem — unless you happen to be using an old computer with 2 MB of RAM or something like that.

A more likely reason why your code is so heavy is that you are looping through cells. There is significant overhead to each communication between VBA and Excel sheet data, and this adds up when you refer to many cells one at a time. In your case, your loop does up to 200,000 separate cell references.

Instead, you should load all your data at once into a Variant array, and then loop through that array, as shown below. This is significantly faster (even though this uses more memory, not less; but again, I don’t think memory is your issue). 

lblStatus.Caption = "Loading to memory"
Dim ArrAuditData() As AData
Dim varTemp As Variant
Dim TotalLookUpCount As Integer

' Load everything into a Variant array. 
varTemp = lookUpRange

ReDim ArrAuditData(1 To UBound(varTemp, 1)) As AData

For J = 1 To UBound(varTemp, 1)

    If varTemp(J, cmbChoice.ListIndex) = "Fail" Then

        ArrAuditData(TotalLookUpCount).AuditType = varTemp(J, cmdAudit2.ListIndex)
        ArrAuditData(TotalLookUpCount).TransTime = varTemp(J, cmbChoice.ListIndex - 1)
        ArrAuditData(TotalLookUpCount).AuditValue = varTemp(J, cmbChoice.ListIndex)
        ArrAuditData(TotalLookUpCount).Slno = varTemp(J, 0)
        TotalLookUpCount = TotalLookUpCount + 1

    ElseIf varTemp(J, cmbChoice.ListIndex) = "" And J > 4 Then
        Exit For

    End If

    DoEvents
Next

ReDim Preserve ArrAuditData(TotalLookUpCount) As AData

For further reading, have a look at this old but still relevant article: http://www.avdf.com/apr98/art_ot003.html

If you still think RAM is the issue, then please show us the AData type declaration.

EDIT: Also, never ReDim Preserve inside a loop like that! ReDim Preserve is a very expensive operation and rarely needs to be done more than once on any given array. Doing it 20,000 times will slow down your code. Here I take it out of the loop, and just use it once at the end to trim off the unused elements. (Notice how I initially ReDim‘ed the array to fit the largest conceivable number of elements.)

More Related Contents:

  1. Excel-VBA – Insert new first column in datafield array without loops or API calls
  2. Creating an Array from a Range in VBA
  3. Expanding column cells for each column cell
  4. How do I slice an array in Excel VBA?
  5. Check if a value is in an array or not with Excel VBA
  6. Matching values in string array
  7. Deleting Elements in an Array if Element is a Certain value VBA
  8. Is it possible to fill an array with row numbers which match a certain criteria without looping?
  9. How to automatically input an array formula as string with more than 255 characters in length into an excel cell using VBA?
  10. Return Index of an Element in an Array Excel VBA
  11. Writing an array to a range. Only getting first value of array
  12. Excel VBA: Range to String Array in 1 step
  13. Faster way of using Sumifs()
  14. How to initialize a multidimensional array variable in vba for excel
  15. VBA using ubound on a multidimensional array
  16. Turn Excel range into VBA string
  17. Get values from union of non-contiguous ranges into array with VBA with a simple command (no loops)
  18. Why am I having issues assigning a Range to an Array of Variants
  19. Subscript out of range error with an array – no idea why?
  20. Performance difference between looping range vs looping array
  21. Create Chart from Array data and not range
  22. Excel VBA Quickest way to sort an array of numbers in descending order?
  23. Simple VBA array join not working
  24. Excel VBA – How to Redim a 2D array?
  25. ReDim Preserve “Subscript Out of Range”
  26. Declare and Initialize String Array in VBA
  27. Automatic array allocation upon assignment in Fortran
  28. In a CUDA kernel, how do I store an array in “local thread memory”?
  29. How Do I Make FLATTEN Work for Non-Contiguous Ranges?
  30. How do I Merge two Arrays in VBA?

Leave a Comment