PS Script is exporting an empty CSVs

by

Unless you read the input file into memory in full, up front, you cannot safely read from and write back to the same file in a given pipeline.

Specifically, a command such as Import-Csv file.csv | ... | Export-Csv file.csv will erase the content of file.csv.[1]

The simplest solution is to enclose the command that reads the input file in (...), but note that:

  • The file’s content (transformed into objects) must fit into memory as a whole.

  • There is a slight risk of data loss if the pipeline is interrupted before all (transformed) objects have been written back to the file.

Applied to your command:

$RootPath = "D:\SomeFolder"

Get-ChildItem $RootPath -Recurse -Include *.csv -OutVariable csvFiles | 
    ForEach-Object{
        (Import-CSV $_.FullName) | # NOTE THE (...)
          Select-Object Test_Name, Test_DataName, Device_Model, Device_FW, 
                        Data_Avg_ms, Data_StdDev | 
            Export-Csv $_.FullName -NoType -Force
}

Note that I’ve used -OutVariable csvFiles in order to collect the CSV file-info objects in output variable $csvFiles. Your attempt to collect the file paths via $csvFilePaths = ... doesn’t work, because it attempts to collects Export-Csv‘s output, but Export-Csv produces no output.

Also, to be safe, I’ve changed the Import-Csv argument from $_ to $_.FullName to ensure that Import-Csv finds the input file (because, regrettably, file-info object $_ is bound as a string, which sometimes expands to the mere file name).

A safer solution would be to output to a temporary file first, and (only) on successful completion replace the original file.

With either approach, the replacement file will have default file attributes and permissions; if the original file had special attributes and/or permissions that you want to preserve, you must recreate them explicitly.

[1] Note that in other contexts (on Windows) you may simply get an error that doesn’t result in the input file’s content getting erased; e.g., the following pipeline: Get-Content file.txt | ForEach-Object { "[$_]" } | Set-Content file.txt reports error The process cannot access the file '...\file.txt' because it is being used by another process, without erasing the content of file.txt

More Related Contents:

  1. Create objects with custom properties containing derived filesystem path information and export them to a CSV – calculated properties
  2. How do I write the value of a single property of a object?
  3. how do I loop through a line from a csv file in powershell
  4. How to remove all quotations mark in the csv file using powershell script?
  5. Merging multiple CSV files into one using PowerShell
  6. ConvertTo-Csv Output without quotes
  7. How to use powershell to reorder CSV columns?
  8. Add Column to CSV Windows PowerShell
  9. Script has two variables when done, but when I pipe to SELECT-object only first one returns data to console
  10. Using Powershell environmental variables as strings when outputting files
  11. Read a Csv file with powershell and capture corresponding data
  12. Remove last line from file with Powershell
  13. Getting “System.Collections.Generic.List`1[System.String]” in CSV File export when data is okay on screen
  14. Unable to Pause or Sleep after Select-Object
  15. What’s the better (cleaner) way to ignore output in PowerShell? [closed]
  16. Using Get-ChildItem -Exclude or -Include returns nothing
  17. ‘kubectl patch’ works on Linux Bash but not in Windows Powershell ISE
  18. Export hashtable to CSV with the key as the column heading
  19. CMD pipe different form Powershell pipe?
  20. What’s the difference between single quote and double quote to define a string in powershell
  21. Convert Unix time with PowerShell
  22. How can I get the file system location of a PowerShell script?
  23. If Statement Against Dynamic Variable [duplicate]
  24. You cannot call a method on a null-valued expression
  25. How do I call the PowerShell CLI robustly, with respect to character encoding, input and output streams, quoting and escaping?
  26. Passing a function to Powershell’s (replace) function
  27. Parsing a Powershell variable
  28. Write output to a text file in PowerShell
  29. Why is “
  30. Writing console output to a file – file is unexpectedly empty

Leave a Comment