Remove trailing spaces from a file using Windows batch?

by

The DosTips RTRIM function that Ben Hocking cites can be used to create a script that can right trim each line in a text file. However, the function is relatively slow.

DosTips user (and moderator) aGerman developed a very efficient right trim algorithm. He implemented the algorithm as a batch “macro” – an interesting concept of storing complex mini scripts in environment variables that can be executed from memory. The macros with arguments are a major discussion topic in and of themselves that is not relevent to this question.

I have extracted aGerman’s algorithm and put it in the following batch script. The script expects the name of a text file as the only parameter and proceeds to right trim the spaces off each line in the file.

@echo off
setlocal enableDelayedExpansion
set "spcs= "
for /l %%n in (1 1 12) do set "spcs=!spcs!!spcs!"
findstr /n "^" "%~1" >"%~1.tmp"
setlocal disableDelayedExpansion
(
  for /f "usebackq delims=" %%L in ("%~1.tmp") do (
    set "ln=%%L"
    setlocal enableDelayedExpansion
    set "ln=!ln:*:=!"
    set /a "n=4096"
    for /l %%i in (1 1 13) do (
      if defined ln for %%n in (!n!) do (
        if "!ln:~-%%n!"=="!spcs:~-%%n!" set "ln=!ln:~0,-%%n!"
        set /a "n/=2"
      )
    )
    echo(!ln!
    endlocal
  )
) >"%~1"
del "%~1.tmp" 2>nul

Assuming the script is called rtrimFile.bat, then it can be called from the command line as follows:

rtrimFile "fileName.txt"

A note about performance
The original DosTips rtrim function performs a linear search and defaults to trimming a maximum of 32 spaces. It has to iterate once per space.

aGerman’s algorithm uses a binary search and it is able to trim the maximum string size allowed by batch (up to ~8k spaces) in 13 iterations.

Unfotunately, batch is very SLOW when it comes to processing text. Even with the efficient rtrim function, it takes ~70 seconds to trim a 1MB file on my machine. The problem is, just reading and writing the file without any modification takes significant time. This answer uses a FOR loop to read the file, coupled with FINDSTR to prefix each line with the line number so that blank lines are preserved. It toggles delayed expansion to prevent ! from being corrupted, and uses a search and replace operation to remove the line number prefix from each line. All that before it even begins to do the rtrim.

Performance could be nearly doubled by using an alternate file read mechanism that uses set /p. However, the set /p method is limited to ~1k bytes per line, and it strips trailing control characters from each line.

If you need to regularly trim large files, then even a doubling of performance is probably not adequate. Time to download (if possible) any one of many utilities that could process the file in the blink of an eye.

If you can’t use non-native software, then you can try VBScript or JScript excecuted via the CSCRIPT batch command. Either one would be MUCH faster.

UPDATE – Fast solution with JREPL.BAT

JREPL.BAT is a regular expression find/replace utility that can very efficiently solve the problem. It is pure script (hybrid batch/JScript) that runs natively on any Windows machine from XP onward. No 3rd party exe files are needed.

With JREPL.BAT somewhere within your PATH, you can strip trailing spaces from file “test.txt” with this simple command:

jrepl " +$" "" /f test.txt /o -

If you put the command within a batch script, then you must precede the command with CALL:

call jrepl " +$" "" /f test.txt /o -

More Related Contents:

  1. At which point does `for` or `for /R` enumerate the directory (tree)?
  2. Batch file: Find if substring is in string (not in a file)
  3. Iterate all files in a directory using a ‘for’ loop
  4. Batch character escaping
  5. How to wait all batch files to finish before exiting?
  6. How to do something to each file in a directory with a batch script
  7. Using a custom Tee command for .bat file
  8. Find out if file is older than 4 hours in Batch file
  9. Batch script: how to check for admin rights
  10. FORFILES date -after- (date calc in cmd file)
  11. Create an empty file on the commandline in windows (like the linux touch command)
  12. How do I echo and send console output to a file in a bat script?
  13. What is the proper way to test if a parameter is empty in a batch file?
  14. Set a persistent environment variable from cmd.exe
  15. IF… OR IF… in a windows batch file
  16. Windows batch script to read an .ini file
  17. Copy text from a Windows CMD window to clipboard
  18. Using another language (code page) in a batch file made for others
  19. How can I debug a .BAT script?
  20. Start multiple tasks in parallel and wait for them in windows?
  21. Counting in a FOR loop using Windows Batch script
  22. cmd is somehow writing Chinese text as output
  23. Special Characters in Batch File
  24. Batch to remove duplicate rows from text file
  25. How to pass environment variables as parameters by reference to another batch file?
  26. Rename multiple files in cmd
  27. Windows shell command to get the full path to the current directory?
  28. windows cmd: problems with for /f with a quoted command with quoted parameters
  29. Batch program to to check if process exists
  30. ERRORLEVEL vs %ERRORLEVEL% vs exclamation mark ERRORLEVEL exclamation mark

Leave a Comment