Out of memory when encoding file to base64

by

You cannot just load the whole file into memory, like here:

byte fileContent[] = new byte[(int) file.length()];
fin.read(fileContent);

Instead load the file chunk by chunk and encode it in parts. Base64 is a simple encoding, it is enough to load 3 bytes and encode them at a time (this will produce 4 bytes after encoding). For performance reasons consider loading multiples of 3 bytes, e.g. 3000 bytes – should be just fine. Also consider buffering input file.

An example:

byte fileContent[] = new byte[3000];
try (FileInputStream fin = new FileInputStream(file)) {
    while(fin.read(fileContent) >= 0) {
         Base64.encodeBase64(fileContent);
    }
}

Note that you cannot simply append results of Base64.encodeBase64() to encoded bbyte array. Actually, it is not loading the file but encoding it to Base64 causing the out-of-memory problem. This is understandable because Base64 version is bigger (and you already have a file occupying a lot of memory).

Consider changing your method to:

public void encode(File file, OutputStream base64OutputStream)

and sending Base64-encoded data directly to the base64OutputStream rather than returning it.

UPDATE: Thanks to @StephenC I developed much easier version:

public void encode(File file, OutputStream base64OutputStream) {
  InputStream is = new FileInputStream(file);
  OutputStream out = new Base64OutputStream(base64OutputStream)
  IOUtils.copy(is, out);
  is.close();
  out.close();
}

It uses Base64OutputStream that translates input to Base64 on-the-fly and IOUtils class from Apache Commons IO.

Note: you must close the FileInputStream and Base64OutputStream explicitly to print = if required but buffering is handled by IOUtils.copy().

More Related Contents:

  1. Decode Base64 data in Java
  2. Encoding as Base64 in Java
  3. Base 64 encode and decode example code
  4. Base64 Java encode and decode a string [duplicate]
  5. Convert base64 string to image
  6. Java – Convert image to Base64
  7. Base64 encoder and decoder
  8. Range of valid character for a base 64 encoding
  9. Storing UUID as base64 String
  10. Does anyone know how to decode and encode a string in Base64 using Base64?
  11. Write Base64-encoded image to file
  12. Convert byte[] to Base64 string for data URI
  13. Java Encode file to Base64 string To match with other encoded string
  14. how to convert hex to base64
  15. How/Can I use base64 as image source in a Jasper Report template?
  16. How to convert an Image to base64 string in java? [duplicate]
  17. Java BufferedImage to PNG format Base64 String
  18. How to Dynamically highlight some words in html format
  19. Java 256-bit AES Password-Based Encryption
  20. How do I measure time elapsed in Java? [duplicate]
  21. How to convert comma-separated String to List?
  22. Scanner error with nextInt() [duplicate]
  23. How to set -source 1.7 in Android Studio and Gradle
  24. Spring Data JPA – “No Property Found for Type” Exception
  25. java.lang.VerifyError: Expecting a stackmap frame at branch target JDK 1.7
  26. JTable -> TableModeListener
  27. java.net.URL read stream to byte[]
  28. Sending non-ASCII text in HTTP POST header
  29. Where to store application data (non-user specific) on Linux
  30. Function pointer to String method in Java

Leave a Comment