Files.walk(), calculate total size

by

No, this exception cannot be avoided.

The exception itself occurs inside the the lazy fetch of Files.walk(), hence why you are not seeing it early and why there is no way to circumvent it, consider the following code:

long size = Files.walk(Paths.get("C://"))
        .peek(System.out::println)
        .mapToLong(this::count)
        .sum();

On my system this will print on my computer:

C:\
C:\$Recycle.Bin
Exception in thread "main" java.io.UncheckedIOException: java.nio.file.AccessDeniedException: C:\$Recycle.Bin\S-1-5-18

And as an exception is thrown on the (main) thread on the third file, all further executions on that thread stop.

I believe this is a design failure, because as it stands now Files.walk is absolutely unusable, because you never can guarantee that there will be no errors when walking over a directory.

One important point to notice is that the stacktrace includes a sum() and reduce() operation, this is because the path is being lazily loaded, so at the point of reduce(), the bulk of stream machinery gets called (visible in stacktrace), and then it fetches the path, at which point the UnCheckedIOException occurs.

It could possibly be circumvented if you let every walking operation execute on their own thread. But that is not something you would want to be doing anyway.

Also, checking if a file is actually accessible is worthless (though useful to some extent), because you can not guarantee that it is readable even 1ms later.

Future extension

I believe it can still be fixed, though I do not know how FileVisitOptions exactly work.
Currently there is a FileVisitOption.FOLLOW_LINKS, if it operates on a per file basis, then I would suspect that a FileVisitOption.IGNORE_ON_IOEXCEPTION could also be added, however we cannot correctly inject that functionality in there.

More Related Contents:

  1. Why is Files.list() parallel stream performing so much slower than using Collection.parallelStream()?
  2. Get List<K> from HashMap<K, V> where V is instance of I [closed]
  3. Java 8 Distinct by property
  4. In Java streams is peek really only for debugging?
  5. Why filter() after flatMap() is “not completely” lazy in Java streams?
  6. Copy a stream to avoid “stream has already been operated upon or closed”
  7. What is difference between Collection.stream().forEach() and Collection.forEach()?
  8. Find first element by predicate
  9. Why is a combiner needed for reduce method that converts type in java 8
  10. Filter Java Stream to 1 and only 1 element
  11. How can I make Cartesian product with Java 8 streams?
  12. Find maximum, minimum, sum and average of a list in Java 8
  13. Does Java SE 8 have Pairs or Tuples?
  14. Most efficient way to get the last element of a stream
  15. Why does findFirst() throw a NullPointerException if the first element it finds is null?
  16. Java streams lazy vs fusion vs short-circuiting
  17. Collectors.summingInt() vs mapToInt().sum()
  18. Java 8 Stream IllegalStateException: Stream has already been operated on or closed
  19. How to check if a Java 8 Stream is empty?
  20. Stream Way to get index of first element matching boolean
  21. How to sum values in a Map with a stream?
  22. Is mapToDouble() really necessary for summing a List with Java 8 streams?
  23. Is it possible to cast a Stream in Java 8?
  24. In Java 8 how do I transform a Map to another Map using a lambda?
  25. Is it possible to get next element in the Stream?
  26. Does collect operation on Stream close the stream and underlying resources?
  27. How to get multiple values from an object using a single stream operation?
  28. non-interference requirement on Java 8 streams
  29. Java 8 Stream and operation on arrays
  30. What is the difference between intermediate and terminal operations?

Leave a Comment