Using Async waterfall in node.js

First identify the steps and write them as asynchronous functions (taking a callback argument)

  • read the file

    function readFile(readFileCallback) {
        fs.readFile('stocktest.json', function (error, file) {
            if (error) {
                readFileCallback(error);
            } else {
                readFileCallback(null, file);
            }
        });
    }
    
  • process the file (I removed most of the console.log in the examples)

    function processFile(file, processFileCallback) {
        var stocksJson = JSON.parse(file);
        if (stocksJson[ticker] != null) {
            stocksJson[ticker].price = value;
            fs.writeFile('stocktest.json', JSON.stringify(stocksJson, null, 4), function (error) {
                if (err) {
                    processFileCallback(error);
                } else {
                    console.log("File successfully written");
                    processFileCallback(null);
                }
            });
        }
        else {
            console.log(ticker + " doesn't exist on the json");
            processFileCallback(null); //callback should always be called once (and only one time)
        }
    }
    

Note that I did no specific error handling here, I’ll take benefit of async.waterfall to centralize error handling at the same place.

Also be careful that if you have (if/else/switch/…) branches in an asynchronous function, it always call the callback one (and only one) time.

Plug everything with async.waterfall

async.waterfall([
    readFile,
    processFile
], function (error) {
    if (error) {
        //handle readFile error or processFile error here
    }
});

Clean example

The previous code was excessively verbose to make the explanations clearer. Here is a full cleaned example:

async.waterfall([
    function readFile(readFileCallback) {
        fs.readFile('stocktest.json', readFileCallback);
    },
    function processFile(file, processFileCallback) {
        var stocksJson = JSON.parse(file);
        if (stocksJson[ticker] != null) {
            stocksJson[ticker].price = value;
            fs.writeFile('stocktest.json', JSON.stringify(stocksJson, null, 4), function (error) {
                if (!err) {
                    console.log("File successfully written");
                }
                processFileCallback(err);
            });
        }
        else {
            console.log(ticker + " doesn't exist on the json");
            processFileCallback(null);
        }
    }
], function (error) {
    if (error) {
        //handle readFile error or processFile error here
    }
});

I left the function names because it helps readability and helps debugging with tools like chrome debugger.

If you use underscore (on npm), you can also replace the first function with _.partial(fs.readFile, 'stocktest.json')

Leave a Comment