Node.js – How to Write a For Loop With Callbacks

Let’s say you have 10 files that you need to upload to your web server. 10 very large files. You need to write an upload script because it needs to be an automated process that happens every day.

You’ve decided you’re going to use Node.js for the script because, hey, it’s cool.

Let’s also say you have a magical upload function that can do the upload:

upload('myfile.ext', function(err){
  if( err ) {
    console.log('yeah, that upload didn't work: '+err)

This upload function uses the callback pattern you’d expect from Node. You give it the name of the file you want to upload and then it goes off and does its thing. After while, when it is finished, it calls you back, by calling your callback function. It passes in one argument, an err object. The Node convention is that the first parameter to a callback is an object that describes any errors that happened. If this object is null, then everything was OK. If not, then the object contains a description of the error. This could be a string, or a more complex object.

I’ll write a post on that innards of that upload function – coming soon!

Right, now that you have your magical upload function, let’s get back to writing a for loop.

Are you a refugee from Javaland? Here’s the way you were thinking of doing it:

var filenames = [...]

try {
for( var i = 0; i < filenames.length; i++ ) { upload( filenames[i], function(err) { if( err ) throw err }) } } catch( err ) { console.log('error: '+err) } Here's what you think will happen: 1. upload each file in turn, one after the other 2. if there's an error, halt the entire process, and throw it to the calling code Here's what you just did: 1. Started shoving all 10 files at your web server all at once 2. If there is an error, good luck catching it outside that for loop – it's gone to the great Event Loop in the sky Node is asynchronous. The upload function will return before it even starts the upload. It will return back to your for loop. And your for loop will move on to the next file. And the next one. Is your website a little unresponsive? How about your net connection? Things might be a little slow when you push all those files up at the same time. So you can't use for loops any more! What's a coder to do? Bite the bullet and recurse. It's the only way to get back to what you actually want to do. You have to wait for the callback. When it is called, only then do you move on to the next file. That means you need to call another function inside your callback. And this function needs to start uploading the next file. So you need to create a recursive function that does this. It turns out there's a nice little recursive pattern that you can use for this particular case:

var filenames = [...]

function uploader(i) {
  if( i < filenames.length ) {
    upload( filenames[i], function(err) {
      if( err ) {
        console.log('error: '+err)
      else {
Do you see the pattern?
repeater(i) {
  if( i < length ) {
     asyncwork( function(){
       repeater( i + 1 )
You can translate this back into a traditional for(var i = 0; i < length; i++) loop quite easily: repeater(0) is var i = 0,
if( i < length ) is i < length, and
repeater( i + 1 ) is i++

When it comes to Node, the traditional way of doing things can mean you lose control of your code. Use recursion to get control back.

This entry was posted in Node.js. Bookmark the permalink.

21 Responses to Node.js – How to Write a For Loop With Callbacks

  1. Narendra says:

    Nice article… I could not understand the final translating to traditional for loop part though…

    • Paul says:

      Yeah, I can’t understand that part either. I think he’s just saying how to understand what it’s doing in terms of a for loop, not how to refactor it into a for loop…

  2. James Coglan says:

    Clear explanation. Worth bearing in mind the edge case when you’re doing work that *might* be asynchronous without causing a stack overflow:

  3. Dhruv Matani says:

    Actually, you can set maxSockets to 1 and subsequent requests will be queued up. I would prefer using a queue in this case and use a for loop instead of recursion. However, the recursion trick is an interesting one!! Another place the for loop will fail you is if you try to use the value of ‘i’ in the callback for upload()

  4. Nice article.. I also quite like the async module, very flexible on flow control:

  5. wmehanna says:

    You are a life saver!!!!!!!!!!!!

    Thanks !

  6. Rafael says:

    Thank’s man, it’s so usefull and simple even for me, “I only speak spanish.” XD

  7. eyar says:

    Saved my life

  8. Sérgio says:

    tks , I done a double loop

    function repeater(i, j) {
    if (j < paises[continentes[i]].length) {
    var continente = continentes[i];
    var pais = paises[continentes[i]][j];
    console.log("pais = " + continente+"/"+ pais );
    fromDB(continentes[i], pais, function(continente, pais, newspaper){
    console.log(continente+"/"+ pais + " " + newspaper.length);
    repeater(i, j+1)
    } else {
    repeater2(i+1, 0);
    function repeater2(i, j) {
    console.log("continente=" + continentes[i] );
    if (i < continentes.length) {
    repeater(i, 0);
    repeater2(0, 0);

  9. Vasant says:

    Nice article….. clear explanation very helpful.

  10. Steve says:

    Thanks a ton man!!! You are a life saver!! I love you!!

  11. Varun Sikka says:

    Great article and it still holds true even in 2015. Thanks for this

  12. Ernesto Malave says:

    Thank you very much for this. It worked like a charm!

  13. Yezarela says:

    Great post dude, you saved my life !! Thanks a lot 😀

  14. Roshan Hadal says:

    Excellent article/method and still works. It helped me where I had to process a batch job which had to execute sequentially, and it had double loops. I just extended the same technique for both loops and it worked.

  15. Avdhoot Saple says:

    Perfect ! impressive approach and helps get away with callback libraries ! Thanks a lot

  16. Thanks , I have just been looking for information about this topic for ages and yours
    is the best I’ve discovered till now. However, what in regards to
    the bottom line? Are you sure about the source?

  17. Nir says:

    from wiki:
    Recursion in computer science is a method where the solution to a problem depends on solutions to smaller instances of the same problem (as opposed to iteration). The approach can be applied to many types of problems, and recursion is one of the central ideas of computer science.

    this just looks like recursion, but it isn’t for two reasons:
    a) there is no smaller problem solved when calling “repeater”
    b) the stack gets cleared and set to initial state each time the async operation is called, so instead of looking like this: repeater->repeater->repeater->….. , it looks like this:
    ->repeater->async call.
    ->anon func->repeater->async call
    ->anon func->repeater->async call

  18. Arun says:

    thanks for sharing!

  19. Pingback: mysql - Nodejs database query async issue - Prosyscom Hosting

Leave a Reply

Your email address will not be published. Required fields are marked *