Background tasks in Meteor

by

2019 update

Before thinking about writing a package for anything, first look if there are existing packages that do what you need. In the Meteor world, this means looking on Atmosphere for “job/queue/task/worker management/scheduling” packages, then on npm for the same search terms. You also need to define your requirements more precisely:

  • do you want persistence, or would an in-memory solution work?
  • do you want to be able to distribute jobs to different machines?

Meteor-specific

  • job-collection – reliable (I used it in 2014 in production at a startup), but currently in maintenance mode. Lets you schedule persistent jobs to be run anywhere (servers, clients).
  • SteveJobs – actively maintained by Max Savin, the author of several powerful Meteor tools
  • littledata:synced-cron – “A simple cron system for Meteor. It supports syncronizing jobs between multiple processes.”

Abandoned packages:

Npm packages

Meteor has been able to use npm packages directly for several years now, so this question amounts to finding job/worker/queue management packages on NPM. If you don’t care about persistence:

  • Async “provides around 70 functions that include the usual ‘functional’ suspects (map, reduce, filter, each…) as well as some common patterns for asynchronous control flow (parallel, series, waterfall…)”
  • d3-queue – minimalistic, written by D3 author Mike Bostock

If you do want persistence, since Meteor uses MongoDB already, it may be advantageous to use a job scheduling package with persistence to MongoDb. The most powerful and popular seems to be Agenda, but unfortunately it hasn’t been maintained in months, and it has a significant backlog of issues.

If you’re willing to add a dependency backed by redis to your project, there are more choices:

Like MongoDB, Redis can also provide high-availability (via Redis Sentinel), and if you want to distribute jobs among multiple worker machines, you can point them all at the same Redis server.

More Related Contents:

  1. How do we or can we use node modules via npm with Meteor?
  2. Npm Package Install Problem (found 23 vulnerabilities)
  3. Where does npm install packages?
  4. Installing a local module using npm?
  5. Meteor.publish: publish collection which depends on other collection
  6. Understanding Meteor Publish / Subscribe
  7. What is the difference between Bower and npm?
  8. What does npm install –legacy-peer-deps do exactly? When is it recommended / What’s a potential use case?
  9. Using Multiple Mongodb Databases with Meteor.js
  10. Maximum call stack size exceeded on npm install
  11. using a variable in mongodb update
  12. Use of @ symbol in Node module names [duplicate]
  13. How does the messages-count example in Meteor docs work?
  14. In meteor is there a way to access array index in spacebars [duplicate]
  15. Deleting `package-lock.json` to Resolve Conflicts quickly
  16. What does ‘x packages are looking for funding’ mean when running `npm install`?
  17. How to use npm module in DENO?
  18. “Meteor code must always run within a Fiber” when calling Collection.insert on server
  19. Angular 2 – How to pass URL parameters?
  20. Meteor JS: use external script
  21. Fix the upstream dependency conflict installing NPM packages
  22. Global variables in Meteor
  23. Is there a way to pass variables into templates in Meteor?
  24. Server cleanup after a client disconnects
  25. How can I dynamically highlight strings on a web page?
  26. Yarn global command not working
  27. JavaScript dependency management: npm vs. bower vs. volo [closed]
  28. How can I manage client-side JavaScript dependencies? [closed]
  29. Error: Cannot find module ‘../lib/cli’
  30. How to use npm modules in browser? is possible to use them even in local (PC)?

Leave a Comment