Meteor Node Process CPU Usage Nears 100%

by

While I don’t have a specific answer to your question, I have experience dealing with CPU issues for our production meteor app for so I can give you a list of things to investigate.

  1. Upgrade to the latest version of meteor and the appropriate node version (see the changelog). As of this writing that’s meteor 0.8.2 and node 0.10.28.

  2. Read this and this article. The latter makes a great point that you really should always try to delay activation of subscriptions until you need them. In particular you may not need to publish anything for users who are not logged in. In my experience, meteor CPU problems have everything to do with subscriptions.

  3. Be careful with observe and observeChanges. These are expensive and are easy to abuse. In particular:

    • Make sure you are calling stop() on your handles when they are no longer needed (consider using a package like publish-with-relations so this is done for you).
    • Fetch only the collections and fields that you absolutely need. Observe works by continually diffing objects (requires lots of CPU). The fewer and smaller objects you have, the less there is to compute.

  4. Consider using smart-collections before it is retired. Use oplog tailing – this can make for a night and day difference in performance and CPU usage in your app.

  5. Consider making some things not reactive (also mentioned in the articles above). For us that was a big win. We had one extremely expensive join that was used on two frequently accessed pages on the site. When it got to the point where the CPU was pegged at 100% about every 30 minutes I gave up on reactivity for that element and just did the join on the server and shipped the data to the client via a method call. I also created a server-side expiring cache for these results and stored them by user (special thanks to Matt DeBergalis for this suggestion).

  6. Do a preventative nightly restart. I have a cron job that tells forever to restart our app once a day in the middle of the night. That brings the CPU down from ~10% to 1%. This seems like black magic, but the fact that the CPU usage changes after a reset leads me to believe this is a good idea.

Updated thoughts (1/13/14)

  • We migrated to oplog tailing as soon as it was available (meteor 0.7) and that made a big difference. Note that in order to get access to the oplog, you’ll probably need to either host your own db or run a dedicated instance on the hosting provider of your choice. I’d also recommend adding the facts package to actually tell if its working.

  • There was a memory leak discovered in publish-with-relations, and as of this writing the atmosphere version (v0.1.5) hasn’t been bumped to reflect these changes. If you are using it in production, I strongly recommend checking out the HEAD version and running it locally.

  • We stopped doing nightly restarts a couple of weeks ago. So far everything has been fine (fingers crossed).

Updated thoughts (7/2/14)

  • A few months ago we switched over to using an Elastic Deployment on mongohq. It’s affordable, the performance has been great, and they even have a blog post which tells you how to enable oplog tailing.

  • I’d strongly recommend checking out kadira to help diagnose performance issues in your app. Also check out the academy articles which have a number of good tips in them.

More Related Contents:

  1. 2013 Meteor NPM Packages
  2. Meteor Subscribe doesn’t update sort order of collection
  3. What’s going on with Meteor and Fibers/bindEnvironment()?
  4. Meteor WebSocket handshake error 400 with nginx
  5. How can I debug my Meteor app using the WebStorm IDE?
  6. im new to making discord bots with node.js, i am having trouble with an error, it says that message is not defined in line 14 and 18
  7. E11000 duplicate key error index in mongodb mongoose
  8. How to share sessions with Socket.IO 1.x and Express 4.x?
  9. Mongoose: findOneAndUpdate doesn’t return updated document
  10. Error: EACCES: permission denied, access ‘/usr/local/lib/node_modules’
  11. React-router with BrowserRouter / browserHistory doesn’t work on refresh
  12. How to include route handlers in multiple files in Express? [duplicate]
  13. How to redirect 404 errors to a page in ExpressJS?
  14. How to npm install to a specified directory?
  15. socket.io and session?
  16. callback to handle completion of pipe
  17. What is the difference between res.end() and res.send()?
  18. Declare multiple module.exports in Node.js
  19. Express next function, what is it really for?
  20. How do I determine the correct “max-old-space-size” for node.js?
  21. MongoDB Node check if objectid is valid
  22. npm ERR! code UNABLE_TO_GET_ISSUER_CERT_LOCALLY
  23. How do you authenticate a server to Firebase?
  24. How can I import an ES module in the Node.js REPL in Node 8?
  25. Idiomatic way to wait for multiple callbacks in Node.js
  26. How to send a pdf file from Node/Express app to the browser
  27. Cache Control for Dynamic Data Express.JS
  28. Setting all queries to raw = true sequelize
  29. How npm start runs a server on port 8000
  30. How random is crypto#randomBytes?

Leave a Comment