Performance Tradeoff – When is MATLAB better/slower than C/C++

by

I have been using Matlab and C++ for about 10 years. For every numerical algorithms implemented for my research, I always start from prototyping with Matlab and then translate the project to C++ to gain a 10x to 100x (I am not kidding) performance improvement. Of course, I am comparing optimized C++ code to the fully vectorized Matlab code. On average, the improvement is about 50x.

There are lot of subtleties behind both of the two programming languages, and the following are some misunderstandings:

  1. Matlab is a script language but C++ is compiled

    Matlab uses JIT compiler to translate your script to machine code, you can improve your speed at most by a factor 1.5 to 2 by using the compiler that Matlab provides.

  2. Matlab code might be able to get fully vectorized but you have to optimize your code by hand in C++

    Fully vectorized Matlab code can call libraries written in C++/C/Assembly (for example Intel MKL). But plain C++ code can be reasonably vectorized by modern compilers.

  3. Toolboxes and routines that Matlab provides should be very well tuned and should have reasonable performance

    No. Other than linear algebra routines, the performance is generally bad.

The reasons why you can gain 10x~100x performance in C++ comparing to vectorized Matlab code:

  1. Calling external libraries (MKL) in Matlab costs time.

  2. Memory in Matlab is dynamically allocated and freed. For example, small matrices multiplication:
    A = B*C + D*E + F*G
    requires Matlab to create 2 temporary matrices. And in C++, if you allocate your memory before hand, you create NONE. And now imagine you loop that statement for 1000 times. Another solution in C++ is provided by C++11 Rvalue reference. This is the one of the biggest improvement in C++, now C++ code can be as fast as plain C code.

  3. If you want to do parallel processing, Matlab model is multi-process and the C++ way is multi-thread. If you have many small tasks needing to be parallelized, C++ provides linear gain up to many threads but you might have negative performance gain in Matlab.

  4. Vectorization in C++ involves using intrinsics/assembly, and sometimes SIMD vectorization is only possible in C++.

  5. In C++, it is possible for an experienced programmer to completely avoid L2 cache miss and even L1 cache miss, hence pushing CPU to its theoretical throughput limit. Performance of Matlab can lag behind C++ by a factor of 10x due to this reason alone.

  6. In C++, computational intensive instructions sometimes can be grouped according to their latencies (code carefully in assembly or intrinsics) and dependencies (most of time is done automatically by compiler or CPU hardware), such that theoretical IPC (instructions per clock cycle) could be reached and CPU pipelines are filled.

However, development time in C++ is also a factor of 10x comparing to Matlab!

The reasons why you should use Matlab instead of C++:

  1. Data visualization. I think my career can go on without C++ but I won’t be able to survive without Matlab just because it can generate beautiful plots!

  2. Low efficiency but mathematically robust build-in routines and toolboxes. Get the correct answer first and then talk about efficiency. People can make subtle mistakes in C++ (for example implicitly convert double to int) and get sort of correct results.

  3. Express your ideas and present your code to your colleagues. Matlab code is much easier to read and much shorter than C++, and Matlab code can be correctly executed without compiler. I just refuse to read other people’s C++ code. I don’t even use C++ GNU scientific libraries because the code quality is not guaranteed. It is dangerous for a researcher/engineer to use a C++ library as a black box and take the accuracy as granted. Even for commercial C/C++ libraries, I remember Intel compiler had a sign error in its sin() function last year and numerical accuracy problems also occurred in MKL.

  4. Debugging Matlab script with interactive console and workspace is a lot more efficient than C++ debugger. Finding an index calculation bug in Matlab could be done within minutes, but it could take hours in C++ figuring out why the program crashes randomly if boundary check is removed for the sake of speed.

Last but not the least:

Because once Matlab code is vectorized, there is not much left for a programmer to optimize, Matlab code performance is much less sensitive to the quality of the code comparing with C++ code. Therefore it is best to optimize computation algorithms in Matlab, and marginally better algorithms normally have marginally better performance in Matlab. On the other hand, algorithm test in C++ requires decent programmer to write algorithms optimized more or less in the same way, and to make sure the compiler does not optimize the algorithms differently.

My recent experience in C++ and Matlab:

I made several large Matlab data analysis tools in the past year and suffered from the slow speed of Matlab. But I was able to improve my Matlab program speed by 10x through the following techniques:

  • Run/profile the Matlab script, re-implement critical routines in C/C++ and compile with MEX. Critical routines are mostly likely logically simple but numerically heavy. This improves speed by 5x.

  • Simplify “.m” files shipped with Matlab tool boxes by commenting all unnecessary safety checks and output parameter computations. Please be reminded that the modified code cannot be distributed with the rest of the user scripts. This improves speed by another 2x (after C/C++ and MEX).

The improved code is ~98% in Matlab and ~2% in C++.

I believe it is possible to improve the speed by another 2x (total 20x) if the entire tool is coded in C++, this is ~100x speed improvement of the computation routines. The hard drive I/O will then dominate the program run time.

Question for Mathworks engineers:

When Matlab code is fully vectorized, one of the performance limiting factor is the matrix indexing operation. For instance, a finite difference operation needs to be performed on Matrix A which has a dimension of 5000×5000:

B = A(:,2:end)-A(:,1:end-1)

The matrix indexing operation makes the Matlab code multiple times slower than the C++ code. Can the matrix indexing performance be improved?

More Related Contents:

  1. Which yields faster code?
  2. How to overload std::swap()
  3. Why is my program slow when looping over exactly 8192 elements?
  4. Why does C++ compilation take so long?
  5. What is the efficient way to count set bits at a position or lower?
  6. Fastest way to determine if an integer is between two integers (inclusive) with known sets of values
  7. Virtual functions and performance – C++
  8. How can I measure CPU time and wall clock time on both Linux/Windows?
  9. How do exceptions work (behind the scenes) in c++
  10. Why are Python Programs often slower than the Equivalent Program Written in C or C++?
  11. Is there any guarantee of alignment of address return by C++’s new operation?
  12. connected components in OpenCV
  13. Getting an accurate execution time in C++ (micro seconds)
  14. Can the use of C++11’s ‘auto’ improve performance?
  15. A fast method to round a double to a 32-bit int explained
  16. linux perf: how to interpret and find hotspots
  17. Is there a 128 bit integer in C++?
  18. What is the Fastest Method for High Performance Sequential File I/O in C++?
  19. Is accessing data in the heap faster than from the stack?
  20. Fastest way to get a positive modulo in C/C++
  21. Do c++ templates make programs slow?
  22. Matlab vs C++ Double Precision
  23. LNK2038: mismatch detected for ‘RuntimeLibrary’: value ‘MT_StaticRelease’ doesn’t match value ‘MD_DynamicRelease’ in file.obj
  24. Initializing a ublas vector from a C array
  25. Multiple classes in a header file vs. a single header file per class
  26. Benefits of using reserve() in a vector – C++
  27. Which is faster/preferred: memset or for loop to zero out an array of doubles?
  28. Is it possible to map string to int faster than using hashmap?
  29. C++ Serialization Performance
  30. Why is pow(int, int) so slow?

Leave a Comment