how to pass large data to web workers

UPDATE

Modern versions of Chrome, Edge, and Firefox now support SharedArrayBuffers (though not safari at the time of this writing see SharedArrayBuffers on MDN), so that would be another possibility for a fast transfer of data with a different set of trade offs compared to a transferrable (you can see MDN for all the trade offs and requirements of SharedArrayBuffers).

UPDATE:

According to Mozilla the SharedArrayBuffer has been disabled in all major browsers, thus the option described in the following EDIT does no longer apply.

Note that SharedArrayBuffer was disabled by default in all major
browsers on 5 January, 2018 in response to Spectre.

EDIT: There is now another option and it is sending a sharedArray buffer. This is part of ES2017 under shared memory and atomics and is now supported in FireFox 54 Nightly. If you want to read about it you can look here. I will probably write up something some time and add it to my answer. I will try and add to the performance benchmark as well.

To answer the original question:

I am working on web workers and I am passing large amount of data to
web worker, which takes a lot of time. I want to know the efficient
way to send the data.

The alternative to @MichaelDibbets answer, his sends a copy of the object to the webworker, is using a transferrable object which is zero-copy.

It shows that you were intending to make your data transferrable, but I’m guessing it didn’t work out. So I will explain what it means for some data to be transferrable for you and future readers.

Transferring objects “by reference” (although that isn’t the perfect term for it as explained in the next quote) doesn’t just work on any JavaScript Object. It has to be a transferrable data-type.

[With Web Workers] Most browsers implement the structured cloning
algorithm, which allows you to pass more complex types in/out of
Workers such as File, Blob, ArrayBuffer, and JSON objects. However,
when passing these types of data using postMessage(), a copy is still
made. Therefore, if you’re passing a large 50MB file (for example),
there’s a noticeable overhead in getting that file between the worker
and the main thread.

Structured cloning is great, but a copy can take hundreds of
milliseconds. To combat the perf hit, you can use Transferable
Objects.

With Transferable Objects, data is transferred from one context to
another. It is zero-copy, which vastly improves the performance of
sending data to a Worker. Think of it as pass-by-reference if you’re
from the C/C++ world. However, unlike pass-by-reference, the ‘version’
from the calling context is no longer available once transferred to
the new context. For example, when transferring an ArrayBuffer from
your main app to Worker, the original ArrayBuffer is cleared and no
longer usable. Its contents are (quiet literally) transferred to the
Worker context.

– Eric Bidelman Developer at Google, source: html5rocks

The only problem is there are only two things that are transferrable as of now. ArrayBuffer, and MessagePort. (Canvas Proxies are hopefully coming later). ArrayBuffers cannot be manipulated directly through their API and should be used to create a typed array object or a DataView to give a particular view into the buffer and be able to read and write to it.

From the html5rocks link

To use transferrable objects, use a slightly different signature of
postMessage():

worker.postMessage(arrayBuffer, [arrayBuffer]);

window.postMessage(arrayBuffer, targetOrigin, [arrayBuffer]);

The worker case, the first argument is the data and the second is the
list of items that should be transferred. The first argument doesn’t
have to be an ArrayBuffer by the way. For example, it can be a JSON
object:

worker.postMessage({data: int8View, moreData: anotherBuffer}, [int8View.buffer, anotherBuffer]);

So according to that your

var worker = new Worker('js2.js');
worker.postMessage(buffer, [ buffer]);
worker.postMessage(obj, [obj.mat2]);

should be performing at great speeds and should be being transferred zero-copy. The only problem would be if your buffer or obj.mat2 is not an ArrayBuffer or transferrable. You may be confusing ArrayBuffers with a view of a typed array instead of what you should be using its buffer.

So if you have this ArrayBuffer and it’s Int32 representation. (though the variable is titled view it is not a DataView, but DataView’s do have a property buffer just as typed arrays do. Also at the time this was written the MDN use the name ‘view’ for the result of calling a typed arrays constructor so I assumed it was a good way to define it.)

var buffer = new ArrayBuffer(90000000);
var view = new Int32Array(buffer);
for(var c=0;c<view.length;c++) {
    view[c]=42;
}

This is what you should not do (send the view)

worker.postMessage(view);

This is what you should do (send the ArrayBuffer)

worker.postMessage(buffer, [buffer]);

These are the results after running this test on plnkr.

Average for sending views is 144.12690000608563
Average for sending ArrayBuffers is 0.3522000042721629

EDIT: As stated by @Bergi in the comments you don’t need the buffer variable at all if you have the view, because you can just send view.buffer like so

worker.postMessage(view.buffer, [view.buffer]);

Just as a side note to future readers just sending an ArrayBuffer without the last argument specifying what the ArrayBuffers are you will not send the ArrayBuffer transferrably

In other words when sending transferrables you want this:

worker.postMessage(buffer, [buffer]);

Not this:

worker.postMessage(buffer);

EDIT: And one last note since you are sending a buffer don’t forget to turn your buffer back into a view once it’s received by the webworker. Once it’s a view you can manipulate it (read and write from it) again.

And for the bounty:

I am also interested in official size limits for firefox/chrome (not
only time limit). However answer the original question qualifies for
the bounty (;

As to a webbrowsers limit to send something of a certain size I am not completeley sure, but from that quote that entry on html5rocks by Eric Bidelman when talking about workers he did bring up a 50 mb file being transferred without using a transferrable data-type in hundreds of milliseconds and as shown through my test in a only around a millisecond using a transferrable data-type. Which 50 mb is honestly pretty large.

Purely my own opinion, but I don’t believe there to be a limit on the size of the file you send on a transferrable or non-transferrable data-type other than the limits of the data type itself. Of course your biggest worry would probably be for the browser stopping long running scripts if it has to copy the whole thing and is not zero-copy and transferrable.

Hope this post helps. Honestly I knew nothing about transferrables before this, but it was fun figuring out them through some tests and through that blog post by Eric Bidelman.