How to get object in WebGL 3d space from a mouse click coordinate

You’re looking for an unproject function, which converts screen coordinates into a ray cast from the camera position into the 3D world. You must then perform ray/triangle intersection tests to find the closest triangle to the camera which also intersects the ray.

I have an example of unprojecting available at jax/camera.js#L568 — but you’ll still need to implement ray/triangle intersection. I have an implementation of that at jax/triangle.js#L113.

There is a simpler and (usually) faster alternative, however, called ‘picking’. Use this if you want to select an entire object (for instance, a chess piece), and if you don’t care about where the mouse actually clicked. The WebGL way to do this is to render the entire scene in various shades of blue (the blue is a key, while red and green are used for unique IDs of the objects in the scene) to a texture, then read back a pixel from that texture. Decoding the RGB into the object’s ID will give you the object that was clicked. Again, I’ve implemented this and it’s available at jax/world.js#L82. (See also lines 146, 162, 175.)

Both approaches have pros and cons (discussed here and in some of the comments after) and you’ll need to figure out which approach best serves your needs. Picking is slower with huge scenes, but unprojecting in pure JS is extremely slow (since JS itself isn’t all that fast) so my best recommendation would be to experiment with both.

FYI, you could also look at the GLU project and unproject code, which I based my code loosely upon: http://www.opengl.org/wiki/GluProject_and_gluUnProject_code

Leave a Comment