Konva – Mouse position + pixelRatio will mess up your day

A user recently had a query regarding how to get the pixel color data for a point on the stage. This is quite easy, but there’s a catch on high pixel density devices – think Apple devices like the MacBook and iPad. Here’s the solution…

TLDR: Here’s the sample code – try it on your retinal display device.

The Mission

As stated, we need to get the color data for the pixel under the mouse – the kind of capability you need for an eye-dropper type gadget.

To get the pixel data we use the HTML5 Canvas element’s getImageData(co-ordinate) method. The trick here is that this is a method of the canvas object – not a Konva method, and it works on absolute co-ordinates. If you think about it, Konva does all kinds of great things with layers, shapes, scale, rotation and translation, but this getImageData() method is fully hammered in to the 1:1 scale and non-transformed co-ordinate system on the web browser page.

The Issue

Ok ok – why the song and dance about getImageData() co-ordinates ?

Glad you asked!

Because in the Konva world we have 2 ways to get the mouse position:

Stage.getRelativePointerPosition()

This always gives you the position of the pointer over the stage, whether the stage is rotated, scaled or translated (dragged). This is not useful for using with getIMageData() unless your stage is positioned at (0, 0) and never scaled or rotated.

Stage.getPointerPosition()

This gives you the position of the pointer in the HTML5 Canvas element. This is the one you want for using with getImageData(), regardless of how you move or rotate the stage.

And here comes the issue…

The Stage.getPointerPosition() method is unaware of the device pixel ratio. The effect of that is the position returned from this method will only be accurate when the pixel ratio is 1. However, on high pixel density display screens such as Apple MacBook and iPads, the pixel ratio is 2 or more.

And the effect of this little point is that the co-ordinates returned by Stage.getPointerPosition() will be wrong by a factor of the pixel ratio.

The magic that you need to solve this is

  const pos = stage.getPointerPosition();
  const canvas = layer.getCanvas()
  const ctx = canvas.getContext('2d');
  const px = ctx.getImageData(pos.x * Konva.pixelRatio, pos.y * Konva.pixelRatio, 1, 1),
        data = px.data,
  colorDesc = `rgba(${data[0]}, ${data[1]}, ${data[2]}, ${data[3] / 255})`;

Line 1 gets the absolute pointer position as discussed above. It gives a point object with x and y properties.

Lines 2 & 3 get the HTML5 Canvas context object which has the getImageData() method. This returns a structure with a data property, which is where the color values live in a 4-slot array.

Line 4 is the important one. Notice the multiplication of the position co-ordinates by the Konva.pixelRatio. This is the magic dust that fixes the issue.

Summary

We’ve seen that if we want to get the color of the pixels under the mouse pointer we use stage.getPointerPosition(), and that this has a blind side when it comes to high pixel density screens. We now know the quick and simple fix which is to multiply the co-ordinates we get by the Konva.pixelRatio value.

Thanks for reading.

VW. Nov 22