What unit of measure does Paint.setStrokeWidth() use?

The stroke width is defined in pixels (yes it's a float, and there's no problem with using fractions of pixels :)

setStrokeWidth uses pixels.

So to convert you dps to pixels for painting:

int dpSize =  10;
DisplayMetrics dm = getResources().getDisplayMetrics() ;
float strokeWidth = TypedValue.applyDimension(TypedValue.COMPLEX_UNIT_DIP, dpSize, dm);
paint.setStrokeWidth(strokeWidth);

One thing to keep in mind if you're attempting to onDraw your own "canvas border" is that even the stroke is clipped. So

paint.style = Paint.Style.STROKE
rect.set(0, 0, width, height)
paint.strokeWidth = 10F
canvas.drawRect(rect, paint)

will result in a "border" that is only 5F wide. This is due to the stroke being centered over the rect, which in this case is the edge of the canvas - resulting in half of it actually being clipped as it's outside the canvas.

To fix this, simply multiply your desired boarder thickness by 2. [In the example above, you'd set strokeWidth = 20F and you'd "see" a stroke of 10F.]