This was an interesting scenario that I came across with. PDF.JS by default can let you select text and copy them to the clipboard. However, since my requirement had us do with creating annotations and since it is messy to mess around the DIV layers of the PDF.JS pages, we had to create a separate Canvas element to draw out annotations in.
Now, the problem is that since there is an extra Canvas layer on top of the existing Canvas layer of PDF.JS where they draw their images if the PDF contains pictures, selecting text is not possible.
However, through a CSS attribute it is possible to let mouse clicks pass through the Canvas element using the pointer-event attribute. The values for these are auto for enabled and none for disabled.