Is screenshot to OCR possible on macOS?

Solution 1:

I have used FastScripts to assign a keyboard shortcut to these commands:

screencapture -i /tmp/tesseract.png
tesseract /tmp/tesseract.png stdout|tr -d \\f|LC_CTYPE=UTF-8 pbcopy

brew install tesseract installs Tesseract. Tesseract prints to STDOUT when the name of the output file is specified as stdout. A form feed character is added to the end of the output.

For non-English text, you have to manually specify the language: for example -l deu for German. To list the available languages, run tesseract --list-langs.

Edit: In cases where Tesseract produces inaccurate results, I now use Google's Cloud Vision API instead. Demo: https://cloud.google.com/vision/docs/drag-and-drop.

curl -sH"Authorization:Bearer $(gcloud auth application-default print-access-token)" -HContent-Type:application/json\;charset=utf-8 https://vision.googleapis.com/v1/images:annotate -d@<(printf %s '{"requests":[{"image":{"content":"'"$(base64 input.png)"'"},"features":[{"type":"TEXT_DETECTION"}]}]}')|jq -r '.responses[0].fullTextAnnotation.text'

Edit 2: My free trial for the Google API expired, so I now use Yandex Translate to do OCR for images that Tesseract has trouble with: https://translate.yandex.com/ocr.

Edit 3: You can do this to also translate the text (https://cloud.google.com/translate/docs/basic/translating-text):

curl -HAuthorization:Bearer\ $(gcloud auth application-default print-access-token) -HContent-Type:application/json -d@<(jq -sR '{q:.,target:"de"}'<<<example) https://translation.googleapis.com/language/translate/v2|jq -r '.data.translations[0].translatedText'

Solution 2:

TextSniper is another app that does screenshot OCR. It recognizes any text from screenshot and copies editable text to clipboard. Does not require internet connection and is lighting fast. Another great feature of TextSniper is that it can turn recognized text to speech, really helpful for people who have difficulties reading text from screen.