---- datatemplateentry publication ----
template            : publications:display_template
title               : ScreenshotMatcher - Taking Smartphone Photos to Capture Screenshots
date_date           : 2021-09-06
template            : publications:display_template
authors_            : [[people:andreas_schmid|Andreas Schmid]], [[people:thomas_fischer|Thomas Fischer]], [[people:alexander_weichart|Alexander Weichart]], Alexander Hartmann, [[people:raphael_wimmer|Raphael Wimmer]]
epub_url            : https://epub.uni-regensburg.de/47814/
publisher_url       : https://dl.acm.org/doi/10.1145/3473856.3474014
pdf_url             : https://epub.uni-regensburg.de/47814/1/screenshotmatcher.pdf
bibtex_url          : https://epub.uni-regensburg.de/cgi/export/eprint/47814/BibTeX/epub-eprint-47814.bib
video_url           : https://www.youtube.com/watch?v=8NxSi1Cx9bs
doi                 : 10.1145/3473856.3474014
photo_img           : projects:smartphone_screenshot:screenshotmatcher_usage.png
short-description   : Application that allows creating screenshots of a computer screen by photographing it with a smartphone.
abstract            : Taking screenshots is a common way of capturing screen content to share it with others or save it for later. Even though all major desktop operating systems come with a screenshot function, a lot of people also use smartphone cameras to photograph screen contents instead. While users see this method as faster and more convenient, image quality is significantly lower. With ScreenshotMatcher, we present a system that allows for capturing a high-fidelity screenshot by taking a smartphone photo of (part of) the screen. A smartphone application sends a photo of the screen region of interest to a program running on the PC which retrieves the matching screen region and sends it back to the smartphone. Comparing four feature matching algorithms and multiple parameters, we identified a combination of ORB keypoint detection (feature limit 2000) and a brute force feature matcher using Hamming distance as the best solution for this task (success rate: 85%, processing time: 90 ms). This raw performance results in a real-world success rate of 47% and a mean response time per screenshot of 878 ms as measured in a remote user study (N=19). Released as open-source code, ScreenshotMatcher may be used as a basis for applications and research prototypes that bridge the gap between PC and smartphone.
published-in        : Proceedings of the Mensch und Computer 2021
project             : screenshotmatcher
----