Ask HN: Is it feasible to observe an LCD with a camera and verify UI elements? | Hacker News Reader

Ask HN: Is it feasible to observe an LCD with a camera and verify UI elements?

2 pointsmaxxxxx5 comments

I thought maybe somebody here has experience witt his.

We do a lot of testing of devices that have LCD screens. Right now the testing is done mainly manually. To automate the tests basically two things have to be done:

- Press buttons. I think that can be done with a 3D printed fixture and something like Lego Robotics.

- Read the screen with a camera: Are there libraries that allow understanding of screen content (buttons, text boxes and so on) even under variable lighting conditions? We have tried screenshots but for that the device had to be in a totally dark enclosure and the brightness couldn't change. I am also interested in screen transitions like progress screens or splash screens that change quickly)

I imagine a webcam capture to a window could then be inspected with a tool such as http://www.sikulix.com/

I was attemping to scrap a Flash newspaper website with PHP and found it wasnt possible easily with Curl, so i wanted to automate navigating chrome and the Flash app using Sikulix about a year ago ... So I imagine Sikulix or a similar tool could be the glue you want to marry your robot tool with information you inspect from the webcam/LCD screens.

Sounds like a fair amount of work though.

Sikulix looks pretty good but it seems it takes screenshots with the OS. For us the tested device is a black box so we need a camera.

Eh? I just mean sikulix would be your driver -- your business logic. So you setup a webcam to record your LCD screen. The webcam feed is displayed on a desktop (OS) and a custom sikulix screen interprets whats on the webcam window and sends commands to your bot or device.

ah. One level of indirection. That can work.

Bug report generated automatically by the system:

BUG #4123 moving little black dots

There are occasionally, random black dots a few pixels wide, moving slowly and apparently in a brownian movement, across the screen. (This is not limited to the tested window).

Reply on news.ycombinator.com