[GSoC 2013]Interactive and Visual Test Automation for SDL 2.0

Hi all,
I’m interested in working on the interactive and visual test automation
idea. I’ve got several years of experience in C and Python. I’ve used SDL
for about two years, although the last time I used it was a couple of years
ago.

Here’s what I’ve done up until now:
I cloned the mercurial repo and built SDL 2.0. Then I built the tests. But
when I run testsprite2, I get a screen with four white rectangles on each
corner of the window and a big green X connecting them. There’s no
movement. I’ve tried increasing the number of sprites and fiddling with the
blend options, which reduces the frame rate, but I still can’t see any
movement. Is this normal?

I would be really grateful if you could help me with this and give me some
me some more information on how to get started with this idea. I’m not
aware of how to run the tests using the SDL test automation framework for
example, so some help with that would be greatly appreciated.

I’m looking forward to hearing from you.
Regards,
Apoorv Upreti

Hi Apoorv,

thanks for the interest in this project.

At a high level, the project idea is to create some validation around
SDLs window, rendering and mouse behavior (as driven by testsprite2).
The content of the window is actually somewhat secondary (but should be
used to validate that everything works as it should). The key goal is to
check what is actually happening on the screen including window manager
stuff, screen content, mouse behavior, title, taskbar state, etc. is
correct.

Check the variety of available parameters:
$ ./testsprite2.exe --help
Usage: I:\Sources\SDL\test\testsprite2.exe [–video driver] [–renderer
driver] [–info all|video|modes|render|event] [–log
all|error|system|audio|video|render|input] [–display N] [–fullscreen |
–fullscreen-desktop | --windows N] [–title title] [–icon icon.bmp]
[–center | --position X,Y] [–geometry WxH] [–min-geometry WxH]
[–max-geometry WxH] [–depth N] [–refresh R] [–vsync] [–noframe]
[–resize] [–minimize] [–maximize] [–grab] [–blend
none|blend|add|mod] [–cyclecolor] [–cyclealpha]

Here are some of the many variations that do change what is shown on the
screen:
$ ./testsprite2.exe --title “Test”
$ ./testsprite2.exe --geometry 100x100
$ ./testsprite2.exe --grab
Note the mouse behavior.
$ ./testsprite2.exe --noframe
$ ./testsprite2.exe --minimize

A goal is to test these variations and window behaviors (all or most of
them) on as many platforms as possible and as automatically as possible.
You may find additional uses of testsprite2.exe (i.e. compile a table of
framerates for various settings).

Here are some steps that could get us what we want:

  1. create a test driver that calls testsprite2 with all/most allowable
    parameters
    [this is basically a “smart” script to check variations - see for
    example http://www.pairwise.org on some theory behind ways to make
    "smart" tests]
  2. add some interactive capabilities to the test driver so one can stop
    testsprite2 (i.e. simulate someone hitting ESC) or control the mouse
    (one can/should modify testsprite2 so it will automatically stop after -
    say - 5 seconds to be safe)
    [do some research into existing frameworks and test code that may do
    what we need - i.e. inject keyboard and mouse events - or roll-your-own
    code]
  3. create a validation framework that can plug into the test driver and
    check that the final screen output is as expected
    [again do some research into existing frameworks and test code that may
    do what we need - i.e. full-screen screenshot capability and some image
    analysis for checking the screen looks as expected - or, again,
    roll-your-own code]
  4. make 1-3 as easy to install and use as possible
  5. make 1-4 as cross-platform as possible

Keep in mind that this is just my rough sketch, it is an example… you
should come up with your own steps and details in your proposal. If you
work on the project, you’d make the architecture and design choices and
as you follow through with an implementation guided by the initial
proposal. We want your ideas! However, input and suggestions from the
mentors and list members are probably very valuable. :slight_smile:

Please prepare for your application by keeping the above info in mind -
see section 4. and 5. on the GSoC FAQ for more info on “How does a
student apply?” and “What should a student proposal look like?” here:
http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page

Good luck!
–Andreas

PS: BTW, I can’t see any visible sprites after a quick check on my
Windows box either (both using a cygwin and VisualStudio build) … is
that your platform? Maybe a candidate for a bug report. ;-)On 4/13/2013 7:01 PM, Apoorv Upreti wrote:

Hi all,
I’m interested in working on the interactive and visual test
automation idea. I’ve got several years of experience in C and Python.
I’ve used SDL for about two years, although the last time I used it
was a couple of years ago.

Here’s what I’ve done up until now:
I cloned the mercurial repo and built SDL 2.0. Then I built the tests.
But when I run testsprite2, I get a screen with four white rectangles
on each corner of the window and a big green X connecting them.
There’s no movement. I’ve tried increasing the number of sprites and
fiddling with the blend options, which reduces the frame rate, but I
still can’t see any movement. Is this normal?

I would be really grateful if you could help me with this and give me
some me some more information on how to get started with this idea.
I’m not aware of how to run the tests using the SDL test automation
framework for example, so some help with that would be greatly
appreciated.

I’m looking forward to hearing from you.
Regards,
Apoorv Upreti


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

testsprite2 should be fixed, thanks for the report!On Sat, Apr 13, 2013 at 7:01 PM, Apoorv Upreti wrote:

Hi all,
I’m interested in working on the interactive and visual test automation
idea. I’ve got several years of experience in C and Python. I’ve used SDL
for about two years, although the last time I used it was a couple of years
ago.

Here’s what I’ve done up until now:
I cloned the mercurial repo and built SDL 2.0. Then I built the tests. But
when I run testsprite2, I get a screen with four white rectangles on each
corner of the window and a big green X connecting them. There’s no
movement. I’ve tried increasing the number of sprites and fiddling with the
blend options, which reduces the frame rate, but I still can’t see any
movement. Is this normal?

I would be really grateful if you could help me with this and give me some
me some more information on how to get started with this idea. I’m not
aware of how to run the tests using the SDL test automation framework for
example, so some help with that would be greatly appreciated.

I’m looking forward to hearing from you.
Regards,
Apoorv Upreti


SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org