Hello SDL community!
The Google Summer of Code project call
ed “SDL Test Coverage” is going forward. Since the last post I made (some
weeks ago) to the list, the test harness has become a bit more mature. It
now supports logging its output in two formats (plain text and xml) and it
can kill hung tests to mention a few. It also has elementary support for
fuzz testing.
During the rest of the project I’ll keep enhacing the fuzzer and fixing
problems found from the harness.
I’m currently adding new data generators to the fuzzer.
Hopefully I’ll manage to write a few tests, too.
If there’s any eager test writers on the list, now would be a perfect time
to jump in and try out the harness I’ve developed during the summer. All
sorts of contributions would be highly appreciated including things like bug
fixes, test cases for SDL, fixing usage guidelines in the project’s wiki.
All sorts of stuff. This would help me to fine tune the project and make it
better.
Check out the wiki for more information about the project (
https://bitbucket.org/Markusk/sdl-gsoc/wiki/)!
Here’s some other project related links:
Sprint 3 retrospective:
https://bitbucket.org/Markusk/sdl-gsoc/wiki/sprintretro3
The current sprintlog:
https://bitbucket.org/Markusk/sdl-gsoc/wiki/sprintlog4
Cheers
Markus Kauppila
The Google Summer of Code project call
ed “SDL Test Coverage” is going forward. Since the last post I made
(some weeks ago) to the list, the test harness has become a bit more
mature. It now supports logging its output in two formats (plain text
and xml) and it can kill hung tests to mention a few. It also has
elementary support for fuzz testing.
Btw: I want to hook this up to an official buildbot some time after the
summer of code. I’m envisioning something that pulls from the main repo,
builds on several platforms, runs the tests, and if everything works
out, pushes the changesets to a “known to work” repo somewhere.
At least, that’s my fantasy at the moment. 
–ryan.
Thanks Markus - it seems quite straightforward to create new tests. I’ve
just added two simple test cases (for SDL_GetNumRenderDrivers and
SDL_CreateRenderer) to the libtestrender suite in like 15min.
$./runner -s libtestrender --log-stdout --verbose
Executing tests from libtestrender
Executing test: render_testCreateRenderer (in libtestrender). Exec
key: 2DCFA0B1D34A14A8
AssertTrue: passed - SDL_Init(SDL_INIT_VIDEO):
AssertPass: passed - Window created
AssertPass: passed - Renderer created
Assert summary: 0 failed, 3 passed (total: 3)
render_testCreateRenderer: ok
Executing test: render_testGetNumRenderDrivers (in libtestrender).
Exec key: EDD397678B3E06DA
AssertTrue: passed - SDL_Init(SDL_INIT_VIDEO):
AssertTrue: passed - Number of renderers >= 1, reported as 2
Assert summary: 0 failed, 2 passed (total: 2)
render_testGetNumRenderDrivers: ok
Suite executed. 2 passed, 0 failed and 0 skipped. Total runtime 0.20500
seconds
Ran 2 tests in 0.20600 seconds from 1 suites.
2 tests passed
0 tests failed
0 tests skipped
Cheers,
AndreasOn 8/6/11 12:28 PM, Markus Kauppila wrote:
Hello SDL community!
The Google Summer of Code project call
ed “SDL Test Coverage” is going forward. Since the last post I made
(some weeks ago) to the list, the test harness has become a bit more
mature. It now supports logging its output in two formats (plain text
and xml) and it can kill hung tests to mention a few. It also has
elementary support for fuzz testing.
During the rest of the project I’ll keep enhacing the fuzzer and
fixing problems found from the harness.
I’m currently adding new data generators to the fuzzer.
Hopefully I’ll manage to write a few tests, too.
If there’s any eager test writers on the list, now would be a perfect
time to jump in and try out the harness I’ve developed during the
summer. All sorts of contributions would be highly appreciated
including things like bug fixes, test cases for SDL, fixing usage
guidelines in the project’s wiki. All sorts of stuff. This would help
me to fine tune the project and make it better.
Check out the wiki for more information about the project (
https://bitbucket.org/Markusk/sdl-gsoc/wiki/)!
Here’s some other project related links:
Sprint 3 retrospective:
https://bitbucket.org/Markusk/sdl-gsoc/wiki/sprintretro3
The current sprintlog:
https://bitbucket.org/Markusk/sdl-gsoc/wiki/sprintlog4
Cheers
Markus Kauppila
SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
2011/8/6 Ryan C. Gordon
The Google Summer of Code project call
ed “SDL Test Coverage” is going forward. Since the last post I made
(some weeks ago) to the list, the test harness has become a bit more
mature. It now supports logging its output in two formats (plain text
and xml) and it can kill hung tests to mention a few. It also has
elementary support for fuzz testing.
Btw: I want to hook this up to an official buildbot some time after the
summer of code. I’m envisioning something that pulls from the main repo,
builds on several platforms, runs the tests, and if everything works out,
pushes the changesets to a “known to work” repo somewhere.
At least, that’s my fantasy at the moment.
–ryan.
Thanks. I’ll keep this in mind.
I have no idea how the official buildbot works but as far as I can imagine
that probably could be done (at least with little work). Currently the test
harness only supports OS X and Linux (tested on Xubuntu). Windows port has
to be done in the future. If all the tests pass, the harness will return 0
as its return value, 1 if any of the tests fail and >1 on other errors…
Well, anyway it shouldn’t be too difficult to satisfy your fantasy with the
harness since it is rather flexible (or tries to be)
– Markus