You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Check if they can be run on an unmodified Jamulus code (I have seen various commented-out code related to "tests" within the code, not sure if this is related)
Check if running them as part of the build is useful and possible
Check if running them as part of CI (e.g. for PRs) is useful and possible
cc @pljones (as you suggested looking at the tests some time ago ;))
The code in testbench.h is a simple template-based fuzzer for the Jamulus protocol - that is, it generates valid messages with completely random field values. It's intended for testing the protocol parsers on the client and server side. To enable it, you uncomment the CTestBench like in main.cpp and recompile, then run Jamulus as usual - this'll cause it to run with an additonal thread that periodically generates a random message and feeds it into the parser.
Because it's non-deterministic, and it needs to run for a very long time to have a chance of finding a problem, it's not really very useful as part of an automated test suite. It also needs updating for the current list of message types (e.g. it can't do PROTMESSID_CLM_RED_SERVER_LIST yet).
Back in mid-2020 I experimented with using it to generate seeds for a coverage-directed fuzzer, afl-fuzz, which is much more efficient (but still non-deterministic). That work is in this branch, which also needs updating. The change to add the -Z option might be worth including in mainline Jamulus, since then the CI tests could at least feed a canned sequence of messages into the parsers...
Activity
atsampson commentedon Apr 1, 2021
The code in testbench.h is a simple template-based fuzzer for the Jamulus protocol - that is, it generates valid messages with completely random field values. It's intended for testing the protocol parsers on the client and server side. To enable it, you uncomment the
CTestBenchlike in main.cpp and recompile, then run Jamulus as usual - this'll cause it to run with an additonal thread that periodically generates a random message and feeds it into the parser.Because it's non-deterministic, and it needs to run for a very long time to have a chance of finding a problem, it's not really very useful as part of an automated test suite. It also needs updating for the current list of message types (e.g. it can't do
PROTMESSID_CLM_RED_SERVER_LISTyet).Back in mid-2020 I experimented with using it to generate seeds for a coverage-directed fuzzer, afl-fuzz, which is much more efficient (but still non-deterministic). That work is in this branch, which also needs updating. The change to add the -Z option might be worth including in mainline Jamulus, since then the CI tests could at least feed a canned sequence of messages into the parsers...
ann0see commentedon Apr 22, 2022
@atsampson thanks for the comment. Could you please quickly also comment on #2428 maybe we can somehow use it with JSON-RPC?
Nevertheless, closing as
Duplicate: #1413