Generate these tests from the listing of directories. This allows us to effectively mark which tests are not working in certain conditions/platforms.