Blue Rose Testing Framework

March 15, 2022

Blue Rose Testing Framework

Blue Rose

Last XMAS, I did a tiny project called Blue Rose which is like a Bin2C with more options. This application converts any files into C++ code file or generates hashes (e.g. SHA-512) also in C++.

Would You Like To Know More? Click Here!

It's not my first data converter. My first one was in Turbo Pascal a long time ago (199X). Even at Ubisoft at the beginning (2005), I created a tiny tool to convert RGB 24 bits images into 15 bits (R5/G5/B5) for the DS console. At that time, we were waiting for the game engine source code to arrive so we spent some time getting familiar with the console by creating some demos. It was pretty nice!

I like to add any kind of tests, validations or protections for all software that I work on. It's a good way to make sure the code works correctly and to avoid any regression when we modify the code. Same thing for bugs. We can fix it by writing a new test that repro the bug, applying the fix and using the test to verify that it will not happen in the future.

For Blue Rose, I also added some tests but I wanted to try something new or different...

Testing Framework

There are a lot of different test types but the three most popular are Unit Testing, Integration Testing and End-to-End Testing (E2E).

Unit testing: Atomic level (one function or one module)
Integration Testing: Composite level (multiple components, functions or modules together)
End-to-End Testing (E2E): User scenarios level / Appplication's workflow from beginning to end

Below is an unit test example with the GoogleTest framework with the function Factorial:


// Tests factorial of 0.

TEST(FactorialTest, HandlesZeroInput) {

EXPECT_EQ(Factorial(0), 1);

}



// Tests factorial of positive numbers.

TEST(FactorialTest, HandlesPositiveInput) {

EXPECT_EQ(Factorial(1), 1);

EXPECT_EQ(Factorial(2), 2);

EXPECT_EQ(Factorial(3), 6);

EXPECT_EQ(Factorial(8), 40320);

}

[Source]

The code above will validate that everytime the test framework calls the Factorial function with some values, it will compare with the expected values.

There is also 2 ways for when to write tests:
- Write product/feature first and tests later
- Write tests first followed by product/feature afterward

Should we write the tests first before creating any feature at the beginning or during the project? It's a personal choice. The thing about writing tests before the features is aligned with Test-driven development (TDD) methodology which I'm not a big fan of. I share the same opinion as Jonathan Blow for this case (video above) where writing the tests before producing anything is counter-productive.

The Challenge

Above we saw the basic elements of testing framework by writing code and tests to improve the product quality. I used a lot of GoogleTest and I'm still using it but like I said above, I wanted to try another way for my app Blue Rose.

And here was the challenge: Is there a way to test my app without writing a single line of code?

The answer is YES !!!

After thinking, I came with a solution where I used Git and a batch file called Tests.bat to test my app Blue Rose.

If you don't know Git: "Git is software for tracking changes in any set of files, usually used for coordinating work among programmers collaboratively developing source code during software development. Its goals include speed, data integrity, and support for distributed, non-linear workflows."
[Wikipedia]

First, I created a folder named "Tests" and I copied all files that I wanted to use for my tests.

After, I added 50 tests inside Tests.bat script file with various parameters to be used with the app like that:


"-i HexTable.txt"

"-sha512 -i ok.txt"

"-p Parse .jpg"

"-md5 -i HexTable.txt -sha1 -last -sha256 -last -sha512 -last"

"-md5 -i zero.txt -sha1 -last -sha256 -last -sha512 -last"

I validated the new files generated from the script and I commited/pushed the new content with Git.

Afterward I added two new thing inside the script for each tests:
- Delete the "result" file(s) before calling Blue Rose with specific test parameters.
- Call git after each test to detect any new or updated content.

Git will detect any difference from the Tests folder (e.g. new file, update file, missing file) and I used this feature to validate if the tests passed or failed.

Result

Above is the video of the Tests.bat script running.

The first call will be green and the second one will be red since I added a dummy file (failfile.txt) and Git detects that as a new content. Git is usually used for development as a source control but I like the idea to be used for other purposes.

Some readers could be confused since I didn't go deeper on some explanation about tests, Git, batch/script file or other stuff. I expect that the reader has some notion on these areas. Otherwise Google Search or Stack overflow could help. You can also email me if you want or need more info!

Thanks for reading,

JS.