Unit Test Framework

[enovak]

I've been researching implementing some automated unit test in my project. I know that there are a number of different frameworks which are available to use. I was curious if anyone has implemented anything using the CCS compiler. Any input on the best framework to use? Are you executing your tests on target or simulating them? Thanks for the input.

[zaz]

I am also looking for unit test framework for CCS project.

[temtronic]

I can't see any UTF or another 'simulator' actually being 100% accurate with using a simple PIC like the 16C84, let alone a modern version, say a 46K22.
The problem I see is that the UTF programmer has no hands on, real world experience with a PIC ? Just HOW does he emulate random cross talk that feeds into the 2nd UART and corrupts the data stream ? Or the effects of a nearby lightning bolt ?

[Ttelmah]

Dead right.
I have used Criterion to make test values to prove functionality of some
code in the past, but the problem is that instrumentation code is far more
likely to have problems from things like EMP, than from general 'code errors'.
I have yet to see any simulator that I would rate as being more than 99%
accurate, and all will miss 'oddities' with particular chips, that the real
instrument has to handle.
So use a tool like Criterion to generate your test sequence values, but do
test these in the real instrument, and these also need to be tested with the
same hardware electrical signals actually present.

[jeremiah]

We tend to distinguish between UNIT test and DVT (Design Verification Test). UNIT test is purely for testing code "paths", so you setup multiple sets of inputs in an attempt to verify every path in the code. For DVT, we hook up to actual signals and such to verify that the code design works as intended, treating the system as a black box and stressing all the inputs physically.

In our world UNIT test doesn't require an emulator. You generally create functions that call other functions/blocks of code with various hard coded input sets (some are generated by tools if we need random input sets) and each output is compared to an expected output of the function / block of code.

[Ttelmah]

Absolutely.
Test things like how the maths behaves with an overflow, and that the
case statements do go where they are meant to go.

[zaz]

What about Unity framework (for embedded software) for testing FW code.

[Ttelmah]

Unity is designed very much for .net languages. I don't know how easy it is
to adapt for languages.

[enovak]

@zaz I did end up using Unity and getting it to work with the Microchip simulator. I did have to mock out some of the HW interactions. Using mocks is going to be key to helping to ensure that all of the code paths are tested and confirmed to be operating correctly.

Here are a couple of links that I referenced in the work that I was doing:
* http://www.throwtheswitch.org/unity
* https://spin.atomicobject.com/2012/10/22/getting-started-with-tdd-for-microchips-pics/

[Ttelmah]

Well done.
As Jeremiah says, you then have to test the actual hardware operation.

[bryant@balancebitsconsult]

Here is an excellent list of unit test frameworks, for the purposes of testing that the code is accurate and returns the proper results.

https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

However, for hardware testing nothing beats running the test on true hardware platforms, and this is specifically distinguished from unit tests.

[Ttelmah]

and for software testing as well...

Unit testing, is great for systems and code that is completely deterministic,
but will not catch faults that are not in the rules used. Unfortunately many
of the faults met inside microcontrollers fall into this category.
They are brilliant for proving that you are not doing something silly in
the handling of maths ranges, but will probably catch less than a quarter
of the real faults that can happen....

[bryant@balancebitsconsult]

[temtronic]

hmm.. 'unit test frameworks' sounds about as reliable and trustworthy as every 'simulation' program I've ever seen.

[bryant@balancebitsconsult]

unit tests are at the crux of advanced software development for large systems.

In particular, test driven development (TDD) is something you may find useful to develop systems faster:
https://en.wikipedia.org/wiki/Test-driven_development

^ details of the benefits and outcomes you can expect when TDD is done properly. Of course, there are lots of ways to do it wrong and not see those benefits.