Wednesday, November 8, 2017

Test Driven Disassembly

When I first started programming, the procedure was simple. You would write the program and then you would test the program. The testing was generally manual testing simply making sure that the program would do what you wanted. This is fine for when working on small or personal projects, but when projects get larger this is not a good way of doing things. Changing things could cause code to break but as it is not tested for it can go unnoticed for a long time and when discovered would require a lot of effort to find and fix.

The idea of automated testing helps solve this problem by making the testing process easy as one just needs to run the test after making changes to see if anything is broken. This does require that the tests exist which can be a problem as writing tests after the code has been completed makes writing the test a chore that can be skipped if one is behind schedule. It also has the problem of the test only testing what is already known to work.

Test driven development moves the testing to the top of the development loop. This has the advantage that the tests are written before the code so code always has test code. You then make sure that the tests fail and then write the code and get the code to pass the tests. You also have the advantage of thinking about how exactly you are going to test things and may uncover issues before you have even started writing the code. A comparison of the three methods is shown in the flowcharts below.

As with pretty much every approach to programming, dogmatism can take over and the advantages of test driven development can quickly be replaced by useless burdens. If you find yourself having to write tests for basic getters and setters then you have fallen into the dogmatism rabbit hole. I have been taking a middle ground with my work coming up with tests before writing code. As some things are simply too difficult to write automated tests for, especially non-deterministic programs such as with many games, manual testing is an option as long as you have a clear test plan. Automated testing is a preference as the tests are always ran so problems are detected earlier.

For my disassembler, the test is simply being able to disassemble known code into the proper instructions. My original plan for the assembly code was to write some test assembly language that would cover all the instructions with the various address modes for the instructions. The code wouldn’t have to perform anything useful, just cover the broad range of assembly instructions. This got me thinking about non-standard instructions.

There are future use operation codes (OP codes) that the 6502 has that when used will do things. As this functionality is unofficial, using such instructions is not wise since the presence is not guaranteed, but some programmers would use these instructions if it would save memory or cycles. As I do want my emulator to work with at least some real cartridges, which may use unofficial instructions, I need my disassembler to be able to detect these instructions and alert me to the use of the instructions so I can figure out what the instruction does and implement it in the future.

This means that all 256 possible OP codes need to be tested. As I want to be able to disassemble from any arbitrary point in memory, this simply meant that my test could be procedurally done. My test memory simply filled memory with the numbers from 0 to 255 so if I set the disassembly address in sequence, I would have all the instructions with the subsequent bytes being the address or value to be used. The fact that the instructions were of different lengths was not a big deal as we would be manually controlling the disassembly address. The list of instructions is something that I have so creating the test result list to compare to was very simple.

When running the test, if there is an error, it is still possible that my test list is incorrect, but this is obvious enough to determine. Once the disassembler is in a state that it can disassemble the complete list, it is probably working so as far as tests are concerned, this is a good way of testing. Once I wrote my disassembler, I did have to fix the list but also did find some issues so overall the test did work. Next week I will go into the disassembler which was surprisingly easy to write.

No comments:

Post a Comment