At RUBRIC we are building what we call a “migration_toolkit” it consists of an upper level directory called migration_toolkit that contains python scripts that are reuseable on many different data migrations.
Below this directory we have folders named after specific data migrations, obviously this is where we put python scripts that are not reuseable on many different data migrations but are only used on one specific data migration.
We use test first programming when ever we write in python and every script has a test that relates to that script
for example the upper directory named migration_toolkit houses not only python scripts but the tests for each script which are run using the command py.test
Enter Problem: When you type py.test python runs all of the tests recursively in directories below the current working directory. In our case because there are tests in the directories for specific data migrations in lower directories python runs all of those tests every time. “So what” you might say. Well the problem is that inside the tests in the lower (specific) directories are sys.path.append commands that point to modules that the script is dependant on in order to run. As we discovered recently the path in the python test eg. sys.path.append(“../../module_name”) is relative to the directory that the py.test command is run from and not from where the test lives.
Enter Solution: If you have paths hard wired into your code you must run the python tests using py.test from the directory that it lives in and therefore the paths will be correct. In our case with the top level directory we are going to create a directory called tests in “migration_toolkit” that way when we run py.test it wont find the other tests because python does not look up for tests eg ../ it only goes through directories recursively in a downwards direction.