Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install FLiT's automated tests #285

Open
mikebentley15 opened this issue Jul 22, 2019 · 2 comments
Open

Install FLiT's automated tests #285

mikebentley15 opened this issue Jul 22, 2019 · 2 comments
Labels
make Involves touching GNU Makefiles python Involves touching python code tests Involves touching tests

Comments

@mikebentley15
Copy link
Collaborator

Feature Request

Describe the new feature:
We install pretty much everything, except we do not install the tests that test FLiT itself. It may be beneficial to install at least the relevant tests (e.g., do not install the test the checks that we can install FLiT).

Currently, the only automated tests installed are those inside of the python scripts (as doctests). One reason to install the tests is so that the doctests within the python scripts can reuse some of the testing infrastructure used by the FLiT automated tests.

Suggested change:

  • Install relevant tests into $PREFIX/share/flit/tests
  • Add a test_dir variable in flitconfig.py
  • Utilize the test_harness.py inside of doctests in the delivered python scripts

This could make for more rigorous testing by running the automated tests after installation to make sure the installation did not break any functionality.

Alternative approaches:
Just install the test_harness.py somewhere to be used by tests and add a variable in flitconfig.py to find it. We don't necessarily need the tests installed. I doubt anyone would even notice that they are even there nor know how to run them. But, if we install test_harness.py somewhere (e.g., $PREFIX/share/flit/test_harness/test_harness.py), then we can comfortably use them from within the doctests. I just worry about having broken doctests after installation.

@mikebentley15 mikebentley15 added make Involves touching GNU Makefiles python Involves touching python code tests Involves touching tests labels Jul 22, 2019
@mikebentley15
Copy link
Collaborator Author

Specifically, I want to simplify the doctests for flit_bisect.py function run_make().

@mikebentley15
Copy link
Collaborator Author

Also, simplify doctests for extract_make_var() and extract_make_vars() in flitutil.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
make Involves touching GNU Makefiles python Involves touching python code tests Involves touching tests
Projects
None yet
Development

No branches or pull requests

1 participant