Option to show log output when running individual tests #1093

Activity

1. jku commented on Aug 3, 2020

   

   jku

   on Aug 3, 2020

   Member

   More actions

   I'd prefer that the test log level was easily modifiable (like V=DEBUG python3 test_updater.py) so the default could be practically silent but tricky issues were easy to debug.

   👍React with 👍1Reacted by Joshua Lock
2. joshuagl commented on Aug 4, 2020

   

   joshuagl

   on Aug 4, 2020

   MemberAuthor

   More actions

   That would also be fine, so long as we can toggle easily.
3. jku commented on Aug 6, 2020

   

   jku

   on Aug 6, 2020

   Last edited by jku

   Member

   More actions

   2. Executing a test "by hand", the logging output remains hidden until an error occurs.

   This is untrue AFAICT: the output is same on error or no error (in an error case there might be more high level output of course, but same logger levels get printed). We do not initialize logging at all when tests are run individually: It think the default is to only print WARNING and above.

   We could initialize the logging based on a env variable (or maybe an argv option) in each if __name__ == '__main__': ... but I would expect this was an issue where python folks would have a Best Practice?

   👍React with 👍1Reacted by Teodora Sechkova
4. joshuagl

   changed the title [-]Show log output when running individual tests[/-] [+]Option to show log output when running individual tests[/+] on Aug 10, 2020
5. jku commented on Aug 13, 2020

   

   jku

   on Aug 13, 2020

   Last edited by jku

   Member

   More actions

   I'll open another issue on logging in general but leaving an example that should work right now here:

   if __name__ == '__main__':
     verbose = '-v' in sys.argv or '--verbose' in sys.argv
     loglevel = logging.DEBUG if verbose else logging.CRITICAL
     logging.basicConfig()
     tuf.log.set_log_level(loglevel)
   
     unittest.main()
   

   Notes:

   • this is only for running test files manually: it does not and should not affect test runners. They set their own logging policies
   • this slightly abuses unittest CLI: '--verbose' is actually unittest argument which we're now using as well
   • had to set default level to CRITICAL since we currently log so much on ERROR (issue logging level consistency #1108 )
   • tuf.log.set_log_level() seems not that useful: I'd just want to set root logger level and expect all components to follow that (and could do logging.getLogger('tuf').setLevel() if I wanted more control) but tuf forces me to call tuf.log.set_log_level() because there is a default that is not logging.NOTSET. Or is this how libraries are supposed to do logging in python?
6. joshuagl

   added

   testing

   on Sep 10, 2020
7. jku

   added 2 commits that reference this issue on Sep 15, 2020

   tests: Configure logging for all test files

   ...

   This issue will close once commit 647e3e0 is merged into the 'develop' branch.

   647e3e0

   tests: Configure logging for all test files

   ...

   This issue will close once commit 03b15fb is merged into the 'develop' branch.

   03b15fb
8. joshuagl

   closed this as completedin #1145on Sep 22, 2020
9. MVrachev

   added a commit that references this issue on Sep 28, 2020

   tests: Configure logging for all test files

   ...

   This issue will close once commit bd53289 is merged into the 'develop' branch.

   bd53289