cli.md

Get rid of return_code checks for CLI

Was experimenting with CLI recently and came to one idea. The main thing that annoys in CLI tests compared with API, is amount of redundant code we have, like this:

# API
content_view = entities.ContentView(id=123).read()

# CLI
result = ContentView.info({'id': 123})
self.assertEqual(result.return_code, 0, "Failed to fetch object")
self.assertEqual(len(result.stderr), 0,
    "There should not be an exception here")
content_view = result.stdout

I'd say we have 30% of CLI tests code just checking if command was executed successfully and didn't return any errors. So i thought what if we move SSHCommandResult checking from the tests to Base class?

The idea is to add one new private function, let it be some _handle_response(), which will capture SSHCommandResult, check return_code, check stderr and return stdout to test. In case something went wrong it will raise an exception. This will give us pretty much API-like style in CLI tests:

# CLI
content_view = ContentView.info({'id': 123})

So for positive tests it decreases amount of code a lot. But what if we expect return_code different from 0? Then instead of

result = ContentView.info({'id': 123})
self.assertNotEqual(result.return_code, 0, "Object wasn't deleted")

we'll have something like

with self.raises(CLIError):
    result = ContentView.info({'id': 123})

so negative tests will work as well.

I've already made some draft version of this change and tried to update one random module. You can see example here: https://github.com/SatelliteQE/robottelo/compare/master...abalakh:cli_remove_return_code#diff-90b76b944e6a4809d4a94fd90f363fbaR28

Forgot to mention, _handle_response() is already there, too:
SatelliteQE/robottelo@master...abalakh:cli_remove_return_code#diff-594daccac11bfc2c58ad084caec25848R98

As @elyezer suggested, it would be nice to have access to return_code, if it's not equal to 0. Cool place to put it is inside the exception, as exception will always be risen when return_code != 0.
That's how it looks like for now:
SatelliteQE/robottelo@master...abalakh:cli_remove_return_code#diff-594daccac11bfc2c58ad084caec25848R24
Example of usage:

>>> try:
...     result = PartitionTable.info({'id': 123123})
... except CLICommandExecutionError as e:
...     print e.return_code
... 
128

Also it can be used like this inside the tests:

with self.assertRaises(CLICommandExecutionError) as e:
    PartitionTable.info({'id': 123})
self.assertEqual(e.exception.error_code, 1)