Why Steak (over Cucumber)

why_steak.md

Why Steak (over Cucumber)

From here

• Plain text have no value if people writing and reading the specs are developers. Code is natural language for developers (specially if that code is Ruby). Having a client who reads the specs is a possible situation and Cucumber is cool there, but in my experience that situation is not the most common one. If in your case there's no value in having your specs in English rather than in Ruby, Cucumber is an additional layer of abstraction in exchange of nothing.

• There's a myth about clients writing the specs. It sounds like a unicorn to me, but let's take it. Still, I wouldn't want it. Clients are experts in problems, I am expert in solutions. I want the client to describe the problem I have to solve, not to describe the solution.

• Some people say Cucumber is cool because you write the stories, the client validates them, and then you can start developing. That is not Agile. The only product from software development that can be validated (if at all) is the software itself. I'm not making that, it's on the second line of the Agile manifesto. In order to get feedback from the client, a quick subfunctional but working prototype is muuuuch better than any plain text story, and thanks to tools like ours (Rails, jQuery, and why not the lighter approach of Steak) we can get it in hours.

• In Cucumber, features and steps maintenance gets harder over time because of that extra layer. That's not bad if it's worth it, but problem is that the main benefit of the stories being in plain text happens in the development phase, not in the manteinance one. You get the benefit at the beginning of the project, and you pay for it for ever (without getting any further benefit).

Thanks for the writeup! I mostly agree with you, but here are some counter arguments for the first two points:

• When not having a client, doing an open source project or even having a client that can read Ruby, writing tests in English might seem a bit pointless. After all, you don't need a business readable DSL in this case since everyone involved is able to read Ruby. What Cucumber does add in this case is executable documentation, which will never go stale. RSpec's command line tests are a great example: https://github.com/rspec/rspec-core/blob/master/features/command_line/line_number_option.feature
• http://jeffkreeftmeijer.com/2010/clients-writing-cucumber-stories/

</devil's advocate>

"Working Agile includes user stories. The client MUST validate them so we can prioritize them. It IS Agile." - https://twitter.com/seivanheidari/status/23762306286493696

Fair enough. However, in this case (if the client does not write them) I personally would choose for less layers of abstraction, purely for the sake of less things that could possibly break. (Not saying that Cucumber might not be stable, just theoretically.)

Also, executable specifications are ‘examples’, that still means you need proper documentation. Now if you have these ‘examples’ in Ruby instead of English, at least you have actual API usage examples.

In my experience Cucumber's limitations to phrasing a story and the extra layer of step definitions also tend to enforce a certain style of phrasing and implementation across the team though ... which is the sole reason for us to choose Cucumber over Steak. It's all about tradeoffs I guess.

I think RSpec is awesome at what it does (after all, most of the time I'm still using RSpec's expectations inside Cucumber "Then" steps), but I stick to Cucumber for acceptance testing.
I agree with @jeffkreeftmeijer that Cucumber is most useful when having a client (although in most CLI tools, which live in a wider environment outside the code, can greatly benefit from this kind of executable docs), and I think it is in a sense that software written for clients is meant to model their specific domain. It matters little whether they write user stories or not. The important thing is that both the developer and the client can talk to each other using the same terms. The developer still "reads minds" from their clients and writes down whatever he understands to be the point of what they said, just as in any translation process, but he can always get feedback on that (since the client can actually validate what the developer has understood). That also enables accountability.

For sure it's an additional layer (maybe we should think of it as a safety net), but I think it's a useful one, since it "caches" a particular understanding of the domain problem and enables other developers to work with it at the same level of abstraction.

I agree with much of what you say. See my article on Ruby Acceptance Testing.

I haven't tried Steak yet, but for me the benefit of Cucumber isn't so much to have an English-like spec but to encourage describing the software's behavior at a higher level than an implementation (which also decouples the spec from the implementation, with step definitions as the adapting glue). This lets you change how things are done without changing your description of what is done.

I also think Cucumber's scenario outlines (similar to FIT tables) are a very handy syntax for describing different permutations of inputs and outputs. I don't think Steak offers an equivalent, other than iterating over an array of arrays.

An argument in favor of Cucumber I've never seen mentioned: Apparently clients and managers love it

Cucumber provides non-technical people with a sense of confidence and control no other tool can provide. Combine Cucumber with Selenium and you'll get a standing ovation in every demo. It's quite arguable whether this sense of control is real or just an illusion, but the point is that, in some contexts, your stakeholders will be happy and you'll look very professional.

@jeffkreeftmeijer I personally find this executable documentation as good as the one you linked: https://github.com/cavalle/steak/blob/master/spec/acceptance/rspec-2/basic_spec.rb

Regarding @Seivanheidari tweet, let's not get into the fallacy of discussing what it is agile and what's not. If writing user stories and using it to prioritize works for you, just do it. But, please, explain what's the point of turning those useful useful stories into executable tests with Cucumber.

@alloy Agreed. I've never seen Cucumber features substituting real, high-quality documentation.

@svenfuchs Sounds like a quite subtle benefit for the extra burden involved. But, of course, it's definitely all about trade-offs

@txus @jeremyw I agree that in software design it is necessary to work at different abstraction levels, and user stories are one of the tools that can be useful in that regard. But, I ask again, what's the point of taking those useful user stories and turning them into executable test? why not just writing separated tests (in ruby) and throw the stories away when they are no longer useful?

@cavalle You can also get clients' and bosses' confidence and sense of control based on other things like honest communication, transparency and continuous delivery.

RE @cavalle “An argument in favor of Cucumber I've never seen mentioned: Apparently clients and managers love it”

At least I did not mention it, because I assume that regardless of whether or not you use Cucumber, or any other comparable tool, you still have user-stories to make sure the communication between you and the client is clear.

To summarize, we do write user-stories, we do have a tool that makes sure they're in a specific format because it parses it and creates HTML for the client [1]. However, we do not make them executable, but check them manually before release, something that we’d have to do anyways.

Regarding the latter point, out of curiosity, do the people that have executable user-stories still test them manually before releasing?

[1] https://github.com/Fingertips/saga

We found the same issues with the abstraction. I blogged about why we agreed to use RSpec only Automated Acceptance Testing with RSpec or Cucumber