Testing CloudTable dependent code using the Azure Storage Emulator and NUnit

TestingCloudTableWithAzureStorageEmulator.cs

public class TestingCloudTableWithAzureStorageEmulator
{
  private CloudTableClient developmentCloudTableClient;
  private CloudTable emulatedCloudTable;
  
  [OneTimeSetUp]
  public void OneTimeSetUp()
  {
      var azureStorageEmulatorProcess = Process.Start("C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\Storage Emulator\\AzureStorageEmulator.exe", "start");
      azureStorageEmulatorProcess?.WaitForExit(2000);
      this.developmentCloudTableClient = CloudStorageAccount.DevelopmentStorageAccount.CreateCloudTableClient();
  }

  [OneTimeTearDown]
  public void OneTimeTearDown()
  {
      var azureStorageEmulatorProcess = Process.Start("C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\Storage Emulator\\AzureStorageEmulator.exe", "stop");
      azureStorageEmulatorProcess?.WaitForExit(2000);
  }
  
  [SetUp]
  public void SetUp()
  {
      this.emulatedCloudTable = this.developmentCloudTableClient.GetTableReference("unittests");
      if (this.emulatedCloudTable.Exists())
      {
          this.emulatedCloudTable.Delete();
      }
      this.emulatedCloudTable.Create();
  }

  [TearDown]
  public void TearDown()
  {
      this.emulatedCloudTable.Delete();
  }
}

That's not a unit test - you're using a storage emulator which is external to your code and therefore you are crossing a boundary. It's an integration test. That's not to say this code is in any way invalid, but call a spade a spade and not a shovel.

You are obviously correct on crossing a boundary, it's right there in the code. What's your solution?

Depends on what you actually want to test, but the CloudTable class can mocked perfectly happily using NSubstitute, Moq and I would expect other libraries too if you want an actual unit test of some functionality that uses a CloudTable. Take away the dependency on the client. But as I said, it's perfectly valid integration test code. The trouble is that being called a unit test, this can be misleading for some people. One of our more junior devs was looking to create some unit tests, found this and thought it was just the job. He's not experienced enough yet to quite grasp that this is not a unit test, so his PR was rejected and he had to go back to the drawing board. Good learning for him, but if this was correctly labelled he might well have saved himself some time and effort, but also he'd have been able to create the integration tests required too at the same time. FWIW I think the client is amenable to mocking too if it's really wanted.

For what it's worth, when I wrote this 3 years ago I wasn't able to mock CloudTable with Moq, I don't remember exactly what the problems where but the best I can recall was that there was no interface and the methods were not marked virtual (I also have notes on checking out the tests in the WebJobs SDK, but I never looked into it). Of course adding another layer of abstraction could allow you to unit test too. That abstraction would still need tested though.

I sympathize with you on the classification of unit tests, especially as someone who prefers a mockist style which it sounds like you do too, but it's important to realize that there are other mindsets too. This reminds me of a good Fowler article Mocks Aren't Stubs that I often find myself referring to when explaining the different between mocks, stubs, and fakes and when each are appropriate. Both you and your junior developer should check it out.

It's also very important for you and your junior dev to understand that gists (and Stack Overflow posts) are not docs and they shouldn't be treated like them. This testing was fine for my project (a couple lines of code in an Azure Function that writes to a CloudTable) and no, I didn't call them unit tests. It's his professional responsibility to write the code that the project needs and to understand the code that he writing (or copying). Considering that wasn't done, I fail to see how any more than a minuscule amount of time and effort was wasted on his part or how this prevented him from writing integration tests, I think you are exaggerating and it's irresponsible on your part to shift that blame to me, especially considering the medium this information is provided on (a gist).

How about you (or your junior dev) create a gist showing mocking the CloudTable with Moq and link it here? That would be helpful to anyone that might stumble on this in the future.

It a fair point about the mockabilty at that time. I suppose we take for granted now that just about everything that interacts with Azure now has an interface or is marked virtual, and it's no surprise that things have moved on in the intervening years. And yes of course as you say your own abstraction on top is possible when not mockable, but I quite agree that this then increases your test overhead. Fortunately we seem to have to do this less and less as time goes on.

I'm well aware of Fowler's article on the subject of mocks and stubs, but I can't see that it's especially relevant to reference it here since the original gist doesn't include either. To my mind, the problem is that what we commonly refer to as mocking libraries actually produce the most basic of test doubles which is a dummy: Looks like the real thing but does nothing at all. You can then enrich that to produce a stub, a spy or a mock depending upon your needs and the way you like to unit test. Of course the exact definition of these things is generally the subject of hot debate (and lets not forget fakes either), but there's no point raking over that particular religious schism here. My favourite explanation is in an article written to epxand on Fowler's original. But because we call them mocking libraries the assumption is often that they produce a mock by default, which is not true unless you make it so. For the record, I am absolutely certain that it looks as though I am teaching grandma to suck eggs here, which is not my intention: I am sure you know all this already. But I think that in any discussion about tests and their classification, it bears repeating for those who follow on.

I use mocking libraries a lot, but I wouldn't describe myself as a mockist because of that: I generally use the output as stubs because I'm interested in the output of a unit given known inputs, not the journey to it, precisely because as Fowler states that ties your test to the implementation of the unit, thereby making the test brittle. That's not to say I don't use mocks or spies at all, because there are cases sometimes where I do care about verifying some aspect of the behaviour, but thats a much more infrequent use case for me. Mocking libraries (or maybe I should start referring to them as 'test double libraries' otherwise I'm just perpetuation confusion) simply mean I don't have to write my own stubs (very often - I'm sure you know as well as I do you can't avoid it all the time). All that said, I am aware that my own use of language defaults to use of the words "mock", "mocked" and "mocking" because we refer to mocking libraries, even though they would perhaps be better named "test double libraries" or something along those lines. If nothing else this is making me think more about my use of those words and I shall try not to default to them in future.

Now, on to the slightly more acerbic part of your response. I am well aware that gists and SO posts are not documentation and the dev in question has also learned that now (I hope - might take another go or two to really sink in). And I absolutely agree that random snippets should never be used verbatim without understanding what the code is really doing, again a lesson learned for the dev in question. This is one of the reasons why we use PRs after all. It would have been much better if we had had the time to pair with him to guide him in the first instance, and we often do, but sometimes you can't because there aren't enough hours in the day when there are people off sick or on annual leave, so you have to trust that the dev has learned enough to have a crack at their task solo. They don't always get it right, and that's fine. But I did not at any point say his time was wased: that was your word, not mine. In fact I see it as a useful learning experience for him and that sort of thing is never wasted time. I also didn't quantify any time spent - I said he might have saved some time - so therefore I don't believe I was exaggerating anything. However if you make a gist public then it is there for all to find, see and potentially criticise, so it's always worth ensuring that it is as well presented as possible, including in this instance the important distinction in what it actually is. I usually find that the community guidelines for SO are a decent yardstick, give or take a little depending upon the context. I see you have now renamed the gist which I appreciate as it's a great example of what it actually is.

The final point is about blame shifting. That was absolutely not my intent - my response was written at the end of a very long and busy day with a tired brain and perhaps my choice of wording was suspect, so I apologise unreservedly for any suggestion of that. What I was trying to do was point out that publishing a public gist is not unlike a blog post or an SO answer. There are lots of them out there that aren't accurate or well written, but yours is bar that (now amended) naming. Shouldn't we call out publicly available information that is wrong or misleading and try to have it corrected in some way, even if you do it clumsily, as it appears I have? I think so. The upshot of all this is that I believe we both approach things in a very similar way and I'm pleased that we can have a civil discussion over something as seemingly trivial as this. I apologise for the wall of words too, by the way!

For completeness, when I get a bit of time later I will indeed create a short gist showing the use of NSubstitute to create a stub for a CloudTable and link it here.

Edit: A simple, contrived example. Not had much time to do anything else. Like you we were using this in an Azure function but utilising the function binding which means we did not need to worry about the client, but I believe it too is marked virtual so it can be substituted.

https://gist.github.com/spettifer/51ba4f7a3ccd80069e3fb7a0502374b3

Thanks for creating and linking a gist! I'm sure someone may find it helpful in the future if they stumble upon this gist and conversation.

I linked to the Classical and Mockist Testing section of the Mocks Aren't Stubs article because it was the first easy to reference article that came to mind that described that there are different mindsets with unit testing. One mindset being to use real objects where ever possible. This mindset is different than how you and I both approach unit testing and while we see fallacies and faults in it, it isn't necessarily wrong. And no, I don't mean to imply that you find it wrong. I could see this conversation going very differently with someone who didn't really care about these nuances. I suppose they would argue on pedanticism or that an emulated db is nothing more than an elaborate test double.

To me, I consider gists a step up from personal notes and have always treated them as such. Below is roughly the order in which I would be concerned with how accurate something is given their various mediums with some links that I found with the most casual of searching (also I'm shocked to see my gist as the 2nd result on my google search, but I suppose these results have been tailored to me).
I have to say though that this interaction has made me reconsider how people may consume gists.

1. Source repositories (Azure/azure-storage-net#465)
2. Documentation and books
3. Blog posts (https://social.technet.microsoft.com/wiki/contents/articles/35320.faking-out-azure-unit-testing-with-microsoft-fakes.aspx)
4. Example repositories
5. SO answers (https://stackoverflow.com/questions/53510000/mocking-cloudstorageaccount-and-cloudtable-for-azure-table-storage)
6. Gists

Shouldn't we call out publicly available information that is wrong or misleading and try to have it corrected in some way, even if you do it clumsily, as it appears I have? I think so

I suppose I would try if it affected me as it has you. I don't think that I would have approached this instance the same way you did though. Rather, I would have found a solution that fit better and offered it up in the comments. My comment would probably be something along the lines of, "This seems to be more of an integration test rather than a unit test since it using a real CloudTable. I see that this was written quite a bit of time ago, but it looks like the CloudTable members are now marked virtual so you can mock it (Azure/azure-storage-net#465). I've done this successfully with Moq/NSubstitute, you can check it out here [link]"

To clarify, the exaggeration, wasted time, and blame shift comments were prompted by my interpretation of your comment "if this was correctly labelled he might well have saved himself some time and effort, but also he'd have been able to create the integration tests required too at the same time". Thanks for clarifying your intent, I appreciate it.

Certainly prompted a good discussion even if approached awkwardly (and you're right, it was by me). I know what you mean about Google - I asked an SO question recently and within 25 hours it was too five in Google for the search terms I was using but as you say, it does get tailored. As a result of all this I have begins to be more careful about how I refer to test objects which is a good thing. It's very easy to fall into bad habits and I certainly had by using terms that have been misappropriated and anything that makes you stop and re-evaluate how you are using language to communicate ideas and concepts is a good thing. Thanks for the discussion. Oh, and I will be a bit more careful about checking the age of stuff before opening my mouth in future 😁

Very interesting discussion and something I've been thinking about how best to test an Azure function with a table binding. I had thought about an approach similar to what's in this gist relying on Azure Storage Emulator but I felt it's not isolated and repeatable enough. Fortunately CloudTableClient exposes a DelegatingHandler property which you can assign a stub handler to providing the JSON response that would come back from the storage account.

Here is an example of how it works https://gist.github.com/vivianroberts/4bf161fdd6a9692d202888130e84df6b

To get the structure of the responses I use Azure Cli with the AZURE_STORAGE_CONNECTION_STRING environment variable set to "UseDevelopmentStorage=true" and then set up a proxy in Postman on 10002 to capture the requests. Next I switch off the proxy and start Azure Storage Emulator and fire the captured requests from Postman to get the response body.

Hopefully this approach provides a good balance between all the suggestions in this discussion 😊