Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years’ experience in .NET and JavaScript. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Amazon and Aliyun) and right now, Shaun is being attracted by JavaScript (Angular.js and Node.js) and he likes it.

Shaun is working at Worktile Inc. as the chief architect for overall design and develop worktile, a web-based collaboration and task management tool, and lesschat, a real-time communication aggregation tool.

MVP

My Stats

  • Posts - 122
  • Comments - 546
  • Trackbacks - 0

Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories


.NET


 

Unit test is very important in modern software development, especially after adopted agile. Microsoft had added the unit test support since the Visual Studio 2008. Then with some enhancement and improvement unit test became easier in Visual Studio 2010. At the mean time, Microsoft provides its own test framework, a.k.a MsTest, which is one of the most popular frameworks. But it’s a little bit strange that Microsoft haven’t shipped any mock framework until now.

From Peter Provost’s post we found that there are two major reasons prevent Microsoft from providing the mock.

The concern is that I have often seen people fall into using behavior verification where none is needed. Overuse of mocks inhibits free refactoring of the code because, as Martin discusses in his essay, it leads to a very tight coupling between the tests and the implementation instead of just to the interface contract. I might blog more on this later, but let’s keep going.

The second reason for my reluctance was that there are quite a few good Mock/Stub implementations available in the community. Moq, Rhino, NMock and others have strong followings and good reputations and I wasn’t sure I wanted to get into the business of competing with them.

But in the latest Visual Studio 2012, we found that Microsoft provides a very powerful weapon. Well I have to say it’s NOT a pure mock framework, but it does helps us solve some real problem. It consisted from Stub and Shim.

 

The Business Logic Going to be Tested

Let’s cerate a simple class library project and add some very simple logic. Assuming this is what we are going to test in the future. We have a business logic class, which can update the information for an existing user through an data access interface. The structure is very straightforward.

image

User.cs contains the user entity class. IUserRepository.cs contains an interface takes the responsible for data access. MyBizLogic.cs is the main business logic that we will test later. The user entity class would be like this.

   1: public class User
   2: {
   3:     public Guid ID { get; set; }
   4:  
   5:     public string FirstName { get; set; }
   6:  
   7:     public string LastName { get; set; }
   8: }

In the IUserRepository.cs we will have four method, add a new user, update an existing user, delete a user and get the user entity by ID.

   1: public interface IUserRepository
   2: {
   3:     User GetByID(Guid id);
   4:  
   5:     void Add(User user);
   6:  
   7:     void Update(User user);
   8:  
   9:     void Delete(Guid id);
  10: }

In the business logic class we will inject the IUserRepository through its constructor. And there’s the first method which will update a user if it exists.

   1: public class MyBizLogic
   2: {
   3:     private IUserRepository _userRepository;
   4:  
   5:     public MyBizLogic(IUserRepository userRepository)
   6:     {
   7:         _userRepository = userRepository;
   8:     }
   9:  
  10:     public User UpdateExisingUser(Guid id, string firstName, string lastName)
  11:     {
  12:         // todo: update the user if it exists
  13:     }
  14: }

In this method we will firstly retrieve the user from the IUserRepository, then set its properties and update.

   1: public User UpdateExisingUser(Guid id, string firstName, string lastName)
   2: {
   3:     var user = _userRepository.GetByID(id);
   4:     if (user != null)
   5:     {
   6:         user.FirstName = firstName;
   7:         user.LastName = lastName;
   8:         _userRepository.Update(user);
   9:     }
  10:     return user;
  11: }

Now let’s create a unit test project in this solution and add reference to the project we had just created. Then let’s have a look on how to test this method.

image

 

Stub

The first test method is to make sure that if the user doesn’t exist, do not update. Since our business logic depends on the IUserRepostory interface, we need to mock it, so that it will have the behavior we want without touching the real database. In Visual Studio 2012, we can simply create the stub classes for a reference by right clicking one it in solution explorer, and selecting “Add Fakes Assembly”.

image

Visual Studio will add a new reference to our test project named “Vs2012StubSuhimDemo.Fakes” and a .fakes file under the new Fakes folder. This fake assembly contains the stub and shim classes we can use in our test code.

image

In fact in unit test glossology, Mock, Stub, Fake, Shim and Dummy has different meaning. But I don’t want to deep into them in this post. And my personally don’t think it’s a big problem to mix them. Basically, except Dummy, they all means to replace a dependent part by a temporary simple implementation. The goal of them are all making the unit test separated from any other 3rd party dependencies and making it run fast.

For more information about them, especially the Mock and Stub, please refer Martin Fowler’s article.

In the new assembly we can find the Stub class for our IUserRepository interface.

   1: // Summary:
   2: //     Stub type of Vs2012StubShimDemo.IUserRepository
   3: [DebuggerDisplay("Stub of IUserRepository")]
   4: [DebuggerNonUserCode]
   5: [StubClass(typeof(IUserRepository))]
   6: public class StubIUserRepository : StubBase<IUserRepository>, IUserRepository
   7: {
   8:     // Summary:
   9:     //     Sets the stub of IUserRepository.Add(User user)
  10:     public FakesDelegates.Action<User> AddUser;
  11:     //
  12:     // Summary:
  13:     //     Sets the stub of IUserRepository.Delete(Guid id)
  14:     public FakesDelegates.Action<System.Guid> DeleteGuid;
  15:     //
  16:     // Summary:
  17:     //     Sets the stub of IUserRepository.GetByID(Guid id)
  18:     public FakesDelegates.Func<System.Guid, User> GetByIDGuid;
  19:     //
  20:     // Summary:
  21:     //     Sets the stub of IUserRepository.Update(User user)
  22:     public FakesDelegates.Action<User> UpdateUser;
  23:  
  24:     // Summary:
  25:     //     Initializes a new instance of type StubIUserRepository
  26:     public StubIUserRepository();
  27: }

This class implemented the IUserRepository interface, which means we can use it to inject into our business logic class. And for each method it has a related member where we can specify the behavior. For example, the Add method now changed to AddUser which is a FakesDelegates.Action<User>. This means we can set how it should work when the Add method was invoked.

Back to our test method, since we wanted to check the situation that the user doesn’t exist. So that when our business logic invoked the IUserRepository.GetByID it should return null.

   1: [TestMethod]
   2: public void UpdateExisingUser_UserNotExisted_DoNotUpdate()
   3: {
   4:     // arrange
   5:     var userRepository = new Fakes.StubIUserRepository();
   6:     userRepository.GetByIDGuid = (id) => null;
   7:  
   8:     // act
   9:  
  10:     // assert
  11: }

Then we also need to make sure that the IUserRepository.Update should not be invoked. This can be done by using the closure feature of lambda. So if the IUserRepository.Update was invoked it will set a local variant to true.

   1: [TestMethod]
   2: public void UpdateExisingUser_UserNotExisted_DoNotUpdate()
   3: {
   4:     // arrange
   5:     var updateExecuted = false;
   6:     var userRepository = new Fakes.StubIUserRepository();
   7:     userRepository.GetByIDGuid = (id) => null;
   8:     userRepository.UpdateUser = (user) => updateExecuted = true;
   9:  
  10:     // act
  11:  
  12:     // assert
  13: }

The full test code would be like this.

   1: [TestMethod]
   2: public void UpdateExisingUser_UserNotExisted_DoNotUpdate()
   3: {
   4:     // arrange
   5:     var updateExecuted = false;
   6:     var userRepository = new Fakes.StubIUserRepository();
   7:     userRepository.GetByIDGuid = (id) => null;
   8:     userRepository.UpdateUser = (user) => updateExecuted = true;
   9:     var bizLogic = new MyBizLogic(userRepository);
  10:  
  11:     // act
  12:     bizLogic.UpdateExisingUser(Guid.NewGuid(), "newFirstName", "newLastName");
  13:  
  14:     // assert
  15:     Assert.IsFalse(updateExecuted);
  16: }

After built the solution we can find the test in the new Test Explorer in Visual Studio. Click Run All and we can see our test passed.

image

 

Stub – Asserts Inside the Custom Behavior

Let’s add another test method which make sure that when user exists it should be updated. Similarly, I used the Stub class for IUserRepository. But in this case when the GetByID invoked I will return a user entity. And then use a local variant to indicate whether the update had been called. Finally I asserted the updated user properties had been changed.

   1: [TestMethod]
   2: public void UpdateExisingUser_UserExists_Updated()
   3: {
   4:     // arrange
   5:     var updateExecuted = false;
   6:     var existingUser = new User()
   7:     {
   8:         ID = Guid.NewGuid(),
   9:         FirstName = "FirstName",
  10:         LastName = "LastName"
  11:     };
  12:     var userRepository = new Fakes.StubIUserRepository();
  13:     userRepository.GetByIDGuid = (id) => existingUser;
  14:     userRepository.UpdateUser = (user) => updateExecuted = true;
  15:     var bizLogic = new MyBizLogic(userRepository);
  16:  
  17:     // act
  18:     var updatedUser = bizLogic.UpdateExisingUser(existingUser.ID, "newFirstName", "newLastName");
  19:  
  20:     // assert
  21:     Assert.IsTrue(updateExecuted);
  22:     Assert.AreEqual("newFirstName", updatedUser.FirstName);
  23:     Assert.AreEqual("newLastName", updatedUser.LastName);
  24: }

If I ran the unit test it passed as I expected. But does this test good enough? Let me added some incorrect code in the business logic.

   1: public User UpdateExisingUser(Guid id, string firstName, string lastName)
   2: {
   3:     var user = _userRepository.GetByID(Guid.NewGuid()); // i passed the wrong id
   4:     if (user != null)
   5:     {
   6:         user.FirstName = firstName;
   7:         user.LastName = lastName;
   8:         _userRepository.Update(new User()); // i passed the wrong instance of user
   9:     }
  10:     return user;
  11: }

As you can see there are two bugs in my code. I passed the wrong ID when getting the user entity. And I passed a blank user entity when performing the update method. But if we ran our test code they all passed.

image

This is because we just specified the behavior in our stub class, but we didn’t check the parameter and validate it’s behavior. For example, we told the stub class that when the GetByID was invoke just return the user entity we want, but regardless the ID I was passed. Similarly, in Update method we were just care about whether it’s being called, but did not check if the user is the one we want to update. So when specifying the behavior of stub class, we also need some validation.

In the code below I had added the asserts inside the stub. In GetByID, we checked if the parameter equals the expected user’s ID. And in Update we checked the properties of the user which will be updated, ensured it must equal the value we expected.

   1: userRepository.GetByIDGuid = (id) =>
   2:     {
   3:         Assert.AreEqual(existingUser.ID, id);
   4:         return existingUser;
   5:     };
   6: userRepository.UpdateUser = (user) =>
   7:     {
   8:         Assert.AreEqual(existingUser.ID, user.ID);
   9:         Assert.AreEqual("newFirstName", user.FirstName);
  10:         Assert.AreEqual("newLastName", user.LastName);
  11:         updateExecuted = true;
  12:     };

This time, our test failed. But if we fixed the bugs in the business logic it will be passed.

image

If we use Mock, for example the Moq, this could be done easier and cleaner. Mock allows us specify the check policy of the parameter values. It can also allow us to verify whether a method had been invoked or not.

 

Shim

Let’s add a new method in business logic which create a new user and add into the user repository.

   1: public User CreateUser(string firstName, string lastName)
   2: {
   3:     var user = new User()
   4:     {
   5:         ID = Guid.NewGuid(),
   6:         FirstName = firstName,
   7:         LastName = lastName
   8:     };
   9:     _userRepository.Add(user);
  10:     return user;
  11: }

And in the related test code, we use the stub user repository and check the user properties when performing the Add method. I also checked the return user’s properties.

   1: [TestMethod]
   2: public void CreateUser_UserAdded()
   3: {
   4:     // arrange
   5:     var firstName = "FirstName";
   6:     var lastName = "lastName";
   7:     var userRepository = new Fakes.StubIUserRepository();
   8:     userRepository.AddUser = (user) =>
   9:         {
  10:             Assert.AreEqual(firstName, user.FirstName);
  11:             Assert.AreEqual(lastName, user.LastName);
  12:         };
  13:     var bizLogic = new MyBizLogic(userRepository);
  14:  
  15:     // act
  16:     var createdUser = bizLogic.CreateUser(firstName, lastName);
  17:  
  18:     // assert
  19:     Assert.AreEqual(firstName, createdUser.FirstName);
  20:     Assert.AreEqual(lastName, createdUser.LastName);
  21: }

But here comes another problem, how can we check the ID of the user created? For example if we forgot to specify the ID how can we detect in our unit test?

We utilized the Guid.NewGuid to retrieve a new Guid from system. But it’s inside our business logic method and the Guid.NewGuid is a static method. This means we cannot use stub or mock. So the Shim feature in Visual Studio 2012 helps us in this case.

Different from the mock and stub, shim leverage the code injection technology to change the code at the runtime. With shim we can customize the behavior against any cases, even though the business code depends on a sealed, internal or static member.

There are many voices that shim is evil. This is because shim can customize the behavior on any kinds of member. If using shim the unit test can be done even though the system we built was not decoupling, extensible and testable. And it violates the principle of test driven design and development. Similar voices had been called out to the TypeMock Isolator.

I don’t want to argue whether shim is good or not. But I have to said, sometimes we have to use an evil to fight against another evil. For example, if we are facing with some legacy code, or some code we cannot change, or it’s very complex to refactor, we must use shim.

The Guid was defined in System assembly (in fact it’s in mscorlib.) so we need to add the fake assembly against it. And then we can find the ShimGuid class in the fake assembly.

image

Then we can specify our behavior through its NewGuid property. But be noticed, the shim code and the codes that utilizes shimed member must be in a ShimsContext.

   1: [TestMethod]
   2: public void CreateUser_UserAdded()
   3: {
   4:     // arrange
   5:     var id = Guid.NewGuid();
   6:     var firstName = "FirstName";
   7:     var lastName = "lastName";
   8:     var userRepository = new Fakes.StubIUserRepository();
   9:     userRepository.AddUser = (user) =>
  10:         {
  11:             Assert.AreEqual(id, user.ID);
  12:             Assert.AreEqual(firstName, user.FirstName);
  13:             Assert.AreEqual(lastName, user.LastName);
  14:         };
  15:     var bizLogic = new MyBizLogic(userRepository);
  16:  
  17:     var createdUser = default(User);
  18:     using (ShimsContext.Create())
  19:     {
  20:         System.Fakes.ShimGuid.NewGuid = () => id;
  21:         // act
  22:         createdUser = bizLogic.CreateUser(firstName, lastName);
  23:     }
  24:  
  25:     // assert
  26:     Assert.AreEqual(id, createdUser.ID);
  27:     Assert.AreEqual(firstName, createdUser.FirstName);
  28:     Assert.AreEqual(lastName, createdUser.LastName);
  29: }

From the updated unit test code we firstly retrieved a Guid. And then within the ShimsContext we set the Guid.NewGuid method so that it will return the Guid we had just created. And then we can assert the user’s ID and make sure it’s the one comes from Guid.NewGuid.

 

Summary

In this post I introduced the new stub and shim feature in Visual Studio 2012 unit test. With stub and shim it would be more easier to create the unit test code especially against the legacy code.

But personally, the stub and shim are not that fantastic. This is more because of the define of stub and shim. Stub focuses on result-based validation, shim let us write unit test code even though the system architecture is not good. So I personally suggest using some mock framework, and do not use shim as possible as we could.

 

You can download the sample code here.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.