In my project there is a platform component which takes the responsible for controlling and monitoring all other components and it need to lead all other components' metadata when starting. We defined metadata in each component assembly with attributes and will be retrieved through .NET reflection. At the beginning of this project ,about 3 years ago, there are only 5 - 6 components and it takes about 10 - 20 seconds to reflect them. But with more and more components were introduced currently there will be more than 30 components installed in our system. Then we met a performance problem when system start up, it takes 60 - 80 seconds to scan all those assemblies to retrieve the metadata in my development workstation. And it takes several minutes in some of customers and testers machine since less hardware capability (slower CPU and less memory).
After reviewed the code I found there is no problem in our implementation. The main problem is we utilizes System.Reflection to retrieve metadata, the performance of System.Reflection is not that satisficed. After some investigation and research I found an alternative solution which is using Mono.Cecil to reflect .NET assembly.
We thought the best solution is to move all metadata information from assembly attributes to database or configuration file so that we can load them without using reflection technology. But since we also need to maintain backward capability, we must allow all existing components works for our new platform component. So we have to continue using attributes to store metadata.
We can find Mono.Cecil here. We also can add Mono.Cecil into our .NET project reference from NuGet package management in Visual Studio.
Below are the operations we are using defined in Mono.Cecil to replace System.Reflection.
| System.Reflection | Mono.Cecil |
| Assembly.LoadFrom() | AssemblyDefinition.ReadAssembly() |
| CustomAttributeData.GetCustomAttributes(Assembly) | AssemblyDefinition.CustomAttributes |
| Aassmebly.GetTypes() | AssemblyDefinition.Modules.SelectMany(m => m.GetTypes()) |
| Type.GetInterfaces() | TypeDefinition.Interfaces |
| CustomAttributeData.GetCustomAttributes(Type) | TypeDefinition.CustomAttributes |
| CustomAttributeData.Constructor.GetParameters() | CustomAttribute.Constructor.Resolve().Parameters |
Most of them are easy to learn and use. We load an assembly from the static class `AssemblyDefinition` and the static method `ReadAssembly` by passing the file path. Different from System.Reflection, it will NOT lock the assembly file. This means by using Mono.Cecil we don't need to create a new AppDomain to load the assembly and unload it at the end.
With the `AssemblyDefinition` we can retrieve attributes and types. Then we can use the `TypeDefinition` and `CustomAttribute` instances to retrieve the type name, interfaces it implements and the value defined in the attribute.
Just one thing need to be highlighted here, by default Mono.Cecil will reflect limited information from the targeting assembly. For example, let's say I have an assembly `MyClass.dll` and there's a class with a custom attribute attached, as well as implemented an interface those are defined in separated assemblies.
When we invoked `AssemblyDefinition.ReadAssembly` to `MyClass.dll`, Mono will only load this assembly but will NOT lock it.
Now we can retrieve the types defined in this assembly and the attributes, interfaces it associates. But since the attribute and interface were defined in other assemblies, when we invoke `CustomAttributes` and `Interfaces` they will return metadata that can be found in this assembly. For example we can retrieve the constructor of `MyAttrib` in type of `MethodReference`, which contains the parameter values and the arguments. But we cannot retrieve the parameter names since the definition of this attribute was in another assembly. To retrieve more information we need to invoke `Resolve` method to let Mono find the definition assembly of this type and load more information. We can also define the behavior how Mono find the relevant assemblies, which path Mono need to lookup. I will show the code later.
Similarly if we want to retrieve more information of the interface our class was implemented, we need to invoke `TypeReference.Resolve()`.
Below is the code I implemented in my project that using Mono.Cecil to retrieve metadata information from assemblies. In order to decouple from my business logic I created a set of interfaces represents all operations I need for reflection as below.
The `IReflector` interface is the main entry to let user load assembly.
1: public interface IReflector
2: {
3: IAssemblyReflector LoadAssembly(string path);
4: }
`IAssemblyReflector` interface isolates operations that interact with assembly, such as retrieving attributes, types, name, file path, etc..
1: public interface IAssemblyReflector
2: {
3: IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute;
4:
5: IEnumerable<ITypeReflector> GetTypes();
6:
7: string Location { get; }
8:
9: string FileName { get; }
10:
11: string FullName { get; }
12: }
With `ITypeReflector` interface we can retrieve attributes attached as well as interfaces it implements.
1: public interface ITypeReflector
2: {
3: IEnumerable<ITypeReflector> GetInterfaces();
4:
5: IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute;
6:
7: string FullName { get; }
8:
9: string Name { get; }
10: }
With `IAttributeReflector` we can get values of the arguments and named property values from its constructor.
1: public interface IAttributeReflector
2: {
3: IDictionary<string, string> Values { get; }
4: }
Below is the Mono.Cecil implementation of these interfaces. Notice that I invoked `Resolve` to load the definition assembly when retrieving interfaces and attribute values. I also tell Mono.Cecil to lookup relevant assemblies in the same folder of the assembly I'm loading.
1: public class MonoReflector : IReflector
2: {
3: public IAssemblyReflector LoadAssembly(string path)
4: {
5: var resolver = new DefaultAssemblyResolver();
6: resolver.AddSearchDirectory(Path.GetDirectoryName(path));
7: var reader = new ReaderParameters()
8: {
9: AssemblyResolver = resolver
10: };
11:
12: var assembly = AssemblyDefinition.ReadAssembly(path, reader);
13: return new MonoAssemblyReflector(assembly);
14: }
15: }
16:
17: public class MonoAssemblyReflector : IAssemblyReflector
18: {
19: private AssemblyDefinition _assembly;
20:
21: public MonoAssemblyReflector(AssemblyDefinition assembly)
22: {
23: _assembly = assembly;
24: }
25:
26: public IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute
27: {
28: if (_assembly.HasCustomAttributes)
29: {
30: var expectedTypeName = typeof(T).Name;
31: return _assembly.CustomAttributes
32: .Where(a => a.AttributeType.Name == expectedTypeName)
33: .Select(a => new MonoAttributeReflector(a))
34: .ToList();
35: }
36: else
37: {
38: return new IAttributeReflector[] { };
39: }
40: }
41:
42: public IEnumerable<ITypeReflector> GetTypes()
43: {
44: var result = new List<ITypeReflector>();
45: var modules = _assembly.Modules;
46: foreach (var module in modules)
47: {
48: var types = module.GetTypes();
49: foreach (var type in types)
50: {
51: result.Add(new MonoTypeReflector(type));
52: }
53: }
54: return result;
55: }
56:
57: public string Location
58: {
59: get
60: {
61: return _assembly.MainModule.FullyQualifiedName;
62: }
63: }
64:
65: public string FileName
66: {
67: get
68: {
69: return _assembly.MainModule.Name;
70: }
71: }
72:
73: public string FullName
74: {
75: get
76: {
77: return _assembly.FullName;
78: }
79: }
80: }
81:
82: public class MonoTypeReflector : ITypeReflector
83: {
84: private TypeDefinition _type;
85:
86: public MonoTypeReflector(TypeDefinition type)
87: {
88: _type = type;
89: }
90:
91: public IEnumerable<ITypeReflector> GetInterfaces()
92: {
93: return _type.Interfaces.Select(i => new MonoTypeReflector(i.Resolve()));
94: }
95:
96: public IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute
97: {
98: if (_type.HasCustomAttributes)
99: {
100: var expectedTypeName = typeof(T).Name;
101: return _type.CustomAttributes
102: .Where(a => a.AttributeType.Name == expectedTypeName)
103: .Select(a => new MonoAttributeReflector(a))
104: .ToList();
105: }
106: else
107: {
108: return new IAttributeReflector[] { };
109: }
110: }
111:
112: public string FullName
113: {
114: get
115: {
116: return _type.FullName;
117: }
118: }
119:
120: public string Name
121: {
122: get
123: {
124: return _type.Name;
125: }
126: }
127: }
128:
129: public class MonoAttributeReflector : IAttributeReflector
130: {
131: private CustomAttribute _attribute;
132: private IDictionary<string, string> _values;
133:
134: public MonoAttributeReflector(CustomAttribute attribute)
135: {
136: _attribute = attribute;
137: }
138:
139: public IDictionary<string, string> Values
140: {
141: get
142: {
143: if (_values == null)
144: {
145: _values = new Dictionary<string, string>();
146: var constructorArguments = _attribute.Constructor.Resolve().Parameters.Select(p => p.Name).ToList();
147: var constructorParameters = _attribute.ConstructorArguments.Select(a => a.Value.ToString()).ToList();
148: for (var i = 0; i < constructorArguments.Count; i++)
149: {
150: _values.Add(constructorArguments[i], constructorParameters[i]);
151: }
152: foreach (var prop in _attribute.Properties)
153: {
154: _values.Add(prop.Name, prop.Argument.Value.ToString());
155: }
156: }
157: return _values;
158: }
159: }
160: }
Below is the implementation by System.Reflection.
1: public class DotNetAssemblyReflector : IAssemblyReflector
2: {
3: private Assembly _assmebly;
4:
5: public DotNetAssemblyReflector(Assembly assmebly)
6: {
7: _assmebly = assmebly;
8: }
9:
10: public virtual IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute
11: {
12: List<CustomAttributeData> returnValue = new List<CustomAttributeData>();
13: var pCustomAttributeType = typeof(T);
14:
15: foreach (CustomAttributeData customAttributeData in CustomAttributeData.GetCustomAttributes(_assmebly))
16: {
17: if (customAttributeData.Constructor.DeclaringType.Name == pCustomAttributeType.Name)
18: {
19: returnValue.Add(customAttributeData);
20: }
21: }
22:
23: return returnValue.Select(x => new DotNetAttributeReflector(x)).ToList();
24: }
25:
26: public string GetVersion()
27: {
28: string version = string.Empty;
29: var assemblyFileVersionCustomAttributeData = GetAttributes<AssemblyFileVersionAttribute>();
30: if (assemblyFileVersionCustomAttributeData.Count() == 1)
31: {
32: try
33: {
34: var assemblyFileVersion = assemblyFileVersionCustomAttributeData.First().Values;
35: version = assemblyFileVersion["version"];
36: }
37: catch (FormatException ex)
38: {
39: // // Console.WriteLine(String.Format("Problem getting the assembly version: {0}", assembly.FullName));
40: // // Console.WriteLine(ex);
41: }
42: }
43: return version;
44: }
45:
46: public IEnumerable<ITypeReflector> GetTypes()
47: {
48: return _assmebly.GetTypes().Select(t => new DotNetTypeReflector(t)).ToList();
49: }
50:
51: public string Location
52: {
53: get
54: {
55: return _assmebly.Location;
56: }
57: }
58:
59: public string FileName
60: {
61: get
62: {
63: return _assmebly.ManifestModule.Name;
64: }
65: }
66:
67: public string FullName
68: {
69: get
70: {
71: return _assmebly.FullName;
72: }
73: }
74: }
75:
76: public class DotNetTypeReflector : ITypeReflector
77: {
78: private Type _type;
79:
80: public DotNetTypeReflector(Type type)
81: {
82: _type = type;
83: }
84:
85: public IEnumerable<ITypeReflector> GetInterfaces()
86: {
87: return _type.GetInterfaces().Select(i => new DotNetTypeReflector(i)).ToList();
88: }
89:
90: public IEnumerable<IAttributeReflector> GetAttributes<T>() where T : Attribute
91: {
92: List<CustomAttributeData> returnValue = new List<CustomAttributeData>();
93: var pCustomAttributeType = typeof(T);
94:
95: foreach (CustomAttributeData customAttributeData in CustomAttributeData.GetCustomAttributes(_type))
96: {
97: if (customAttributeData.Constructor.DeclaringType.Name == pCustomAttributeType.Name)
98: {
99: returnValue.Add(customAttributeData);
100: }
101: }
102:
103: return returnValue.Select(a => new DotNetAttributeReflector(a)).ToList();
104: }
105:
106: public string FullName
107: {
108: get
109: {
110: return _type.FullName;
111: }
112: }
113:
114: public string Name
115: {
116: get
117: {
118: return _type.Name;
119: }
120: }
121: }
122:
123:
124: public class DotNetAttributeReflector : IAttributeReflector
125: {
126: private CustomAttributeData _attribute;
127: private IDictionary<string, string> _values;
128:
129: public IDictionary<string, string> Values
130: {
131: get
132: {
133: if (_values == null)
134: {
135: Dictionary<string, string> returnValue = new Dictionary<string, string>();
136: try
137: {
138: ParameterInfo[] ConstructorParameters = _attribute.Constructor.GetParameters();
139: for (int i = 0; i < ConstructorParameters.Length; i++)
140: {
141: returnValue.Add(ConstructorParameters[i].Name, _attribute.ConstructorArguments[i].Value.ToString());
142: }
143: }
144: catch (KeyNotFoundException ex)
145: {
146: // something is not matching up with constructor parameters and arguments
147: // this will be a FormatException for our purposes, so we will wrap and rethrow
148: throw;
149: }
150: foreach (CustomAttributeNamedArgument argument in _attribute.NamedArguments)
151: {
152: returnValue.Add(argument.MemberInfo.Name, argument.TypedValue.Value.ToString());
153: }
154: _values = returnValue;
155: }
156: return _values;
157: }
158: }
159:
160: public DotNetAttributeReflector(CustomAttributeData attribute)
161: {
162: _attribute = attribute;
163: }
164: }
165:
166: public class DotNetReflector : IReflector
167: {
168: public IAssemblyReflector LoadAssembly(string path)
169: {
170: return new DotNetAssemblyReflector(Assembly.LoadFrom(path));
171: }
172: }
Finally let's have a look on the performance result. I'm using Mono.Cecil and System.Reflection to retrieve the metadata information from all 38 components in my system. In .NET 4.0 and 4.5.1, Mono.Cecil took 4 seconds while System.Reflection took 21 seconds.
Hope this helps,
Shaun
All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.
On 16th Jan, the "Gu" announced several features in Windows Azure in his blog, one of them is "Website Staging Support ". With this feature we can deploy our application to the staging slot of our Windows Azure Website (a.k.a. WAWS) for test purpose. And if everything is fine we can simply "SWAP" to the production slot without any down time and within few seconds. If you have been working with Windows Azure Cloud Service, this feature is very similar.
But since this feature is still in preview phase, there's some restricts. If you specified the deployment source of your WAWS, when you opened the staging feature, you might be encounter some problem and in this post I'd like to explain and the workaround.
Common Deployment Pattern before Staging Feature
Before we have staging feature available, if we'd like to continue deploy the Beta version our application to WAWS, we might be using the branch feature of our source code provider. For example, when we are using GitHub, we can create two branches in my repository: master and beta. Then I created two Website one of it connected with master another with beta.
In this way I can deploy the new version in beta branch and keep my beta WAWS deployed automatically for test purpose. If the test was passed I will merge my changes from beta to master branch. This will trigger the master WAWS deployed so that the new version will be available in my production site.
Staging Slot and Swap
By enable WAWS Staging feature, we don't need to create two websites and two branches as below. First let's enable the staging feature in my WAWS. Just in the general tab of my WAWS, let's click "Enable staged publishing" button.
You have to upgrade your WAWS to standard mode to enable this feature. Free and shared mode doesn't support staging right now.
Then in the WAWS list we will find another item appeared under my original WAWS with "staging" in its name. Also it has a URL with "-staging" as well.
Then let's click into the staging item, set the deployment from my GitHub repository.
Once I push any code changes the staging site will pull the code and deploy them into the staging website. Below is a MVC website deployed to my staging site.
And when I visited the staging website I will find the application had been deployed. In order to make future discussion clear I displayed the application version in the site content.
Now if we opened the website production URL there will be nothing since our code was not deployed.
Now I can clicked the "SWAP" button in azure portal. This will switch my staging and production site, very similar as the feature in Cloud Service.
After that we will find the website appeared in my production slot while my staging slot turned to empty.
Problem When Next Commit
Now let's change our code and commit again. Since I had enabled the GitHub deployment in my staging slot, in theory it should be deployed to my staging slot automatically. But after waiting for several minutes there's no deployment appeared.
And in the production slot since we didn't enabled the deployment source there's no "deployment" tab we can click.
So this is the problem if we are using GitHub deployment against the staging feature.
I didn't tested other deployment sources such as TFS, local Git but I think they should have the same problem since all of them are using the same deployment mechanism.
(My Guessing) Reason and How Staging Implemented
From the phenomenon of swapping slots I guess this is because deployment repository on WAWS are separated between production and staging slots and the swap operation just changes the DNS setting.
So when we enabled the staging feature in my WAWS and integrated with my GitHub, my staging slot contains the deployment repository and staging slot was register as "integrated". Then when I committed the code staging slot was get updated successfully.
When I clicked "Swap" the DNS setting was changed.
Now if I committed code changes, my staging slot was registered as integrated but since currently it was linked with my production environment, the source code cannot be pulled since no repository available. My production slot linked with the environment which contains repository but it was not registered as integrated. So I think this is the reason why my next code commit was not getting deployed once swapped.
If I swapped back and committed another version then my staging slot deployed automatically.
And the website are updated as well.
But this is NOT what I wanted. I wanted the staging slot getting deployed automatically while production slot should be updated only by swapping.
Workaround by David Ebbo
This problem was reported in Windows Azure MVP mail-list. David Ebbo from Microsoft confirmed that this is an issue of WAWS staging feature and will be solved in the future releases. For now he provided a workaround to make staging slot continue deployed with GitHub only.
Now let's create a new WAWS and enable the staging feature. After that we need to enable GitHub deployment on both of the production and staging slot.
I also recommended to create a new clean GitHub repository before you set up the deployment. Because once you set up the deployment WAWS will fetch your code and run the deployment. If you have existing code in your repository both staging and production will be deployed and this will cause some problem in the future.
Next step, we need to go to GitHub and cut down the hook to the WAWS production slot.
Selected "Service Hooks" and "Webhooks URLs" there should be two items. We should find "__staging" string in one of them which is the hook to our WAWS staging slot, and the other one is the hook to production slot. What we should do is to remove the hook WITHOUT "__staging" and click "Update settings."
After this step our GitHub repository hooks should be like this.
Now let's create a new MVC website and push the codes to GitHub then we will find only the staging slot got deployed.
Then swap the staging and production slot.
Now let's commit some changes and push to GitHub. We will find the staging slot got deployed with version 2 while the production slot still in version 1.
Now swap again to move the version 2 to production slot.
And then update our code to version 3 and push, the staging getting deployed again successfully.
(My Guessing) How Workaround Works
The workaround mentioned above utilizes the GitHub hook feature. Firstly we enabled the deployment on both staging and production slots so that in Windows Azure side both of them could connect to GitHub to deploy our application.
When we removed the production hook in GitHub, this slot will not be able to deployed automatically due to failed to connect to GitHub.
Then we when pushed a new version, only staging slot got the hook notification and began the deployment.
When we swapped the slots, the DNS was changed but the hook still connected with the staging slot with the URL (I guess). So the staging still be able to get the GitHub hook notification now.
Now if we pushed a new version to GitHub it will notify the hook to the staging URL and now the staging URL connected to another slot (previously production slot). So the staging can be deployed again.
And when we swapped the DNS switched again and the new version will be deployed to staging if we pushed version 3.
If we went back to azure portal, the deployments will be shown in our WAWS. If we visited the staging slot the deployments should be initial commit, v1 and v3.
While the production slot shows initial commit and v2. Just the same as what I guessed above.
Summary
In order to make the cloud environment testing easier Microsoft introduced the staging feature in Windows Azure Website. With this new feature we can deploy our application to staging slot and test, then swap to the production slot within few seconds.
If we want only staging slot continue deployed with our source control source we need some workaround at this stage. But I think in the near future this problem will be fixed, maybe when it's GA.
I only covered the case when using GitHub deployment. But I think we can leverage the same approach against other source control services like Visual Studio Online, CodePlex, etc..
Hope this helps,
Shaun
All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.