Shaun Xu

The Sheep-Pen of the Shaun


News

logo

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years experience in .NET. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Aliyun) as well as WCF and ASP.NET MVC. Recently he's falling in love with JavaScript and Node.js.

Currently Shaun is working at IGT Technology Development (Beijing) Co., Ltd. as the architect responsible for product framework design and development.

MVP

My Stats

  • Posts - 98
  • Comments - 352
  • Trackbacks - 0

Tag Cloud


Recent Comments


Recent Posts


Archives


Post Categories



My working in a company which was inside a Crop. network in China. As you may know, in China there are some websites we cannot connect such as Facebook, Twitter, YouTube. And the connectivity with Google is always unstable, too. As a software developer I need Google everyday to search technical actuals, look for resolution and best practices. Besides, our Crop. network only support 80 and 443 port. This means I cannot use FTP, Remote Desktop and SSH, etc. from my workstation to the service on the cloud. But it was changed when I began to use Azure RemoteApp.

Azure RemoteApp was powered by application virtualization provided Microsoft. It means we can use, in theory, any application installed in our Windows virtual machine in Azure from any of our device: Windows or Mac OS, iOS or Android.

 

Create New RemoteApp Account

There is a blog post by Scott Guthrie introducing how to create a RemoteApp account. We just need to go to azure portal, select RemoteApp, specify the name of my service, where it will be provisioned (region), price plan and the template (Windows Server 2012 or Office 365). Then that's all. Several minutes later it will be ready.

image

It's very simple to use RemoteApp. Just open it from azure portal and download the client application.

image

By default it will show the client download link for the device we are using now. But we can check downloads for all clients available here.

image

After downloaded and installed the client we can launch and login with the Microsoft Account in Azure. It will show some programs already published: Calculator, cmd, Internet Explorer, Paint, PowerShell, PowerShell ISE.

Now let me open Internet Explorer and check my IP. Since I selected East Asia region when I created the account, the IP shows that I'm in Hong Kong. Also this Internet Explorer was launched as a normal application in my workstation, an icon was in my task bar. Only different is that, there is a indicator shows that this Internet Explorer was a remote application instead of local.

image

 

Publish More Applications

RemoteApp allows us publish applications through Start Menu or path in azure portal. In this case I want to view the file system of the virtual machine my RemoteApp located. So I need to publish Windows Explorer.

Go to my RemoteApp on azure portal and select "Publishing" tab, we will find applications already exposed. Click "Publish" button in the bottom and select "Publish program using path".

image

Specify the application name and path. We use some environment variants when specifying the path.

image

Then Click "OK", RemoteApp will try to find the application and publish it. After several seconds we can see Windows Explorer was in the list.

If RemoteApp found your application successfully, it will refresh the icon with the one application uses. When you find the icon was not changed accordingly it might because the path you specified was incorrect.

image

Next, back to the RemoteApp client and refresh we can see Windows Explorer appeared.

image

 

Publish More 3rd Party Applications

As you can see I had published some 3rd party applications in my RemoteApp account such as PuTTY, FileZilla. It's very simple as well.

First of all, we need to ensure the application we want to publish can be used by simple copy-and-paste. Which means we don't need to install them on RemoteApp machine. This is because the virtual machine the RemoteApp was located contains an administrator account named "RDSAdmin" which we don't have the credential. The user when we launched a remote application was a guest account which doesn't have the permission to install or uninstall a program.

Something in details. I logged in azure portal with shaun@live.com and created my RemoteApp account, it will create a new virtual machine, install and configured Remote Desktop Service and added this account as guest permission to the system. Then when I launched any applications in RemoteApp client it will firstly access the virtual machine with this guest account in background and launch the application I selected, and show in my desktop. But if I tried to install something, the UAC will be popped up ask for the password of the administrator named "RDSAdmin" that I don't know.

Second, find the application we want to publish. We can open Internet Explorer via RemoteApp and download it directly to the virtual machine. Alternatively we can copy files from our local machine to RemoteApp machine through the "remote-ed" Windows Explorer.

Finally, publish this application by path in azure portal, same as what we did for Windows Explorer.

 

For example, below I copied "putty.exe" from my local machine to RemoteApp machine through the "remote-ed" Windows Explorer.

image

And published on azure portal as below.

image

Refresh my local RemoteApp client and launched the PuTTY from my local machine.

image

Note that even though I'm in the Crop. network that doesn't allow port 22, I can use it to connect a Linux machine on another Azure region in West US.

image

 

Remote Desktop to Virtual Machine with App Installed

In the section above I mentioned in RemoteApp we cannot install application due to the guest account restriction. But we can resolve it by introducing another virtual machine.

Normally we uses RemoteApp as below.

image

Now let's create a new virtual machine in Azure. Since this is a standard virtual machine we have administrator rights. So we can launch Remote Desktop from RemoteApp, connect to this virtual machine, install applications we want and use them. In this mode we can launch any application (copy-and-paste, or install) and use them inside the Crop. network via RemoteApp.

image

Let's launch "cmd" from my RemoteApp client, execute "mstsc".

image

Specify the virtual machine IP address or DNS name with its Remote Desktop port (normally it's NOT the default 3389), type login and password them we can see the desktop and launch Visual Studio.

image

 

Summary

In this post I introduced Azure RemoteApp feature on how to create a new account, how to publish an existing application. I also introduced on how to publish 3rd party application by copy-paste them into the virtual machine RemoteApp located.

At last I shared how to use RemoteApp to connect to another virtual machine in Azure so that we can use to install and launch more applications.

Azure RemoteApp supports hybrid deployment that allows us to publish applications need to be installed. But it's complex and time-consuming. I think by using my approach (remote to remote VM) it become very easy to use any application regardless which device we are using, what kind of network and firewall in front of.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.


When I'm developing "My DocumentDB" I decided to enhance the JSON input part by introducing a designer. After Google-ed I found JSONEditor is good to me. It's a web-based tool allows to view, edit and format JSON with simple API to integrate into a web page. Then I was going to use this cool thing into my project.

 

Use Directive

I firstly created a directive that will apply JSONEditor in its DOM. The JSON data will be specified from the "ng-model" attribute. The code is very simple as below.

   1: app.directive('uiJsonEditor', [function (jsoneditorOptions) {
   2:     'use strict';
   3:     return {
   4:         restrict: 'A',
   5:         scope: {
   6:             json: '=ngModel'
   7:         },
   8:         link: function (scope, elem) {
   9:             var opts = {
  10:                 change: function () {
  11:                         if (scope.editor) {
  12:                             scope.$apply(function () {
  13:                                 scope.json = scope.editor.get();
  14:                             });
  15:                         }
  16:                     }
  17:             };
  18:             scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  19:         }
  20:     };
  21: }]);

One thing need to be paid attention. I specified JSONEditor "change" event function so that it will update the JSON value back to the scope variant. Since this event triggered outside of AngularJS event-loop I need to wrap the update code into "scope.$apply".

Then I can use JSONEditor in my page. Below is the web page I used for prototype. I attach this directive in a DIV. And I also displayed the scope variant that ensured JSON value was updated accordingly.

   1: <!DOCTYPE html>
   2: <html ng-app="MyApp">
   3: <head>
   4:     <link rel="stylesheet" href="jsoneditor.css" />
   5: </head>
   6:  
   7: <body>
   8:     <h1>Hello AngularJS-JSONEditor</h1>
   9:  
  10:     <div ng-controller="MyCtrl">
  11:     <p>
  12:         <div data-ui-json-editor data-ng-model="json"></div>
  13:         <p>
  14:             {{json}}
  15:         </p>
  16:     </p>
  17:     </div>
  18:  
  19:     <script src="http://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.js"></script>
   1:  
   2:     <script src="http://cdnjs.cloudflare.com/ajax/libs/angular.js/1.2.20/angular.js">
   1: </script>
   2:     <script src="jsoneditor.js">
   1: </script>
   2:  
   3:     <script>
   4:         var app = angular.module('MyApp', []);
   5:  
   6:         app.controller('MyCtrl', function($scope) {
   7:  
   8:             $scope.json = {
   9:                 firstName: 'Shaun',
  10:                 lastName: 'Xu',
  11:                 skills: [
  12:                     'C#',
  13:                     'JavaScript'
  14:                 ],
  15:                 roles: [
  16:                     'dev',
  17:                     'speaker'
  18:                 ]
  19:             };
  20:         });
  21:  
  22:         app.directive('uiJsonEditor', [function (jsoneditorOptions) {
  23:             'use strict';
  24:             return {
  25:                 restrict: 'A',
  26:                 scope: {
  27:                     json: '=ngModel'
  28:                 },
  29:                 link: function (scope, elem) {
  30:                     var opts = {
  31:                         change: function () {
  32:                                 if (scope.editor) {
  33:                                     scope.$apply(function () {
  34:                                         scope.json = scope.editor.get();
  35:                                     });
  36:                                 }
  37:                             }
  38:                     };
  39:                     scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  40:                 }
  41:             };
  42:         }]);
  43:     
</script>
  20: </body>
  21:  
  22: </html>

After launched the web page JSON data will be shown in both JSONEditor and the text area.

image

If I changed something in JSONEditor we will find the data was changed automatically.

image

 

Use Module

This is good for "My DocumentDB" project, but I was thinking if I can change it as a standalone UI control being used in any my and others' projects. This is not a big deal. In AngularJS we can use module to group controllers, factories, services and directives. In this case what I need to do is to create a module and put the directive into it, and then changed my main module that depends on it.

   1: <script>
   1:  
   2:     angular.module('ui.jsoneditor', [])
   3:         .directive('uiJsonEditor', [function (jsoneditorOptions) {
   4:             'use strict';
   5:             return {
   6:                 restrict: 'A',
   7:                 scope: {
   8:                     json: '=ngModel'
   9:                 },
  10:                 link: function (scope, elem) {
  11:                     var opts = {
  12:                         change: function () {
  13:                                 if (scope.editor) {
  14:                                     scope.$apply(function () {
  15:                                         scope.json = scope.editor.get();
  16:                                     });
  17:                                 }
  18:                             }
  19:                     };
  20:                     scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  21:                 }
  22:             };
  23:         }]);
</script>
   2:  
   3: <script>
   1:  
   2:     var app = angular.module('MyApp', ['ui.jsoneditor']);
   3:     app.controller('MyCtrl', function($scope) {
   4:  
   5:         $scope.json = {
   6:             firstName: 'Shaun',
   7:             lastName: 'Xu',
   8:             skills: [
   9:                 'C#',
  10:                 'JavaScript'
  11:             ],
  12:             roles: [
  13:                 'dev',
  14:                 'speaker'
  15:             ]
  16:         };
  17:     });
</script>

In HTML part I don't need to change anything the page loaded successfully and JSONEditor works well.

image

 

Better Configuration

This is better, but not perfect. I knew JSONEditor allows developer specify some options. This can be done by introducing more scope variants into the directive. In the code below I added "options" variant. So we can tell the directive which scope variant will be used as the JSONEditor configuration.

   1: angular.module('ui.jsoneditor', [])
   2:     .directive('uiJsonEditor', [function (jsoneditorOptions) {
   3:         'use strict';
   4:         return {
   5:             restrict: 'A',
   6:             scope: {
   7:                 json: '=ngModel',
   8:                 options: '=options'
   9:             },
  10:             link: function (scope, elem) {
  11:                 var opts = scope.options || {};
  12:                 opts.change = opts.change || function () {
  13:                             if (scope.editor) {
  14:                                 scope.$apply(function () {
  15:                                     scope.json = scope.editor.get();
  16:                                 });
  17:                             }
  18:                         };
  19:                 scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  20:             }
  21:         };
  22:     }]);

In HTML part I specified which scope variant will be used as the options as below.

   1: <p>
   2:     <div data-ui-json-editor data-ng-model="json" data-options="options"></div>
   3:     <p>
   4:         {{json}}
   5:     </p>
   6: </p>

And in the controller I specified the options, defined the root node text and modes of JSONEditor.

   1: var app = angular.module('MyApp', ['ui.jsoneditor']);
   2: app.controller('MyCtrl', function($scope) {
   3:  
   4:     $scope.options = {
   5:         name: 'root',
   6:         modes: ['tree', 'text']
   7:     };
   8:  
   9:     $scope.json = {
  10:         firstName: 'Shaun',
  11:         lastName: 'Xu',
  12:         skills: [
  13:             'C#',
  14:             'JavaScript'
  15:         ],
  16:         roles: [
  17:             'dev',
  18:             'speaker'
  19:         ]
  20:     };
  21: });

Refresh the web page we will see the options was changed.

image

Now it's almost perfect. But if I have more than one JSONEditor controls in my application I might  wanted to have a default options. This can be done by using AngularJS provider. In the help page it said

You should use the Provider recipe only when you want to expose an API for application-wide configuration that must be made before the application starts. This is usually interesting only for reusable services whose behavior might need to vary slightly between applications.

Provider holds some variants which can be exposed by a special function named "$get". We can defined functions in it which can be used before the application starts, typically in our AngularJS main module's "config" function.

In our case we just need to create a provider, define a local variant stores JSONEditor options, expose two function. "$get" function will return this variant, "setOptions" function will set options into this value.

And in JSONEditor directive we referenced the provider just created, retrieved the value and merged with the scope variant. Now we have a global options, and developer can specify some options for a particular JSONEditor control as well.

   1: angular.module('ui.jsoneditor', [])
   2:     .directive('uiJsonEditor', ['jsoneditorOptions', function (jsoneditorOptions) {
   3:         'use strict';
   4:         return {
   5:             restrict: 'A',
   6:             scope: {
   7:                 json: '=ngModel',
   8:                 options: '=options'
   9:             },
  10:             link: function (scope, elem) {
  11:                 var opts = angular.extend({}, jsoneditorOptions, scope.options);
  12:                 opts.change = opts.change || function () {
  13:                             if (scope.editor) {
  14:                                 scope.$apply(function () {
  15:                                     scope.json = scope.editor.get();
  16:                                 });
  17:                             }
  18:                         };
  19:                 scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  20:             }
  21:         };
  22:     }])
  23:     .provider('jsoneditorOptions', function () {
  24:         'use strict';
  25:         this.options = {};
  26:  
  27:         this.$get = function () {
  28:             var opts = this.options;
  29:             return opts;
  30:         };
  31:  
  32:         this.setOptions = function (value) {
  33:             this.options = value;
  34:         };
  35:     });

Then back to the main module we can define default options of JSONEditor in "app.config" function.

   1: app.config(['jsoneditorOptionsProvider', function (jsoneditorOptionsProvider) {
   2:     jsoneditorOptionsProvider.setOptions({
   3:         name: 'root',
   4:         modes: ['text', 'tree']
   5:     });
   6: }]);

After refresh the web page we will see that the JSONEditor options changed even I had removed the options from controller scope.

image

And if I specified the options in the controller scope it will be updated, but the global options still remained.

   1: $scope.options = {
   2:     name: 'this'
   3: };

image

 

Summary

In this post I introduced how to create an AngularJS module wraps a UI control. In AngularJS we should use directive when dealing with DOM. Then I moved the code into a standalone module to make it useable for any other projects. At the end I added the functionality for global configuration.

The full sample code is as below.

   1: <!DOCTYPE html>
   2: <html ng-app="MyApp">
   3: <head>
   4:     <link rel="stylesheet" href="jsoneditor.css" />
   5: </head>
   6:  
   7: <body>
   8:     <h1>Hello AngularJS-JSONEditor</h1>
   9:  
  10:     <div ng-controller="MyCtrl">
  11:     <p>
  12:         Default Options
  13:     </p>
  14:     <p>
  15:         <div data-ui-json-editor data-ng-model="json"></div>
  16:     </p>
  17:     <p>
  18:         Instance Options
  19:     </p>
  20:     <p>
  21:         <div data-ui-json-editor data-ng-model="json" data-options="options"></div>
  22:     </p>
  23:     <p>
  24:         {{json}}
  25:     </p>
  26:     </div>
  27:  
  28:     <script src="http://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.1/jquery.js"></script>
   1:  
   2:     <script src="http://cdnjs.cloudflare.com/ajax/libs/angular.js/1.2.20/angular.js">
   1: </script>
   2:     <script src="jsoneditor.js">
   1: </script>
   2:  
   3:     <script>
   4:         angular.module('ui.jsoneditor', [])
   5:             .directive('uiJsonEditor', ['jsoneditorOptions', function (jsoneditorOptions) {
   6:                 'use strict';
   7:                 return {
   8:                     restrict: 'A',
   9:                     scope: {
  10:                         json: '=ngModel',
  11:                         options: '=options'
  12:                     },
  13:                     link: function (scope, elem) {
  14:                         var opts = angular.extend({}, jsoneditorOptions, scope.options);
  15:                         opts.change = opts.change || function () {
  16:                                     if (scope.editor) {
  17:                                         scope.$apply(function () {
  18:                                             scope.json = scope.editor.get();
  19:                                         });
  20:                                     }
  21:                                 };
  22:                         scope.editor = new JSONEditor(elem[0], opts, scope.json || {});
  23:                     }
  24:                 };
  25:             }])
  26:             .provider('jsoneditorOptions', function () {
  27:                 'use strict';
  28:                 this.options = {};
  29:  
  30:                 this.$get = function () {
  31:                     var opts = this.options;
  32:                     return opts;
  33:                 };
  34:  
  35:                 this.setOptions = function (value) {
  36:                     this.options = value;
  37:                 };
  38:             });
  39:     
</script>
   1:  
   2:  
   3:     <script>
   4:         var app = angular.module('MyApp', ['ui.jsoneditor']);
   5:  
   6:         app.config(['jsoneditorOptionsProvider', function (jsoneditorOptionsProvider) {
   7:             jsoneditorOptionsProvider.setOptions({
   8:                 name: 'root',
   9:                 modes: ['text', 'tree']
  10:             });
  11:         }]);
  12:  
  13:         app.controller('MyCtrl', function($scope) {
  14:  
  15:             $scope.options = {
  16:                 name: 'this'
  17:             };
  18:  
  19:             $scope.json = {
  20:                 firstName: 'Shaun',
  21:                 lastName: 'Xu',
  22:                 skills: [
  23:                     'C#',
  24:                     'JavaScript'
  25:                 ],
  26:                 roles: [
  27:                     'dev',
  28:                     'speaker'
  29:                 ]
  30:             };
  31:         });
  32:     
</script>
  29: </body>
  30:  
  31: </html>

And you can find the module I created in GitHub.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.


What's DocumentDB

DocumentDB is a fully-managed, scalable, NoSQL document database service in Microsoft Azure. It provides rich query and indexing capabilities over a schema-free JSON data model. It also provides configurable and reliable performance, native JavaScript transactional processing, and is built for the cloud with elastic scale.

DocumentDB was announced on 22nd October as a preview feature. User can provision through the new azure portal.

If you are not familiar or this is the first time you hear about DocumentDB, I'd like suggest you to read my previous post. I also recommend you to read some documents about it in Microsoft Azure website.

 

What's "My DocumentDB" & Why I Created It

"My DocumentDB" is a web-based DocumentDB Management tool I had just created. The reason I created is because, in azure portal there's very few features for us to manage a DocumentDB.

As you can see it's easy to create or delete a DocumentDB account in azure portal as below.

image

And inside an existing DocumentDB account there are many monitors and status reports. We can also create and delete a databases, and view the collections' basic information under a certain database.

image

But that's all. I found that we cannot create or delete a collection, cannot create, update or delete a document under a collection. This makes me a little bit hard to use when I'm working with DocumentDB. Hence this is the reason why I created "My DocumentDB". And in order to make it cross-platform and easy to use, I decided to create a web-based tool rather than a client application.

 

"My DocumentDB" Features (Currently)

Currently there are three features in "My DocumentDB":

- Database Management: List, create and delete a database.

- Collection Management: List, create and delete a collection under a certain database.

- Document Management: List, create, update and delete a document under a certain collection.

 

Below is the home page of "My DocumentDB". On the top right corner we need to specify the host and key we want to connect.

image

The host and key can be copied from the DocumentDB details page from azure portal as below.

image

After specified the host and key, press the "Connect" button. Then there will be two menu items appeared as below. The "Explorer" menu navigates us to the main management pages. The "Console" page content is still under developed.

image

When clicked "Connect", this application will NOT connect to your DocumentDB. It just stored the credentials inside local JavaScript variants. If it's incorrect you will got error message when clicked "Explorer" menu.

Ref the credentials and potential security risks please refer to the security section later in this post.

Click "Explorer" menu will list all databases in the DocumentDB account. And we can create and delete a database by clicking the related button.

image

Note that when we are going to delete a database, it asked us to re-input the database ID for double verification.

image

Click a database ID will list all its collections. In this page we can create and delete collections as well. Similarly, when deleting a collection we also need to re-input the ID.

image

Click a document ID will list all its documents. In this page the document ID and resource ID will be displayed by default. If you want to view the content of the document you can click the arrow icon in front of each row as below.

image

We can create, edit and delete a document from this page as well. When deleting a document we also need to re-input its ID. When create or edit a document we need to provide the ID as well as the content. Since the document is a JSON, in the popup window we need to specify in JSON format. Currently we only support raw JSON string in the "Raw" tab. In the future we might introduce a JSON editor.

imageThe ID property was specified from the separated input box in this window so we don't need to put it into the content box. And if we specified ID in both places the one in the content will be replaced. For example, I specified ID in top input box with value "123" and in the content input box you specified as below.

   1: {
   2:   "id": "456",
   3: }

Then the document inserted into DocumentDB will be

   1: {
   2:   "id": "123"
   3: }

Below is the new document I had just created.

image

 

  • Architecture

  • Below is a brief architecture diagram of this project. Basically all request will be sent from "My DocumentDB" web pages to backend "My DocumentDB" API, then sent to Azure DocumentDB REST service.

  • image

 

Source Code & Deployment

You can find the source code of "My DocumentDB" on GitHub. And Currently there are three ways to use "My DocumentDB".

The first one is to use the live demo I deployed in Azure Website located at North Europe data center. This is the most easy way. But it might introduce connection legacy and cost if your DocumentDB was created in other data centers.

The second way is to deploy "My DocumentDB" in your Azure account. This is simple, too. Since it's designed for Azure, what you need to do is to download the source code from GitHub and deploy to any Azure Website you have.

The last choice, you can deploy "My DocumentDB" on premise. Similar as the previous one, download the source code to you desktop, make sure you have Node.js installed and execute the command as below. Since "My DocumentDB" is hosted by Node.js, you can run it under Windows, Linux and Mac.

   1: node.exe server.js --port=[port number] [--debug]

When deploying "My DocumentDB" locally you need to ensure the Internet connectivity without proxy. Currently we don't support local deployment under a proxy Internet connectivity environment.

image

 

Security Risks

I created "My DocumentDB" and worked in spare time so that I didn't pay more effort in security consideration. There might be some risks when you are using it please be aware. And since "My DocumentDB" was released with MIT license I will NOT provide any warranties.

No SSL connection.

Currently "My DocumentDB" will be hosted with HTTP instead of HTTPS. This means the data between your browser and "My DocumentDB" server can be seen by others. But the connection to Azure DocumentDB is HTTPS. You can change the source code to enable HTTPS if you are going to deploy to your own azure service or local.

Wildcard SSL Certificate

Azure Website support wildcard certificate for its all sites. This means you can simply use https://mydocdb.azurewebsites.net for security connection on the live demo. But if you need more security request I'd like you deploy to your own azure account with your certificate.

Credentials will be stored in local JavaScript variants.

Currently the DocumentDB host and key will be stored in the local JavaScript variants and passed to "My DocumentDB" service through HTTP headers. This means if you forgot to disconnect others would be able to see your credentials by opening the debug mode of your browser.

 

Based on these risks I personally recommend

1. Click "Disconnect" button when you don't use it, or when you need to leave for a moment.

2. Local deployment provides better security boundary than using live demo since all sensitive data are stored in your machine, and the connection to Azure DocumentDB is HTTPS.

 

Roadmap

As I mentioned below this is the first version of "My DocumentDB". Based on your feedback and usage there might be more features in pipeline I'm going to implement. Below are some of them in my mind. Hopefully I have passion and energy to work them out.

1. Console

In the console page user can execute DocumentDB query console against database, collection or document. Developers can try their query and check the result before implementation.

2. JSON editor

Currently user has to use plaint JSON string when create or edit a document. In the future I will add a JSON designer in document edit window.

3. Paging

Currently I didn't specify any paging options in requests. But if there are many documents in a collection we'd better let users specify how many items per page they like.

4. More to manage

We only support databases, collections and documents management, but there are many more things in DocumentDB such as users, stored procedures, triggers and attachments, etc. They should be able to be managed as well in the future.

 

Feedbacks & Issues

I encourage everyone to try "My DocumentDB", either from the live demo, or self host on azure or local. If you have any questions or issues please feel free to contact in the GitHub page.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.


Microsoft just released a bunch of new features for Azure on 22nd and one of them I was interested in most is DocumentDB, a document NoSQL database service on the cloud.

 

Quick Look at DocumentDB

We can try DocumentDB from the new azure preview portal. Just click the NEW button and select the item named DocumentDB to create a new account.

Screen Shot 2014-08-23 at 11.19.27

Specify the name of the DocumentDB, which will be the endpoint we are going to use to connect later. Select the capacity unit, resource group and subscription. In resource group section we can select which region our DocumentDB will be located.

Same as other azure services select the same location with your consumers of the DocumentDB, for example the website, web services, etc..

Screen Shot 2014-08-23 at 11.23.21

After several minutes the DocumentDB will be ready. Click the KEYS button we can find the URI and primary key, which will be used when connecting.

Screen Shot 2014-08-23 at 11.34.13

Now let's open Visual Studio and try to use the DocumentDB we had just created. Create a new console application and install the DocumentDB .NET client library from NuGet with the keyword "DocumentDB".

You need to select "Include Prerelase" in NuGet Package Manager window since this library was not yet released.

Screen Shot 2014-08-24 at 18.37.46

Next we will create a new database and document collection under our DocumentDB account. The code below created an instance of DocumentClient with the URI and primary key we just copied from azure portal, and create a database and collection. And it also prints the document and collection link string which will be used later to insert and query documents.

   1: static void Main(string[] args)
   2: {
   3:     var endpoint = new Uri("https://shx.documents.azure.com:443/");
   4:     var key = "LU2NoyS2fH0131TGxtBE4DW/CjHQBzAaUx/mbuJ1X77C4FWUG129wWk2oyS2odgkFO2Xdif9/ZddintQicF+lA==";
   5:  
   6:     var client = new DocumentClient(endpoint, key);
   7:     Run(client).Wait();
   8:  
   9:     Console.WriteLine("done");
  10:     Console.ReadKey();
  11: }
  12:  
  13: static async Task Run(DocumentClient client)
  14: {
  15:  
  16:     var database = new Database() { Id = "testdb" };
  17:     database = await client.CreateDatabaseAsync(database);
  18:     Console.WriteLine("database link = {0}", database.SelfLink);
  19:  
  20:     var collection = new DocumentCollection() { Id = "testcol" };
  21:     collection = await client.CreateDocumentCollectionAsync(database.SelfLink, collection);
  22:     Console.WriteLine("collection link = {0}", collection.SelfLink);
  23: }

Below is the result from the console window. We need to copy the collection link string for future usage.

Screen Shot 2014-08-24 at 19.43.46

Now if we back to the portal we will find a database was listed with the name we specified in the code.

Screen Shot 2014-08-24 at 19.45.13

Next we will insert a document into the database and collection we had just created. In the code below we pasted the collection link which copied in previous step, create a dynamic object with several properties defined. As you can see we can add some normal properties contains string, integer, we can also add complex property for example an array, a dictionary and an object reference, unless they can be serialized to JSON.

   1: static void Main(string[] args)
   2: {
   3:     var endpoint = new Uri("https://shx.documents.azure.com:443/");
   4:     var key = "LU2NoyS2fH0131TGxtBE4DW/CjHQBzAaUx/mbuJ1X77C4FWUG129wWk2oyS2odgkFO2Xdif9/ZddintQicF+lA==";
   5:  
   6:     var client = new DocumentClient(endpoint, key);
   7:  
   8:     // collection link pasted from the result in previous demo
   9:     var collectionLink = "dbs/AAk3AA==/colls/AAk3AP6oFgA=/";
  10:  
  11:     // document we are going to insert to database
  12:     dynamic doc = new ExpandoObject();
  13:     doc.firstName = "Shaun";
  14:     doc.lastName = "Xu";
  15:     doc.roles = new string[] { "developer", "trainer", "presenter", "father" };
  16:  
  17:     // insert the docuemnt
  18:     InsertADoc(client, collectionLink, doc).Wait();
  19:  
  20:     Console.WriteLine("done");
  21:     Console.ReadKey();
  22: }

the insert code will be very simple as below, just provide the collection link and the object we are going to insert.

   1: static async Task InsertADoc(DocumentClient client, string collectionLink, dynamic doc)
   2: {
   3:     var document = await client.CreateDocumentAsync(collectionLink, doc);
   4:     Console.WriteLine(await JsonConvert.SerializeObjectAsync(document, Formatting.Indented));
   5: }

Below is the result after the object had been inserted.

Screen Shot 2014-08-24 at 19.53.02

Finally we will query the document from the database and collection. Similar to the insert code, we just need to specify the collection link so that the .NET SDK will help us to retrieve all documents in it.

   1: static void Main(string[] args)
   2: {
   3:     var endpoint = new Uri("https://shx.documents.azure.com:443/");
   4:     var key = "LU2NoyS2fH0131TGxtBE4DW/CjHQBzAaUx/mbuJ1X77C4FWUG129wWk2oyS2odgkFO2Xdif9/ZddintQicF+lA==";
   5:  
   6:     var client = new DocumentClient(endpoint, key);
   7:  
   8:     var collectionLink = "dbs/AAk3AA==/colls/AAk3AP6oFgA=/";
   9:  
  10:     SelectDocs(client, collectionLink);
  11:  
  12:     Console.WriteLine("done");
  13:     Console.ReadKey();
  14: }
  15:  
  16: static void SelectDocs(DocumentClient client, string collectionLink)
  17: {
  18:     var docs = client.CreateDocumentQuery(collectionLink + "docs/").ToList();
  19:     foreach(var doc in docs)
  20:     {
  21:         Console.WriteLine(doc);
  22:     }
  23: }

Since there's only one document in my collection below is the result when I executed the code. As you can see all properties, includes the array was retrieve at the same time. DocumentDB also attached some properties we didn't specified such as "_rid", "_ts", "_self" etc., which is controlled by the service.

Screen Shot 2014-08-24 at 20.14.29

 

DocumentDB Benefit

DocumentDB is a document NoSQL database service. Different from the traditional database, document database is truly schema-free. In a short nut, you can save anything in the same database and collection if it could be serialized to JSON.

We you query the document database, all sub documents will be retrieved at the same time. This means you don't need to join other tables when using a traditional database. Document database is very useful when we build some high performance system with hierarchical data structure.

For example, assuming we need to build a blog system, there will be many blog posts and each of them contains the content and comments. The comment can be commented as well. If we were using traditional database, let's say SQL Server, the database schema might be defined as below.

image

When we need to display a post we need to load the post content from the Posts table, as well as the comments from the Comments table. We also need to build the comment tree based on the CommentID field.

But if were using DocumentDB, what we need to do is to save the post as a document with a list contains all comments. Under a comment all sub comments will be a list in it. When we display this post we just need to to query the post document, the content and all comments will be loaded in proper structure.

   1: {
   2:     "id": "xxxxx-xxxxx-xxxxx-xxxxx",
   3:     "title": "xxxxx",
   4:     "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
   5:     "postedOn": "08/25/2014 13:55",
   6:     "comments": 
   7:     [
   8:         {
   9:             "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  10:             "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  11:             "commentedOn": "08/25/2014 14:00",
  12:             "commentedBy": "xxx"
  13:         },
  14:         {
  15:             "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  16:             "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  17:             "commentedOn": "08/25/2014 14:10",
  18:             "commentedBy": "xxx",
  19:             "comments":
  20:             [
  21:                 {
  22:                     "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  23:                     "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  24:                     "commentedOn": "08/25/2014 14:18",
  25:                     "commentedBy": "xxx",
  26:                     "comments":
  27:                     [
  28:                         {
  29:                             "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  30:                             "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  31:                             "commentedOn": "08/25/2014 18:22",
  32:                             "commentedBy": "xxx",
  33:                         }
  34:                     ]
  35:                 },
  36:                 {
  37:                     "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  38:                     "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  39:                     "commentedOn": "08/25/2014 15:02",
  40:                     "commentedBy": "xxx",
  41:                 }
  42:             ]
  43:         },
  44:         {
  45:             "id": "xxxxx-xxxxx-xxxxx-xxxxx",
  46:             "content": "xxxxx, xxxxxxxxx. xxxxxx, xx, xxxx.",
  47:             "commentedOn": "08/25/2014 14:30",
  48:             "commentedBy": "xxx"
  49:         }
  50:     ]
  51: }

 

DocumentDB vs. Table Storage

DocumentDB and Table Storage are all NoSQL service in Microsoft Azure. One common question is "when we should use DocumentDB rather than Table Storage". Here are some ideas from me and some MVPs.

First of all, they are different kind of NoSQL database. DocumentDB is a document database while table storage is a key-value database.

Second, table storage is cheaper. DocumentDB supports scale out from one capacity unit to 5 in preview period and each capacity unit provides 10GB local SSD storage. The price is $0.73/day includes 50% discount. For storage service the highest price is $0.061/GB, which is almost 10% of DocumentDB.

Third, table storage provides local-replication, geo-replication, read access geo-replication while DocumentDB doesn't support.

Fourth, there is local emulator for table storage but none for DocumentDB. We have to connect to the DocumentDB on cloud when developing locally.

But, DocumentDB supports some cool features that table storage doesn't have. It supports store procedure, trigger and user-defined-function. It supports rich indexing while table storage only supports indexing against partition key and row key. It supports transaction, table storage supports as well but restricted with Entity Group Transaction scope. And the last, table storage is GA but DocumentDB is still in preview.

 

Summary

In this post I have a quick demonstration and introduction about the new DocumentDB service in Azure. It's very easy to interact through .NET and it also support REST API, Node.js SDK and Python SDK.

Then I explained the concept and benefit of  using document database, then compared with table storage.

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.


There are many advantages build our own proxy server on the cloud. For instance, in Microsoft Azure, the price is pay-as-you-go, which means we only need to pay when we need a proxy server and turned it on. Second, it's very easy to scale up and down. If the proxy is just used by myself, I can create a minimum virtual machine with small CPU, memory, disk and network bandwidth. But we can scale it up if we need, for example when we need to watch World Cup videos. Last, there are many Azure data centers around the world. This means we can create proxy server in US, Euro, HK, Japan or Brazil, etc..

 

Create a proxy server in Microsoft Azure is very easy. First of all we need to create virtual machine in Microsoft Azure. In this case I'm going to use Ubuntu.

Screen Shot 2014-06-28 at 22.00.50

Next, specify the name, size and authentication of the machine. Since this proxy server will be use by myself, I specified a small size which should be OK to view web pages. And in order to make it simple I created a user with password rather than upload a certificate for authentication.

Screen Shot 2014-06-28 at 22.01.24

Next, we need to select a region where our proxy machine will be hosted. In the screenshot below we can find there are 11 data centers in the world. And if you have account in Azure China there will be two more, Beijing and Shanghai. I selected Japan West which is close to me.

Then I need to specify an endpoint for the proxy. Just create a new endpoint at the bottom of this page, specify the number you like with TCP protocol.

Recommend to specify endpoint number larger than 1024 since in Linux, you must use "sudo" to start an application listening port less than 1024.

Screen Shot 2014-06-28 at 22.02.32

Now let's click OK to provision our virtual machine. After several minutes when the machine was ready, go to its details page and copy its public IP address. This is our proxy address and it will NOT be change until we stop the machine.

Screen Shot 2014-06-28 at 22.06.30

Next, we need to login to our virtual machine and install the proxy software. In Windows we can use Putty, an SSH and telnet client. In Mac or Linux there is a build-in SHH command line, that just need to type the command as below. The argument is "ssh [your login]@[virtual machine public IP]", which are the login I specified when created the virtual machine and the public IP I copied previously.

Screen Shot 2014-06-28 at 22.06.58

Type the password I specified when created the virtual machine and now we logged into the Ubuntu in Azure. Now I'm going to install the proxy software Squid through "apt-get".

   1: sudo apt-get install squid

After the Squid was installed we will modify its configuration. Go to the configuration folder, backup the original one and create an empty configuration file. Then launch "vim" to edit. Just follow the command below.

   1: cd /etc/squid3
   2: sudo cp squid.conf squid.conf.bak
   3: sudo rm squid.conf
   4: sudo touch sqiud.conf
   5: sudo vim squid.conf

Then in "vim" I will use the simplest configuration, which allows all clients to connect and allows all destinations to communicate. And specify the port Squid is listening, which must be the same one as what we specified when created machine and save it.

If you are not familiar with "vim", you need to type "a" to enter the append mode and paste the configuration below. Then press ESC to back to the command mode and press ":wq" to save and quit.

   1: http_access allow all
   2: http_port 21777

Next, restart Squid service to apply our configuration.

   1: sudo service squid3 restart

Then you will see the process ID of Squid.

Screen Shot 2014-06-28 at 22.11.45

To test our proxy, just back to my laptop and connect the proxy endpoint through "telnet" as below.

Screen Shot 2014-06-28 at 22.12.14

 

If you see the message in terminal as above your proxy is up and running. If you are using Chrome there is a awesome extension for smart proxy configuration named SwitchySharp. In the screenshot below I specified the proxy setting to my server in Azure Japan. Just copied the virtual machine public IP as the HTTP proxy and the Squid endpoint as the proxy port.

Screen Shot 2014-06-28 at 22.13.18

Below is the IP detection result. As you can see I'm not at Japan with Microsoft Network.

Screen Shot 2014-06-28 at 22.35.02

 

Hope this helps,

Shaun

All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.