Hadoop (HDInsight) on Windows 8

The following link will describe how to install Hadoop on Windows 8 clearly: http://marktab.net/datamining/2012/10/31/install-microsoft-hdinsight-server-hadoop-windows-8-professional/.

Windows Performance Analyzer Series

There are tools for Windows that can be used to visualize the trace files from ETW. It is Windows Performance Analyzer (WPA) which is a part of Windows Performance Toolkit.
These links below are the WPA series from channel9:

• Capturing and analyzing performance traces
• Customizing WPA Trace Views
• Understanding pool usage using Windows Performance Analyzer
• Understanding heap data using Windows Performance Analyzer
• Understanding VirtualAlloc usage using Windows Performance Analyzer

It is really suit for who new to performance tracing to know how to analyze data using WPA.

Hadoop on Windows Azure

This post is to show you overview about Hadoop on Azure.



References:

1. Channel9: Hadoop on Windows Azure
2. MSDN: Windows Azure - Hadoop on Windows Azure

How to measure UI Rendering time and FPS for Windows Store Apps (C# / XAML)

To measure performance for Windows Store Apps, if you have no experience before, I would recommend you to look at this session first: http://channel9.msdn.com/Events/Build/2012/4-103. He will explain the concepts to capture Windows Store Apps performance tips.

In short, XAML Operations are broken down into frames, and follows these 3 main steps:

1. Parsing: parse XAML for any objects

2. Layout: put objects, and determine their layout

3. Rendering: draw visible objects

To capture UI rendering time easier, think about the followings things:
1. Scenario: understand and know which parts that want to trace. Try to break down into atomic parts, it will be more clear for design scenario steps.

2. Steps of the trace scenario: it will notice us where our trace begins or ends in the trace file.

3. If in the same scenario contains sub parts such as loading items then scrolling down to the last items, try to make a self-marker (known by yourself) to separate sub tasks clearly.
4. When draw objects don't have to be re-draw, Frame will not be produced. Mouse interaction will produce frames only it trigger events on specific objects such as mouse over on objects then that object will have hover animation (re-draw frames).

Here is a command to capture UI Frame behavior:

@if %_echo%!==! echo off

set UserOptions=-BufferSize 1024 -MinBuffers 32 -MaxBuffers 32 -Buffering
set KernelOptions=-BufferSize 1024 -MinBuffers 64 -MaxBuffers 64 -Buffering

xperf -start xaml_user -on "Microsoft-Windows-XAML::5"+perftrack %UserOptions%
xperf -on base+cswitch %KernelOptions%

xperf -m BeginTrace
echo Event listeners started, run desired scenario then press any key
pause
xperf -m EndTrace

echo stopping xaml_kernel...
xperf -flush -f xaml_kernel.etl
xperf -stop

echo stopping xaml_user...
xperf -flush xaml_user -f xaml_user.etl
xperf -stop xaml_user

echo merging to xaml_merged.etl...
xperf -merge xaml_user.etl xaml_kernel.etl xaml_merged.etl

The command you see is a xperf command to capture XAML behavior which we can use it to monitor UI Frame Rendering time.

Now, let's see the real example. I will use Kona project in codeplex to demonstrate how to trace UI Rendering time and FPS. You can install and follow the example scenario I given.

Example Scenario: Capture UI rendering time of Bikes Category page, then scroll to the right and left

Steps:

1. Click at the header Bikes category
    

2. Waiting for all items are loaded into the page
3. Scroll page to the right using mouse wheel
    

4. Scroll page to the left using mouse wheel

Open trace file using WPA (Windows Performance Analyzer)

1. On the left side select System Activity, then there are more items expanded.
2. Select Generics events, an analysis tab will be shown.

3. Select Provider Name Microsoft-Windows-XAML.
4. Select Task Names that related to the scenario e.g. Frame, PointerDown, PointerUp, PointerLeave, PointerUpdate, PointerWheel.

5. Right click, and select filter selection.

Analyze trace file

1. We know that scenario start from the first click at Bikes category header, and page will be navigated after mouse is up then we do long wait for UI rendering. So, we know immediately when it starts and stops.
    

2. Zoom in, and we know that page is rendered after XAML is parsed. So, the Frame interval followed ParseXaml is the UI rendering frames for drawing objects on the screen (Duration 1.1 seconds).
    

3. The next sub part is after we do long wait and begin-end using mouse wheel. We can easily look at where PointerWheel start and stop that is the interval for scrolling items to right and left. To measure scrolling, it is difficult to measure in term of finding the longest gap between frames or longest interval because of UI thread will not produce frames when no re-draw objects (need to scroll mouse wheel accurately and continuously), which lead to calculate using FPS instead.

Calculate FPS UI Thread

1. In some scenarios, they don't create or load any objects just move or animate around the page. In order to measure it and detect UI glitch in overall using FPS (frames per second) is may be a good choice.
2. From scrolling, zoom in to that part. And open Frame tab to see how many frames in that time range.
    

3. There are 300 frames during 6.77 seconds, so the scrolling FPS is 44.3.

In conclusion, I still don't know the best way to trace UI glitch or at least find out the root problem if it is glitched yet (to know it is glitched because of what). However, based on the link I gave above, we can adjust the number of elements shown on the screen and remove overdraw elements or animations to improve application performance.

Last but not least, information in this post, I have tried by myself and no guarantee that this is right or wrong. Please read and consider by yourself.

Delete project from TFS 2012

If you have ever used TFS, you can create a new project from Visual Studio directly. However, to delete a project, you need to use Team Foundation Server Console in a remote server which is not available in sometimes such as https://tfs.visualstudio.com/. You cannot access the server directly. So, the solution is we need to access remotely using command line tool from Visual Studio.
- Open Developer Command Prompt for VS2012
- The syntax command is TFSDeleteProject [/q] [/force] [/excludewss] /collection:URL TeamProjectName
- Example: TFSDeleteProject /collection:"https://xxx.visualstudio.com/DefaultCollection/" "Project X"

Change IE security setting to normal in Windows Server 2012

By default Internet Explorer (IE) in Windows Server is protected with some security to prevent unintentionally attacked during browse internet. Sometimes, we want it to be just a normal IE that trusts how user use it. In order to do that in Windows Server 2012 is described as follows:

Firstly, opens Server Manager page.

Next, on the top of Dashboard under welcoming message, clicks at "Configure this local server" as shown picture below.

Then, on properties tab, you will see information is divided into 2 sides. Look at the right side. You will see "IE Enhanced Security Configuration", it should be on if your IE is still protected. So, we gonna change it off. by click at it.

After you click it, a popup will be opened. Your job is changing to off as shown below.

Click OK and wait for a while during system setting is changing. When it finished, you will notice on properties tab "IE Enhanced Security Configuration" is changed to Off.

Remove Remote Desktop Entries from Remote Desktop Connection

Steps to remove are described as follows:

1. Click start or press Windows button, it will show application screen.
2. Type "regedit", it will search and show only one program.
3. Select that "regedit.exe".
4. On the left hand side of the program, it is a navigation panel. Then you can easily navigate to "HKEY_CURRENT_USER\Software\Microsoft\Terminal Server Client\Default".
5. On the right hand side of the program, they are entries of all connection Remote Desktop Connection remembered. In order to remove an entry, just select which connection you want to delete using right click at its name then select delete command. It will automatically remove (no need to save again).

Reference: http://support.microsoft.com/kb/312169

AutoResetEvent to synchronize threads

Sometimes when we are dealing with multithreading application, we need to lock some threads and waiting for another thread finished execution, in order to do that it needs some lock mechanism to lock other threads . So, in this post we have a solution that we can use AutoResetEvent class to block threads from execution. AutoResetEvent is a class to control lock mechanism and guarantee that after it is set, it will allow only one thread to continue, then reset automatically (become lock again). There is also another class named ManualResetEvent, which it concept is similar to AutoResetEvent class, however, the difference is we need to reset the lock mechanism manually. So, it means that it will allows threads at the same time to continue until we tell it to stop (reset it). By the way, in this post I would like to talk about AutoResetEvent class only. The usage scenario for this class is easy to follow as described in the following website:

References:

• http://msdn.microsoft.com/en-us/library/system.threading.autoresetevent.autoresetevent.aspx
• What is the difference between ManualResetEvent and AutoResetEvent in .NET?

Task the new way to parallel jobs, a replacement for BackgroundWorker

Task class is an improvement version for BackgroundWorker. It supports nesting (parent/child tasks), new cacellation API, task continuations, etc. The following website is an example to compare Task and BackgroundWorker.

References: 

• Reporting Progress from Tasks
• Task parallel library replacement for BackgroundWorker?
• Continuation Tasks
• Parallel Programming: Task Schedulers and Synchronization Context

Image POST Client and Sever in Python and C#

From last time, I had told how to encode image into base64 string[1], and how to do POST message to HTTP server[2][3] This post will continue from those posts. First, we will create HTTP POST server in Python. I based on the concept that a message will be encoded using url-encode format in a form of base64 string, and the image string will be sent to a variable named "img". It will decode an image string from that variable.

import SimpleHTTPServer
import SocketServer
import cgi
from base64 import decodestring

PORT = 8000

class ServerHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):

 def _writeheaders(self):
  self.send_response(200)
  self.send_header('Content-type', 'text/html')
  self.end_headers()

 def do_GET(self):
  filename = 'temp.png'
  f = open(filename, 'rb')
  encode = f.read().encode('base64')
  img = '<img height="200" src="data:image/png;base64,' + encode + '" width="250" />'
  self._writeheaders()
  self.wfile.write("""
  <html><head><title>Simple Server</title></head>
   <body>
    It worked!!!!
 
    %s
   </body>
  </html>"""  % (img))
 def do_POST(self):
  form = cgi.FieldStorage(
   fp=self.rfile,
   headers=self.headers,
   environ={'REQUEST_METHOD':'POST',
      'CONTENT_TYPE':self.headers['Content-Type'],
     })
  encode = form['img'].value
  decode = decodestring(encode)
  output = open('temp.png', 'wb')
  output.write(decode)
  output.close()
  # After posting image has done, it will response GET message to show that image
  self.do_GET()

Handler = ServerHandler

httpd = SocketServer.TCPServer(("", PORT), Handler)

print "serving at port", PORT
httpd.serve_forever()

Then, we will create HTTP POST client to upload an image to our HTTP POST server.

Python:

import urllib, urllib2

uri = 'http://localhost:8000'
filename = 'Leafa.jpg'
f = open(filename, 'rb')
encode = f.read().encode('base64')
params = { 'img' : encode }
data = urllib.urlencode(params)

p = urllib2.urlopen(uri, data)
print p.read()

C# WPF:

string fileName = "Leafa.jpg";
StreamResourceInfo sri = null;
Uri uri = new Uri(fileName, UriKind.Relative);
sri = Application.GetResourceStream(uri);

using (var memoryStream = new MemoryStream())
{
    sri.Stream.CopyTo(memoryStream);
    byte[] result = memoryStream.ToArray();
    var base64 = "img=" + HttpUtility.UrlEncode(System.Convert.ToBase64String(result));

    MessageBox.Show(base64);

    using (var wc = new WebClient())
    {
        wc.Headers["Content-Type"] = "application/x-www-form-urlencoded";
        wc.UploadStringCompleted += wc_UploadStringCompleted;
        wc.UploadStringAsync(new Uri(url), "POST", base64);
    }
}

The server will response a web page, which the client will show that response as a text format, so, if you want to make sure that your image are already uploaded, just check it the "server.py" directory. You will see the image named "temp.png", which is converted from jpg image in the client side. Moreover, you can use a web browser request to http://localhost:8000 to let GET response show you that uploaded one.

The sample project is uploaded, can download from http://www.mediafire.com/?1okv3lc6nk1jsc7.

Leafa.jpg

References:

1. Encode an Image into base64 string in C# / Python
2. Simple HTTP POST Server / Client with Python
3. HTTP POST data via WebClient and WebRequest