Veerapat's IT journey: November 2012

Friday, November 30, 2012

Simple HTTP POST Server / Client with Python

Continue from the last post, it's about simple HTTP server with Python. Today, I will explain more about create POST server with Python. From last time to response GET request, we need to override do_GET in class SimpleHTTPRequestHandler. Similarly, to response POST request, we just override do_POST method. However, to receive parameters from POST there are 2 popular formats: url-encode, and multipart as I described in my previous post.

In this post I will describe only url-encode. It's like encrypt the data in url format before sending to the server. So, your data will be like "{key1}={value1}&{key2}={value2} ...:". Basically, it's not that difficult with Python because all we need, are already had.

HTTP POST Client:

import urllib, urllib2

uri = 'http://localhost:8000'
name = "Mark"
# Encode data in base64 string
encode = name.encode('base64')
params = { 'name' : encode }
# Pack key-value pairs in form of url-encode format
data = urllib.urlencode(params)

# urlopen with data will use POST method by default
p = urllib2.urlopen(uri, data)
print p.read()

HTTP POST Server:

import SimpleHTTPServer
import SocketServer
import cgi
from base64 import decodestring

# Server port
PORT = 8000

class ServerHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
  
        def do_POST(self):
                # Use cgi module to retrieve data from POST as a form
                form = cgi.FieldStorage(
                        fp=self.rfile,
                        headers=self.headers,
                        environ={'REQUEST_METHOD':'POST',
                                 'CONTENT_TYPE':self.headers['Content-Type'],
                                 })
                # We can get value from the form key like we did in dictionary class
                encode = form['name'].value
                # Decide the value from base64 string
                decode = decodestring(encode)
                self.wfile.write('Hello, ' + decode + '.')
                
Handler = ServerHandler

# Initialize server object
httpd = SocketServer.TCPServer(("", PORT), Handler)

print "serving at port", PORT
httpd.serve_forever()

You will get the message "Hello, Mark." in return.

References:

Wednesday, November 28, 2012

Simple HTTP Server with Python

To create HTTP server in Python, we need to bind to a port & IP address, listen to that port for responding requests. To do that, we can use built-in module named BaseHTTPServer to handle HTTP requests. The main methods are do_GET and do_HEAD, we will need them to construct our header and body.

Here is a server code:

import SimpleHTTPServer
import SimpleHTTPServer
import SocketServer

# Server port
PORT = 8000

class ServerHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):

 def _writeheaders(self):
  self.send_response(200)
  self.send_header('Content-type', 'text/html')
  self.end_headers()

 def do_GET(self):
  # Handle GET request
  self._writeheaders()
  self.wfile.write("""
  <html><head><title>Simple Server with Python</title></head>
   <body>
    Hello World!!!!
   </body>
  </html>""")

Handler = ServerHandler

# Initialize server object
httpd = SocketServer.TCPServer(("", PORT), Handler)

print "serving at port", PORT
httpd.serve_forever()

For handling POST request, just create do_POST method in ServerHandler class. I will explain more in the next post. For this post, just want to ground the basic HTTP server knowledge.

References:

Nokia Lumia 920 Test Drive - 25-27 Nov 2012

In Thailand, Nokia open the campaign Test Drive that allows people to book and play Nokia Lumia 920 for 3 days. It's an amazing product. If you're looking for a new phone within 22k baht with full features and services. Lumia 920 is one of smart choice. It has a nice color: grey, white, red, yellow. For my first touch, I feel that it is a hi-end product and designed for human. It looks natural, and solid. I don't need any cover and screen protector at all, if I do, I feel like I'm hiding the coolest thing in Lumia 920. I have tried to test in many real usage situations to test user experience when using Lumia 920.

First thing that impressed me is the display. With Pure Motion HD+ technology and IPS, the display is very seamless to real world compared to the real objects. For me, the first sight I saw the tile screen in demo phone. I had thought that it is a mock up phone, but, actually it is a real phone!!! It is a kind of shocked, I can't distinguish between real and mock up phones. Moreover, it has a large display, you can read e-books from kindle app comfortably with amazing display.

Second test is battery, Nokia Lumia 920 comes with 2000 mAh. I opened 3G all day, and play the phone only movement (morning, noon, evening). It still remained 50%. So, I think Nokia provides battery size for using it at least one day without charging. However, If you manage wisely, I think it can continue to the end of a next day too.

Third test is sound, Nokia Lumia 920 speaker provide acceptable and loud sound but it's not good like dedicate speaker - -*. Compared to my phone (HTC One X), Lumia 920 has a louder voice. Then, I compared them with the same earphones, in my opinion, they provide the same level of voice and sound :D

Fourth test is mobility, it is a kind of big for me to bring it into a pocket properly. Also, Lumia 920 is heavier than my One X. So, it's a bit difficult to keep into a pocket for some trousers. My solution is hold it on my hand, and play during walk.

Fifth test is apps, productivity, and integration. I think Windows Phone is one of platforms which is able to let me feel comfortable and convenience to work within a few touches, compared to Android it's a bit messy like Joe Belfiore said. For me, I only concern about well-known apps e.g. foursquare, Kindle, map, Facebook, Twitter. Those are contained in Windows Phone store. So, the number of application is not a point. My point is the quality of these apps are still buggy such as Facebook notification doesn't work well as Android and iOS do. However, it has a XBox music and other interesting services included in Windows Phone features.

Sixth test is phone function. For a Nokia, they don't forget to bring the quality of basic phone features. Microphone has less noise even using the speaker mode. It also provides a good quality of respond sound especially when using a small talk.

The last test is camera, like Nokia claims, Lumia 920 provides fantastic features to take a shot such as low light - this one is very cool, however it's a bit brighter than eyes see. The panorama app is still not good, my short is blur.

In conclusion, I think Nokia Lumia 920 can be one of the best smart phones in this year, it depends on could Microsoft make Windows Phone OS more complete, and be able to fight others OSs e.g. Apps to play with, etc.

Example pictures taken by Lumia 920:

Indoor

Outdoor with movement objects

Outdoor normal shot

Focus object

Night

References: Nokia Lumia 920 detailed specifications
Other Review: Blognone (TH)

Monday, November 26, 2012

HTTP POST data via WebClient and WebRequest

To request HTTP with POST method in .NET, there are 2 classes applicable to do that: WebClient, and WebRequest. I will describe both ways in post. Normally to send the data back via POST method, there are 2 kinds of content-type (that I know) are url-encode, multipart. These two content types have an effect for the specific format for a receiver and how to construct a form message. In this post I will explain only url-encode. Firstly, we should know how to construct a format. For urlencode format, the format is very similar to querystring format like "{key1}={value1}&{key2}={value2}...". Moreover, we should encode the values like a GET parameter string (not necessary but for some characters, they need to).

To specific content-type for urlencode is set its value to "application/x-www-form-urlencoded".

Example 1: using WebClient

using System.Web;
...
private void wc_UploadStringCompleted(object sender, UploadStringCompletedEventArgs e)
{
     // Show a return message when upload is completed
     var result = (string)e.Result;
     MessageBox.Show(result);
}

private void Do_Something()
{
      string name = "Hello";
      string url = "<host ip>";
      var base64 = "name=" + HttpUtility.UrlEncode(result);

      var wc = new WebClient();
      wc.Headers["Content-Type"] = "application/x-www-form-urlencoded";
      wc.UploadStringCompleted += wc_UploadStringCompleted;
      wc.UploadStringAsync(new Uri(url), "POST", base64);
}

Example 2: using WebRequest

using System.Net;
...
private void Do_Something()
{
      string name = "Hello";
      string url = "<host ip>";
      var base64 = "name=" + HttpUtility.UrlEncode(result);

      var encode = Encoding.UTF8.GetBytes(base64);

      var request = WebRequest.Create(new Uri(url));
      request.Method = "POST";
      request.ContentType = "application/x-www-form-urlencoded";
      request.ContentLength = encode.Length;
      Stream dataStream = request.GetRequestStream();
      dataStream.Write(encode, 0, encode.Length);
      dataStream.Close();

      using (var response = request.GetResponse())
      {
           using (var reader = new StreamReader(response.GetResponseStream()))
           {
                // Show a response message
                string responseText = reader.ReadToEnd();
                MessageBox.Show(responseText );
            }
      }
}

The different from these 2 methods in my opinion are details for handling data i.e. We need to create delegate function for handling task when uploading is done, but WebRequest doesn't need to. Moreoever, WebClient has higher level of data sending compared to WebRequest. WebRequest need to pack data into byte array, but WebClient can send them as a pack of string directly.

References:

Friday, November 23, 2012

What the different between Web Browser, Web Client, WebRequest, WebResponse

I found the answer from stack overflow. He said as the following description:

WebBrowser is actually in the System.Windows.Forms namespace and is a visual control that you can add to a form. It is primarily a wrapper around the Internet Explorer browser (MSHTML). It allows you to easily display and interact programmatically with a web page. You call the Navigate method passing a web URL, wait for it to complete downloading and display and then interact with the page using the object model it provides.

HttpWebRequest is a concrete class that allows you to request in code any sort of file over HTTP. You usually receive it as a stream of bytes. What you do with it after that is up to your application.

HttpWebResponse allows you to process the response from a web server that was previously requested using HttpWebRequest.

WebRequest and WebResponse are the abstract base classes that the HttpWebRequest and HttpWebResponse inherit from. You can't create these directly. Other classes that inherit from these include Ftp and File classes.

WebClient I have always seen as a nice helper class that provides simpler ways to, for example, download or upload a file from a web url. (eg DownloadFile and DownloadString methods). I have heard that it actually uses HttpWebRequest / HttpWebResponse behind the scenes for certain methods.

If you need more fine grained control over web requests and responses, HttpWebRequest / HttpWebResponse are probably the way to go. Otherwise WebClient is generally simpler and will do the job.

In my opinion, I prefer to use WebClient because it's much more higher level, and easy to use in case we need to specific setting parameters. However, if I only use it for crawl data, mostly I just use WebRequest because it's short, and simple don't need to have delegate for async function.

Reference: http://stackoverflow.com/questions/1780679/net-webbrowser-webclient-webrequest-httpwebrequest-argh

Convert string to byte array and byte array to string (C#)

To convert from string into byte array, we can use Encoding class to convert into specific format such as ASCII or UTF8.

Example:

using System.Text;
...
string str = "Hello, World!!!";
byte[] byteData = Encoding.UTF8.GetBytes(str);

On the opposite way, to convert back from byte array into string, we can simply use StreamReader to read it from MemoryStream.

Example:

string output = string.Empty;
byte[] byteData = ...;
using (var memoryStream = new MemoryStream(byteData))
{
     using (var streamReader = new StreamReader(memoryStream))
     {
          output = streamReader.readToEnd();
     }
}

What is MemoryStream, and its purpose (C#)

MemoryStream represents a pure data in memory as a stream of data. It is useful to use with other types such as BinaryReader to improve performance. It can be constructed from array of bytes. To use MemoryStream, it will store in a memory which is faster than disk or network accesses, and use it as temporary data on memory.

Reference: http://www.dotnetperls.com/memorystream

Wednesday, November 21, 2012

Design Patterns in .NET

Take a look at the following links:

Thursday, November 15, 2012

HTML Element addEventListener

Take a look at the following links:

Wednesday, November 14, 2012

Using the aria-labelledby attribute

The aria-labelledby attribute can be used for indicating label for an object which can be a list of id (separated by space). It is useful for rich internet applications to describe objects. There are 2 sources, and 1 example websites that may helpful.

Monday, November 5, 2012

Crawl HTTP 403 Forbidden Error page (may solve)

Sometime, to crawl specific page, the web servers may block you crawl their contents using a simple method by looking at UserAgent header request from a client request. In well-known web clients like IE, FireFox, Chrome, Opera, those clients are trusted by anyone, so servers will allows them to access the contents. Specific programs, however, won't be allowed to do that just because they don't have signature for it. To solve this program, you can add UserAgent header into that program using signature from well-known web clients. In case that servers block for specific clients to prevent some bots that don't have UserAgent header, This way is the easiest solution.

The following code is an example of adding UserAgent header in C#:

HttpWebRequest request = (HttpWebRequest)WebRequest.CreateDefault(uri);
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5";

And this is an example code for Python:

import urllib2
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
ob = opener.open('http://www.google.com/')
print ob.read()

Veerapat's IT journey

Pages

Profile