Profile

Click to view full profile
Hi, I'm Veerapat Sriarunrungrueang, an expert in technology field, especially full stack web development and performance testing.This is my coding diary. I usually develop and keep code snippets or some tricks, and update to this diary when I have time. Nowadays, I've been giving counsel to many well-known firms in Thailand.
view more...
Showing posts with label Performance Tracing. Show all posts
Showing posts with label Performance Tracing. Show all posts

Wednesday, July 9, 2014

Run a set of Visual Studio load tests at once using Command Line

Normally, if you have only one load test environment and have a bunch of load tests that need to run, you need to wait for each test finished then start to run the next test manually.

In order to run them all at one time, we can use Visual Studio Command Prompt to run load tests. But we need to prepare .loadtest files for each load test run separately. It means that if you want to run 15 load tests, you need 15 of .loadtest files.

A command to execute load test has shown as follows:
mstest /TestContainer:[Load Test Name].loadtest /resultsfile:[Load Test Result Location].trx
Then, to run a next load test after another finished, create a command prompt script to run them automatically.

For example - create a loadtest.cmd then add the following lines into the script:
mstest /TestContainer:TestA.loadtest /resultsfile:C:\TestA.trx
mstest /TestContainer:TestB.loadtest /resultsfile:C:\TestB.trx
In case run a distributed load test using test controller and agents from the command-line, it need to add "/testsettings:[Test Setting Name].Testsettings" after specify result file in a command.

For more information about load test, you can take a look at http://msdn.microsoft.com/en-us/library/ms182588(v=vs.110).aspx.

Thursday, October 24, 2013

Web Test Recorder on Internet Explorer 11

Recently, I have got a problem about using Web Test Recorder after upgrading to Windows 8.1. It has come with the new version of internet explorer i.e. version 11. That doesn't compatible with current Web Test Recorder version (we have to wait until new release to support IE11). However, there is a workaround for this problem. Try to disable Enhanced Protected Mode, then it should work. If not, follow this link: http://blogs.msdn.com/b/visualstudioalm/archive/2013/09/16/using-internet-explorer-11-and-not-able-to-record-a-web-performance-test-successfully.aspx. It contains information to disable something new which doesn't support on Web Test Recorder.

Friday, March 22, 2013

Windows Performance Analyzer Series

There are tools for Windows that can be used to visualize the trace files from ETW. It is Windows Performance Analyzer (WPA) which is a part of Windows Performance Toolkit.
These links below are the WPA series from channel9:
It is really suit for who new to performance tracing to know how to analyze data using WPA.

Monday, March 18, 2013

How to measure UI Rendering time and FPS for Windows Store Apps (C# / XAML)

To measure performance for Windows Store Apps, if you have no experience before, I would recommend you to look at this session first: http://channel9.msdn.com/Events/Build/2012/4-103. He will explain the concepts to capture Windows Store Apps performance tips.

In short, XAML Operations are broken down into frames, and follows these 3 main steps:
1. Parsing: parse XAML for any objects
2. Layout: put objects, and determine their layout
3. Rendering: draw visible objects

To capture UI rendering time easier, think about the followings things:
1. Scenario: understand and know which parts that want to trace. Try to break down into atomic parts, it will be more clear for design scenario steps.
2. Steps of the trace scenario: it will notice us where our trace begins or ends in the trace file.
3. If in the same scenario contains sub parts such as loading items then scrolling down to the last items, try to make a self-marker (known by yourself) to separate sub tasks clearly.
4. When draw objects don't have to be re-draw, Frame will not be produced. Mouse interaction will produce frames only it trigger events on specific objects such as mouse over on objects then that object will have hover animation (re-draw frames).

Here is a command to capture UI Frame behavior:
@if %_echo%!==! echo off

set UserOptions=-BufferSize 1024 -MinBuffers 32 -MaxBuffers 32 -Buffering
set KernelOptions=-BufferSize 1024 -MinBuffers 64 -MaxBuffers 64 -Buffering

xperf -start xaml_user -on "Microsoft-Windows-XAML::5"+perftrack %UserOptions%
xperf -on base+cswitch %KernelOptions%

xperf -m BeginTrace
echo Event listeners started, run desired scenario then press any key
pause
xperf -m EndTrace

echo stopping xaml_kernel...
xperf -flush -f xaml_kernel.etl
xperf -stop

echo stopping xaml_user...
xperf -flush xaml_user -f xaml_user.etl
xperf -stop xaml_user

echo merging to xaml_merged.etl...
xperf -merge xaml_user.etl xaml_kernel.etl xaml_merged.etl
The command you see is a xperf command to capture XAML behavior which we can use it to monitor UI Frame Rendering time.

Now, let's see the real example. I will use Kona project in codeplex to demonstrate how to trace UI Rendering time and FPS. You can install and follow the example scenario I given.

Example Scenario: Capture UI rendering time of Bikes Category page, then scroll to the right and left
Steps:
  1. Click at the header Bikes category
     

  1. Waiting for all items are loaded into the page
  2. Scroll page to the right using mouse wheel
     

  1. Scroll page to the left using mouse wheel

Open trace file using WPA (Windows Performance Analyzer)
  1. On the left side select System Activity, then there are more items expanded.
  2. Select Generics events, an analysis tab will be shown.


  1. Select Provider Name Microsoft-Windows-XAML.
  2. Select Task Names that related to the scenario e.g. Frame, PointerDown, PointerUp, PointerLeave, PointerUpdate, PointerWheel.


  1. Right click, and select filter selection.

Analyze trace file
  1. We know that scenario start from the first click at Bikes category header, and page will be navigated after mouse is up then we do long wait for UI rendering. So, we know immediately when it starts and stops.
     

  1. Zoom in, and we know that page is rendered after XAML is parsed. So, the Frame interval followed ParseXaml is the UI rendering frames for drawing objects on the screen (Duration 1.1 seconds).
     

  1. The next sub part is after we do long wait and begin-end using mouse wheel. We can easily look at where PointerWheel start and stop that is the interval for scrolling items to right and left. To measure scrolling, it is difficult to measure in term of finding the longest gap between frames or longest interval because of UI thread will not produce frames when no re-draw objects (need to scroll mouse wheel accurately and continuously), which lead to calculate using FPS instead.


Calculate FPS UI Thread
  1. In some scenarios, they don't create or load any objects just move or animate around the page. In order to measure it and detect UI glitch in overall using FPS (frames per second) is may be a good choice.
  2. From scrolling, zoom in to that part. And open Frame tab to see how many frames in that time range.
     

  1. There are 300 frames during 6.77 seconds, so the scrolling FPS is 44.3.

In conclusion, I still don't know the best way to trace UI glitch or at least find out the root problem if it is glitched yet (to know it is glitched because of what). However, based on the link I gave above, we can adjust the number of elements shown on the screen and remove overdraw elements or animations to improve application performance.

Last but not least, information in this post, I have tried by myself and no guarantee that this is right or wrong. Please read and consider by yourself.