Featured | Leadership | Web Dev (.NET, HTML, etc)

ASP.NET Website Performance Basics – Part 2

Challenge:

Recap : In last part of ASP.NET Website performance basics, Dax shared basic methodology of troubleshooting ASP.NET Website performance challenges!
Where he shared, what all things we need to do, and why we should do it (Remember W-W-H theory?). And just pointers of How to do it! And as Dax promised, that he will help us to understand how part more [We all know, how well Dax keeps his promises! :)] and here he is.
In this post [Would also name it “ASP.NET Basics”], we will look at following things (Which are very IMP. to diagnose performance issues):

  1. When you open any website from your browser, what all happens, behind the scenes? — Not related to ASP.NET just basics, how it reaches till your ASP.NET Application
  2. Now, we’ve reached to ASP.NET Application, after that we will look at — ASP.NET Application LifeCycle — All about — HttpApplication,HttpContext,HttpRuntime, HttpModules etc.
  3. Request Queuing
  4. Major Performance killers ( Hanging Requests, Session can slowdown your application — Thread Agility Issue, Exceptions and performance co-relation, MaxConnections, High GC, Contentions, App Pool Recycle, Memory leak)

Above topics sounds similar? Very Basic? (Back to Basics, Stick to basics. Because a basic always works!) But still unknown? (During our interview sessions, we ask these questions to most of our candidate, And so less can answer it)Don’t worry, we will learn it, So, as Dax and his team!
Here we go!

Solution:

When you open any website from your browser, what all happens, behind the scenes?

You might know this question or learnt it during your college days. And If you are pretty much confident you know this you can skip it. But Dax recommends that even though you know it. It would be a good idea to brush up this stuff.
Because while working on performance stuff, you need to know about everything starting from Client’s Browser, DNS, Firewall, Switch — You no need to be expert in this subject. But when you troubleshoot it, You should know all these players very well.
My favorite — one is from MCTS 70-528 exam book – Chapter1-Lesson1 – Understanding the PlayersMajor-Players
Another good articles:
http://www.codeproject.com/Articles/121096/Web-Server-and-ASP-NET-Application-Life-Cycle-in-D
http://www.codeproject.com/Articles/73728/ASP-NET-Application-and-Page-Life-Cycle

Good to remember about M-H-P-M

http://www.codeproject.com/Articles/87316/A-walkthrough-to-Application-State
Thanks a lot to author of above articles. It clarifies everything. So, now you know what all happens — from browser till server — a request goes through a journey and then gets served!
Here you must’ve seen about HttpApplication object, this guy is very critical while serving your requests.  HttpApplication consists of HttpModules.
index
 
http://blog.andreloker.de/post/2008/05/HttpApplication-instances.aspx
HttpApplication Gotchas:

  1. ASP.NET may instantiate many instances of HttpApplication as required (. In fact, it will create an instance of the class for each request that is handled in parallel on the server.
  2. “ASP.NET maintains a pool of HttpApplication instances over the course of a Web application’s lifetime. ASP.NET automatically assigns one of these instances to process each incoming HTTP request that is received by the application. The particular HttpApplication instance assigned is responsible for managing the entire lifetime of the request and is reused only after the request has been completed. This means that user code within the HttpApplication does not need to be reentrant.”
  3. You can observe asp.net pipeline instance count in the performance counters to see how many instances of your HttpApplication class are pooled at the moment.
  4. In Integrated Application mode — HttpApplication will called once during application initialization and another during the first request
  5. When ASP.NET Creates Instance of the HttpApplication class that represents your application, instance of any odules that have been registered are created. When a Module is created, its Init method is called and the module initializes itself. — and the custom module will run for all resource handler, even though resource handler is not an ASP.NET handler
  6. If you see More number of requests [You can check it via IIS log or performance counter] and Heavy HttpApplication Instances is Normal, Because it means that current HttpApplication Pool was not enough and ASP.NET need to spawn worker processes.
  7. But if you see less number of requests, and still see heavy HttpApplication instances then it is abnormal. We need to find out slow pages/handlers etc.
  8. ASP.NET run-time keeps two pools of HttpApplication objects. First is a special pool with HttpApplication objects used for application level events (such as Application_Start, Application_End), Second pool contains instances used in requests to serve all other types of events
  9. Try this out — http://lowleveldesign.wordpress.com/2011/07/20/global-asax-in-asp-net/
  10. HttpApplication Events

HttpApplication-Events
Other good reads:

Huh! Load of things, correct? Useful? Dax says these information is really useful and it will clarify your understanding more on ASP.NET internals!

Request Queuing

It is really good to know about Request Queuing, lot of us already know about Request Queuing — Simple, a Request is queuing. Yes, my dear friend you are right. But there are different level of Queuing happens. And untill and unless you know your request is queued at which level, it won’t be easy to fix it!
Great post! — http://blog.leansentry.com/2013/07/all-about-iis-asp-net-request-queues/
Basically, Request Queuing can happen at mainly 4 levels:

  1. HTTP.SYS: Application pool queue
  2. IIS worker process: completion port
  3. ASP.NET: CLR threadpool queue
  4. ASP.NET: Integrated mode global queue
  5. [OR]ASP.NET: Classic mode application queue

Request_Timeline2 Req-Q
Source — http://fullsocrates.wordpress.com/2013/02/28/asp-net-threadstuning-thread-parameters-12-2/
To diagnose, your level of Queuing. Best thing is performance counters [Dax, is going to share more on performance counters in his upcoming posts] and once found you can use tools like FRT, Dump analysis etc. to find a main bottleneck [Yes, Yes — Dax will post on these topics in future for you!]

Major Performance Killers

  1. Application Pool Recycle/Crash — If your application pool is crashing is or recycling [How Can I check that? You can check EventLog OR if your application got log do log entry [Just a note : Application_End will not get called, when your application crashed unexpectedly a.k.a. Crash] — see — http://weblogs.asp.net/scottgu/archive/2005/12/14/433194.aspx] periodically than it’s not good for your application’s health. Because when application pool gets recycled everything in memory [Session/Cache/Static/Application objects] gets lost, and your application need to server everything from Database and rebuild cache. You can solve this issue by checking EventLog and Configuring+Analyzing Crash Dump.http://kiranpatils.wordpress.com/2012/06/20/is-it-necessary-to-recycle-worker-process-periodically/ and http://kiranpatils.wordpress.com/2010/05/25/goo-to-know-on-asp-net-application-restarts/
  2. Hanging Requests — Your users are complaining that the site is loading slowly. Requests to your application are hanging. With so many possible causes of request hangs, its difficult to know where to even start. — http://mvolo.com/troubleshoot-iis-hanging-requests/
  3. Exceptions, handle them gracefully — Exceptions can slowdown your application. Because when your application throws an exception — It Creates objects — CLR needs to do object allocation — And after that GC for those allocation! [Analyze your application logs to see exceptions, and handle them gracefully] — http://blogs.msdn.com/b/tess/archive/2005/11/30/498297.aspx
  4. %Time in GC — When this time is high, your requests will get paused for a while and your end users will see slow response time during this time. If it is >=10% then you need to dig in to this issue.  Also, check for Memory Leak — http://kiranpatils.wordpress.com/2012/06/05/basics-of-memory-leak/
  5. Contentions — If your CPU is not being utilized and you still see low throughput, you may be suffering from high contention rate. Means your code has lock on shared resources, which is taking time to process, and due to this other requests are going in Queue. Simple concept in locking — “Acquire lock as late as possible, and release it as soon as possible” http://kiranpatils.wordpress.com/2013/01/13/thread-synchronization-basics/
  6. Session lock – Thread agility issue — Disable session for the pages/handlers, where you don’t need them. https://github.com/angieslist/AL-Redis http://stackoverflow.com/questions/3629709/i-just-discovered-why-all-asp-net-websites-are-slow-and-i-am-trying-to-work-out
    http://stackoverflow.com/questions/8068925/redis-backed-asp-net-sessionstate-provider

Good to read resources:

  1. http://msdn.microsoft.com/en-us/library/ff647787.aspx
  2. Most recommended — https://www.leansentry.com/try#course
  3. http://blogs.msdn.com/b/mcsuksoldev/archive/2011/01/19/common-performance-issues-on-asp-net-web-sites.aspx
  4. http://www.codeproject.com/Articles/23306/10-ASP-NET-Performance-and-Scalability-Secrets
  5. http://blog.leansentry.com/2013/07/the-server-logs-you-need-to-know-to-fix-any-iis-aspnet-error/
  6. http://mvolo.com/fix-the-3-high-cpu-performance-problems-for-iis-aspnet-apps/

Sorry for such a long post. But It’s good to clarify basics. Because once it is clear, you can fix any issue!
These are the common performance killers, In upcoming posts, we will see how to diagnose them! Till than Happy Performance Tunning! 🙂
 

ASP.NET | Web Dev (.NET, HTML, etc)

ASP.NET Website Performance Basics

Challenge:

In a small city, there lived a developer named as Dax. And his boss name was Gabe. Their main goal was to develop a faster running websites and have happy clients! (So, as you!?) And they were best in the town to deliver faster running websites!
One fine morning, they faced a challenge. And this blog post is about how they overcome that challenge.

Disclaimer — All characters in this publication are entirely fictitious and any resemblance to real persons, living or dead, is purely coincidental.

Solution:

Read it from left to right
Basics-1 Basics-22
Gabe asked Dax to fix, their ASP.NET website performance issue and while working on it, Dax made a mistake, which most of us (Including me) make, it is fixing a performance issue by blindly guessing the things along with worse part, making code change which “they think”  it is causing slowness! But this “stab in the dark” method doesn’t work most of the time! [Just a note : I believe that don’t fix any issue, until and unless you’ve reproduced it or understood it thoroughly, If you do it without understanding it, you never know when it will come back to you! But surely, it will!]
Y’day night Dax, sent me an e-mail and asked to share his and his team’s performance learnings with you! [You know, Jesus requested him to share his learnings with you? How honest, Dax is!]
Whatever stuff, you work on, If you follow W (What)- W (Why)- H (How) theory you can solve anything!

What are Website Performance standards?

Have seen lot of people who work on performance issues, without having a measures [Statistics and numbers are your best friends while working on performance! Because if you can’t measure anything, you haven’t improved it yet!]. How fast is really fast? [Carpentry quote : Measure twice, cut once!]
So, here are some numbers, which Dax came across while doing research and reading on website performance standards blogs and articles:
Before, we move on, let’s have a look at, good to know terminologies:

  1. Response time of your server : The time taken, by server to process your request, it includes time taken by Database call, File IO Operation, Time taken by .NET CLR, Request Queuing etc.
  2. TTFB : Time to load first byte, this is the time when browser requests HTML of your page, and server builds your whole page’s HTML to browser, and after that browser starts rendering of your page by requesting other resources like CSS/JS/images etc.
  3. Page Load time : This is the total time taken by your browser to completely load a page.

So, to ensure that your site is performing as per web standards, check following values for your website (Just a note : these values are standard values, it might not fit in to your environment. so, do understand your environment  before you start investigating performance issues):

  • TTFB should be <= 100 Milliseconds; Because if it is high then your users will see a white browser page, and nothing will get loaded until and unless it receives first HTML data from server.
  • Response Time should be <= 2 seconds
  • Start Render Time should be <= 2 seconds
  • Page Load time should be between 2-3 seconds

Now, you must be curious to know, how we can check these values? Going at each layer, and checking logs at each level or there is a smart way? Luckily, there is a smart way available for you! We will have a look at it in “How” section soon!
Good to read:
http://www.webperformancetoday.com/2012/02/13/non-geeky-guide-to-performance-measurement/
http://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/

Why Website Performance matters?

http://www.strangeloopnetworks.com/resources/infographics/web-performance-and-user-expectations/poster-visualizing-web-performance/
http://www.webperformancetoday.com/2012/03/21/neuroscience-page-speed-web-performance/
Above articles nicely explained everything!
In summary, just take a case, that if you got 20 Million visitors per month, and if your website is slow by just 1 second, then 20*1= 20 M seconds gets wasted, which they can spend somewhere else — with their family, friends or If you are running e-commerce, website then they might place multiple orders because of your faster running website! 🙂
And as per research, in this impatient era, where everyone wants everything to be done instantly  [5G!], If your site is running slow, then people will not complaint you about it. But they will go to your competitor.

How to Improve Website Performance?

Now, how to measure, these values, there are smart tools available, which you can use to know your website’s performance values, we will categorize it in two categories, Server-Side and Client-Side:
Performance problem can be divided mainly in two categories:

  1. Serve-Side : If your server is not serving requests faster, then your website is suffering from server-side/back-end performance issues. Server-Side performance issues, are really very tough to find. Because in most of the cases, you will not have a specific scenario to reproduce, You will not have Visual Studio on live server, where you can debug your code! :-). But thanks to APM tools, Performance counters, Log4Net, DebugDiag and WinDbg – these are the tools who will help you to fix your back-end issues!
  2. Client-Side : If your website is not using minify/merge than it falls in client-side/front-end performance issues, Because server has sent response rapidly. But client [Browser] is taking time to render a page, and it happens if you are not following performance best practices while building your website. It is also known as “Front-End” time. And as per stats. “80-90% time goes in front-end!”. So, If you’ve faster servers, But if your front-end is not following performance rules, then your site will never get loaded faster!

Server-Side performance testing tools:

  1. APM  Tools– APM stands for Application performance management, there is lot of APM tools available in market. You need to install their lightweight agent on your server, that’s it! It will report you everything via a nice web-based dashboard! But Dax’s favorite is, and tried one is New Relic [It’s costly. But worth! — Indeed “A web & mobile Developer’s Best Friend”]:  http://newrelic.com/
  2. Profiling – You can run profiling tool on your website, mostly in local, this tool will help you to find performance issues. It mostly helps when you’ve specific scenario to reproduce. For unknown scenarios it doesn’t help much, again Dax’s favorite is ANTS Performance Profiler from Rad Gate — http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/
  3. Performance Counters – “Performance counters is your computers Google” Nice quote from Performance webinar, Just add performance counters [Which one to add, how to analyze that something we will see in upcoming posts]
  4. DebugDiag and WinDbg — Using performance counter, If you found that there is a memory leak in your application or when Request Queuing goes high your application goes slow. And now, you would like to see what is there in heap when such thing happens — then DebugDiag and WinDbg are your friends! — http://blogs.msdn.com/b/tess/archive/2008/02/04/net-debugging-demos-information-and-setup-instructions.aspx

Have Dax missed anything? Just feel free to comment!
Client-Side performance testing tools:

  1. WebPageTest.Org —  www.webpagetest.org
  2. Pingdom Tools — http://tools.pingdom.com/
  3. Google PageSpeed Tools — https://developers.google.com/speed/pagespeed/
  4. Browser based tools — YSlow, HttpWatch, Firebug

Have Dax missed anything? Just feel free to comment!
Good to read:
http://yslow.org/user-guide/
http://www.webperformancetoday.com/2010/07/09/waterfalls-101/
http://www.stevesouders.com/blog/2012/10/09/webperfdays-performance-tools/
http://blog.newrelic.com/2012/09/04/improving-site-performance-with-yslow/
Still this point of time, we’ve seen how to delve in to performance issues more and find out, where you need to focus on server-side or client-side or BOTH.
Now, let’s have a look at “How” to solve them

  1. Client-side/Front-End site performance issues are relatively easy to find and fix! Just use any tool, follow the given rules, re-test and you are done!
  2. Server-side/back-end performance issues are relatively complex to find out, and once you found it fixing them might be easy! But finding back-end issue involves following level of understanding:
    1. Basics of ASP.NET — You must be saying this guy is crazy, come on, we are ASP.NET Professional and we’ve lot of years of experience, how ASP.NET works, we’ve developed and deployed lot of websites, And you are asking us to learn ASP.NET Basics? I agree with your thoughts, and I know you guys are experts in this field. But ASP.NET is a big topic, and when we develop something we work on more abstract layer, just drag and drop control. add some code behind stuff and we are done! But to work on performance stuff, you need to know lot of ASP.Net Internals, Like How ASP.NET spawns worker threads? What is HttpApplication? What is HttpModule? What is co-relation between them? When HttpApplication instance gets created? How many requests your ASP.NET application can handle? What is Request Queuing? What is Request Queued? What is Contention? You know Thread agility issue? Lot of questions correct, bouncer? Don’t worry Dax also had same questions. But after investing sometime he learnt it, and he will share those learnings with you in upcoming posts!
    2. Basics of Performance counters – Counters are your best friends! Just do learn some basics of performance counter, introduce yourself to them and vice-versa! These guys will tell you lot of secrets! (So, as your best friend!)
    3. Basics of IIS Tools : IIS got plenty of nice tools, like FRT [Failed Request Tracing] which will help you to see what is going on?
    4. Basics of Performance Testing Tools : Before fixing anything on live server, do measure it locally, and you will need performance testing tools like JMeter,WCAT,ACT,AB.exe etc. Keep them handy!
    5. Basics of APM : As discussed earlier, good to have APM tool in your armory!
    6. Basic of Dump analysis : When no one helps you, and disappoints you. Dump is a light of hope, and it won’t disappoint you for sure!

Will expand these topics in upcoming posts! So, just keep visiting, Keep reading, And if you’ve good explanation of any topic listed here, do share with us! So, as Dax! 🙂

Good to read links:

http://www.webperformancetoday.com/2013/01/31/web-performance-101-developers/
http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/entrypage/avoid-find-fix-asp-problems
http://www.softdevtube.com/2013/01/15/ten-web-performance-tuning-tricks-in-60-minutes/
http://stackoverflow.com/questions/4310719/asp-net-processmodel-configuration
http://www.guidanceshare.com/wiki/ASP.NET_2.0_Performance_Guidelines_-_Threading
http://www.aspnet101.com/2010/03/50-tips-to-boost-asp-net-performance-part-i/
Now, Dax and Gabe are really happy and they are enjoying their time! I’m sure they will keep sharing such things with us!
Have a happy faster running websites! 🙂

1
.NET | ASP.NET | Web Dev (.NET, HTML, etc)

Fast and furious way to LoadXML document

Challenge:

While loading XMLDocument using LoadXML method we found that sometimes it takes a long time to load xml. Is is a same challenge you are facing? Then this post is for you!

Solution:

http://codinglight.blogspot.in/2009/06/how-to-load-xmldocument-and-completely.html
This article helped us to do so! Basically, when you hit LoadXml Method it, internally validates XML by loading it’s DTD — and if your DTD server is busy it may take time to load.
So, the solution (If it is not breaking your functionality) is set XmlReaderSetting’s — Set XmlResolver to null, and ProhibitDtd to false.
That’s it!
Happy XML Loading! 🙂