Scalability tests

Topics: User Forum
Jan 18, 2007 at 7:46 PM
Anyone done any stress tests when they use this factory in their web application?

Any kind of feedback on any problems w/ enterprise level web applications?

Thanks
Coordinator
Jan 20, 2007 at 3:55 AM
We realize that performance of a web application is a critical non-functional requirement. As part of the development process of the factory, we performed multiple scaleout, scaleup, stress and performance tests. We made changes to the code based upon the results of the tests.

At this point we do not have results in a format that can be shared with the community. Early next week we will look at the results and determine what is needed to put the results in a format that can be shared with the community and how much time it will take.
Jan 22, 2007 at 2:27 PM
Howdy,
Thanks for the response. I am really looking forward to the results because i would like to know how this factory will perform in an enterprise wide scenario and scalability is one of the top questions asked.

Also, blainew are you in dallas?

Thanks
Jun 29, 2007 at 2:21 PM
blainew,

Have these numbers been posted yet? I would very much like to see them.

Thanks
Jul 17, 2007 at 4:11 AM
Edited Jul 17, 2007 at 9:45 PM
We haven't posted the results yet and are still looking into this. What we can say is that our testing showed that CWAB in itself adds very little overhead above ASP.NET (around 5%). PageFlow on the other hand did impact the number of responses more significantly, though this was not due to overhead added by PageFlow itself but rather due to PageFlow's use of Windows Workflow Foundation. That being said we have some very large customers that are using PageFlow and WorkFlow in their web environments. All of the results are based on the scenarios we have looked at and may or may not apply to your particular needs so we strongly recommend you do your own internal testing.

Regards
Glenn
Oct 10, 2007 at 9:30 PM
I have found that the pageflows use of WWF has been exteremely slow, especially starting and stopping and serialization of objects so they are persisted in the workflow database. Is there anything that we can do to improve the performance?

Thanks,
Gavin
Coordinator
Oct 15, 2007 at 5:54 PM
To second what Blaine said, we did a lot of perf tuning on the Page Flow engine to get it to perform at its current level. Believe it or not, we increased performance by an order of magnitude (or more) from the initial version. Without these changes, we would not have released it.

To answer gav_bar's question: You could write one of the following:
  • An alternative Page Flow provider that does not use Workflow Foundation. This is relatively simple, as the Page Flow interfaces are fairly simple.
  • An in-memory workflow persistence store for Workflow Foundation (they do provide an interface for their persistence store).
  • If neither of the above are doable, look for something that will better fit your performance and scalability needs. (I know, this is odd coming form someone on the project team, but this may be a case of a square peg and a round hole for your situation).

Michael Puleio - patterns & practices
Webhttp://msdn.microsoft.com/practices/
Bloghttp://blogs.msdn.com/mpuleio/
Dec 21, 2007 at 9:43 AM
I read that the following items can have significant impact on performance:

Runtime: EnablePerformanceCounters (default true) and ValidateOnCreate (defaults to true)
TransactionService: EnableRetries
SchedulerService: MaxSimultaneousWorkflows
PersistenceService: EnableRetries, LoadIntervalSeconds, OwnershipTimeoutSeconds
TrackingService: EnableRetries, IsTransactional (has nothing to do with real transactions, but will batch up tracking database writes), UseDefaultProfile (default profile will track all workflow level events and activity level events), PartitionOnCompletion
DisableWorkflowDebugging

Now, I can set DisableWorkflowDebugging by doing this:
<system.diagnostics>
<switches>
<add name="DisableWorkflowDebugging" value="true"/>
</switches>
</system.diagnostics>

How can I set the other switches?
For example ValidateOnCreate could be set like this:
<workflowRuntime name="WorkflowServiceHostRuntime" validateOnCreate="false" enablePerformanceCounters="false">
But WCSF does not create a section like that in our web.config...

Is this worth investigating or is the pageflow implementation already tuned with this?

Regards,

Mike
Dec 21, 2007 at 9:44 AM
Edited Dec 21, 2007 at 9:45 AM
I read that the following items can have significant impact on performance:

Runtime: EnablePerformanceCounters (default true) and ValidateOnCreate (defaults to true)
TransactionService: EnableRetries
SchedulerService: MaxSimultaneousWorkflows
PersistenceService: EnableRetries, LoadIntervalSeconds, OwnershipTimeoutSeconds
TrackingService: EnableRetries, IsTransactional (has nothing to do with real transactions, but will batch up tracking database writes), UseDefaultProfile (default profile will track all workflow level events and activity level events), PartitionOnCompletion
DisableWorkflowDebugging

Now, I can set DisableWorkflowDebugging by doing this:
<system.diagnostics>
<switches>
<add name="DisableWorkflowDebugging" value="true"/>
</switches>
</system.diagnostics>

How can I set the other switches?
For example ValidateOnCreate could be set like this:
<workflowRuntime name="WorkflowServiceHostRuntime" validateOnCreate="false" enablePerformanceCounters="false">
But WCSF does not create a section like that in our web.config...

Is this worth investigating or is the pageflow implementation already tuned with this?

Regards,

Mike
sorry for the doublepost, I can't find a delete button
Dec 24, 2007 at 3:39 PM
Hi Michael / Mike,

Thanks for the info, we tried several approaches to improving performance in the persistence of session data:

  • Changing the persistence property on the workflow state to discard.
  • Storing session state data in memory only
  • Persisting of session state data using ASP.NET serialization and SQL server 2005 as the data store

We found that the most gains were made by using no persistence at all for session data as you would expect. Using Persistence using ASP.NET serialization then a marginal difference in performance was noticed, however the difference between using Workflow Persistence and ASP.NET in built persistence was in the order of seconds not milliseconds. Changing the persistence property made no real difference.

We are now using Pageflow to control our page flow only and in built ASP.NET Session object for persistence of session data. We hope that future versions of the WebSF will not have performance issues that we have seen, therefore enabling us to move back to using Pageflow to persist our data.

On a separate note we have also found issues with the Discard persistence property on the PageStates. Even though we set the state to Discard, the workflow session object is still stored as running/active in the database. To overcome this we have had to put code in place that will terminate workflows.

We'll try further tuning the configuration to see if it improves performance.

Thanks,
Gavin