Build Google IG like Ajax Start Page in 7 days using ASP.NET Ajax and .NET 3.0

Introduction

I will show you how I built a start page similar to Google IG in 7 nights using ASP.Net Ajax, .NET 3.0, Linq, DLinq and XLinq. I have logged my day to day development experience in this article and documented all the technical challenges, interesting discoveries and important design & architectural decisions. You will find the implementation quite close to actual Google IG. It has drag & drop enabled widgets, complete personalization of the pages, multi page feature and so on. It's not just a prototype or a sample project. It's a real living and breathing open source start page running at http://www.dropthings.com/ which you can use everyday. You are welcome to participate in the development and make widgets for the project.

 

What is an Web 2.0 Ajax Start page

Start page allows you to build your own homepage by dragging & dropping widgets on the page. You have complete control over what you want to see, where you want to see and how you want to see. The widgets are independent applications which provides you with a set of features like to-do-list, address book, contact list, RSS feed etc. Start pages are also widely known as RSS aggregators or in general term "content aggregators" from variety of web sources. But you can not only read RSS feeds using your start page but also organize your digital life with it. Ajax start pages are one step ahead of old school start pages like My Yahoo by giving you state-of-the-art UI with lots of Javascript effects. They give you a desktop application like look & feel by utilizing Ajax and lots of advanced Javascript & DHTML techniques.

Some of the popular Ajax Start Pages are Pageflakes, Live, Google IG, Netvibes, Protopage, Webwag etc. Among these, Google IG is the simplest one. The one I have built here is something between real Google IG and Pageflakes in terms of Ajax and client side richness. Google IG is mostly web 1.0 style postback model and it's not really that much of Ajax. For example, you see it postback on switching page, adding new modules, changing widget properties etc. But the one I have built here is a lot more Ajax providing rich client side experience close to what you see in Pageflakes.

Features

Build your page by Dragging & Dropping widgets. You can completely personalized the page by putting what you want and where you want. You can add, remove widgets on your page. You can drag & drop them where you like. Close your browser and come back again, you will see the exact setup as you left it. You can use it without registering as long as you like.

 

Once you put a lot of content on your page, you will find one page is not enough. You have the option to use multiple pages. You can create as many pages as you like.  

Widgets

Widgets provide you with an interesting architecture where you can focus on providing the features relevant to the widget and never worry about authentication, authorization, profile, personalization, storage, framework etc. All these are something widgets get for granted from their host. Moreover, you can build widgets independent of the host project. You do not need the whole host web application source code in your local development computer in order to build widgets. Just create a regular ASP.NET 2.0 Web site, create a user control, make it do what it's supposed to do in regular postback model without worrying about javascripts, implement a little interface and you are done! I have tried my best to create an architecture where you need not worry about Ajax and Javascripts at all. Also the architecture allows you to use regular ASP.NET 2.0 controls, Ajax Control Toolkit controls and any extender in ASP.NET Ajax. You also get full server side programming support and utilize .NET 2.0 or 3.0. You can use regular ViewState and store temporary states. You can also use ASP.NET Cache in order to cache data for Widgets. It is far better than what you find in current start pages where you have to build the whole widget using Javascripts and you need to abide by specific API guidelines and strict "no postback" model. Those who have built widgets for current start pages must know what a traumatizing experience widget development really is for them.

Technologies

The client side is built using ASP.NET Ajax RC and Ajax Control Toolkit. Several custom extenders are used to provide the specialized drag & drop feature.

Middle tier is built using Windows Workflow foundation and the data access layer uses Dlinq and SQL Server 2005.

Web Layer

There's basically only one page, the Default.aspx. All the client side feature you see are all available in Default.aspx and in the form of Widgets. We cannot do postback or too much navigation between pages because that would kill the Web 2.0ness. So, all the features must be provided in one page which never post back and does not redirect to other pages.

 

 

The tabs are just simple

and
• inside an UpdatePanel. When you change page title or add new page, it does not post back the whole page because only the UpdatePanel which contains the tab refreshes. Other part of the page remains as it is.

  public UserPageSetup NewUserVisit( )
  {
  var properties = new Dictionary();
  properties.Add("UserName", this._UserName);
  var userSetup = new UserPageSetup();
  properties.Add("UserPageSetup", userSetup);
  WorkflowHelper.ExecuteWorkflow( typeof( NewUserSetupWorkflow ), properties );
  return userSetup;
  }

  Here we pass the UserName (which is basically a Guid for a new user) and we get back a UserPageSetup object which contains the user settings and pages and widgets on first page that is rendered on screen.

  Similarly on second visit, it just loads user's setup by executing UserVisitWorkflow.

  public UserPageSetup LoadUserSetup( )
  {
  var properties = new Dictionary();
  properties.Add("UserName", this._UserName);
  var userSetup = new UserPageSetup();
  properties.Add("UserPageSetup", userSetup);
  WorkflowHelper.ExecuteWorkflow( typeof( UserVisitWorkflow ), properties );
  return userSetup;
  }

  But how about performance? I did some profiling on the overhead of workflow execution and it is very fast for synchronous execution. Here's proof from the log you get in Visual Studio output window:

  334ec662-0e45-4f1c-bf2c-cd3a27014691        Activity: Get User Guid        0.078125
  b030692b-5181-41f9-a0c3-69ce309d9806        Activity: Get User Pages        0.0625
  b030692b-5181-41f9-a0c3-69ce309d9806        Activity: Get User Setting        0.046875
  b030692b-5181-41f9-a0c3-69ce309d9806        Activity: Get Widgets in page: 189        0.0625
  334ec662-0e45-4f1c-bf2c-cd3a27014691        Total: Existing user visit        0.265625

  First four entries are the time taken by individual activities during data access. The time entries here are in seconds and the first four entries represent duration of database operations inside activities. The last one is the total time for running a workflow with 5 activities and some extra code. If you sum up all the individual activity execution time for database operations, it is 0.25 which is just 0.015 sec less than the total execution time. This means, executing the workflow itself takes around 0.015 sec which is almost nothing.

  Data Access using Dlinq

  DLinq is so much fun. It's so amazingly simple to write data access layer that generates really optimized SQL. If you have not used DLinq before, brace for impact!

  When you use DLinq, you just design the database and then use SqlMetal.exe (comes with Linq May CTP) in order to generate a Data Access class which contains all the data access codes and entity classes. Think about the dark age when you had to hand code all entity classes following the database design and hand code data access classes. Whenever your database design changed, you had to modify the entity classes and modify the insert, update, delete, get methods in data access layer. Of course you could use third party ORM tools or use some kind of code generators which generates entity classes from database schema and generates data access layer codes. But do no more, DLinq does it all for you!

  The best thing about DLinq is it can generate something called Projection which contains only the necessary fields and not the whole object. There's no ORM tool or Object Oriented Database library which can do this now because it really needs a custom compiler in order to support this. The benefit of projection is pure performance. You do not SELECT fields which you don't need, nor do you contruct a jumbo object which has all the fields. DLinq only selects the required fields and creates objects which contains only the selected fields.

  Let's see how easy it is to create a new object in database called "Page":

  var db = new DashboardData(ConnectionString);
  var newPage = new Page();
  newPage.UserId = UserId;
  newPage.Title = Title;
  newPage.CreatedDate = DateTime.Now;
  newPage.LastUpdate = DateTime.Now;
  db.Pages.Add(newPage);
  db.SubmitChanges();
  NewPageId = newPage.ID;

  Here, DashboardData is the class which SqlMetal.exe generated.

  Say, you want to change a Page's name:

  var page = db.Pages.Single( p => p.ID == PageId );
  page.Title = PageName;
  db.SubmitChanges();

  Here only one row is selected.

  You can also select a single value:

  var UserGuid = (from u in db.AspnetUsers
  where u.LoweredUserName == UserName && u.ApplicationId == DatabaseHelper.ApplicationGuid
  select u.UserId).Single();

  And here's the Projection I was talking about:

  var users = from u in db.AspnetUsers
  select { UserId = u.UserId, UserName = u.LoweredUserName };
  foreach( var user in users )
  {
  Debug.WriteLine( user.UserName );
  }

  If you want to do some paging like select 20 rows from 100th rows:

  var users = (from u in db.AspnetUsers
  select { UserId = u.UserId, UserName = u.LoweredUserName }).Skip(100).Take(20);
  foreach( var user in users )
  {
  Debug.WriteLine( user.UserName );
  }

  If you are looking for transaction, see how simple it is:

  using( TransactionScope ts = new TransactionScope() )
  {
  List pages = db.Pages.Where( p => p.UserId == oldGuid ).ToList();
  foreach( Page page in pages )
  page.UserId = newGuid;
  // Change setting ownership
  UserSetting setting = db.UserSettings.Single( u => u.UserId == oldGuid );
  db.UserSettings.Remove(setting);
  setting.UserId = newGuid;
  db.UserSettings.Add(setting);
  db.SubmitChanges();
  ts.Complete();
  }

  Unbelievable? Believe it.

  You may have some mixed feelings about DLinq performance. Believe me, it generates exactly the right SQL that I wanted it to do. Use SqlProfiler and see the queries it sends to the database. You might also think all these "var" stuffs sounds like late binding in old COM era. It will not be as fast as strongly typed code or your own hand written super optimal code which does exactly what you want. You will be surprised to know that all these DLinq code actually gets transformed into pure and simple .NET 2.0 IL by the Linq compiler. There's no magic stuff or no additional libraries in order to run these codes in your existing .NET 2.0 project. Unlike many ORM tools, DLinq also does not heavily depend on Reflection.

  Day 1: Building the Widget container using UpdatePanel

  There are two concepts here, one is the Widget Container and the other is the Widget. Widget Container provides the frame which has a header and a body area. The actual widget is loaded in the body area. WidgetContainer is a server control which is dynamically created on the page for each widget instance. Actual Widget is also a server control which is loaded dynamically inside the widget container.

  Each Widget contains several update panels which helps smaller part of the widgets get updated without whole page refresh or whole Widget refresh. For example, the actual widget which is hosted inside the container is loaded inside an UpdatePanel. So, no matter how many times the actual widget postbacks, the whole widget does not postback or whole the column.

  Finding the right combination of UpdatePanel and distribution of Html Elements inside UpdatePanel was difficult. For example, I first put the whole widget inside one UpdatePanel. It worked nicely, there was only one UpdatePanel per widget so the overhead was small. But the problem was with the extenders which are attached with the Html elements inside UpdatePanel. When UpdatePanel refreshes, it removes existing Html Elements and creates new ones. As a result, all the extenders attached to the previous Html elements get lost unless the extenders are also inside the UpdatePanel. Putting extenders inside UpdatePanel means whenever UpdatePanel is refreshed, new instance of extenders are created and initialized. This makes UI experience very slow. You can actually see the slowness visually when you do something on the widget which makes it postback itself.

  So, the final idea was to separate the header area and the body area between multiple UpdatePanel. One UpdatePanel hosts the header area and the other UpdatePanel hosts the actual Widget. This way if you do something on the widget and the widget body refreshes, it does not refresh the header area and the extenders attached to the header does not get lost. The CustomFloatingBehavior extender is attached with the header. So, the extender itself needs to be inside the UpdatePanel. But putting extender inside UpdatePanel means every time the UpdatePanel refreshes, the extender is created and initialized again. This gives poor performance.

  So, the optimal solution so far is, have 2 UpdatePanel per WidgetContainer, one contains contents of the header, not the whole header itself. So, when the header UpdatePanel refreshes, the DIV which contains the whole header does not get recreated as it is outside the UpdatePanel. This way we can put the CustomFloatingBehavior extender outside the UpdatePanel too. Thus the extender can attach with the header container DIV.

  The WidgetContainer is quite simple. It has the header area where the title and the expand/collapse/close buttons are and the body area where the actual Widget is hosted. In the solution, the file "WidgetContainer.ascx" is the WidgetContainer.

  Collapse

  When the page is loaded, for each widget instance, first a widget container is created and then the widget container hosts the actual widget inside it. WidgetContainer works as a gateway between the core framework and the actual Widget and provides convenient API for storing state, or changing the state of the widget like expanding/collapsing etc. WidgetContainer also conveys important messages to the actual widget like when it's collapsed or when it is closed etc.

  protected override void OnInit(EventArgs e)
  {
  base.OnInit(e);
  var widget = LoadControl(this.WidgetInstance.Widget.Url);
  widget.ID = "Widget" + this.WidgetInstance.Id.ToString();
  WidgetBodyPanel.Controls.Add(widget);
  this._WidgetRef = widget as IWidget;
  this._WidgetRef.Init(this);
  }

  Here the widget container first loads the actual widget from the Url provided in Widget definition. Then it puts the widget inside a body panel. It also passes its own reference as IWidgetHost to the actual widget.

  WidgetContainer implements IWidgetHost interface which helps the actual widget to communicate with the framework and the container:

  public interface IWidgetHost
  {
  void SaveState(string state);
  string GetState();
  void Maximize();
  void Minimize();
  void Close();
  bool IsFirstLoad { get; }
  }

  The implementations are quite simple. For example, the IWidgetHost.Minimize collapses the widget body area:

  void IWidgetHost.Minimize()
  {
  DatabaseHelper.Update(this.WidgetInstance, delegate(WidgetInstance i)
  {
  i.Expanded = false;
  });
  this.SetExpandCollapseButtons();
  this._WidgetRef.Minimized();
  WidgetBodyUpdatePanel.Update();
  }

  First we update the WidgetInstance row and then we refresh the UI. The actual widget also gets a callback via the IWidget interface.

  All the functionality of IWidgetHost was easy to implement except the Close one. When Close is called, we need to remove the widget from the page. This means, the WidgetContainer on the page and the WidgetInstance row in the database needs to be removed. Now this is something the WidgetContainer itself cannot do. It needs to be done by the column container which contains WidgetContainer. The Default.aspx is the container of all WidgetContainers. So, whenever Close is called, WidgetContainer raises an event to the Default.aspx and Default.aspx does the actual work for removing the widget and refreshing the column.

  Day 2: Building a custom drag & drop extender and multicolumn drop zone

  Ajax Control Toolkit comes with a DragPanel extender which you can use to provide drag & drop support to panels. It also has a ReorderList control which you can use to provide reordering of items in a single list. Our widgets are basically panels with a header which acts as the drag handle and flows vertically in each column. So, it might be possible that we can create reorder list in each column and use the DragPanel to drag the widgets. But I could not use ReorderList because:

  • The ReorderList strictly uses Html Table to render its items
  • The ReorderList takes Drag Handle template creates a drag handle for each item. We already have drag handle created inside a Widget, so we cannot allow ReorderList to create another drag handle.
  • I need client side callback on drag & drop and reordering of items so that I can make Ajax calls and persist the widget positions.

  Next trouble was with the DragPanel extender. The default implement of Drag & Drop in Ajax Control Toolkit has some problems:

  • When you start dragging, the item becomes absolutely positioned, but when you drop it, it does not become static positioned. There's a small hack needed for restoring the original position to "static".
  • It does not put the dragging item on top of all items. As a result, when you start dragging, you see the item is being dragged below other items which makes the drag get stuck sometimes especially when there's an IFRAME.

  So, I have made a CustomDragDropExtender and a CustomFloatingExtender. CustomDragDropExtender is for the column containers where widgets are placed. It provides the reordering support. It allows any item right under the container to be ordered which are marked with a specific class name. Here's how it works:

  LeftPanel becomes a widget container which allows widgets to be dropped on it and reordered. The DragItemClass attribute on the extender defines the items which can be ordered. This prevents from non-widget Html Divs from getting ordered. Only the DIVs with the class "widget" are ordered. So, say there are 5 DIVs with the class named "widget". It will allow reordering of only these five divs:

  
   ... 
   ... 
   ... 
   ... 
   ... 
  This DIV will not move
  
  

  It also takes a DropCallbackFunction which it calls when a widget is dropped on the container.

  function WidgetDropped( container, item, position )
  {
  var instanceId = parseInt(item.getAttribute("InstanceId"));
  var columnNo = parseInt(container.getAttribute("columnNo"));
  var row = position;
  WidgetService.MoveWidgetInstance( instanceId, columnNo, row );
  }

  This allows me to get the widget which was dropped or reordered, the column and the position. I can then call a web service and asynchronously inform server what just happened. Server updates the position of the widgets according to the new placement.

  Note: I am not doing a postback, instead calling Web service on Drag & Drop. If I do postback, say postback the column UpdatePanel, then the whole column will refresh which gives a poor drag & drop experience. This is why the drag & drop does not refresh any part of the page and silently calls a web service in the background in order to save the position of the dropped widget.

  The Html output contains the column number inside the column DIV as an attribute and each widget DIV contains the widget instance ID. These two IDs help the server identify what the column is and which widget has been moved.

  columnNo="0">
  InstanceId="151" id="WidgetContainer151_Widget" class="widget">

  The additional attributes are generated from the server side.

  Now, making the first extender is really hard. I generally do not openly admit if something was hard for me, so trust me, when I say hard, it is "H A R D". The architecture is just so overwhelming when you start with. But gradually you will grasp the idea and you will surely try hard to appreciate the OOPS style super slow Javascript object model that ASP.NET Ajax provides.

  Day 3: Building data access layer and site load

  It was so easy to build the data access layer using Dlinq. First I designed the database:

  User contains a collection of pages. Each page contains a collection of WidgetInstance. WidgetInstance represents one Widget. Widget table contains the definition of the widget, e.g. Name of the widget and the user control file name which has the code for the widget. WidgetInstance represents an instance of a widget on a column and row of a page. UserSetting stores some user level setting.

  After designing the database, I used SqlMetal.exe and generated the data access class named DashboardData which contains all the entity classes and Dlinq implementations for working with the database. DashboardData inherits from DataContext class which is a base class in System.Data.Dlinq namespace for all data access classes. It has all the methods for insert, update, delete, select, transaction management, connection management etc.

  I also created a convenient DatabaseHelper class which contains convenient methods for Insert, Update and Delete. One of the issue with Dlinq is that, if your entity travel through multi-tier, then they get detached from the DataContext from where they were initially loaded. So, when you try to update entities again using a different DataContext, you first need to attach the entity instance with the data context, then make the changes and call SubmitChanges. Now the problem is, from business layer, you do not have access to the DataContext which will be created by data access layer while updating the entity object. Business Layer will just send the entity object to the data access component and then the data access layer will do the update by creating a new DataContext. But Dlinq requires you to attach the entity object "before" making changes to them. But regular business layer will make the modifications first and then send to data access component in order to update the object. So, a traditional attempt like this will fail:

  Page p = DashboardData.GetSomePage();
  ...
  ...
  // Long time later may be after a page postback
  p.Title = "New Title";
  DashboardData.UpdatePage( p );

  Somehow you need to do this:

  Page p = DashboardData.GetSomePage();
  ...
  ...
  // Long time later may be after a page postback
  DashboardData.AttachPage( p );
  p.Title = "New Title";
  DashboardData.UpdatePage( p );

  But this is not possible because this means you cannot make DashboardData stateless. You need create DataContext inside methods and somehow you need to store the reference to DataContext between function calls. This might be ok for single user scenario, but not an acceptable solution for multiuser websites.

  So, I did this approach:

  Page p = DashboardData.GetSomePage();
  ...
  ...
  // Long time later may be after a page postback
  DashboardData.Update( p, delegate( Page p1 )
  {
  p1.Title = "New Title";
  });

  Here, the Update<> method first attaches the page object with the DataContext and then calls the delegate passing the reference to the attached object. You can now modify the passed object as if you were modifying the original object inside the delegate. Once the delegate completes, it will be updated using DataContext.SubmitChanges();

  The implementation of the Update<> method is this:

  public static void Update(T obj, Action update)
  {
  var db = GetDashboardData();
  db.GetTable().Attach(obj);
  update(obj);
  db.SubmitChanges();
  }

  Here's an example usage:

  WidgetInstance widgetInstance = DatabaseHelper.GetDashboardData().WidgetInstances.Single( wi =>
  wi.Id == WidgetInstanceId );
  DatabaseHelper.Update( widgetInstance, delegate( WidgetInstance wi )
  {
  wi.ColumnNo = ColumnNo;
  wi.OrderNo = RowNo;
  });

  The delegate gives us a benefit that you are in the context of the business layer or the caller. So, you can access UI elements or other functions/properties which you need in order to update the entity's properties.

  For convenience, I have made Insert<>, Delete<> also. But they are not required as they do not have such "Attach first, modify later" requirement.

  public static void Delete(Action makeTemplate) where T:new()
  {
  var db = GetDashboardData();
  T template = new T();
  makeTemplate(template);
  db.GetTable().Remove(template);
  db.SubmitChanges();
  }

  Day 4: Building Flickr photo and RSS widget using Xlinq

  The first widget we will build is a nice Flickr widget.

  It downloads Flickr photos as Xml feed from Flickr website and then renders a 3X3 grid with the pictures.

  First step is to download and parse the Xml using Xlinq. Here's an easy way to prepare a XElement from an URL:

  var xroot = XElement.Load(url);

  Now we convert each photo node inside the Xml to an object of PhotoInfo class for convenient processing:

  var photos = (from photo in xroot.Element("photos").Elements("photo")
  select new PhotoInfo
  {
  Id = (string)photo.Attribute("id"),
  Owner = (string)photo.Attribute("owner"),
  Title = (string)photo.Attribute("title"),
  Secret = (string)photo.Attribute("secret"),
  Server = (string)photo.Attribute("server"),
  Farm = (string)photo.Attribute("Farm")
  })

  But from the screenshot you see you can navigate between the photos because Flickr actually returns more than 9 photos. So, we need to prepare objects of PhotoInfo class from only those Xml Nodes which belong to current paging index.

  Here's how paging is done on the Xml:

  var photos = (from photo in xroot.Element("photos").Elements("photo")
  select new PhotoInfo
  {
  Id = (string)photo.Attribute("id"),
  Owner = (string)photo.Attribute("owner"),
  Title = (string)photo.Attribute("title"),
  Secret = (string)photo.Attribute("secret"),
  Server = (string)photo.Attribute("server"),
  Farm = (string)photo.Attribute("Farm")
  }).Skip(pageIndex*Columns*Rows).Take(Columns*Rows);

  We take only 9 photos from the current pageIndex. Page index is changed when user clicks next or previous links. The Skip method skips the number of items in the Xml and the Take method takes only the specified number of nodes from Xml.

  Once we have the photo objects to render, a 3X3 Html Table renders the photos:

  Collapse

  foreach( var photo in photos )
  {
  if( col == 0 )
  table.Rows.Add( new HtmlTableRow() );
  var cell = new HtmlTableCell();
  var img = new HtmlImage();
  img.Src = photo.PhotoUrl(true);
  img.Width = img.Height = 75;
  img.Border = 0;
  var link = new HtmlGenericControl("a");
  link.Attributes["href"] = photo.PhotoPageUrl;
  link.Attributes["Target"] = "_blank";
  link.Attributes["Title"] = photo.Title;
  link.Controls.Add(img);
  cell.Controls.Add(link);
  table.Rows[row].Cells.Add(cell);
  col ++;
  if( col == Columns )
  {
  col = 0; row ++;
  }
  count ++;
  }

  The reason why I used HtmlGenericControl instead of HtmlLink is that, HtmlLink does not allow you to add controls inside its Controls collection. This is a limitation of the HtmlLink class.

  This was very easy to make using XLinq. Then I built the RSS Widget which shows RSS Feeds from a feed source. First I get the URL of the feed from Widget State and then download the feed XML:

  string url = State.Element("url").Value;
  int count = State.Element("count") == null ? 3 : int.Parse( State.Element("count").Value );
  var feed = Cache[url] as XElement;
  if( feed == null )
  {
  feed = XElement.Load(url);
  Cache.Insert(url, feed, null, DateTime.MaxValue, TimeSpan.FromMinutes(15));
  }

  Then I bind the XML to a DataList which shows a list of Hyperlink:

  FeedList.DataSource = (from item in feed.Element("channel").Elements("item")
  select new
  {
  title = item.Element("title").Value,
  link = item.Element("link").Value
  }).Take(this.Count);

  The DataList is very simple:

  
  
  
  <%# Eval("title") %>
  
  
  

  And that's all!

  But there's a bit tweaking with the state. Each RSS Widget stores the URL in it's State. The Widget table has a DefaultState column which contains predefined URL for RSS Widgets. When a RSS widget is created on the page, the default state is copied to the widget instance's state. XLinq makes it very easy to deal with the simple Xml fragments. For example, here's how I read the Url:

  public string Url
  {
  get { return State.Element("url").Value; }
  set { State.Element("url").Value = value; }
  }

  The state Xml is like this:

  
  3
  ...
  

  The State property parses the Xml and returns it as XElement which refers to the root node :

  private XElement State
  {
  get
  {
  if( _State == null ) _State = XElement.Parse(this._Host.GetState());
  return _State;
  }
  }

  Day 5: Building workflows in business layer

  Here's an workflow which shows what happens when a user visits the site:

  First we get the UserGuid from user's name. Then we use that Guid to load pages, user setting and widgets in the current page. Finally we prepare a UserPageSetup object which contains all the information required to render the page.

  Now what happens when user visits the site for the first time? We need to create an anonymous user and create a default page setup for the user and then load user's page setup again. This is done inside the new user visit workflow which is like this:

  The last activity named "CallWorkflow" calls the User Visit workflow again in order to load the user setup which is just created. So, here we can see some reuse of workflow.

  The activities do very small amount of work. For example, create new page activity creates a new page and returns the ID:

  protected override ActivityExecutionStatus Execute(ActivityExecutionContext executionContext)
  {
  DashboardData db = DatabaseHelper.GetDashboardData();
  var newPage = new Page();
  newPage.UserId = UserId;
  newPage.Title = Title;
  newPage.CreatedDate = DateTime.Now;
  newPage.LastUpdate = DateTime.Now;
  db.Pages.Add(newPage);
  db.SubmitChanges(ConflictMode.FailOnFirstConflict);
  NewPageId = newPage.ID;
  return ActivityExecutionStatus.Closed;
  }

  The DashboardFacade which is the entry point to the business layer is quite simple. It knows which workflows to invoke on which operations. It just takes the parameters and invokes the right workflow for the operation. For example, it has a NewUserVisit function which does nothing but to execute NewUserVisitWorkflow.

  public class DashboardFacade
  {
  private string _UserName;
  public DashboardFacade( string userName )
  {
  this._UserName = userName;
  }
  public UserPageSetup NewUserVisit( )
  {
  var properties = new Dictionary();
  properties.Add("UserName", this._UserName);
  var userSetup = new UserPageSetup();
  properties.Add("UserPageSetup", userSetup);
  WorkflowHelper.ExecuteWorkflow(
  typeof( NewUserSetupWorkflow ), properties );
  return userSetup;
  }

  There were 3 major headaches I had to solve while implementing the business layer using Workflow and Dlinq.

  • Synchronous execution of workflow in ASP.NET
  • Getting objects out of workflow after execution is complete
  • Invoke one workflow from another synchronously

  Synchronous execution of workflow in ASP.NET

  Workflow is generally made for asynchronous execution. WorflowRuntime is usually created only once per Application Domain and the same instance of runtime is used everywhere in the same app domain. In ASP.NET, the only way you can ensure single instance of a WorkflowRuntime and make it available every where is by storing it in the HttpApplication. Also you cannot use the Default scheduler service which executes workflows asynchronously. You need to use ManualWorkflowSchedulerService which is specially made for synchronous workflow execution.

  There's a handy class called WorkflowHelper which does Workflow creation and execution. Its ExecuteWorkflow function executes a workflow synchronously.

  public static void ExecuteWorkflow( Type workflowType,
  Dictionary properties)
  {
  WorkflowRuntime workflowRuntime =
  HttpContext.Current.Application["WorkflowRuntime"] as
  WorkflowRuntime;
  ManualWorkflowSchedulerService manualScheduler =
  workflowRuntime.GetService
  ();
  WorkflowInstance instance =
  workflowRuntime.CreateWorkflow(workflowType, properties);
  instance.Start();
  manualScheduler.RunWorkflow(instance.InstanceId);
  }

  It takes the type of workflow to execute and a dictionary of data to pass to the workflow.

  Before running any workflow, first WorkflowRuntime needs to be initialized once and only once. This is done in the Global.asax in Application_Start event.

  void Application_Start(object sender, EventArgs e)
  {
  // Code that runs on application startup
  DashboardBusiness.WorkflowHelper.Init();
  }

  The WorkflowHelper.Init does the initialization work:

  public static WorkflowRuntime Init()
  {
  var workflowRuntime = new WorkflowRuntime();
  var manualService = new ManualWorkflowSchedulerService();
  workflowRuntime.AddService(manualService);
  var syncCallService = new Activities.CallWorkflowService();
  workflowRuntime.AddService(syncCallService);
  workflowRuntime.StartRuntime();
  HttpContext.Current.Application["WorkflowRuntime"] = workflowRuntime;
  return workflowRuntime;
  }

  Here you see two services are added to the workflow runtime. One is for synchronous execution and another is for synchronous execution of one workflow from another.

  Invoke one workflow from another synchronously

  This was a major headache to solve. The InvokeWorkflow activity which comes with Workflow Foundation executes a workflow asynchronously. So, if you are calling a workflow from ASP.NET which in turn calls another workflow, the second workflow is going to be terminated prematurely instead of executing completely. The reason is, ManualWorkflowSchedulerService will execute the first workflow synchronously and then finish the workflow execution and return. If you use InvokeWorkflow activity in order to run another workflow from the first workflow, it will start on another thread and it will not get enough time to execute completely before the parent workflow ends.

  Here you see only one activity in the second workflow gets the chance to execute. The remaining two activities do not get called at all.

  Luckily I found an implementation of synchronous workflow execution at:

  http://www.masteringbiztalk.com/blogs/jon/PermaLink,guid,7be9fb53-0ddf-4633-b358-01c3e9999088.aspx

  It's an activity which takes the workflow as input and executes it synchronously. The implementation of this Activity is very complex. Let's skip it.

  Getting objects out of workflow after execution is complete

  This one was the hardest one. The usual method for getting data out of workflow is to use the CallExternalMethod activity. You can pass an interface while calling a workflow and the activities inside the workflow can call host back via the interface. The caller can implement the interface and get the data out of the workflow.

  It is a requirement that the interface must use intrinsic data types or types which are serializable. Serializable is a requirement because the workflow can go to sleep or get persisted and restored later on. But, Dlinq entity classes cannot be made serializable. The classes that SqlMetal generates are first of all not marked as [Serializable]. Even if you add the attribute manually, it won't work. I believe during compilation, the classes are compiled into some other runtime class which does not get the Serializable attribute. As a result, you cannot pass Dlinq entity classes from activity to workflow host.

  The workaround I found was to pass object references as properties in the dictionary that we pass to workflow. As ManualWorkflowSchedulerService runs the workflow synchronously, the object references remain valid during the lifetime or the workflow. There is not cross app domain call here, so there is no need for serialization. Also modifying the objects or using them does not cause any performance problem because the objects are allocated in the same process.

  Here's an example:

  public UserPageSetup NewUserVisit( )
  {
  var properties = new Dictionary();
  properties.Add("UserName", this._UserName);
  var userSetup = new UserPageSetup();
  properties.Add("UserPageSetup", userSetup);
  WorkflowHelper.ExecuteWorkflow( typeof( NewUserSetupWorkflow ), properties );
  return userSetup;
  }

  So far so good. But how do you write Dlinq code in a WinFX project? If you create a WinFX project and start writing Linq code, it won't compile. Linq requires special compiler in order to generate C# 2.0 IL out of Linq code. There's a specialized C# compiler in "C:\Program Files\Linq Preview\bin" folder which MSBuild uses in order to compile the Linq codes. After long struggle and comparison between a Linq project file and a WinFX project file, I found that WinFX project has a node at the end:

  "$(MSBuildExtensionsPath)\Microsoft\Windows Workflow Foundation\v3.0\
  Workflow.Targets" />

  And Linq project has the node:

  "$(ProgramFiles)\LINQ Preview\Misc\Linq.targets" />

  These notes select the right MSBuild script for building the projects. But if you just put Linq node in a WinFX project, it does not work. You have to comment out the first node:

  After this, it built the code and everything successfully ran.

  But workflows with Conditions and Rules did not run. At runtime, the workflows threw "Workflow Validation Exception". When I use code in the rule, it works. But if I use Declarative Rules in condition, then it does not work. Declarative rules are added as Embedded Resource under the workflow or activity which contains all the rules defined in an Xml format. It appears that the .rules file does not get properly embedded and workflow runtime cannot find it while executing a workflow.

  Now this was a dead end for me. If I create regular WinFX project, then it works fine. But then again, I cannot write Linq code in a regular WinFX project. So, I have to create mix of Linq and WinFX project and use no Declarative rules. But I so desperately wanted to write rules in workflows and activities. I struggled whole night on this problem but found no solution. It was so frustrating. Then in the dawn, when there was absolute silence everywhere and the sun was about to rise, I heard divine revelation to me from the heaven:

  Thou shalt bring forth the source of misery above thy

  So, I did. I brought the .rules file (source of misery) from under the .cs file to one level upward on the project level. It then looked like this:

  For this, I had to open the Project file (.csproj) in notepad and remove the node under the node:

  
  "Activities\CreateDeafultWidgetsOnPageActivity.rules">
  
  

  And it worked! There's absolutely no way in the world I could have known that, right?

  Day 6: Page switch problem

  Widgets need to know whether it's first time load of the widget or is it a postback. Normally when it's a first time load, widgets load all settings from their persisted state and render the UI for the first time. Upon postback, widgets don't restore settings from persisted state always, instead sometimes they update state or reflect small changes on the UI. So, it is important for user's to know when they are being rendered for the first time and when it is a postback.

  However, when you have multiple tabs, the definition of first time load and postback changes. When you click on another tab, it's a regular postback for ASP.NET because a LinkButton gets clicked. This makes the Tab UpdatePanel postback asynchronously and on the server side we find out which tab is clicked. Then we load the widgets on the newly selected tab. But when widgets load, they call Page.IsPostBack and they get true. So, widgets assume they are already on the screen and try to do partial rendering or try to access ViewState. But this is not true because they did not appear on screen yet and there's no ViewState for the controls on the widget. As a result, the widgets behave abnormally and all ViewState access fails.

  So, we need to make sure during tab switch, although it's a regular ASP.NET postback, Widgets must not see it as postback. The idea is to inform widgets whether it is a first time load or not via the IWidget interface.

  On Default.aspx, there's a function SetupWidgets which creates the WidgetContainer and loads the widgets. Here's how it works:

      private void SetupWidgets(Func isWidgetFirstLoad)
  {
  var setup = Context.Items[typeof(UserPageSetup)] as UserPageSetup;
  var columnPanels = new Panel[] {
  WidgetViewUpdatePanel.FindControl("LeftPanel") as Panel,
  WidgetViewUpdatePanel.FindControl("MiddlePanel") as Panel,
  WidgetViewUpdatePanel.FindControl("RightPanel") as Panel};
  // Clear existin widgets if any
  foreach( Panel panel in columnPanels )
  {
  List widgets =
  panel.Controls.OfType().ToList();
  foreach( var widget in widgets ) panel.Controls.Remove( widget );
  }

  Skip the Func<> thing for a while. First, I clear the columns which contains the WidgetContainer so that we can create the widgets again. See the cool Linq way to find out only the WidgetContainer controls from the Panel's Controls collection.

  Now, we create the WidgetContainers for the widgets on the newly selected tab:

          foreach( WidgetInstance instance in setup.WidgetInstances )
  {
  var panel = columnPanels[instance.ColumnNo];
  var widget = LoadControl(WIDGET_CONTAINER) as WidgetContainer;
  widget.ID = "WidgetContainer" + instance.Id.ToString();
  widget.IsFirstLoad = isWidgetFirstLoad(instance);
  widget.WidgetInstance = instance;
  widget.Deleted +=
  new Action(widget_Deleted);
  panel.Controls.Add(widget);
  }

  While creating, we set a public property IsFirstLoad of the WidgetContainer in order to let it know whether it is being loaded for the first or not. So, during first time load of Default.aspx or during tab switch, the widgets are setup by calling:

  SetupWidgets( p => true );

  What you see here is called Predicate. This is a new feature in Linq. You can make such predicates and avoid creating delegates and the complex coding model for delegates. The predicate returns true for all widget instances and thus all widget instances see it as first time load.

  So, why not just send "true" and declare the function as SetupWidgets(bool). Why go for the black art in Linq?

  Here's a scenario which left me no choice but to do this. When a new widget is added on the page, it is a first time loading experience for the newly added widget, but it's a regular postback for existing widgets already on the page. So, if we pass true or false for all widgets, then the newly added widget will see it as a postback just like all other existing widgets on the page and thus fail to load properly. We need to make sure it's a non-postback experience only for the newly added widget but a postback experience for the existing widget. See how it can be easily done using this Predicate feature:

          new DashboardFacade(Profile.UserName).AddWidget( widgetId );
  this.SetupWidgets(wi => wi.Id == widgetId);

  Here the predicate only returns true for the new WidgetId, but returns false for existing WidgetId.

  Day 7: Signup

  When user first visits the site, an anonymous user setup is created. Now when user decides to signup, we need to copy the page setups and all user related settings to the newly signed up user.

  The difficulty was to get the anonymous user's Guid. I tried Membership.GetUser() passing Profile.UserName which contains the anonymous user name. But it does not work. It seems Membership.GetUser only returns a user object which exists in aspnet_membership table. For anonymous users, there's no row in aspnet_membership table, only in aspnet_users and aspnet_profile tables. So, although you get the user name from Profile.UserName, but you cannot use any of the methods in Membership class.

  The only way to do it is to read the UserId directly from aspnet_users table. Here's how:

  AspnetUser anonUser = db.AspnetUsers.Single( u => u.LoweredUserName == this._UserName
  && u.ApplicationId == DatabaseHelper.ApplicationGuid );

  Note: You must use LoweredUserName, not the UserName field and must include ApplicationID in the clause. Aspnet_users table has index on ApplicationID and LoweredUserName. So, if you do not include AapplicationID in the criteria and do not use the LoweredUserName field, the index will not hit and the query will end up in a table scan which is very expensive. Please see my blog post for details on this:

  Careful-when-querying-on-aspnet

  Once we have the UserId of the anonymous user, we just need to update the UserID column in Page and UserSetting table to the newly registered user's UserId.

  So, first get the new and old UserId:

              MembershipUser newUser = Membership.GetUser(email);
  // Get the User Id for the anonymous user from the aspnet_users table
  AspnetUser anonUser = db.AspnetUsers.Single( u => u.LoweredUserName == this._UserName
  && u.ApplicationId == DatabaseHelper.ApplicationGuid );
  Guid oldGuid = anonUser.UserId;
  Guid newGuid = (Guid)newUser.ProviderUserKey;

  Now update the UserId field of the Pages of the user:

                  List pages = db.Pages.Where( p => p.UserId == oldGuid ).ToList();
  foreach( Page page in pages )
  page.UserId = newGuid;

  But here's a catch. You cannot change the field value if it's a primary key using Dlinq. You have to delete the old row using the old primary key and then create a new row using new primary key:

                  UserSetting setting = db.UserSettings.Single( u => u.UserId == oldGuid );
  db.UserSettings.Remove(setting);
  setting.UserId = newGuid;
  db.UserSettings.Add(setting);

  See DashboardFacade.RegisterAs(string email) for the full code.

  Web.config walkthrough

  The web project is a mix of WinFX, Linq and ASP.NET Ajax. So, the web.config needs to be configured in such a way that it allows harmonious co-existence of these volatile technologies. The web.config itself requires a lot of explanation. I will just highlight the areas which are important.

  You need to use Linq compiler so that default C# 2.0 compiler does not compile the site. This is done by:

  Then you need to put some extra attributes in the node:

  Now you need to include the ASP.NET Ajax assemblies and WinFX assemblies:

  You also need to put "CSharp3CodeDomProvider.dll" in the "bin" folder and add reference to System.Data.DLinq, System.Data.Extensions, System.Query and System.Xml.Xlinq. All these are required for Linq.

  I generally remove some unnecessary HttpModule from default ASP.NET pipeline for faster performance:

  How slow is ASP.NET Ajax

  Very slow, especially in IE6. In fact it's slowness is so bad that you can visually see on local machine while running on your powerful development computer. Try pressing F5 several times on a page where all the required data for the page are already cached on the server side. You will see the total time it takes to fully load the page is quite long. ASP.NET Ajax provides a very rich object oriented programming model and a strong architecture which comes at high price on performance. From what I have seen, as soon as you put UpdatePanels on the page and some extenders, the page becomes too slow. If you just stick to core framework for only web service call, you are fine. But as soon as you start using UpdatePanel and some extenders, it's pretty bad. ASP.NET Ajax performance is good enough for simple pages which has say one UpdatePanel and one or two extenders for some funky effects. May be one more data grid on the page or some data entry form. But that's all that gives acceptable performance. If you want to make a start page like website where one page contains almost 90% of the functionality of the whole website, the page gets heavily loaded with javascripts generated by extenders and UpdatePanels. So, Start Page is not something that you should make using UpdatePanel and Extenders. You can of course use the core framework without doubt for webservice calls, XML HTTP, login/logout, profile access etc.

  When you make a start page, it is absolutely crucial that you minimize network roundtrip as much as possible. If you study Pageflakes, you will see on first time load, the Wizard is visible right after 100KB of data transfer. Once the wizard is there, rest of the code and content download in the background. But if you close the browser and visit again, you will see total data transfer over the network is around 10 KB to 15 KB. Pageflakes also combines multiple smaller scripts and stylesheets into one big file so that the number of connection to the server is reduced and overall download time is less than large number smaller files. You really need to optimize to this level in order to ensure people feel comfortable using the start page everyday. Although this is a very unusual requirement, but this is something you should try in all Ajax applications because Ajax applications are full of client side code. Unfortunately you cannot achieve this using ASP.NET Ajax unless you do serious hacking. You will see that even for a very simple page setup which has only 3 extenders, the number of files downloaded is significant:

  All the files with ScriptResource.axd are small scripts in Ajax Control Toolkit and my extenders. The size you see here is after gzip compression and they are still quite high. For example, the first two are nearly 100 KB. Also all these are individual requests to the server which could be combined into one JS file and served in one connection. This would result in better compression and much less download time. Generally each request has 200ms of network roundtrip overhead which is the time it takes for the request to reach server and then first byte of the response to return to client. So, you are adding 200ms for each connection for nothing. It is quite apparent to ScriptManager which scripts are needed for the page on server side because it generates all the script references. So, if it could combine them into one request and serve them gzipped, it could save significant download time. For example, here 12 X 200ms = 2400ms = 2.4 sec is being wasted on the network.

  However, one good thing is that, all of these gets cached and thus does not download second time. So, you save significant download time on future visits.

  So, final statement, UpdatePanel and Extenders are not good for websites which push client side richness to the extreme like Ajax Start Pages, but definitely very handy for not so extreme websites. It's very productive to have designer support in Visual Studio and very good ASP.NET 2.0 integration. It will save you from building an Ajax framework from scratch and all the javascript controls and effects. In Pageflakes, we realized there was no point building a core Ajax framework from scratch and we decided to use Atlas runtime for XmlHttp and Webservice call. Besides the core Ajax stuff, everything else is homemade including the drag & drop, expand/collapse, fly in/out etc. These are just too slow using UpdatePanel and Extenders. Both Speed & smoothness are very important to start pages because they are set as browser homepage.

  How to run the code

  • Install .NET 3.0 Framework
  • Install ASP.NET Ajax RC1
  • Install Linq May CTP
  • Restore the database in your SQL Server as "Dashboard" or whatever name you like.
  • Open the web.config in "src\Dashboard" folder and specify correct connection string according to your SQL server configuration
  • Load the Dashboard.sln and run.

  Remember, you cannot just copy the website to a server and run it. It will not run. Something wrong with ScriptResource handler in ASP.NET Ajax RC version. You will have to Publish the website and copy the precompiled site to a server.

  Next Steps

  If you like this project, let's make some cool widgets for it. For example, a To-do-list, Address book, Mail Widget etc. This can become a really useful start page if we can make some useful widgets for it. We can also try making a widget which runs Google IG modules or Pageflakes' flakes on it.

  Conclusion

  Ajax Start Page is a really complex project where you push DHTML & Javascript to their limits. As you add more and more features on the client side, the complexity of the web application increases geometrically. Fortunately ASP.NET Ajax takes away a lot of complexity on the client side so that you can focus on your core features and leave the framework and Ajax stuffs to the runtime. Moreover, Dlinq and cool new features in .NET 3.0 makes it a lot easier to build powerful data access layer and business logic layer. Making all these new technologies work with each other was surely a great challenge and rewarding experience.

  Shameless disclaimer: I am co-founder & CTO of Pageflakes, the coolest Web 2.0 Ajax Start Page. I like building Ajax websites and I am really, really good at it.

  About Omar Al Zabir

  I am Co-founder and CTO of www.pageflakes.com , a Web 2.0 personalized start page. It defeated Microsoft's Live.com and Google IG and became #1 in Web 2.0 awards this year. I also got the Visual C# MVP award twice. I remember when I was 15, instead of playing football with my friends, I started working in projects for Bank Of America, Federel Reserved Bank, HSBC, CitiBank, WAMU, FNB, HP etc. I have been a software developer for more than a decade. You will fall asleep reading the long list of projects in my resume (total 28 I think). I graduated from Computer Science last year. Ready to do something serious now. I heard those who see my site www.oazabir.com get their mouth wide open for a while until something gets inside. See if that happens to you also. Email: OmarALZabir at gmail dot com Blog: http://www.msmvps.com/omar Open Source Projects: Smart UML RSS Feeder Store Front in JSP but ASP.NET style I live in Dhaka, Bangladesh. I love traveling. I have traveled around the world twice in 40 days (by international airlines). Click here to view Omar Al Zabir's online profile.