Nikhil's Space

My wife sent me this heart touching snippet from a story written by W. Livingston Larned. In this crazy complex world of deadlines, stress and worries, I have many times done exactly what I should not have done - Forgotten that my son is just 2 years old. It brought tears to my eyes and I hope to never always remember that they are just children which is why they keep on doing what they are supposed to do. We as adults need to just spend a few moments in understanding them better.

Listen, son; I am saying this as you lie asleep, one little paw crumpled under your cheek and the blond curls stickily wet on your damp forehead. I have stolen into your room alone. Just a few minutes ago, as I sat reading my paper in the library, a stifling wave of remorse swept over me. Guiltily I came to your bedside.

There are things I was thinking, son: I had been cross to you. I scolded you as you were dressing for school because you gave your face merely a dab with a towel. I took you to task for not cleaning your shoes. I called out angrily when you threw some of your things on the floor.

At breakfast I found fault, too. You spilled things. You gulped down your food. You put your elbows on the table. You spread butter too thick on your bread. And as you started off to play and I made for my train, you turned and waved a hand and called, "Goodbye, Daddy!" and I frowned, and said in reply, "Hold your shoulders back!"

Then it began all over again in the late afternoon. As I came Up the road, I spied you, down on your knees, playing marbles. There were holes in your stockings. I humiliated you before your boyfriends by marching you ahead of me to the house. Stockings were expensive - and if you had to buy them you would be more careful! Imagine that, son, from a father!

Do you remember, later, when I was reading in the library, how you came in timidly, with a sort of hurt look in your eyes? When I glanced up over my paper, impatient at the interruption, you hesitated at the door. "What is it you want?" I snapped.

You said nothing, but ran across in one tempestuous plunge, and threw your arms around my neck and kissed me, and your small arms tightened with an affection that God had set blooming in your heart and which even neglect could not wither. And then you were gone, pattering up the stairs.

Well, son, it was shortly afterwards that my paper slipped from my hands and a terrible sickening fear came over me. What has habit been doing to me? The habit of finding fault, of reprimanding - this was my reward to you for being a boy. It was not that I did not love you; it was that I expected too much of youth. I was measuring you by the yardstick of my own years.

And there was so much that was good and fine and true in your character. The little heart of you was as big as the dawn itself over the wide hills. This was shown by your spontaneous impulse to rush in and kiss me good night. Nothing else matters tonight, son. I have come to your bedside in the darkness, and I have knelt there, ashamed!

It is a feeble atonement; I know you would not understand these things if I told them to you during your waking hours. But tomorrow I will be a real daddy! I will chum with you, and suffer when you suffer, and laugh when you laugh. I will bite my tongue when impatient words come. I will keep saying as if it were a ritual: "He is nothing buy a boy - a little boy!"

I am afraid I have visualized you as a man. Yet as I see you now, son, crumpled and weary in your cot, I see that you are still a baby. Yesterday you were in your mother's arms, your head on her shoulder. I have asked too much, too much.

W. Livingston Larned.

I was reading an article in ComputerWorld  which really caught my attention. Here is an excerpt that I found so true across various companies I have worked. This is part of an interview between Kathleen Melymuka and the Harwell Thrasher, author of the book "Boiling the IT Frog".

Why do most projects fail? In my experience, there are six primary reasons. First, you’re doing the wrong project — it’s not what the business really needs. Second, you’re missing prerequisites in the proposal. You start and then realize you have to beef up the infrastructure, for example, so you’re in trouble right from the beginning. Third, you’re going for home runs instead of base hits; for example, a global rollout of a major system instead of chipping away at it a little at a time. Fourth, the project’s duration is greater than the job tenure of the sponsoring executive. If the project isn’t completed before he or she leaves, you may be in deep trouble with the successor. Fifth, you gather requirements instead of negotiating them. If you go around asking all the stakeholders what they want and your project goal is the sum of all those things, you will undoubtedly have contradictions — for example, a simple system that must do everything. Sixth, there’s not enough contingency planning. You’ve got to anticipate things that can go wrong and take them into account in your plan.

Boiling the IT Frog

One of the common problems I have seen is to bulk upload data to a SQL Server database. If you have the flexibility to directly run your code in SQL, you have a ton of options. But let's say that you have to massage the data before you throw it in to the database, then you have to really know your SQL (well to do it in SQL).

Let's say you have to read data from an RSS feed, parse it and then load it into SQL. Let's assume further that this feed updates every 2 hours. It would be a trivial task to write a C# app that reads and parses the feed. One crude way to upload this data would be to do a single row insert for each data element. This would be terribly inefficient. The other option would be to use .Net framework's SqlBulkCopy class.

The basic template would be something like

private void WriteToDatabase()
{
    // get your connection string
    string connString = "";
    // connect to SQL
    using (SqlConnection connection = 
            new SqlConnection(connString))
    {
        // make sure to enable triggers
        // more on triggers in next post
        SqlBulkCopy bulkCopy = 
            new SqlBulkCopy
            (
            connection, 
            SqlBulkCopyOptions.TableLock | 
            SqlBulkCopyOptions.FireTriggers | 
            SqlBulkCopyOptions.UseInternalTransaction,
            null
            );

        // set the destination table name
        bulkCopy.DestinationTableName = this.tableName;
        connection.Open();

        // write the data in the "dataTable"
        bulkCopy.WriteToServer(dataTable);
        connection.Close();
    }
    // reset
    this.dataTable.Clear();
    this.recordCount = 0;
}

The above code snippet shows you the API usage. But before you actually do that, you need to follow a couple of steps to setup your data table.

First, let's look at a simple record structure (as reflected in C# class):

using System;
using System.Data;
using System.Configuration;

/// <summary>
/// Summary description for MyRecord
/// </summary>
public class MyRecord
{
    public int TestInt;
    public string TestString;

	public MyRecord()
	{
	}

    public MyRecord(int myInt, string myString)
    {
        this.TestInt = myInt;
        this.TestString = myString;
    }
}

Now, let's start dissecting the class that we will use to upload the data:

using System;
using System.Data;
using System.Collections.Generic;

using System.Data.SqlClient;
using System.Configuration;
using System.IO;

namespace SqlExamples.FileLoader
{
/// <summary>
/// Summary description for BulkUploadToSql
/// </summary>
public class BulkUploadToSql
{
    private List<MyRecord> internalStore;

    protected string tableName;
    protected DataTable dataTable = new DataTable();
    protected int recordCount;
    protected int commitBatchSize;

Note that we have an internal List data structure as well as the DataTable. This is redundant and you can avoid using the internalStore if your application does not need to massage the data before it's sent to SQL.

I then define 2 private constructors. The reason is that we want to use the factory pattern to return our object to the caller.

    private BulkUploadToSql(
        string tableName, 
        int commitBatchSize)
    {
        internalStore = new List<MyRecord>();

        this.tableName = tableName;
        this.dataTable = new DataTable(tableName);
        this.recordCount = 0;
        this.commitBatchSize = commitBatchSize;

        // add columns to this data table
        InitializeStructures();
    }

    private BulkUploadToSql() : 
        this("MyTableName", 1000) {}

Note that we set the commit batch size. This is a very important factor that needs to be fine tuned for your database. What this defines is the number of records that we would send in one shot to the database.

The next step is to Initialize the data table with columns that reflect the actual table structure.

private void InitializeStructures()
{
    this.dataTable.Columns.Add("TI", typeof(Int32));
    this.dataTable.Columns.Add("TS", typeof(string));
}

I then provided a factory method to load data into my internal structure from a data source. In the example code below, I use a Stream, but this can be any data source from where you wish to populate your data.

public static BulkUploadToSql Load(Stream dataSource)
{
    // create a new object to return
    BulkUploadToSql o = new BulkUploadToSql();
    
    // replace the code below
    // with your custom logic 
    for (int cnt = 0; cnt < 10000; cnt++)
    {
        MyRecord rec = 
            new MyRecord
            (
            cnt, 
            string.Format("string{0}", cnt)
            );
        o.internalStore.Add(rec);
    }

    return o;
}

This would make sure that our class is properly initialized and loaded with data. Once the caller has a valid object, they can now "Flush" the data as shown below:

public void Flush()
{
    // transfer data to the datatable
    foreach (MyRecord rec in this.internalStore)
    {
        this.PopulateDataTable(rec);
        if (this.recordCount >= this.commitBatchSize)
            this.WriteToDatabase();
    }
    // write remaining records to the DB
    if (this.recordCount > 0)
        this.WriteToDatabase();
}

private void PopulateDataTable(MyRecord record)
{
    DataRow row;
    // populate the values
    // using your custom logic
    row = this.dataTable.NewRow();

    row[0] = record.TestInt;
    row[1] = record.TestString;

    // add it to the base for final addition to the DB
    this.dataTable.Rows.Add(row);
    this.recordCount++;
}

In the example above, the call to Flush() actually massages the data (and at the same time loads it into the actual data table). As I mentioned before, you can actually skip this step if your application does not require massaging.

As a example of an app that uses this class:

using System;
using System.Collections.Generic;
using System.Text;
using SqlExamples.FileLoader;
using System.IO;

namespace DemoApp
{
class Program
{
    static void Main(string[] args)
    {
        using (Stream s = 
            new StreamReader(@"C:\TestData.txt"))
        {
            BulkUploadToSql myData = 
                BulkUploadToSql.Load(s);
            myData.Flush();
        }
    }
}
}

As always, this is JUST demo code to explain a concept. This is NOT production quality code and please make sure to follow the coding guidelines in your team.

Happy coding....

One of the most tiresome (but important) things when developing web services is handling un-handled exceptions. A good design principle forces you to catch and cast relevant exceptions raised by your web methods into more meaningful SOAP exceptions. But exceptions will occur.

It is quite tedious to wrap each web method in a try/catch loop. This dictates a need for a common framework to handle unhandled web services exceptions. In this blog post, I will guide you through a step by step process for building one.

Summary:

1. Extend SoapExtension class and override the ProcessMessage() method.
2. In ProcessMessage(), add special handler code for message stage of SoapMessageStage.AfterSerialize.
3. Modify the web service's web.config file and add a <soapExtensionTypes> node to <webServices> section.

Details:

Ok, now let's dig deeper into the code.

Step 1:

The first step in the whole process is to extend the SoapExtension class override the ProcessMessage() method.

using System.IO;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.Xml;
using System;

public class TelspaceSoapExtension : SoapExtension
{
    private Stream originalStream;
    private Stream updatedStream;

    public override object GetInitializer(Type serviceType)
    {
        return null;
    }

    public override object GetInitializer
    (LogicalMethodInfo methodInfo, SoapExtensionAttribute attribute)
    {
        return null;
    }

    public override void Initialize(object initializer)
    {
    }

    public override Stream ChainStream(Stream stream)
    {
        originalStream = stream;
        updatedStream = new MemoryStream();
        return updatedStream;
    }

    private void Transfer(Stream inStream, Stream outStream)
    {
        StreamReader sr = new StreamReader(inStream);
        StreamWriter sw = new StreamWriter(outStream);
        sw.Write(sr.ReadToEnd());
        sw.Flush();
    }

    public override void ProcessMessage(SoapMessage soapMessage)
    {
        switch (soapMessage.Stage)
        {
            case SoapMessageStage.BeforeDeserialize:
                Transfer(originalStream, updatedStream);
                updatedStream.Position = 0;
                break;

            case SoapMessageStage.AfterSerialize:
                if ((soapMessage.Exception != null))
                {
                    ExceptionProcessor processor = new ExceptionProcessor();

                    string details;

                    // handle our exception, and get the SOAP <detail> string
                    details = processor.HandleWebServiceException(soapMessage);

                    // read the entire SOAP message stream into a string
                    updatedStream.Position = 0;
                    TextReader tr = new StreamReader(updatedStream);

                    // insert our exception details into the string
                    string s = tr.ReadToEnd();
                    s = s.Replace("<detail />", details);

                    // overwrite the stream with our modified string
                    updatedStream = new MemoryStream();
                    TextWriter tw = new StreamWriter(updatedStream);
                    tw.Write(s);
                    tw.Flush();
                }

                updatedStream.Position = 0;
                Transfer(updatedStream, originalStream);
                break;
        }
    }

}

Step 2:

The next step is to handle the exception and get out meaningful details from the exception. For this purpose, let's dig deeper into the ExceptionProcessor class. This class has one public method: HandleWebServiceException(System.Web.Services.Protocols.SoapMessage sm). This method is called from our ProcessMessage()case SoapMessageStage.AfterSerialize.

Here is the code:

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.Web.Services.Protocols;
using System.Runtime.InteropServices;
using System.Diagnostics;
using System.Reflection;
using SASMTPLib;
using System.IO;

class ExceptionProcessor
{
    private string currentExceptionDetails;
    private string currentExceptionTypeName;

    private const string RootExceptionName = "System.Web.HttpUnhandledException";
    private const string RootWsExceptionName = "System.Web.Services.Protocols.SoapException";

    public string HandleWebServiceException(System.Web.Services.Protocols.SoapMessage sm)
    {
        HandleException(sm.Exception);
        XmlDocument doc = new XmlDocument();
        XmlNode detailNode = doc.CreateNode(XmlNodeType.Element, SoapException.DetailElementName.Name, SoapException.DetailElementName.Namespace);
        XmlNode typeNode = doc.CreateNode(XmlNodeType.Element, "ExceptionType", SoapException.DetailElementName.Namespace);
        typeNode.InnerText = currentExceptionTypeName;
        detailNode.AppendChild(typeNode);

        XmlNode messageNode = doc.CreateNode(XmlNodeType.Element, "ExceptionMessage", SoapException.DetailElementName.Namespace);
        messageNode.InnerText = sm.Exception.Message;
        detailNode.AppendChild(messageNode);

        XmlNode infoNode = doc.CreateNode(XmlNodeType.Element, "ExceptionInfo", SoapException.DetailElementName.Namespace);
        infoNode.InnerText = currentExceptionDetails;
        detailNode.AppendChild(infoNode);
        return detailNode.OuterXml.ToString();
    }

    private void HandleException(Exception ex)
    {
        try
        {
            currentExceptionDetails = NormalizeException(ex);
            currentExceptionTypeName = ex.GetType().FullName;

            // ignore root exceptions
            if (((currentExceptionTypeName == RootExceptionName) || (currentExceptionTypeName == RootWsExceptionName)))
            {
                if (ex.InnerException != null)
                {
                    currentExceptionTypeName = ex.InnerException.GetType().FullName;
                }
            }
        }
        catch (Exception e)
        {
            currentExceptionDetails = string.Format("Error \'{0}\' while generating exception string", e.Message );
        }

        ExceptionToEmail();
    }

    private string NormalizeException(Exception ex)
    {
        StringBuilder sb = new StringBuilder();

        if (ex.InnerException != null)
        {
            if (((ex.GetType().ToString() == RootExceptionName) || (ex.GetType().ToString() == RootWsExceptionName)))
            {
                return NormalizeException(ex.InnerException);
            }
            else
            {
                sb.Append(NormalizeException(ex.InnerException));
                sb.Append(Environment.NewLine);
                sb.Append("(Outer Exception)");
                sb.Append(Environment.NewLine);
            }
        }

        // get exception-specific information
        sb.Append("Exception Type:\t\t");
        try
        {
            sb.Append(ex.GetType().FullName);
        }
        catch (Exception e)
        {
            sb.Append(e.Message);
        }
        sb.Append(Environment.NewLine);

        sb.Append("Exception Message:\t\t");
        try
        {
            sb.Append(ex.Message);
        }
        catch (Exception e)
        {
            sb.Append(e.Message);
        }
        sb.Append(Environment.NewLine);

        sb.Append("Exception Target Method:\t");
        try
        {
            sb.Append(ex.TargetSite.Name);
        }
        catch (Exception e)
        {
            sb.Append(e.Message);
        }
        sb.Append(Environment.NewLine);

        sb.Append("Stack Trace:\t\t");
        sb.Append(Environment.NewLine);
        try
        {
            sb.Append(ex.StackTrace);
        }
        catch (Exception e)
        {
            sb.Append(e.Message);
        }
        sb.Append(Environment.NewLine);

        return sb.ToString();
    }

    private void ExceptionToEmail()
    {
        // Send e-mail code
    }

}

Step 3:

The final step is to modify web.config of the web services to include the following section (under <system.web> node)

<webServices>
  <soapExtensionTypes>
    <add type="TelspaceSoapExtension, ExceptionHandler" priority="1" group="High"/>
  </soapExtensionTypes>
</webServices>

One of the biggest pain points that I have seen for Windows Live Spaces users is the in-ability to change a blog post date (once posted).

Well, for those of you (including me) who have been bitten by this problem more than once, cry no more!!! Windows Live Writer team recently launched their flag ship product (read about it here) that has a "mechanism" to change the blog post date. Here it goes... (don't get scared by the number of steps I have listed below. I have broken the process down in very minute steps).

1. Install Windows Live Writer from here and set it up for Windows Live Spaces and make it point to your space (e.g. Nikhil's space)
2. Click on "Open" (from either File->Open Post... OR the Open button on the tool bar.
3. Select your Space from the list on the left side. There will be a brief wait while the Writer is getting blog posts from your space. You will then be shown your latest 5 blog entries. Let's say you want to edit post #2 which you had posted a week back.
4. Double click on the post. The post opens in edit mode in a new window. There are 2 ways to reach to the "Change date" section - either press "F2" or navigate to the bottom of the page and click onthe "Properties" line.
5. You will see a Date/Time: edit field. Check the checkbox next to the date and update the date/time to what ever you desire. Make sure you do not specify a date greater than NOW.
6. Click on "Publish" at the top on the tool bar.

YOU ARE DONE. It's that's simple. Enjoy!!!

This post has been long overdue on my part. I have been planning to write this up for now over a year...

Windows Live Spaces search feature is a very powerful tool that allows you to to search Spaces contents in a very precise way. Let me first introduce some basic concepts:

Scoping (a.ka. Drill down): Spaces search results can be scoped to specific object types. Currently, 4 types of scoping is permitted: "All", "Spaces", "People" and "Blog entries". The following image shows the various options.

When you first do a keyword search (more on this below), we try to give you a "preview" into all areas of Spaces that would have a match for your given search term. From here you can narrow down (aka drill down) into one specific area that interests you. For example, if you are just looking for personal spaces home page (e.g. nikhil.spaces.live.com), narrowing down to Spaces is what you would do. On the same idea, if you are looking for profiles (e.g. http://nikhil.spaces.live.com/personalspace.aspx?_c01_memberprofiletile=showdefault&_c=memberprofiletile), you would select "People" and if your interest lies in blog entries (e.g. http://nikhil.spaces.live.com/blog/cns!A6E2B32DB60BF014!925.entry), you would use "Blog entries" scope.

There are 2 ways to drill down to your specific scope (or result type) - By using the drop down box (as shown in the above image) or by clicking the name in the section display (see image below).

Search Types: Now that we have some understanding of what scoping is and how it works, let's change our focus on different types of searches that Spaces supports. The first type is very obvious - Keyword search. This is the search that happens when you issue a search query from the Header (sorry for the squished up image).

Keyword search is what you would generally do on MSN Search (or Google). When you do a keyword search, the page text crawled is searched. Also, for a keyword search, you can scope to all different types of searches.

The second type of search you can do is "Interests" or "Tag" search. This is a very specific search that you can initiate from various UI elements - From the "Spaces interests" page, or from the interests section of a personal profile page or from "Advanced People Search" (more on this one below). When you do an "Interests" search, we (Spaces) issues a special "META" search for people's profiles that have been "tagged" with those interests. As we are searching for interests, we only show you "Profile" scoped results.

The third type of search is the most advanced, complicated and well, interesting. We call it Advanced People Search or APS for short. You can do an APS from the Spaces home page.

This is again a people profile search so you would get only "People" results and you cannot change the scope. This is a specialized search that searches for people's profiles for very specific items. You can search for Display Name, First Name, Last Name, Gender, Age range, Location, Occupation and Interests or any combination there of. This allows you to search for, lets say, Females, in the age range of 25-29, living in Seattle, who are software engineers and are interested in having fun :-). This is a very powerful search mechanism that let's you drill down on very specific attributes for people you are looking for.

Other UI elements:

• Saved session search: Once you have issued a search, it will be saved for your session and you can always visit it back by clicking on the "Search" tab on the left navigation panel on the Spaces home page.

• Contact control: Spaces search is very different than other search engines as we do not show you just the search results. For each search result, we show you a "Contact control" which allows you to see that person's photo (if available) and a bunch of other options like viewing their space, profile, contact card, friends list, etc. etc.

• "Safe" search: We take protecting our minors very seriously. Only "gray area" words are allowed if you are signed in and are of an adult age as defined by your market (for example 18 years in en-US).
• Search RSS feed: Spaces search results are also available as RSS feeds (if you are so inclined). Navigate down to the bottom of the page and you will see a cool looking RSS icon which will let you subscribe to that search result (properly scoped and typed) as an RSS feed. We limit the RSS feed items to the top 10 items.

• "Collapsible" sections: The search results for each scope are contained within a section which can be collapsed/expanded by either clicking the +/- symbol on the left or on the blue section header.

• Ads: Of course, we have to make money too :-)

What sucks: Aha... The section that every developer hates to talk about. But I will be brave and hopefully you will also be understanding enough as we work through the issues and improve the experience. My 2 biggest sore points are result count and  paging controls. We are working on both these issues and hopefully will have a better story on them.

• Result count: Although we show that for a given query we have 137838 results (see image above), we do not show any results over page 16 (or 17). Basically, we cap out at 250 results (we are not responsible for this - more on this in another post). It SUCKS.
• Paging: This is a direct result of the above problem. The paging model allows you to navigate to the "last" page of search results, but since we cap the result as 250, this paging model does not make sense.

Hopefully, these notes would help you understand how spaces search cab be used by you in a more meaningful way that relates more to what you are looking for. After all, Windows Live Spaces is your personal reflection to the world.

 An excellent collection of radio stations...

Quote

List of Hindi & Punjabi radio stations online

I thought I would share some a few of the resources I use to listen to music and relax.  Here is the list of online Hindi & Punjabi radio stations that I listen to frequently. Copy the URL in your media player and enjoy !!

RadioTarana-NewZealand ==> mms://l1.r2.co.nz/tarana-1

106.2 HUMFM - UAE ==> http://www.humfm.com/humfm.asx 

RadioOfIndia - Bollywood ==> http://www.radioofindia.com/asf/bollywood.asx

RadioOfIndia - Bhajans ==> http://www.radioofindia.com/asf/bhajans.asx

RadioOfIndia - Classical ==> http://66.238.65.109/classical

DesiSoundz - India ==> http://desisoundz.com:8000

Sabras radio - UK ==> http://ct1.fast-serv.com:8744 

Haagstad Radio - Holand ==> mms://81.205.146.32:21/haagstadradio 

RadioTeenTaal - Paris ==> http://www.radioteentaal.com/masala128.wax 

ApnaRadio - USA ==> http://www.apnaradio.com/live/media24/ApnaRadio.asx

BombayBeats FM ==> http://www.1.fm/player/energybbfm32k.asx

Punjabi Radio - UK ==> http://azul.streamguys.com/panjabradio?MSWMExt=.asf

Amrit Bani - UK ==> http://62.25.97.192/amritbani?MSWMExt=.asf

Yarr Radio - UK ==> mms://193.218.160.20/yaarradio

Sunrise FM - UK ==> http://62.25.96.7/sunrise

Radio XL - UK ==> http://www.vtuner.com/vTunerweb/mms/m3u13219.m3u

Asian Gold Radio - UK ==> http://62.25.96.7/asiangold

Asian Sound Radio - UK ==> http://www.vtuner.com/vtunerweb/mms/mms15278.asx

Sanskar Radio - UK ==> http://www.vtuner.com/vTunerweb/mms/m3u18290.m3u

Trishul 90.5 FM ==> http://www.vtuner.com/vtunerweb/mms/mms14734.asx

Radio Apni Awaz ==> mms://67.15.80.29/radioapniawaz

Radio India - Canada ==> mms://live.radioindiabroadcasting.com/liveradio

City 101.6 FM - Dubai ==> http://asx.abacast.com/arabian_radio-city-24.asx

DDLive Video - India ==> http://164.100.51.209/ddlive?MSWMExt=.asf 

AajKal - Asian Network ==>

http://stream.servstream.com/ViewWeb/BBCRadio_music/Event/BBCAsianNetwork_hi.asx

BBC News ==> http://www.bbc.co.uk/worldservice/meta/tx/nb/live_news_au_nb.asx