let generateKey (chars : char[]) = new string(chars)

let lookup =  words |> Seq.groupBy(fun x -> generateKey(x.ToCharArray() |> Array.sort))
 

The lookup function expects a list of strings.  It works – although I’ve only tried it with fairly short lists of words.  Next exercise is to try calling this (or something like it) from C#.

The first step is to try (part of) a potential solution – given a list of words, group the words by the constituent letters (so that, for instance, horse and shore would be grouped together.)  That way, I won’t have to try every single combination of letters and compare against a list of words.  My intention this week was to write a simple program in C# to do this.  Here’s the method that does the work:

private Dictionary<string, List<string>> ProcessWordList(string[] words)

{

    Dictionary<string, List<string>> result = new Dictionary<string, List<string>>();

    foreach (string word in words)

    {

        IOrderedEnumerable<char> ordered = from char a in word.ToCharArray()

                                           orderby a

                                           select a;

        string orderedString = new string(ordered.ToArray());

        if (result.ContainsKey(orderedString))

        {

           result[orderedString].Add(word);

        }

        else

        {

            List<string> list = new List<string>();

            list.Add(word);

            result.Add(orderedString, list);

        }

    }

    return result;

}

Next exercise is either to extend the program towards solving anagrams or to write the method above in F#.

Firstly, it’s helpful to have some words to count.  Here’s a simple approach:

string test = "The cat sat on the mat.";

string[] words = test.Split(' ');

(In F# I populated a list of words directly, so there’s an extra line of C# here – largely because I started with my previous C# code.)  Right, now to the counting in one line:

var result = from word in words

             let strippedWord = StripPunctuation(word).ToLower()

             where strippedWord.Length > 0

             group word by strippedWord into grouped

             select new { Word = grouped.Key, Count = grouped.Count() };

You  may have noticed a call to StripPunctuation – a utility function I had in my previous C# code.  Here it is (declared static as I was running it in a console application:

private static string StripPunctuation(string word)

{

    string result = word;

    if (result.Length > 0)

    {

        if (char.IsPunctuation(result[0]))

        {

            result = result.TrimStart(result[0]);

        }

        if (result.Length > 0)

        {

            if (char.IsPunctuation(result[result.Length - 1]))

            {

                result = result.TrimEnd(result[result.Length - 1]);

            }

        }

    }

    return result;

}

And now, with a little sprinkling of dynamic capability, outputting the results to the console:

foreach (dynamic entry in result)

{

    Console.WriteLine("{0}\t{1}", entry.Word, entry.Count);

}

So it is possible to apply the more functional approach courtesy of LINQ, although there’s still more code than I had in F#.  The C# is doing a couple of extra things (it strips out punctuation and is case insensitive) – but the point isn’t really the comparison between the two examples so much as the fact that by grasping some functional concepts can result in a change to your C# – which is a good reason to learn some F#.

let words = ["the"; "cat"; "sat"; "on"; "the"; "mat"];;

(the double semicolons are signal to F# Interactive the completion of a statement.)  After reading a bit about processing sequences in F#, I spotted that there is a function to count elements in a list – it can easily be used against the whole list like this:

let count = words |> Seq.countBy(fun x -> x);;

The countBy function takes a function to generate a key – in this case we can use each individual word.  To see if that has worked, we can print out the contents of the result:

printfn "%A" count;;

And in this case, I got the following result:

seq [("the", 2); ("cat", 1); ("sat", 1); ("on", 1); ...]
val it : unit = ()

Which means it worked as intended.  There’s work to be done to make it the equivalent of the C# code, but the core counting is implemented in one line of code.

So, this time around, I decided to go back to my tried and trusted method which consists simply of reading and experimenting.  In support of which I bought a couple of books: Programming F# and Functional Programming.  Other books are available.  As for online resources, I tend to read articles that are suggested via the maelstrom of near real time social online noise-makers or search for relevant stuff.  However, there’s a few links worth noting as good starting places:

F# Developer Centre

Don Syme’s blog

Tomas Petricek’s blog

hubFS forums

Of course, you’ll find your own places and resources as I’m sure I will as the journey continues.

Originally, I intended to read the data with IronPython.  Reading CSV data with Python is very simple – there’s a built in CSV module.  However, this module is written in C, which means it’s not available in IronPython – see here for more info.  There is aproject called IronClad that allows Python modules written in C to be used from IronPython.  At the moment, it’s built against .NET 2, which means that I could get it to work in .NET 2, but I had plans to use .NET 4 and the dynamic support in C#.   Time for another approach.

Using the CsvReader class,it’s easy to access the data in a CSV file.  I started with the Player Summaries sheet.  To make this dynamic (and, therefore, useful for each of these sheets and, potentially, other as yet unknown sheets) I created a class to hold each row of data.  Here it is:

public class DynamicDataObject : DynamicObject

{

    private readonly Dictionary<string, dynamic> data;

    public DynamicDataObject(Dictionary<string, dynamic> data)

    {

        this.data = data;

    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)

    {

        result = data[binder.Name];

        return (result != null);

    }

}

By inheriting from DynamicObject, it will be possible to call this class dynamically – meaning that I can use the names of the data fields as defined properties on the class.  Next I created a DataReader class that reads the data from the CSV file and stores it as an IEnumerable<DynamicDataObject>.  Here’s that class:

public class DataReader : IEnumerable<DynamicDataObject>

{

    private readonly List<DynamicDataObject> dataList;

    public DataReader(string filename)

    {

        this.dataList = new List<DynamicDataObject>();

        using (StreamReader streamReader = new StreamReader(filename))

        {

            using (CsvReader reader = new CsvReader(streamReader, true))

            {

                string[] headers = reader.GetFieldHeaders();

                Dictionary<string, string> cleanHeaders = CleanHeaders(headers);

                while (reader.ReadNextRecord())

                {

                    Dictionary<string, dynamic> data = new Dictionary<string, dynamic>();

                    foreach (string header in headers)

                    {

                        int result;

                        dynamic value;

                        if (int.TryParse(reader[header],  out result))

                        {

                            value = result;

                        }

                        else

                        {

                            value = reader[header];

                        }

                        data.Add(cleanHeaders[header], value);

                    }

                    this.dataList.Add(new DynamicDataObject(data));

                }

            }

        }

    }

    private Dictionary<string, string> CleanHeaders(string[] headers)

    {

        Dictionary<string, string> result = new Dictionary<string, string>();

        foreach (string header in headers)

        {

            string cleanheader = header.Replace(' ', '_');

            cleanheader = cleanheader.Split('(')[0];

            result.Add(header, cleanheader);

        }

        return result;

    }

    #region IEnumerable<DynamicDataObject> Members

    public IEnumerator<DynamicDataObject> GetEnumerator()

    {

        return this.dataList.GetEnumerator();

    }

    #endregion

    #region IEnumerable Members

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()

    {

        return this.dataList.GetEnumerator();

    }

    #endregion

}

There’s a couple of things worth pointing out.  The first is that I’ve cleaned up the field names so that they can be used in code (by replacing spaces with underscores and removing anything in brackets).  The second is that if a value is an integer, I’m storing it as an integer.  I’m storing these values of type dynamic, which will come in handy when we want to query the data.

Speaking of querying that data, I wanted to use LINQ.  Here’s some simple code I wrote in a console application to try it out:

static void Main(string[] args)

{

    DataReader reader = new DataReader(@"C:\Users\Mark\Downloads\Eng-USA Data\Player Summaries.csv");

    var result = from dynamic player in reader

                 where player.Goals > 0

                 select player;

    foreach (dynamic player in result)

    {

        Console.WriteLine(player.Player_Name + " - " + player.Goals + " goals");

    }

    Console.WriteLine("Press any key to exit...");

    Console.ReadKey();

}

And here’s the output:

By using the dynamic support in C#, the LINQ query just works and I can reference properties dynamically without having to create a class specifically for each sheet of data.  It’s important in the LINQ query to declare the player of type dynamic – otherwise C# will revert to its statically typed ways and inform you that the Goals property doesn’t exist, which, given that it only exists at runtime, is correct.  Now I can analyse the data easily.  Doesn’t change the result though…