Implementing (Parts Of) ASP.NET Identity Using F#

I started working though this and this article for implementing security in a new Web Api 2 site I am spinning up.  Everything is working out OK – not great but better than prior ASP.NET implementations I have done.  I do think the ASP.NET team has made security better and I am really excited about token based security.  My biggest gripe is that there is still too much magic going on and it is still hard to introduce non-out of the box implementations.  For example, the sample code that you can add via Nuget has a placeholder for en email provider.  The article has the code for a specific implementation of a company called SendGrid (the sample is here and the email is here).
The CSharp implementation looks like this (I did add some constructor injection b/c I am opposed to touching the Configuration (or any part of the file system for that matter) outside of Main on sanity grounds):
1 public class SendGridEmailProvider: IIdentityMessageService 2 { 3 String _mailAccount = String.Empty; 4 String _mailPassword = String.Empty; 5 String _fromAddress = String.Empty; 6 7 public SendGridEmailProvider(String mailAccount, String mailPassword, String fromAddress) 8 { 9 _mailAccount = mailAccount; 10 _mailPassword = mailPassword; 11 _fromAddress = fromAddress 12 } 13 14 public System.Threading.Tasks.Task SendAsync(IdentityMessage message) 15 { 16 var sendGridMessage = new SendGridMessage(); 17 sendGridMessage.From = new MailAddress(_fromAddress); 18 List<String> recipients = new List<string>(); 19 recipients.Add(message.Destination); 20 sendGridMessage.AddTo(recipients); 21 sendGridMessage.Subject = message.Subject; 22 sendGridMessage.Html = message.Body; 23 sendGridMessage.Text = message.Body; 24 25 var credentials = new NetworkCredential(_mailAccount, _mailPassword); 26 var transportWeb = new Web(credentials); 27 if (transportWeb != null) 28 return transportWeb.DeliverAsync(sendGridMessage); 29 else 30 return Task.FromResult(0); 31 } 32 }

I then decided to implement a SMS Text Provider along the same lines.  The 1st company I came to was CDyne but when I went to their sample code, they are still using a SOAP-based service and the last thing I wanted to do was to clutter up my project with all of the WSDL and files that you needs when consuming a service like that.
I then thought, this is stupid.  I might was well use FSharp type providers to do the implementation.  Less Code, less files, less clutter, more goodness.  I first swapped out the Email to a FSharp implementation:
1 type SendGridEmailService(account:string, password:string, fromAddress:string) = 2 interface IIdentityMessageService with 3 member this.SendAsync(identityMessage) = 4 let sendGridMessage = new SendGridMessage() 5 sendGridMessage.From = new MailAddress(fromAddress) |> ignore 6 let recipients = new List<string>() 7 recipients.Add(identityMessage.Destination) 8 sendGridMessage.AddTo(recipients) 9 sendGridMessage.Subject <- identityMessage.Subject 10 sendGridMessage.Html <- identityMessage.Body 11 sendGridMessage.Text <- identityMessage.Body 12 13 let credentials = new NetworkCredential(account, password) 14 let transportWeb = new Web(credentials) 15 match transportWeb with 16 | null -> Task.FromResult(0):> Task 17 | _ -> transportWeb.DeliverAsync(sendGridMessage)

I then did a SMS text provider using type providers.
1 type cDyneService = Microsoft.FSharp.Data.TypeProviders.WsdlService<"http://sms2.cdyne.com/sms.svc?wsdl"> 2 3 type CDyneSMSService(licenseKey:Guid) = 4 interface IIdentityMessageService with 5 member this.SendAsync(identityMessage) = 6 let cDyneClient = cDyneService.Getsms2SOAPbasicHttpBinding 7 let client = cDyneService.Getsms2SOAPbasicHttpBinding() 8 match client with 9 | null -> Task.FromResult(0):> Task 10 | _ -> client.SimpleSMSsendAsync(identityMessage.Destination,identityMessage.Body,licenseKey):> Task

Compared to a CSharp implementation, this is joyous.  Less noise, more signal.  And thanks to Lee on Stack Overflow for helping with the upcast of Task…

Consuming Azure ML web api endpoint from an array

Last week, I blogged about creating an Azure ML experiment, publishing it as a web service, and then consuming it from F#.  I then wanted to consume the web service using an array – passing in several values and seeing the results.  I created added on to my existing F #script with the following code

1 let input1 = new Dictionary<string,string>() 2 input1.Add("Zip Code","27519") 3 input1.Add("Race","W") 4 input1.Add("Party","UNA") 5 input1.Add("Gender","M") 6 input1.Add("Age","45") 7 input1.Add("Voted Ind","1") 8 9 let input2 = new Dictionary<string,string>() 10 input2.Add("Zip Code","27519") 11 input2.Add("Race","W") 12 input2.Add("Party","D") 13 input2.Add("Gender","F") 14 input2.Add("Age","47") 15 input2.Add("Voted Ind","1") 16 17 let inputs = new List<Dictionary<string,string>>() 18 inputs.Add(input1) 19 inputs.Add(input2) 20 21 inputs 22 |> Seq.map(fun i -> invokeService(i)) 23 |> Async.Parallel 24 |> Async.RunSynchronously 25

And sure enough, I can run the model using multiple inputs:

image

Sql Server and “Saving Changes Are Not Permitted”

Dear Future Jamie:

If you are trying to add a identity column to a table after it is created and you get this message:

image

Make the change in Management Studio here:

image

 

Love,

Current Jamie

Remote Debugging On Azure

I decided that I needed to learn a bit about remote debugging on Azure.  I know that you are supposed to use the Azure emulator to hash out your problems before hand, but I still would like the ability to debug remotely in case the need comes up.

I Googled on Bing how to do it and I ran across this article that seems to be a good place to start.  It looks like I need to upload the VS2010 remote tools to Azure as a necessary but not sufficient step.  The thing is, I have no idea how to do that.  Do I upload the tools only once to my azure account?  Is it site && project specific?  I don’t know and Binging on Google doesn’t seem to help.

I then looked at this article but it assumes that I am using a Virtual Machine and installing the MSVSMON.exe to the VM.  If that was the case, I would just install VS2010 to the VM and debug there.  So that article is of no help.

I am stuck on this line of the 1st article: “You can find the tools on Microsoft Download Center here, and then upload to your storage account using whatever Windows Azure Storage account tool of your choice. “  Going over to my Azure account, I see a “Storage” section.

So I create a new storage like so:

image

The illogical meter is running high.  Why should I need to install Data Services to install MSVSMON?  I can do Computer-> Cloud Service, but that us the same as creating a Cloud Service.

I decided to work in the other direction.  I created a new Azure hosted WCF Service in VS2010 like so:

 

image

 

and after the default template of this:

image

I changed Service1 name and then added a single method that returns the sum of two numbers:

public class AddingMachine : IAddingMachine
{
    public Int32 Add(Int32 number1, Int32 number2)
    {
        return number1 + number2;
    }

}

I then deployed his to the Azure server via Visual Studio – 1st I set up the deployment parameters

image

 

Then I added a certificate

image

And during the deployment I got this:

image

After a couple of minutes, I got the site up on Azure:

image

Which I assume might be the way to remote debug?  I then hit myself in the head that the Build Configuration is Release (the default) and not debug and I need the debug symbols to remote debug.  I deployed again and then I opened attach to process in Visual Studio:

image

I then pumped in the name of the site:

image

Doh!

Time to go to Brian Hitney’s office hours!

Web Performance Test

 

So I decided I wanted to learn more about the testing capabilities of VS2010 – esp. load testing a web service.  I fired up Visual Studio and created an out of the box WCF Service with the following 1 method:

public class AddingMachine : IAddingMachine
{
    public int GetData(int value1, int value2)
    {
        Random random = new Random();
        Int32 delay = random.Next(15);
        Thread.Sleep(TimeSpan.FromSeconds(delay));

        return value1 + value2;
    }
}

I then added a new test project using a Web Performance Test template:

image

I then hit record, closed IE which automatically popped up, and navigated to the service using the WCF Test Client. 

image

I am sure you are not surprised that nothing was recorded.  The problem is that the out of the box Web Performance Test assumes that you are calling a web site so I couldn’t just hit a recording and navigate to a webservice the same way I would navigate to a website.

Instead, I had to right click and add a new web service request.  I then changed the Url property to the service location and tried to run the test.

image

I then binged on Google and ran across a couple of Stack Overflow posts and this nugget on MSDN.  Apparently, I need to use Fiddler to capture the request.  I fired up Fiddler and WCF Test Client (remember the “.” after local host) and sure enough and can make the call and intercept the traffic:

image

I copied the entire request from Fiddler:

image

But I got this when I ran the test I got the same exception.  The problem is that the article assumes an .asmx web service and I am using WCF.  Digging into the article’s code sample, I realized that I needed a Header like so:

image

and I needed to put only the body from Fiddler into the String Body property

Fiddler:

image

WebTest:

image

And green is good

image

Calling Command

I know I am supposed to be using (and loving) power shell, but I ran across a problem this weekend and the good-old command window worked fine.  I was building a one-click application that moved data from one location to another and then manipulating the data.  As part of the workflow, I created a DTS/SSIS package.  To execute this package, I used the following code to shell out to the command prompt and fire up the package:

            String sourceConnectionString = CreateSourceConnectionString();
            String targetConnectionString = CreateTargetConnectionString();

            Process process = new Process();
            process.StartInfo.FileName = "cmd";
            StringBuilder stringBuilder = new StringBuilder();
            stringBuilder.AppendFormat(@"/k");
            stringBuilder.AppendFormat(@"dtexec");
            stringBuilder.AppendFormat(@" /F");
            stringBuilder.AppendFormat(@" TransferAllTables.dtsx");

            stringBuilder.AppendFormat(" /Conn {0};\"{1}\"", "SourceConnectionOLEDB", sourceConnectionString);
            stringBuilder.AppendFormat(" /Conn {0};\"{1}\"", "DestinationConnectionOLEDB", targetConnectionString);
            process.StartInfo.Arguments = stringBuilder.ToString();

            process.Start();

I also want to thank my friend Ian (who doesn’t have a blog yet so I can’t point you to him) for mentioning  the StringBuilder.AppendFormat() function.  Append() + String.Format() in 1 place.  Nice! Thanks  Ian!

NOAA and how not to do a web service

I came across the NOAA API when I was looking at various providers of weather data via the programmable web.  Thinking that the government might be a great place to get (free) data, I dove into their API.  I am glad I didn’t go head-first.  The API is, well, wretched.  In fact, it probably is the worst public API I have come across in my limited travels.

Why is so bad?

1) Ambiguous Website.  Their use of jargon is over-whelming.  To understand the API, you need to learn about the NDFD .  What is that?  What about current weather?  Nope, I need to know about the National Digital Forecast Database.  How about the NCDC?  What is DWML? On to issue #2.

2) They invented their own version of SOAP: Digital Weather Markup Language.  Enough said.

3)  The web site is rife with links that show graphics that no one can use.  The API help is in clear language?  No where to be seen.

4) Hooking up to their WSDL is not much better.  I made a connection and this is what I got back:

  image

Got that?  You need to create an instance of ndfdXMLPortTypeClient.  Say that 3 times fast.  How about a weather class?  A forecast class?  Nope, this API assumes that other developers give a hoot about their internal implementation (and no one does). 

5) I tried a simple call to the web service just to see what it sent back:

public WeatherReading GetReading(string zipCode)
{
    ndfdXMLPortTypeClient client = new ndfdXMLPortTypeClient();
    String output = client.LatLonListZipCode("27519");
    return null;
}

And what did I get?

image

Got that – a non-standard encoding.  PTF (Palm to face…)

So I am giving up on the government and trying some of the other providers.

Using Pointers In Managed Code

I am getting ready for my Kinect presentation at RDU’s code camp.  One of the techniques that you have to absolutely use with the tidal wave of data that the Kinect sends you is pointers.  For example, I have some code that is straight from this book where I turn the video image from the the Kinect ColorSensor a darker shade of blue.

In a default WPF application, I added the following class-level variables:

KinectSensor kinectSensor = null;
WriteableBitmap colorImageBitmap = null;
Byte[] colorData = null;

I then wired up the Kinect:

kinectSensor = KinectSensor.KinectSensors[0];
kinectSensor.ColorStream.Enable();
kinectSensor.ColorFrameReady += new EventHandler<ColorImageFrameReadyEventArgs>(kinectSensor_ColorFrameReady);
kinectSensor.Start();

I then added the following code to the event handler as the video frames come in (30 per second, each frame about 1.5 megs):

if (colorImageFrame != null)
{
    if (colorImageBitmap == null)
    {
        colorImageBitmap = new WriteableBitmap(
            colorImageFrame.Width,
            colorImageFrame.Height,
            96, 96,
            PixelFormats.Bgr32,
            null);
        this.kinectImage.Source = colorImageBitmap;
    }
    if (colorData == null)
    {
        colorData = new Byte[colorImageFrame.PixelDataLength];
    }

    colorImageFrame.CopyPixelDataTo(colorData);
    
   
    Int32 newColor = 0;
    for (int i = 0; i < colorData.Length; i = i + 4)
    {
        Int32 oldColor = colorData[i];
        newColor = (Int32)colorData[i] + 50;
        if (newColor > 255)
        {
            newColor = 255;
        }
        colorData[i] = (byte)newColor;
    }  
  

    colorImageBitmap.WritePixels(
        new Int32Rect(0, 0, colorImageFrame.Width, colorImageFrame.Height),
        colorData,
        colorImageFrame.Width * colorImageFrame.BytesPerPixel, 0);
    
}

This code works – but it gets too slow when using an under powered computer.  To use pointers, I first needed to mark my project as unsafe:

image

I then added the unsafe keyword to the event handler:

unsafe void kinectSensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)

And finally, I replaced the managed loop with a loop that uses pointers:

int noOfPixelBytes = colorData.Length;
fixed (byte* imageBase = colorData)
{
    byte* imagePosition = imageBase;
    byte* imageEnd = imageBase + noOfPixelBytes;

    Int32 newColor = 0;
    while (imagePosition != imageEnd)
    {
        newColor = *imagePosition + 50;
        if (newColor > 255)
        {
            newColor = 255;
        }
        *imagePosition = (byte)newColor;
        imagePosition += 4;
    }
}

This speed things up considerably.  A couple of things to note:

The fixed keyword pins the location of imageBase to 1 location so the garbage collector doesn’t move it around.  Also, note that I refer the to value of the current byte via imagePosition* (*imagePosition = (byte)newColor) – when I want to move 4 bytes over, I increment the imagePosition (imagePosition += 4)

I am looking forward to Saturday  hopefully this presentation will go off without a hitch…

VS2012, EF, and Mocking

A couple of years ago I presented at TriNug’s code camp a way to build an interface for EF 1.0 using partial classes.  I thought I would revisit that with VS2012 to see if it is still possible using EF 5.0 and the mocking framework in VS2012.

My first stop was to  install SQL Server Express and then Run Northwind.exe.  Note that you can’t just attach to the Northwind .mdf because of version incompatibilities.  Rather, you need to open SQL Server Express and run the scripts that came with the install.  In any event, once it is installed, it was straight forward to add an EF representation to the project.  The one cool thing that I noticed is that you can change the color of different tables via the property panel:

image

Sure enough, by changing the fill color, you get a really cool effect like this:

image

With EF added to the project, coded up a factory like this:

public List<Territory> GetTerritoriesForARegionId(Int32 regionId)
{
    List<Territory> territories = null;

    if (regionId < 0)
    {
        throw new ArgumentOutOfRangeException("regionId cannot be less than 0");
    }

    using (NorthwindEntities entities = new NorthwindEntities())
    {
        territories = entities.Territories.Where(t => t.RegionID == regionId).ToList();
    }

    return territories;
}

I then went to right click –> add Tests like you do in VS2010 and… Oh No!  That feature is not in VS2012

I then went to hand-code a unit test project, class, and method like this:

[TestMethod]
public void GetTerritoryForARegionIdWithRegionIdOne_ReturnsThreeRecords()
{
    TerritoryFactory factory = new TerritoryFactory();
    List<Territory> territories = factory.GetTerritoriesForARegionId(1);

    Int32 expected = 3;
    Int32 actual = territories.Count;

    Assert.AreEqual(expected, actual);
}

When I ran it, I got this:

image

I needed to copy/paste the .config over.  (Which is the 1st clue that my test is is actually an integration test).  I then ran the test and got red:

image

So there are 19 records in the database.  I can do a couple of things here.  I could check that any value is coming out of the database:

[TestMethod]
public void GetTerritoryForARegionIdWithRegionIdOne_ReturnsThreeRecords()
{
    TerritoryFactory factory = new TerritoryFactory();
    List<Territory> territories = factory.GetTerritoriesForARegionId(1);
    Assert.IsNotNull(territories);

}

The problem is that the working code will always return a instance – unless an exception is thrown. I suppose I can leave the list as null and put a try…catch around the lambda, but that doesn’t really get you anything.  So really, what do you need to test? It is the lambda expression in the working code:

territories = entities.Territories.Where(t => t.RegionID == regionId).ToList();

The problem is that the working code (and therefore the test) is dependent on the entity framework. Breaking the expression into 2 parts, I don’t need to test if the EF does its job, I only need to test my code:

t => t.RegionID == regionId

So how to I make sure you are writing the correct statement to pull down the results you expect? I need to isolate the statement and following functional programming, guarantee that the result will be the same.

To that end, I need to remove the dependency on volatile data. So no database. So how do you make a EF that is not database specific? You sub it out. There are a couple of options (xUnit Test Patterns p.171) . The way I favor is to first inject that context into any class that is using it (and I prefer property injection over constructor injection). 

Note that the dependent class no longer controls the lifetime of the context, so no “using” statement in it.

public class TerritoryFactory
{
    public NorthwindEntities NorthwindEntities { get; set; }

    public List<Territory> GetTerritoriesForARegionId(Int32 regionId)
    {
        List<Territory> territories = null;

        if (regionId < 0)
        {
            throw new ArgumentOutOfRangeException("regionId cannot be less than 0");
        }

        territories = this.NorthwindEntities.Territories.Where(t => t.RegionID == regionId).ToList();

        return territories;
    }
}

Once I am injecting the context, I can pass in a local instance of an EF Context. A If I want to sub out a different version, you need to implement an interface.

The problem is that EF is auto generated – so you can’t just make an interface and have it implement it.

image

This is the problem I solved at the code camp 2 years ago (Note that this would be so much easier if MSFT added interfaces to their auto generated classes.  Just sayin’).  In any event, the way around it is to use a partial class and then extract the interface from that partial class.  To that end, I added a new class to the project with the following code:

public partial class NorthwindEntities 
{

}

I then used Refactor –> Extract Interface and I got an interface:

interface INorthwindEntities
{
System.Data.Entity.DbSet<Category> Categories { get; set; }
System.Data.Entity.DbSet<CustomerDemographic> CustomerDemographics { get; set; }
System.Data.Entity.DbSet<Customer> Customers { get; set; }
System.Data.Entity.DbSet<Employee> Employees { get; set; }
System.Data.Entity.DbSet<Order_Detail> Order_Details { get; set; }
System.Data.Entity.DbSet<Order> Orders { get; set; }
System.Data.Entity.DbSet<Product> Products { get; set; }
System.Data.Entity.DbSet<Region> Regions { get; set; }
System.Data.Entity.DbSet<Shipper> Shippers { get; set; }
System.Data.Entity.DbSet<Supplier> Suppliers { get; set; }
System.Data.Entity.DbSet<Territory> Territories { get; set; }
}

I then changed my unit test code to use the interface like so:

[TestMethod]
public void GetTerritoryForARegionIdWithRegionIdOne_ReturnsThreeRecords()
{
    TerritoryFactory factory = new TerritoryFactory();
    INorthwindEntities entities = new NorthwindEntities();
    factory.NorthwindEntities = entities;
    List<Territory> territories = factory.GetTerritoriesForARegionId(1);

    Int32 expected = 3;
    Int32 actual = territories.Count;

    Assert.AreEqual(expected, actual);
}

When I ran the test, I got the same red (red>green>refactor?  how about red > redder > refactor).

image

So the last thing is the implementation.  How do we build a “fake” in memory instance of the Northwind Entities that returns 3 Territories?  Enter Mocking Framework!

image

 

And then switch out the implementation for the Mock:

[TestMethod]
public void GetTerritoryForARegionIdWithRegionIdOne_ReturnsThreeRecords()
{
    TerritoryFactory factory = new TerritoryFactory();

    //Replace the actual implementation with the Fake
    //INorthwindEntities entities = new Tff.NorthwindApp.NorthwindEntities();
    INorthwindEntities entities = new Fakes.StubNorthwindEntities();

    //Add in Stub Data
    entities.Territories.Add(new Territory { TerritoryID = "1", RegionID = 1 });
    entities.Territories.Add(new Territory { TerritoryID = "2", RegionID = 1 });
    entities.Territories.Add(new Territory { TerritoryID = "3", RegionID = 1 });

    factory.NorthwindEntities = entities;
    List<Territory> territories = factory.GetTerritoriesForARegionId(1);
    Int32 expected = 3;
    Int32 actual = territories.Count;

    Assert.AreEqual(expected, actual);

}

I half expected that running this would get green.  I assumed that the default constructor of EF (which is what the Mocking framework would use) would just give an empty graph without going to the database.  Alas, I was wrong:

image

So the Stub is still going out to the database.

clip_image001

I then spent a couple of hours of trying to get around this “feature” of the EF being so tightly coupled to the database and I gave up.  I then tried the same technique with Linq to Sql.  Unfortunately, I ran into the same problem.  I then went to stack overflow and got nothing.

To me, this is a real limitation of EF.  EF should be able to be stubbed easily.  MSFT missed a real opportunity here…

The Annotated Turing

Based on Bob Martin’s recommendation in his Clean Coder’s webcast, I picked up The Annotated Turing by Charles Petzold. 

image

What a fun read!  Petzold really does a good job of breaking down the original paper into manageable chunks – and gives just enough background to place the paper’s ideas in context.  I really had fun digging though my gray cells of the difference among the different kids of numbers (real, imaginary, etc…) .  I am about half way through the book  and I plan to re-read it once I finish it.  If you work in IT for a living, this is worth a read – and re-read….