WebApi | Jamie Dixon's Home

Setting up an OData Service on WebAPI2 to be used by F# Type Providers

January 7, 2014 1 Comment

I am prepping for the F#/Data Analytics workshop on January 8th and wanted to get the data that I used for the Road Alert application beck to better shape. By better, I mean out of that crusty WCF SOAP that I have had it in for the last 2 years. To that end, I jumped over to Mike Wasson’s Creating an OData tutorial. All in all, it is a good step by step guide, but I had to make some changes to get it working for me.

Change #1 is that I am not using a local database, I am using a database located on WinHost. Therefore, I had to swap out the EF connection string.

One of the things I can appreciate about the template is the comments to get the routing set up (I guess there is not attribute-based routing for O-Data?)

        /*
To add a route for this controller, merge these statements into the Register method of the WebApiConfig class. Note that OData URLs are case sensitive.
 
using System.Web.Http.OData.Builder;
using ChkickenSoftware.RoadAlertServices.Models;
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.EntitySet<TrafficStop>("TrafficStop");
config.Routes.MapODataRoute("odata", "odata", builder.GetEdmModel());
*/

Things were looking good when I took a departure from the tutorial and added in a couple of unit tests (Change #2). The 1st one was fairly benign:

[TestClass]
public class TrafficStopControllerIntegrationTests
{
    [TestMethod]
    public void GetTrafficStopUsingKey_ReturnsExpected()
    {
        TrafficStopController controller = new TrafficStopController();
        var trafficStop = controller.GetTrafficStop(1);
        Assert.IsNotNull(trafficStop);
    }
}

Note that I had to add an app.config to the test project b/c this is an integration test and I am making a real database call – a unit test would using a mocking framework. In any event, when I went to run the test, I got a compile error – I needed to add a reference to System.Web.Http.OData to resolve the return value from the controller. Not big thing, though I wish I could install packages from Nuget via their .dll name and not just their package name:

In any event, I then ran the test and I got this exception:

So this is another reason why EF drives me nuts. I have to add a reference to Entity Framework (and throw some crap in the .config file)

<entityFramework>
  <defaultConnectionFactory type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework" />
  <providers>
    <provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
  </providers>
</entityFramework>

– even thought the calling application has nothing to do with EF. In 2014, we have such dependency drip? Really? In any event, once I added a reference to EF and updated the .config file, my unit/integration test ran green so I was on the right track.

I then went to fiddler and tried to call the controller:

Yikes, it looks like my model has to match the EF exactly

The database:

And the model:

public class TrafficStop
{
    public Int32 Id { get; set; }
    public double CadCallId { get; set; }
    public DateTime StopDateTime { get; set; }
    public Int32 DispositionId { get; set; }
    public String DispositionDesc { get; set; }
    public double Latitude { get; set; }
    public double Longitude { get; set; }
}

– I assume that I should be able to override this behavior – another thing to research.

So after matching up field names, I ran fiddler and sure enough:

So that was pretty painless to get an OData Service up and running. I then removed everything but the read methods and I added an auth header (you can see the value in the screen shot above), feel free to hit up the service now that it is deployed to WinHost:

One of the coolest things about OData is that it has a .WSDL type discovery:

http://chickensoftware.com/RoadAlert/odata/$metadata

I was really missing that when we went from SOAP Services to REST

Note that I had to do a couple of more things in Tsql (remember that?) to the original data to get it ready for general consumption (and analytics). I had to create a real date/time from the 2 varchar fields:

Update [XXXX].[dbo].[TrafficStops]
Set StopDateTime = Convert(DateTime, right (left([Date],6),2) + ‘/’ + right([Date],2) + ‘/’ + left([Date],4) + ‘ ‘ + left(Time,2) + ‘:’ + Right(left(Time,4),2) + ‘:’ + Right(left(Time,6),2))

I also had to add an integral value for when we do statistical analysis:

Update [XXXXX].[dbo].[TrafficStops]
Set dispositionId =
CASE
     WHEN dispositionDesc = ‘FURTHER ACTION NECESSARY’ THEN 1
     WHEN dispositionDesc = ‘UNABLE TO LOCATE’ THEN 2
     WHEN dispositionDesc = ‘FALSE ALARM’ THEN 3
     WHEN dispositionDesc = ‘WRITTEN WARNING’ THEN 4
     WHEN dispositionDesc = ‘OTHER    SEE NOTES’ THEN 5
     WHEN dispositionDesc = ‘REFERRED TO PROPER AGENCY’ THEN 6
     WHEN dispositionDesc = ‘VERBAL WARNING’ THEN 7
     WHEN dispositionDesc = ‘NULL’ THEN 8
     WHEN dispositionDesc = ‘ARREST’ THEN 9
     WHEN dispositionDesc = ‘NO FURTHER ACTION NECESSARY’ THEN 10
     WHEN dispositionDesc = ‘CIVIL PROBLEM’ THEN 11
     WHEN dispositionDesc = ‘COMPLETED AS REQUESTED’ THEN 12
     WHEN dispositionDesc = ‘INCIDENT REPORT’ THEN 13
     WHEN dispositionDesc = ‘UNFOUNDED’ THEN 14
     WHEN dispositionDesc = ‘CITATION’ THEN 15
     WHEN dispositionDesc = ‘FIELD CONTACT’ THEN 16
     WHEN dispositionDesc = ‘BACK UP UNIT’ THEN 17
     WHEN dispositionDesc = ‘CITY ORDINANCE VIOLATION’ THEN 18
END

So now I am ready to roll with doing the analytics.

So when I say “ready to roll”, I really meant to say “ready to flail.” When we last left the show, I was ready to start consuming the data from OData using the F# type providers. Using Fiddler, I can see the data coming out of the OData service

The problem started when I went to consume the data using the F# OData Type Provider as documented here. I got the red squiggly line of approbation when I went to create the type:

with the following message:

Error 1 The type provider ‘Microsoft.FSharp.Data.TypeProviders.DesignTime.DataProviders’ reported an error: error 7001: The element ‘DataService’ has an attribute ‘DataServiceVersion’ with an unrecognized version ‘3.0’.

I went over to the F#-open source Google group to seek help and Isaac Abraham had this response:

WebAPI 2 now pushes out OData 3 endpoints by default, which are actually not even backwards compatible with the OData 2 standard. OData 3 was (AFAIK) released some time after the OData Type Provider was written, so I suspect it doesn’t support OData 3.

So I am stuck. I really want to use type providers but they are behind. I thought about if I could downgrade my WebAPI2 OData to go to OData2 standard (whatever that is).

My 1st thought was to trick out the client by removing the DataServiceVersion header like so:

public class HeadersHandler : DelegatingHandler
{
async protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
HttpResponseMessage response = await base.SendAsync(request, cancellationToken);
response.Content.Headers.Remove("DataServiceVersion");
return response;
}
}

The header was removed, but alas, the RSLA is still with me with the same message. I then thought, perhaps I can go back to the old version of Json so I modified the header like so:

public class HeadersHandler : DelegatingHandler
{
async protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
request.Headers.Add("Accept", "application/json;odata=verbose");
HttpResponseMessage response = await base.SendAsync(request, cancellationToken);
response.Content.Headers.Remove("DataServiceVersion");
return response;
}
}

So the Json is now the “old” version, but I am still getting the RSLA. I then ran fiddler when creating the type provider and I see this:

Crap. I need to have Entity Framework use a lower version (I am using EF 6.0). I guess? My 1st thought was to remove EF from the situation entirely, which is always a good idea. My next, and more time-efficient, thought was to ask Stack Overflow – which is what I did here. While I wait for Stack Overflow to come to the rescue. I decided to press on. I just exposed the data via a normal controller like so:

public class TrafficStopSearchController : ApiController
{
public List<TrafficStop> Get()
{
DataContext context = new DataContext();
return context.TrafficStops.ToList<TrafficStop>();
}
public TrafficStop Get(int id)
{
DataContext context = new DataContext();
return context.TrafficStops.Where(ts => ts.Id == id).FirstOrDefault();
}
[HttpGet]
[Route("api/TrafficStopSearch/Sample/")]
public List<TrafficStop> Sample()
{
DataContext context = new DataContext();
return context.TrafficStops.Where(ts => ts.Id < 100).ToList();
}
}

The reason I threw in the Sample method is that the F# JSON type provider uses a sample to infer types and I didn’t want to send the entire set of data across the wire for that. Once that was done, the traffic stop data was consumable in my F# application like so:

type roadAlert = JsonProvider<"http://chickensoftware.com/roadalert/api/trafficstopsearch/Sample">
type AnalysisEngine =
static member RoadAlertDoc = roadAlert.Load("http://chickensoftware.com/roadalert/api/trafficstopsearch")

Once/if I get the OData set up, I will swap this out but this is good enough for now – after all the interesting piece is not getting the data – but doing something with it!

Filed under F#, OData, WebApi

Handling images in WebApi

October 1, 2013 15 Comments

I am digging into WebApi (v1) and one of the issues I was facing was images – specifically how to handle images. I spent a fair amount of time researching the possibilities over the last couple of days and this is what I think I think.

1) You can include your image in your JSON payload like this:

public class Person
{
    public Int32 PersonId { get; set; }
    public String FirstName { get; set; }
    public byte[] Image { get; set; }
}

or you can include the imageUri in your JSON payload like this:

public class Person
{
    public Int32 PersonId { get; set; }
    public String FirstName { get; set; }
    public String ImageUri { get; set; }
}

The advantage of the former is that you can make 1 POST or GET call and get all of the data. The disadvantages are that the image gets serialized into a large size and your service can get bogged down processing images when the rest of the payload is very small.

The advantage of the later is that you can separate the image into its own location – in fact not even use a Controller, and the processing is snappy-pappy. The downside is that you need to make 2 GETs and POSTs for any given person (in this example). For the POST, there is the additional factor that you have to wait for the 1st post to complete so you can get some kind of association key returned so that both POSTs can be associated together I was talking to fellow TRINUG member Ian Hoppes about problem and he came up with a great idea – have the client generate a GUID and then push both POSTs at the same time.

In any event, if you separate the payload into two POSTS, you need a way of handling the image alone. In WebApi V1, there seems to be 3 different ways you can do this:

A Controller
A Handler
A Formatter

I wasn’t sure what was the <right> answer so I posted this question to stack overflow – and no one answered it (at the time of this writing). Left to my own devices, I decided to use an image controller so that the API has consistent implementation.

I newed up an Image Controller and added 3 methods: GET, POST, and DELETE. The GET was pretty straight foreward (I used a Int32 on the parameter because I haven’t implemented the GUID yet):

public HttpResponseMessage Get(int id)
{
    var result = new HttpResponseMessage(HttpStatusCode.OK);
    String filePath = HostingEnvironment.MapPath("~/Images/HT.jpg");
    FileStream fileStream = new FileStream(filePath, FileMode.Open);
    Image image = Image.FromStream(fileStream);
    MemoryStream memoryStream = new MemoryStream();
    image.Save(memoryStream, ImageFormat.Jpeg);
    result.Content = new ByteArrayContent(memoryStream.ToArray());
    result.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
 
    return result;
}

The DELETE was event easier:

public void Delete(int id)
{
    String filePath = HostingEnvironment.MapPath("~/Images/HT.jpg");
    File.Delete(filePath);
}

The POST, not so easy. My first attempt was to read the steam and push it into an Image:

public void Post()
{
    var result = new HttpResponseMessage(HttpStatusCode.OK);
    if (Request.Content.IsMimeMultipartContent())
    {
        StreamContent content = (StreamContent)Request.Content;
        Task<Stream> task = content.ReadAsStreamAsync();
        Stream readOnlyStream = task.Result;
        Byte[] buffer = new Byte[readOnlyStream.Length];
        readOnlyStream.Read(buffer, 0, buffer.Length);
        MemoryStream memoryStream = new MemoryStream(buffer);
        Image image = Image.FromStream(memoryStream);
    }
    else
    {
        throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.NotAcceptable, "This request is not properly formatted"));
    }
}

No dice, because the stream includes things other than the image binrary. My second attempt was to use the MutlipartFormDataStreamProvider (which is the most popular way apparently) like so:

public Task<HttpResponseMessage> Post(int id)
{
    if (!Request.Content.IsMimeMultipartContent())
    {
        throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
    }
 
    string root = HttpContext.Current.Server.MapPath("~/Images");
    var provider = new MultipartFormDataStreamProvider(root);
 
    var task = Request.Content.ReadAsMultipartAsync(provider).
        ContinueWith<HttpResponseMessage>(t =>
        {
            if (t.IsFaulted || t.IsCanceled)
            {
                Request.CreateErrorResponse(HttpStatusCode.InternalServerError, t.Exception);
            }
 
            foreach (MultipartFileData file in provider.FileData)
            {
                //Trace.WriteLine(file.Headers.ContentDisposition.FileName);
                //Trace.WriteLine("Server file path: " + file.LocalFileName);
            }
            return Request.CreateResponse(HttpStatusCode.OK);
        });
 
    return task;
}

I don’t like this because I am writing to disk first – and I don’t need to use the file system – in fact, it might be locked down in some scenarios. So finally, I cobbled together some posts and found a way to read the parsed stream and create an image without using disk:

public HttpResponseMessage Post()
{
    var result = new HttpResponseMessage(HttpStatusCode.OK);
    if (Request.Content.IsMimeMultipartContent())
    {
        Request.Content.ReadAsMultipartAsync<MultipartMemoryStreamProvider>(new MultipartMemoryStreamProvider()).ContinueWith((task) =>
        {
            MultipartMemoryStreamProvider provider = task.Result;
            foreach (HttpContent content in provider.Contents)
            {
                Stream stream = content.ReadAsStreamAsync().Result;
                Image image = Image.FromStream(stream);
                var testName = content.Headers.ContentDisposition.Name;
                String filePath = HostingEnvironment.MapPath("~/Images/");
                String[] headerValues = (String[])Request.Headers.GetValues("UniqueId");
                String fileName = headerValues[0] + ".jpg";
                String fullPath = Path.Combine(filePath, fileName);
                image.Save(fullPath);
            }
        });
        return result;
    }
    else
    {
        throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.NotAcceptable, "This request is not properly formatted"));
    }
 
 
}

Note that I am using a disk right now on the POST and GET but I am going to swap it out with Azure Blob Storage. That way I can just create the image and push it over to Azure without my local disk being used at all.

Sure enough, this works like a champ. The GET

and the POST (using Fiddler – you have to use the most recent Fiddler so you get the [Upload File] link):

Another note is that I am pushing the UniqueID of the image in the request header – not in the content’s header.

This works:

(String[])Request.Headers.GetValues("UniqueId");

This does not:

var testName = content.Headers.GetValues("UniqueId");

Filed under WebApi

Jamie Dixon's Home

Setting up an OData Service on WebAPI2 to be used by F# Type Providers

Handling images in WebApi

Categories

Recent Posts

Archives

Blogroll

Meta