Skip to content

Instantly share code, notes, and snippets.

@awright18
Last active March 27, 2020 11:24
Show Gist options
  • Save awright18/d4873a8ff3b018181b7ce89586d5144e to your computer and use it in GitHub Desktop.
Save awright18/d4873a8ff3b018181b7ce89586d5144e to your computer and use it in GitHub Desktop.
F# http reqeust with HttpClient
// Learn more about F# at http://fsharp.org
open System
open System.Text
open System.Net.Http
open Rss
[<EntryPoint>]
let main argv =
let doXmlStuff = async {
let client = new HttpClient()
let! response = client.GetByteArrayAsync("http://pablojuan.com/feed/")
|> Async.AwaitTask
let content = Encoding.UTF8.GetString(response,0, (response.Length - 1))
let parsedStuff = deserializeXml content
do printfn "%s" parsedStuff.channel.title
return ()
}
doXmlStuff |> Async.RunSynchronously
0 // return an integer exit code
module Rss
open System.IO
open System.Text
open System.Xml
open System.Xml.Serialization
[<CLIMutable>]
type RssItem = {
title:string;
link:string;
description:string;
author:string;
category:string option;
enclosure: string option;
guid: string option;
pubDate: string option;
source: string option;
}
[<CLIMutable>]
type RssChannel = {
title:string;
link:string;
description:string;
language: string option;
copyright: string option;
managingEditor: string option;
webMaster: string option;
pubDate: string option;
lastBuildDate: string option;
category:string option;
generator:string option;
docs: string option;
cloud: string option;
ttl: string option;
image: string option;
textInput: string option;
skipHours: string option;
skipDays: string option;
item: RssItem[];
}
[<CLIMutable>]
type Rss = {channel: RssChannel}
let deserializeXml str =
let root = XmlRootAttribute("rss")
let serializer = XmlSerializer(typeof<Rss>,root)
use reader = new StringReader(str)
serializer.Deserialize reader :?> Rss
<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp1.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="rss.fs" />
<Compile Include="Program.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.Core" Version="4.1.*" />
<PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
<!-- You need to add this for System.Net.Http to work. After adding this do a dotnet restore -->
<PackageReference Include="System.Net.Http" Version="4.3.*" />
<PackageReference Include="System.Xml.XmlSerializer" Version="4.3.*" />
</ItemGroup>
</Project>
@yelluw
Copy link

yelluw commented Jul 5, 2017

@awright18 I'm thinking more of getting the RSS feed xml and turning it into JSON in the same place. That way all data inside the backend is in the same format (JSON).

@yelluw
Copy link

yelluw commented Jul 7, 2017

Ok, so this is now spitting out the XML as a string. At least it sort of works. :)

open System
open System.Net.Http

open System.Xml

open System.Text

open Newtonsoft.Json


[<EntryPoint>]
let main argv =

    let rssFeedToJson = async { 
            let client = new  HttpClient()
            let! response = client.GetByteArrayAsync("http://pablojuan.com/feed")                         
                            |> Async.AwaitTask

            let content = Encoding.UTF8.GetString(response,0, (response.Length - 1))
            return content
        }

    rssFeedToJson 
    |> Async.RunSynchronously
    |> printfn "%A"      
    0 // return an integer exit code

@yelluw
Copy link

yelluw commented Jul 7, 2017

@awright18

I have added some more stuff, gets you a little farther, but the Item array isn't getting serialized may need a different collection type instead of array.

I'll try to see if I can just dump it to json

@yelluw
Copy link

yelluw commented Jul 7, 2017

Seems I'm able now to dump it to json. Although the array issue on the code is still now solved. Progress! :)

// Learn more about F# at http://fsharp.org

open System
open System.Text
open System.Net.Http
open Rss

open Newtonsoft.Json

[<EntryPoint>]
let main argv =

    let doXmlStuff = async { 
            let client = new  HttpClient()
            let! response = client.GetByteArrayAsync("http://pablojuan.com/feed/")                         
                            |> Async.AwaitTask                                                     
            let content = Encoding.UTF8.GetString(response,0, (response.Length - 1))                 
           
            let parsedStuff = deserializeXml content 
            let json = JsonConvert.SerializeObject(parsedStuff)
            return json
            
        }

    doXmlStuff 
    |> Async.RunSynchronously
    |> printfn "%s"       
    0 // return an integer exit code
module Rss 

    open System.IO
    open System.Text
    open System.Xml
    open System.Xml.Serialization   

    [<CLIMutable>]
    type RssItem = {
        title:string; 
        link:string;
        description:string;
        author:string;
        category:string option;
        enclosure: string option;
        guid: string option;
        pubDate: string option;
        source: string option;
      }

    [<CLIMutable>]
    type RssChannel = {
        title:string;
        link:string;
        description:string;
        language: string option;
        copyright: string option;
        managingEditor: string option;
        webMaster: string option;
        pubDate: string option;
        lastBuildDate: string option;
        category:string option;
        generator:string option;
        docs: string option;
        cloud: string option;
        ttl: string option;
        image: string option;
        textInput: string option;
        skipHours: string option;
        skipDays: string option;
        item:  RssItem[];
        }

    [<CLIMutable>]        
    type Rss = {channel: RssChannel}
   
    let deserializeXml str =
        let root = XmlRootAttribute("rss")
        let serializer = XmlSerializer(typeof<Rss>,root)
        use reader = new StringReader(str)
        serializer.Deserialize reader :?> Rss

@yelluw
Copy link

yelluw commented Jul 7, 2017

In this line in Rss.fs:

item: RssItem[];

removing the [] returns the first item. I'm now working on adding all items to that array as you noted.

@yelluw
Copy link

yelluw commented Jul 7, 2017

@awright18

This was the issue (Mitch solved it):

        [<XmlElement>]
        item:  RssItem[];

You had to add that notation. :) It is now working as expected.

@yelluw
Copy link

yelluw commented Jul 7, 2017

I will now take this code and port it over to Nikeza. Thanks to all who helped!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment