Skip to content

Instantly share code, notes, and snippets.

@enricosada

enricosada/fsharp and .net core sdk status.md

Last active March 29, 2020 13:13
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save enricosada/539da221ef3d87f0151d395f7889f94c to your computer and use it in GitHub Desktop.
Save enricosada/539da221ef3d87f0151d395f7889f94c to your computer and use it in GitHub Desktop.
Download ZIP
.NET Core sdk, msbuild, fsharp, fsc, etc
Raw
fsharp and .net core sdk status.md

.NET Core sdk, msbuild, fsharp, fsc, etc

How to finish/polish the integration between msbuild based .net core sdk and f# (fsc, FscTask, etc)

This doc contains current issues i known and a proposed solution (already implemented in a PR, and works) on how to fix these (obv ihmo).

AFAIK all the solution i choose are based on where dotnet/sdk, microsoft/msbuild and dotnet/cli are going with development, and updated to latest release (preview4), and vnext in development now.

TL;DR

Current f# compiler package is not enough on integrate with dotnet cli without hacks. And are not needed.

I'll try to explain why. If there are other way to do it, feedback welcome.

My PR dotnet/fsharp#2250 fix all the following issues already, if ok my reasoning, everyhing is already implemented!

Just check the new package created by my PR, download it from myget feed Is not too different, but fix all these issue

Considerations:

  • fsc is just a normal console app, like FSharp.Core is a normal library
  • When i say "from nuget packages", i mean added with a <PackageReference />
  • The new sdk (Msbuild 15) give new features for msbuild authoring
    • Extensibility with auto-import of target files and .props files from nuget packages based on a naming convention.
    • Assemblies who contains MSBuild tasks can be included from nuget packages (distinct both .net and .net core), and are not considered like fsc references
    • Sdks (like FSharp.NET.Sdk;Microsoft.NET.Sdk) are normal nuget packages, imported first, but atm (wip) need to be bundled with .net core sdk
    • Microsoft.NET.Sdk has an extensibility to add new languages
  • with msbuild project, the dotnet-compile-fsc should die, in favor of using the normal FscTask

Proposed solution (explained below), and already implemented in my PR:

  • fsc deployed as normal framework dependent app, inside a compiler package
  • the compiler package has the right layout and a msbuild .props file, to make it easy to hook the fsc from FSharp.NET.Sdk package

XPlat console app with .net core, how to

With a .net fw console app, just xcopy the myapp.exe, and run it with myapp.exe or mono myapp.exe.

This show the current way to deploy an xplat console app, who can be xcopied in any os where .net core runtime is installed.

So after installing .NET Core Sdk preview4 (see links archive for preview4 )

mkdir myapp
cd myapp
dotnet new -t console -l fsharp
dotnet restore
dotnet publish -o deploy

this create a deploy directory who contains an .net core console app as framework dependent deployment

That directory contains:

  • myapp.dll the entry point of console app
  • myapp.runtimeconfig.json who specify runtime options, like GC tuning, or framework used
  • myapp.deps.json metadata from the lock file used in build
  • all the assemblies referenced but not embedded in Microsoft.NETCore.App pkg, like FSharp.Core.dll
  • a runtimes directory and subdir with all os specific files (also assemblies).

You can xcopy this directory content in any os where the .net core runtime is installed (full sdk is not needed) and can be run with dotnet ./path/to/myapp.dll

PRO: doesnt need multiple version per os CONS: need dotnet runtime installed (not an issue for fsc)

Current behaviour using preview2/4 with dotnet-compile-fsc, and why i say hacks

The Microsoft.FSharp.Compiler.netcore doesnt contains a real FDD deploy of fsc because some files (like fsc.deps.json) are missing (doesnt matter the package version). If you have it in nuget cache, and you run

dotnet "C:\Users\%USERNAME%\.nuget\packages\Microsoft.FSharp.Compiler.netcore\1.0.0-rc-170122\lib\netstandard1.6\fsc.exe"

Give an error The library 'hostpolicy.dll' required to execute the app... etc

The dotnet build it's fsc all the way down, but how? who call it? why works?

To see it, just do dotnet -v build /v:n (yes sucks, -v for netcoresdk verbose, and /v:n for msbuild verbose). For preview2, just do dotnet -v build instead.

Atm the dotnet build process (semplified, args omitted) is:

dotnet build -> dotnet compile-fsc -> dotnet /path/to/dotnet-compile-fsc.dll -> dotnet /path/to/fsc.exe

the trick used to make dotnet /path/to/fsc.exe is (semplified)

dotnet.exe exec --depsfile /path/to/dotnet-compile-fsc.deps.json /path/to/fsc.exe @dotnet-compile-fsc.rsp

Pratically is invoking the fsc.exe exe passing the deps.json file of the dotnet-compile-fsc tool. Because they have similar dependencies (or all are added), that works, sort of. but is hacky. Workaround to use a different deps file for another app. There is no need to do that, if fsc is a normal console app.

This has some big issues:

  • dotnet-compile-fsc need to die, was an useful wrapper only for preview2 (project.json).
  • preview4 (msbuild) should use instead the msbuild FscTask from FSharp.Build (implemented already in my PR), like old fsproj
  • If FscTask need to call fsc, how can do that? Two issue:
    • where is fsc installed? the dotnet-compile-fsc can infer location because compiler package is a deps of the dotnet-compile-fsc cli tool. FscTask cannot do the same, because is different (library vs cli tool invocation).
    • how fsc can be run? As before, if i know the path of fsc, just dotnet /path/to/fsc.exe doesnt works.

Let's asses how to fix that.

Options for fsc usage

Can we do like c# compiler? NO, and we shouldnt ihmo

What c# does with csc compiler? it's bundled with .net core sdk. Can be run from install dir of netcore sdk /sdk/1.0.0-preview4-004233/RunCsc.cmd. That's just a .bat wrapper for dotnet "csc.exe" %* (like RunCsc.sh). And the csc.exe is like myapp.dll, a netcore console app. The bundled directory contains a framework indipendend deploy of csc, but that's because the .NET Core sdk is build for every os, and they reuse the shared fw dlls also for other things (msbuild/nuget/etc).

We dont need that for fsc, we should not publish multiple version for os, NOR need to bundle fsc inside sdk.

If we package fsc inside a nuget package, we get two issues:

  • How make it xplat, easy to run
  • How to discover location, because a package can be installed anywhere (csc is in a fixed path relative to sdk install, doesnt have this issue)

Two options left for fsc deployment:

  1. As dotnet cli tools (<DotNetCliToolsReference>), so per project (can be auto included by f# sdk pkg) and after that can be run just with dotnet fsc:
    • PRO no need to know fsc location, dotnet resolve that when doing dotnet fsc command.
    • CONS need dotnet/cli#4722, otherwise we need to rename fsc as dotnet-fsc
    • it's anyway a console app, just include dependencies as package dependency (installed on dotnet restore) instead of bundled inside package
  2. Add the fsc published as framework dependent deply inside a nuget package
    • PRO easy to discover location, if we add a .props file inside the package (see my PR dotnet/fsharp#2250 )
    • CONS all deps (but not framework assemblies!, it's FDD) are inside the package, not reused from nuget cache. But nupkg is just 7MB, so negligible
    • PRO can optionally contain also .net fw version of fsc, like FSharp.Compiler.Tools nupkg, so msbuild.exe can run the .NET version, and dotnet msbuild can run the netcore version
    • PRO unzip the nupkg (like FSharp.Compiler.Tools), and you can do dotnet /path/to/unzipped/build/netcoreapp1.0/fsc.dll and will works!

I think the best solution is the 2. My PR implement that. The generated package works like described there.

NOTE Same FDD publish should be done ihmo also for FSI, so dotnet /path/to/unzipped/fsi.dll will work. NOTE Btw ihmo fsi should be a dotnet cli global command, when these are implemented. if is a netcore console app, is easier afaik

Info about new msbuild sdk extensibility

Auto imported files based on naming convention. Mechanisms has some details, but high level is like the following.

Example: You have a MyPlugin nuget package, referenced as PackageReference

 <ItemGroup>
    <PackageReference Include="MyPlugin" Version="*" />

RULES:

  • if exists inside the nupkg a build/MyPlugin.props file, is auto imported at BEGINNING of msbuild project.
  • if exists inside the nupkg a build/MyPlugin.target file, is auto imported at END of msbuild project.
<Project .. >
  <!--AUTO IMPORTED PROPS-->

  my msbuild code

  <!--AUTO IMPORTED TARGETS-->
</Project>

The msbuild project also has the Sdk attribute (<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk" ToolsVersion="15.0">) These are nuget packages with same rules as before (like MyPlugin) but:

  • are included before any PackageReference props/target. That's good because they contains this auto-import mechanism, compile/pack/publish target (Microsoft.NET.Sdk)
  • The f# stuff (FSharp.NET.Sdk), is imported before Microsoft.NET.Sdk because set a property require for integration used there Basically tell the to Microsoft.NET.Sdk what file contains the f# CoreCompile target. That's the extension point in sdk to add new languages (just implementing a CoreCompile target)
  • ATM (it's temporary) are embedded in sdk bundle. If it's not inside .netcore sdk directory sdk\1.0.0-preview4-004233\Sdks, cannot be used. In rtm (or after dunno) these should be directly restored from nuget too. So like <Project Sdk="FSharp.NET.Sdk/2.0.0;Microsoft.NET.Sdk/1.0.0" ToolsVersion="15.0"> and instead of looking inside intalled sdk, the FSharp.NET.Sdk version 2.0.0 package will be used.
  • For f# <Project Sdk="FSharp.NET.Sdk i use a trick with props files, so real file with f# CoreCompile target is not bundled, but is instead inside <PackageReference Include="FSharp.NET.Sdk" Version="1.0.0-beta-*">. Why? because like that we can update it with just a new FSharp.NET.Sdk on nuget.org, without waiting a new sdk bundle. Pratically i partially implement the <Project Sdk="FSharp.NET.Sdk/2.0.0;, adding a <PackageReference (final rtm should not need this duplication)

Usually extension with <PackageReference are enough, but for some stuff, can be useful Sdk attribute

For msbuild Task from assemblies, can be put inside this or another nuget package too. In the target file, i can just import a tasks assembly using a path relative to target file.

So having:

  • build/Myplugin.target
  • build/Myplugin.Tasks.dll

In the auto-imported Myplugin.target, i can just

  <UsingTask AssemblyFile="$(MSBuildThisFileDirectory)/Myplugin.Tasks.dll" TaskName="DoSomething" />

In my PR i dot that for FscTask in FSharp.Build.dll (inside FSharp.Compilers.netcore package), who is imported and used inside a target of FSharp.NET.Sdk package

So packages can use assemblies/targets/props of other nuget packages. No hardcoded local filesystem paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment