CI/CD for .NET with GitHub/Travis-CI/AppVeyor/Codecov/NuGet

I recently setup continuous integration for a few open source projects. I used GitHub, AppVeyor, Travis-CI and Codecov. The resulting packages are deployed to NuGet.org. 

Here I aim to describe the process in an attempt to document it both for myself and hopefully help others in getting started with it.

Scenario

I have some .NET Core projects on GitHub that are actually just .NET Standard libraries.

Ultimately, once they are built and tested on Windows, MacOS and Linux, I want them published to NuGet.org.
I also want to use GitHub's Release feature to document what I am pushing to NuGet.

Some of these projects actually generate more than one NuGet package. That's the case for Greentube.Messaging, Greentube.Serialization and Greentube.Monitoring. Depending on the change I make, I might be looking at publishing any number of those packages. I don't want to publish all packages created while building the repository every time.

TL;DR 

Travis-CI is used only to build and run tests on MacOS and Linux.
AppVeyor, which runs on Windows, builds, run tests while tracking code coverage and sends the results to codecov.io. The result of AppVeyor's build are artifacts, .nupkg files (aka: NuGet packages) ready to be sent to Nuget.org.

If AppVeyor is building a tag, it changes the build version to be the value of the tag. That sets the actual version of the NuGet packages created. Also, AppVeyor then creates a GitHub release in draft mode.

At that point from AppVeyor I can publish individual packages to NuGet with a button click.
The release notes can be added to the GitHub Release and then it can be published.

How?

Nearly all settings to AppVeyor, Travis-CI and Codecov are defined on their respective configuration files. Although it's possible to configure all of those via their Web UI, I personally rather use the text configuration file which is version controlled.

Travis-CI

Previously I mentioned there's no code coverage coming from the Travis-CI build. That’s because as of today there's no way to do it. OpenCover and vstest don't support Linux or MacOS because there's still no released profiling API for those platforms. That seems to be going to change soon though.
Nonetheless, Travis-CI is a very nice, free for open source, CI system. Changes pushed to GitHub automatically trigger a build in both Linux and MacOS:

Using the Greentube.Messaging Travis-CI configuration file as an example: .travis.yml

Firstly note that mono is set to none since I'm building with dotnet here. Since tests run CoreCLR, only .NET Core SDK is required. I'm bulding the specific projects although from the SDK version 2.0.0 forward or so it's possible to build a solution file. That is because all I want here is to run my tests. Running dotnet test will build the dependent projects which in turn will run a dotnet restore if required. Again, this was not possible with .NET Core SDK 1.x.

The ulimit -n 512 is there to solve a problem when restoring NuGet packages which open too many connections. Also while trying to solve a problem, I've added osx_image: xcode8.1 to be able to build with .NET Core SDK 2.0.

There're tons of other examples on how to get Travis-CI to build a .NET Core project.

AppVeyor

Here there's more going on. Besides building all commits, including pull requests, AppVeyor will take code coverage and send it to codecov, create GitHub Releases and publish packages to NuGet.

I'm going to start by describing things before spitting the configuration file in front of you:

Code coverage

I've tried to summarize the code coverage setup on StackOverflow before but it was never marked as an answer so here goes another try. :)

Configuration

All project files have an extra configuration called Coverage with: DebugType=full

The reason for the full PDB configuration is that by default it compiles with portable pdbs and those are still not supported.

Run

Code coverage is tracked with OpenCover. I wrote a simple powershell script that will download the necessary tools and, using the .NET Core CLI, it runs all tests while tracking code coverage.

It has two optional arguments:

1. generateReport
• Generates an HTML report useful for testing things before sending to Codecov
2. uploadCodecov
• Sends the coverage data to Codecov.io

Previously I mentioned that nearly all settings are source controlled. That is not true to the API KEY used to send code coverage results to Codecov. That value is defined as an environment variable (named CODECOV_TOKEN) on AppVeyor. 

Packaging

The projects being built don't have their version on csproj defined before committing to GitHub.

They are always set to 0.0.0 and that gets replaced (aka patched) by AppVeyor before building and packaging it.

As mentioned before, the result of AppVeyor builds, as other CI systems, are called artifacts. What is included in the artifacts list is specified via the configuration key with that same name. A glob pattern to include app nupkg is enough then.: '**\*.nupkg'

Packages are created with the dotnet pack command. That is true to all but 1 package: metapackage

Metapackage

One thing I've left out of the summary is the creation of a metapackage. That's an attempt to ease into the API adoption by providing the simplest way to get started.

Note that obviously the packages referenced by this metapackage are not included in the runtime package store. That means that using this will in fact include the dependent packages in your publish output.

I'm publishing two metapackages, each from their own repository:

1. Greentube.Serialization.All
2. Greentube.Messaging.All

Someone wants to try the API?  Run a single dotnet add package command and all of the building blocks are available to them. 

I don't expect it to be used in production though. I advise the fine grained approach instead (pay-for-play).

Create

Runing dotnet pack on a project that has no code yields no package! 

The CLI will not build up an empty package. To create a metapackage you need to use a nuspec file.

To pack this one up, there's a line in the configuration invoking nuget pack instead.

Creating a Release

I've added to .appveyor.yml (see below) a piece of powershell which resets the version of the build to the tag if that's what's being built.

Pushing a tag is the method to express the intent of a creating release.

I'm using semver and NuGet already takes packages versioned N.N.N-something as a pre-release package.

That means a tag 1.0.0-beta will publish a pre-release package. On the other hand, if the tag is simply 1.0.0, that'd be a plain release.

GitHub on the other hand doesn't handle that automatically. AppVeyor creates the draft release on GitHub but at the bottom of the page before publishing, I still need to select that option:

AppVeyor will push every artifact of that build to GitHub's drafted release. Considering that I do not always publish all packages to NuGet, I remove whatever package I didn't publish to NuGet before finalizing the GitHub release.

In other words:

1. Push a tag to GitHub, 
2. Publish desired packages to NuGet through AppVeyor UI (more below).
3. Remove from the GitHub release the packages not deployed on the next step
4. Publish release on GitHub (green button above)

NuGet

Publishing to NuGet is not done automatically as a result of the release creation mentioned above. The reason is that each build results in multiple packages as artifacts and publishing is a all or nothing approach.

I'm looking for publishing packages in a selective way. AppVeyor has environments which can be configured to deploy specific artifacts from specific projects. Those can also be defined via the configuration file but you'd still need to define which 'environment' (aka: package) you want to deploy anyway. At that point I've chosen to use AppVeyor's UI to set-up these environments.

The way I've done it is: one AppVeyor's environment for each package:

To get something deployed I can click a single button which pushes that single package to NuGet.org.

Differentiating each of those is done via regular expressions. For example the RegEx for the first item on that image is:

/Greentube\.Messaging\.\d.*\.nupkg/

I'm still trying to push the markdown docs automatically together to NuGet. I've asked about it on StackOverflow a few days ago but still not answer.

One downside of this approach is that the dependencies between the packages have to be tracked by yourself.

For example:

If I want to publish Greentube.Messaging.DependencyInjection.Redis, I need to take into consideration that it depends on Greentube.Messaging.DependencyInjection which in turn depends on Greentube.Messaging of the same version.

For that reason, the 3 packages have to deployed otherwise restoring the first one will fail.

Configuration

Again using the Greentube.Messaging as an example, here's the .appveyor.yml

Note about the pace of changes

Everything about .NET Core changes rapidly. The tooling around it specially. A lot of configuration I've come up with was added as a reaction to some issue I encountered. I'm sure many of these issues will be solved soon so configuration could be simplified. An example is the full pdbs for coverage and the profiling API on MacOS. Take that into account when getting your own setup ready.