A NuGet package workflow using GitHub Actions

Background

I’ve had a few ideas for this year’s C# Advent, but this one feels the most relevant at this time. The client I’m currently working with, builds their NuGet packages on developers’ machines; it’s a practice that generally works and is not too labour intensive, but occasionally steps are missed and problems can occur. The relative infrequency of builds means developers are not sufficiently motivated to automate the task, but as we’ll see, it’s not too difficult. This post walks through my preferred solution.

Is this for me?

However you build your NuGet packages, I hope you’ll find this post useful. I’m going to walk through how to build a simple NuGet package (along with a test project) in GitHub Actions. We will progress from building every push to a main branch, through building Pull Requests to publishing to a NuGet package repository. Later on I’ll demonstrate how to build pre-release NuGet packages and end by showing you my attempt at auto-generating release notes.

TL;DR;

The code and workflows demonstrated here can be found in GitHub at https://github.com/acraven/blog-nuget-workflow-github-actions. Feel free to fork the repository, push some commits, cross your fingers, and watch the workflows run. I’ve tried to keep the commits in line with the sections in this post so you can refer back if needs be.

Getting Started

I’ve created a baseline solution containing a library project and a test project. I’ve pushed that as the first commit to my repository — hopefully, if your existing repository is not too different, we can add the workflows together.

Build and Test Every Push to Main

The first thing we should do is ensure that the code can be built, and all tests run successfully after pushing to the main branch. We are going to configure GitHub Actions to do this by adding a new file.

Create a new file .github/workflows/ci.yml in your repository containing the following snippet and push it to the main branch. It consists of two simple steps, one to build the code and another to test it.

on:
  push:
    branches:
    - main
jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Build
      run: dotnet build --configuration Release
    - name: Test
      run: dotnet test --configuration Release --no-build

One of the best features of GitHub Actions is that it automatically discovers workflows that are pushed to any branch; and if the push matches the trigger condition the workflow is immediately run without the need for any other setup. With GitHub Actions, the repository becomes responsible for its own integrity and artifact creation, which is great from a DevOps point of view.

GitHub Actions workflows seem to run almost instantly, even in the free tier which is awesome, so by now your build is most likely underway; you can see its progress by navigating to the Actions tab in your repository (that is https://github.com/acraven/blog-nuget-workflow-github-actions/actions for me).

Click on the link to view the progress — when it’s completed, you should see something like this:

This is a good start; the code is built and all the tests are run whenever anyone pushes to the main branch. However, it’s far from a workflow, it doesn’t publish to a NuGet package repository nor does it support running tests prior to merging Pull Requests.

Build and Test Your Pull Requests

Replace the trigger condition in the ci.yml file as shown below and commit the change, but don’t push just yet.

on:
  push:
    branches:
    - main
  pull_request:
    branches:
    - main

The on: block of a workflow file defines the trigger conditions; the pull_request block added instructs GitHub Actions to run the workflow whenever a Pull Request (PR) is created that targets the main branch, or, whenever additional commits are pushed to those PRs. Instead of pushing the commit to the main branch, if you create and push a new branch for it, and create a PR for the new branch, you should see the workflow running and shown as a successful check when complete.

I know it can be controversial, but I would always protect the main branch, so that changes to it must be made via a PR and any checks must pass before merging. This can be painful for small changes (and can be avoided by administrators), but I think the benefit gained by always having the HEAD of main in a state of green is worth it. But that’s a story for another day, so just go ahead and merge the PR into the main branch.

What we have now is valuable, but we are still missing a crucial step for NuGet packages; publication, the one that enables not just you, but others, to consume your code.

How to Publish to a NuGet Package Repository

Create a new file .github/workflows/release.yml in your repository containing the following snippet, but don’t commit just yet. Before we continue, you’ll need to change the name of the package in the push step as well as changing the username in the [package repository] source attribute.

on:
  push:
    tags:
    - "v[0-9]+.[0-9]+.[0-9]+"
jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Verify commit exists in origin/main
      run: |
        git fetch --no-tags --prune --depth=1 origin +refs/heads/*:refs/remotes/origin/*
        git branch --remote --contains | grep origin/main
    - name: Set VERSION variable from tag
      run: echo "VERSION=${GITHUB_REF/refs\/tags\/v/}" >> $GITHUB_ENV
    - name: Build
      run: dotnet build --configuration Release /p:Version=${VERSION}
    - name: Test
      run: dotnet test --configuration Release /p:Version=${VERSION} --no-build
    - name: Pack
      run: dotnet pack --configuration Release /p:Version=${VERSION} --no-build --output .
    - name: Push
      run: dotnet nuget push NuGet.Workflow.${VERSION}.nupkg --source https://nuget.pkg.github.com/acraven/index.json --api-key ${GITHUB_TOKEN}
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This is obviously more complicated than the original ci.yml file, so I’ll run through the differences.

• Perhaps most importantly, the trigger has changed, this workflow will run whenever a tag is pushed to the repository that matches the v#.#.# pattern (where # is one or more digits).
• The verify step ensures that the commit tagged with the version number exists in the main branch. This could easily be removed, but it protects me and others (but mostly me) from incorrectly tagging a short lived branch.
• The version (without the v prefix) is extracted from the tag and used to set the VERSION environment variable for the remaining steps.
• The build and test steps are the same except that the VERSION is embedded in the package build.
• The pack step prepares the package for pushing to a package repository. The .nuget extension is the convention, but it’s just a zip file. If you were to add a .zip extension you can view the contents, which can be useful for debugging purposes.
• Finally the push step pushes the package to the GitHub Packages package repository. You could push to another NuGet package repository by changing the source attribute, but you’ll need to provide an access token via an explicit GitHub Secret.

Before we commit and push the release workflow, the following snippet needs to be added to your package .csproj file and changed appropriately. This adds meta-data to the .nuget package in the pack step in order to be able to push it to a package repository.

  <PropertyGroup>
    <PackageId>NuGet.Workflow</PackageId>
    <RepositoryType>git</RepositoryType>
    <RepositoryUrl>https://github.com/acraven/blog-nuget-workflow-github-actions</RepositoryUrl>
  </PropertyGroup>

Go ahead, commit and push these two files to the main branch. You can now tag that commit with v0.0.1, making sure it’s a remote tag (ie. it’s pushed) otherwise the workflow won’t run.

If you head to the Actions tab you should see a workflow in progress for your new version. Once it completes you will be able to see the package in the package repository and pull it into your projects.

In order to be able to consume this package though, you would need to setup a NuGet package source in your IDE with credentials to access the GitHub package repository. To use it in another workflow, you would have to use the setup-dotnet action (before the build step) to specify those same credentials via a GitHub secret. The credentials would be in the form of a Personal Access Token with at least read:packages scope.

Should you push your package to NuGet.org instead then this wouldn’t be required, but your package would be publicly available.

What about Pre-release packages?

There are times when developers need to be able share packages that aren’t considered complete. The NuGet specification enables this with the concept of a Pre-release flag — adding a - and a suffix to the version number of the package is enough to set this flag. I happen to use -preview### but other patterns including alpha### and beta### are equally as good.

The trigger in the release.yml file only matches against v#.#.#, so rather than changing the existing file, I have a separate file to manage the pre-release workflow. It would certainly be possible to combine both workflows into one file using conditional steps, but I prefer the separation.

Create a new file .github/workflows/release-preview.yml in your repository containing the following snippet and push it to the main branch — again remembering to change the name of the package and the source attribute.

on:
  push:
    tags:
    - "v[0-9]+.[0-9]+.[0-9]+-preview[0-9][0-9][0-9]"
jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Set VERSION variable from tag
      run: echo "VERSION=${GITHUB_REF/refs\/tags\/v/}" >> $GITHUB_ENV
    - name: Pack
      run: dotnet pack --configuration Release /p:Version=${VERSION} --output .
    - name: Push
      run: dotnet nuget push NuGet.Workflow.${VERSION}.nupkg --source https://nuget.pkg.github.com/acraven/index.json --api-key ${GITHUB_TOKEN}
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

There are some notable absences here in order to speed up the process — the branch check is removed as the commit is more likely to belong to a branch other than main, and the tests have been removed too. I’m happy that the PR workflow in ci.yml will catch any failing tests; this workflow for me is about sharing unfinished work quickly.

You will now be able to tag any commit in any branch with the pattern v#.#.#-preview### and a pre-release package will be pushed to the package repository.

This does mean that there is untested code in circulation. I haven’t got a problem with that as developers have to opt-in to receive it; it does however mean we have to be careful not to merge code that consumes a pre-release package to its own main branch.

This approach supports including pre-release packages in other pre-release packages too, for example when separating abstractions and concrete classes into two packages, although unwinding the “stack” with full versions can be a bit painful.

What about Release Notes?

I was challenged by a colleague to add release notes to the package build, which I accepted, and kind of achieved, but it feels like I cheated a little as you’ll see. It’s very much a work-in-progress so feel free to skip this section.

The versioning strategy used in this post doesn’t lend itself well to maintaining release notes in the repository, as the version is dictated by the tag used to trigger the release build. Often release notes are embedded in the committed content of the repository in aRELEASE-NOTES.md file or similar, but with this workflow the repository content does not have the luxury of knowing its version at the point of committing changes.

For sure it would be possible to maintain such a file and add version numbers to it, but that creates a point of failure where the tagged and built version number may not match the version number in the release notes.

We need to make a few changes; replace the checkout and verify steps in release.yml with the following snippet.

    - name: Checkout
      uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - name: Verify commit exists in origin/main
      run: git branch --remote --contains | grep origin/main
    - name: Extract release notes
      run: |
        git log --pretty=format:'%d %s' ${GITHUB_REF} | perl -pe 's| \(.*tag: v(\d+.\d+.\d+(-preview\d{3})?)(, .*?)*\)|\n## \1\n|g' > RELEASE-NOTES.txt

Add the following snippet to your package .csproj file.

  <Target Name="PrepareReleaseNotes" BeforeTargets="GenerateNuspec">
    <PropertyGroup>
      <PackageDescription>$([System.IO.File]::ReadAllText("$(MSBuildProjectDirectory)/../../RELEASE-NOTES"))</PackageDescription>
    </PropertyGroup>
  </Target>

And an empty RELEASE-NOTES file to the root of your repository. You may need to adjust the path I’ve used in the .csproj file above depending on the layout of your own repository.

The change to the checkout step pulls all commits from the repository, rather than just the tagged commit. This has the benefit of simplifying the verify step, but be warned that pulling large numbers of commits will obviously slow down the workflow for larger repositories.

The new release notes step is my cheat — I’m using git-log and regular expressions to parse commit messages and version tags. The limiting factor here is my knowledge of regular expressions, so this comes with a big caveat — there may well be certain tags, branch names or commit messages that this solution can’t handle.

Commit those changes, push to the main branch and tag the commit with v0.0.2. Once the workflow has completed, you can view the package along with its release notes.

I tried, but failed, to create a regular expression that ignored my pre-release tags. I would prefer them not to appear here, but as remote tags are tricky to remove from repositories, a change to the regular expression would be welcomed.

Summary

We’ve seen how to automate workflows for developing and publishing NuGet packages. There are many other ways of achieving this, but I’ve found this approach successful for managing departmental NuGet packages where there are generally only double digits of consumers with good communication channels. For public NuGet packages with potentially 10s or 100s of 1000s of consumers, skipping testing for pre-release packages is probably not desirable and the depth of the release notes would no doubt be found to be lacking.

I hope you’ve found this post useful and interesting. Be sure to check out the other posts that are part of the 4th C# Advent series.

Credits

Credit is due to https://dev.to/j_sakamoto and https://dev.to/dimay for their post https://dev.to/j_sakamoto/writing-a-nuget-package-release-notes-in-an-outside-of-a-csproj-file-3f94 and comment respectively. Their solution of how to add an external release notes file into a .csproj file was most useful.

And a big shout out to https://medium.com/@dcook_net for providing a valuable review.