Wednesday, December 30, 2020

[new blog](https://garcia.in)


Dear Blogger.com, we need to talk.


Since I met markdown, I've been wanting to tell you. It's not going to work between us anymore.


I'll now blog on https://garcia.in

Friday, September 11, 2020

Move repositories on GitHub without breaking links to PRs

 Since it took me way too much time to figure this out, I'll write it down for future references.


I needed to move a git repository from one GitHub repository to another one that already had code, past PRs, etc. It wasn't a simple rename which would make GitHub redirect everything. It was adding a new history from one repository to another, while not getting rid of the past tags, releases.

When you squash a PR on GitHub by default it'll write the commit message (#1)

Where the number following the # character is the PR number. When viewing this message on GitHub it'll link to the PR page on the repo.

If you simply push that git repo to another github repository, it'll link to the wrong PR page.


To fix that you need to rename the #1 to a fully qualified PR link that GitHub understands.

It has the format: organization/repository#1

 

For that, I've used a tool called git-filter-repo.

 

git filter-repo --commit-callback '
message = commit.message
new_msg = re.sub(br"(#(\d*))", br"org/repo\1", message, flags=re.MULTILINE)
print(new_msg)
commit.message = new_msg
' --refs HEAD


On the target repository I created an orphan branch:


git checkout --orphan main


Pull the rewritten repository into main branch, and pushed to remote. That's it.

Sunday, September 6, 2020

Swift adventures: CocoaLumberjack logger

I haven't had much contact with Swift other than writing a few unit tests on a project. So when someone asked about sending CocoaLumberjack logs to Sentry on the forum I thought it was a good excuse to write some Swift on the weekend.

To create an integration between two libraries written in Objective-C, to make sure you can support the most number of apps and platforms, the choice for a language is obvious: Objective-C.

The best thing about such a hobby project is:

Making sure you can support old code bases, where the developers have not invested in keeping things up-to-date doesn't not become a priority to you.

 

With that in mind, I went ahead with the latest stable Swift version: 5.2.

 

At the start I learned that I actually don't need xcode at all. So as part of my experiment, I decided to go without any of those xcworkspace, pbxproj, xcscheme etc, and use only Visual Studio Code.

And the Swift CLI:

 

swift build

and

swift run

 

With the right Visual Studio tasks and launch configuration I was even able to step through the code with the debugger. Generally really nice.


In the process I also learned about the Swift Package Index which builds your project with different Swift versions and targets and creates a badge describing the compatibility.

 

On top of all the fun learning a new language and ecosystem, I got a good reminder that dogfooding is an incredible tool. Specially when you're building APIs and SDKs. I would even say that it makes sense to have a project at work that the team could work together a few hours a week to get a good verification of the experience of using what you're building.


The code is here if you're interested: https://github.com/bruno-garcia/SentryCocoaLumberjack


The low res gif I recorded:



Tuesday, July 16, 2019

NuGetTrends: .NET libraries download trends

I joined sentry.io just over a year ago. Soon after I started, I was tasked with writing a new .NET SDK for Sentry.

Throughout the previews, I was always curious if the releases were being downloaded at all.

I found myself checking nuget.org and looking at the Statistics for total downloads. It was obvious we in the .NET ecosystem were missing some package download stats website.

Welcome NuGetTrends, to the .NET community!

NuGetTrends is a website with historical total download count for NuGet packages on nuget.org.

There's data since 2013 which was contributed by ZZZ projects, the company behind the EF Extensions. Shout out to Jonathan! Thanks a lot! Unfortunately there's a gap in 2017 of about 10 months, though.

Also, the UI so far has predefined filter for as far back as 2 years and result is grouped by week. Query string takes months as an integer though so URL hack to have some fun.

The NuGetTrends workers go through the nuget.org's catalog API so all the package's metadata are available in its database. That means there's the potential to build some new cool stats like:
  • How many packages are signed.
  • Are the DLLs in the packages strong named.
  • Packages with unoptimized DLLs.
  • Stats package adoption of source link.
  • TFM adoption (ran some queries for this and it's great how .NET Standard 2.0 picked up fast).
For some of those feature we'd still need to download the actual packages and inspect its contents. There's some work there but this is the call for help!
 
We're online on Gitter, if you'd like to chat.

It'd be great to get some help too!

Let's build something cool out of this? It would be great to see that contributors list grow.

Sunday, April 15, 2018

.NET runtime information in different systems

Edit:

I've created a repository on GitHub to expose an API which simplifies getting runtime, operating system and other platform information: https://github.com/getsentry/dotnet-sentry-platform-abstractions
If you are interested in contributing, please let me know. A few of issues were created to discuss API design and features, feel free to drop a message there. 
There is already code to detect the different installations of the .NET Framework installed and also what runtime the code is running on. Preview versions are available on NuGet.org.

An output from the sample app is included under the samples directory.

Read on for details:

.NET Core is great multi-platform framework. In my previous job, most of the team members were working on Windows, I worked on a MacBook and we deployed stuff to CentOS and Windows Servers. Build servers had a mix of Windows Server 2012 R2, Windows Server 2016 and a lonely CentOS and it all worked just fine.

The truth is that there are lots of business applications which are built with .NET Framework in a variety of versions with active development. At my new job (it's been 2 weeks), there's a focus in integrating all kinds of technologies and frameworks/versions to make developer's life's easier. If for example you maintain a Windows Form application built 6 years ago, targeting .NET Framework 4.5 which Microsoft has dropped support to 2 years ago, it's possible that a tool like Sentry is even more vital to you. I'd still recommend you upgrade to 4.5.2 though, at least, if possible.

Because of that, it's one of Sentry's .NET SDKs goals to target as many framework versions as possible. It currently even supports .NET 3.5 which was released 10 years ago and cannot be built with the new msbuild SDK.

While working on adding a feature to Sentry's SDK which augments the events sent to Sentry with data about the system where the app is running, I was presented with a different set of APIs available to different versions of the framework. For example, when targeting netstandard2.0, it's possible to retrieve the framework information with:

System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription

This API does not exist in versions prior to 4.5. My first idea was to copy the CoreFX implementation but once I saw the #ifdef's I realized it's not possible as I'm not generating assemblies just to have .NET Native or .NET Framework etc. This was just one example, there are many more APIs I'd like to use that will differ depending on the target framework.

CoreThrower


To help me test different scenarios, I wrote a script (batch and bash versions) to build and run a simple app that throws an error with the Sentry .NET SDK restored from some local path. It has the following targets:

  1. netcoreapp2.0
  2. net471
  3. net46
  4. net452
  5. net45
  6. net40
Considering that all .NET 4.x are in-place installations, all I'm doing here is making sure I can build the app and load the NuGet package against those versions. Compiling uses reference assemblies to ensure compatibility. In runtime, it'll use the v4.0 folder with the CLR 4.0.
The .NET Core 2 version runs with CoreCLR on all platforms. The full framework versions will run with Mono on macOS and Linux and on Windows, the desktop (full) .NET Framework .

This tool is useful to me to get a locally built Sentry SDK and run in different environments.
I ran it on:

  1. macOS 10.13.3 High Sierra
  2. Windows 10 Pro Fall Creator
  3. Ubuntu 16.04.3 LTS (Windows Subsystem for Linux)
  4. Windows 7 32bit
  5. CentOS 7
All of them have .NET Core SDK 2.1.104.

While dealing with the full framework builds and runs I had a few surprises though:

Windows 7 x86


I verified (through the Windows registry) that the Windows 7 machine has version .NET Release number 378389. That means the plain .NET Framework 4.5. Although 4.7.1 is compatible with Windows 7 32 bits, I decided not to update it to verify which versions of the program and the SDK would work properly. I copied the reference assemblies to allow me to build all versions:

Windows 10 (64 bits): C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework
To the Win7 (32 bits): C:\Program Files\Reference Assemblies\Microsoft\Framework

At this point running run.cmd successfully compiled all versions and ran all versions! That was to my surprise since the versions ahead of 4.5 could very well fail. It does make sense though since the application itself is not using any .NET 4.6+ API and Sentry's SDK is targeting net45 as the latest besides netstandard2.0. 

That also means that the netstandard2.0 version was not restored to any of the net4x versions. Since Considering I'm using the .NET Core 2.0 SDK I would expect the targets net46 and net471 to restore the netstandard2.0 version of the Sentry SDK. Another possibility is that I have no idea why it worked ;)

At sentry.io I see:

That surely needs some work: The events sent by the .NET Core version of the app shows the CoreCLR version (4.6.26212.01) followed by the .NET Framework 4.6+ version (4.0.30319.42000) which is not even installed on the machine. The middle entry shows 4.0.30319.18063. At least the value looks correct. It represents .NET 4.5 on Windows 7 SP1.

The values presented have two parts where the first is runtime name and the second is the version which was coming from the API:

System.Environment.Version

Microsoft does have a disclaimer on this API:

For the .NET Framework 4.5 and later, we do not recommend using the Version property to detect the version of the runtime; instead, you can determine the version of the common language runtime by querying the registry. For more information, see How to: Determine Which .NET Framework Versions Are Installed.
From MSDN

The version name and number get combined and depending on the version of the framework the code compiled against it shows the latest .NET Framework installed or simply the CLR version.

Needs work but for now lets see what shows up on the other systems:

Ubuntu 16.04 (WSL)


The WSL installation of Ubuntu I have, included OOTB Mono version 4.6.2:


$ mono --version

Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com

       TLS:       __thread

       SIGSEGV:       altstack

       Notifications: epoll

       Architecture:  amd64

       Disabled:      none
       Misc:       softdebug
       LLVM:       supported, not enabled.
       GC:       sgen

This version does not implement netstandard2.0. That was only introduced in version 5.4 back in November 2017. I took the same approach I had with the Win7 box: Try to build and run with an older version.

... /usr/share/dotnet/sdk/2.1.101/Microsoft.Common.CurrentVersion.targets(2052,5): warning MSB3245: Could not resolve this reference. Could not locate the assembly "System.Net.Http". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors. [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] CSC : error CS0006: Metadata file '/mscorlib.dll' could not be found [/mnt/c/Users/bruno/git/CoreThrower/CoreThrower.csproj] 63 Warning(s) 5 Error(s)

With the exception of the netcoreapp2.0 target, nothing would build. Using FrameworkPathOverride I configured the projects' csproj to look into mono's path for the reference assemblies but realized that there was actually nothing there. Considering Mono dropped support to pre-net45 reference assemblies on Mono 4 I had to install anyway the package for net40:

apt-get install mono-reference-assemblies-4.0

After that, I was still missing the newer packages. Makes sense since I was on an older version of Mono. This can be solved by taking them directly from GitHub:


cd /usr/lib/mono


sudo cp -r reference-assemblies/v* .

sudo rm -rf reference-assemblies/

The directories still needs renaming, to the expected format: remove v prefix and add -api suffix:

sudo sh -c 'for dir in v*; do mv "$dir" "${dir#v}-api"; done'



Build successful! That doesn't mean it will run and indeed it didn't.
First error:

Running: bin/Release/net46/CoreThrower.exe

Unhandled Exception:
System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0
  at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0
  at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0
  at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0
  at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

The first thread I found on the topic pointed out that I need to have libmoono-system-core4.0-cil installed. 30 seconds later I was running it again:


Running: bin/Release/net46/CoreTh

rower.exe


Unhandled Exception:

System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0

 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0

 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0
 at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0
 at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1650 + 0x0010f> in <filename unknown>:0
 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc1610 + 0x00019> in <filename unknown>:0
 at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine] (System.Runtime.CompilerServices.TStateMachine& stateMachine) <0x7ffc15e0 + 0x00019> in <filename unknown>:0
 at CoreThrower.Program.Main (System.String[] args) <0x7ffc1080 + 0x001db> in <filename unknown>:0
 at CoreThrower.Program.<Main> (System.String[] args) <0x7ffc0f10 + 0x0001b> in <filename unknown>:0

As pointed out further down in the same thread I linked above, mono-complete does the job:

sudo apt-get install mono-complete
Setting up mono-complete (4.2.1.102+dfsg2-7ubuntu4) ...

Wait, what? Mono 4.2? I didn't change any of Ubuntu's package feed I don't think, did it just downgrade from 4.6.2 to 4.2.1? 

Mono JIT compiler version 4.2.1 (Debian 4.2.1.102+dfsg2-7ubuntu4)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

It looks like it did and if I now: ./run.sh
Finally, it works! 

At sentry.io I see:


Although I've installed the same SDK version on both Win7 and Ubuntu, the latter is returning only 4.6.0.0 compared to 4.6.26212.01 returned on Windows.

Windows 10 Pro



Straight forward. I have .NET Framework 4.7.1 and Visual Studio on this box so it built and ran without issues.


The first item is created by the net40 version of the app. Since it doesn't have the same APIs as net45+. It's using Environment.Version mentioned above which is far from ideal.
The rest of the results are OK with the exception of the "or higher" which came as a result of the version being higher than 461308. I'll need to rework this too.

CentOS 7


Using the official .NET Core and Mono install links I got their SDKs installed really quickly.
Latest Mono version: 5.10.1.20 and .NET Core SDK 2.1.101

Pull the code, pushed the nuget package to the checkout directory and ran ./run.sh
Works fine and I get some events at sentry.io.


Like the Ubuntu version, .NET Core only include the major and minor version numbers of the CoreCLR.

macOS


The macOS device is my work computer so I might have installed reference assemblies before, I don't remember. Things ran just fine and the Mono version there is 5.8:

It seems that only on Windows the API RuntimeInformation.FrameworkDescription returns the whole version number including build and revision.

Other system information


Above I've discussed only the 'Runtime' context information that Sentry handles. There's much more data that Sentry's SDK will extract and send with events. Lots of them are challenging when multi-targeting from netstandard2.0 all the way down to net35 and running on Windows, macOS, Linux, Tyzen, iOS, WebAssembly (blazor?) and whatever else we can use a .NET NuGet package in now and in the future.

Conclusion


I still have a lot of work ahead to get this context data sent to sentry in a unambiguous way. This was just the first try though.

Sunday, March 4, 2018

Getting started with SonarQube and .NET Core

SonarQube 


I decided to check SonarQube now that it finally supports .NET Core and the new MSBuild SDK format.

SonarQube is an open source product from SonarSource. They have a hosted instance called SonarCloud which is free for open source projects! Besides SonarQube, there's also an IDE plugin called SonarLint.

SonarLint

I started testing SonarLint out to see if it was even worth setting up SonarCloud. As a test project I picked a tiny OSS library I wrote a few months ago for ASP.NET Core Basic Authentication

SonarLint will run the analysis while using the IDE's Error/Warning panels to notify you when it finds something. Although they support IntelliJ IDEA, the support doesn't include Rider. The support to VSCode doesn't include C# either. Bummer. But honestly there's still no code coverage for CoreCLR yet outside of Windows either.

I tested the Visual Studio plugin and ironically, I was happy to see a warning:


Indeed. I should make that class static.

I happen to have ReSharper which is a great (paid) tool by JetBrains. I ran its Inspection/Code issues in Solution and it didn't pick this one up. JetBrains Rider which I use on my work computer (macOS) has the same inspection feature. They do give awesome feedback even though there's no warning for this specific issue.
Please note that both are amazing tools. In any case, it's hard to expect that all contributors of an OSS project will have licenses. JetBrains provide licenses to open source project maintainers but not for such small projects like the ones I have.

Running the analysis


Now that I had a project with a warning, I could test running the analysis scanner on my machine and see what the results look like on the hosted instance SonarCloud. The analysis scanner (although requires Mono, .NET Core and Javacan be executed on macOS and Linux but as today I'm on a Windows 10 box, I followed this guide. The only thing I did differently was that I didn't invoke MSBuild directly, I used the CLI: dotnet build instead. It's really just 3 commands including your build command.

I was sad to discover that it requires Java in order to run the MSBuild scanner. I've got a clean Windows 10 install a few months ago and I was very happy not to have Java installed. SonarQube is nice enough to convince me to install it though. Can't wait to see those Java update notifications every week!

The analysis took quite a while to run (over 5 minutes). Perhaps because it was the first run, but the report was sent!

The link to the results is part of the output:


Now I can browse the results at: https://sonarcloud.io/dashboard?id=Bazinga.AspNetCore.Authentication.Basic



It was a good quick getting started for me. I'll consider it now for bigger projects too and explore the many features it has.

Monday, February 12, 2018

CentOS build agent and .NET Core SDK versions

Our build server running on CentOS ends up building different repos, which use different versions of the .NET Core SDK.

The SDK version is defined via global.json and instructs the CLI which version to load in that context.

Surely the CLI is smart enough to roll forward. That means in case you target version 2.0.2 and the machine only has 2.0.3, it will (should) work just fine with that version.

I've tested setting my global.json to 2.0.2 where only SDK version 2.0.0 was available:
$ dotnet --version
2.0.0

Surprising to me it built and tests passed without warnings. So it also rolls backwards somehow.

When the exact SDK version defined via global.json is available, that's the one used by the CLI.
I believe that's a safer approach to have on a build server. Which means having all SDK versions in all agents.

That is, regardless which agent the build runs, it'll always use the same SDK version. Otherwise potentially one agent, created at a later time, could lack the exact SDK version used via global.json.

Installing all SDK versions


Using yum's wildcard, it's easy enough to install every single .NET Core SDK available on the official Microsoft feed, simply by:

$ sudo yum install dotnet-sdk-*

As an example, this agent had only the SDK version 2.1.4 (the latest one).

By running yum install dotnet-sdk-* on this machine:


Easy enough.

All SDK versions made available by Microsoft are now installed.