Episode 126 - Slimming Down .NET: The Unofficial Experiments of Michal Strehovský
Embedded Player
The .NET Core Podcast
Episode 126 - Slimming Down .NET: The Unofficial Experiments of Michal Strehovský
Supporting The Show
If this episode was interesting or useful to you, please consider supporting the show with one of the above options.
Episode Summary
This episode features an interview with Michal Strehovský, a developer on the .NET runtime team who has been experimenting with reducing the size of .NET applications. Strehovský’s experiments have led him to create BFlat and Flattened.NET, personal projects that allow .NET developers to play with the technology and non-.NET developers to get into .NET.
One of his experiments involved creating a self-contained WinForms Snake game in C# that was under 8KB in size. By using unsupported territories like ahead-of-time compilation and trimming, and even writing his own core library to work around missing pieces of the runtime, Strehovský was able to achieve this impressive feat.
The standard .NET publishing process includes the entire runtime and base class libraries, resulting in a large executable, but trimming can be used to remove unnecessary components. However, the runtime itself cannot be trimmed. Native AoT can be used to compile the entire app ahead of time, resulting in a smaller runtime and smaller app size.
BFlat is an ahead-of-time compiler for C# that can create natively compiled executables for any supported platform, including Windows and Linux, x64 or Arm 64. The compiler was created by taking the C# compiler and the ahead-of-time compiler pieces that are part of the SDK and combining them into one tool that can compile C# to native code. BFlat includes all the dependencies needed, such as the framework and any other OS dependencies, in a relatively small 70MB package that can be downloaded and used without requiring the dotnet SDK.
As a sample project for BFlat, the Snake game was made to run on bare metal hardware without an operating system, demonstrating the ability to run something written in C# in a low-level environment. BFlat encourages developers to take ownership of third-party dependencies by submoduling the Git repo that has the dependency and building it themselves with BFlat, treating it as their own source code and potentially forking it if necessary.
While these experiments are not officially supported by Microsoft and rely on undocumented implementation details, they demonstrate the potential for significantly reducing the size of .NET applications. BFlat is targeted more towards developers who are not .NET developers and want to try out C#, but do not want to learn Ms build and prefer to use make files or their own way of doing things.
In conclusion, Strehovský’s experiments have shown that with the use of unsupported territories and ahead-of-time compilation, the size of .NET applications can be significantly reduced. BFlat is a personal project that allows developers to play with the technology and non-.NET developers to get into .NET. While these experiments are not officially supported by Microsoft, they demonstrate the potential for reducing the size of .NET applications and the versatility of the technology.
Episode Transcription
Hello everyone and welcome to THE .NET Core Podcast. An award-winning podcast where we reach into the core of the .NET technology stack and, with the help of the .NET community, present you with the information that you need in order to grok the many moving parts of one of the biggest cross-platform, multi-application frameworks on the planet.
I am your host, Jamie “GaProgMan” Taylor. In this episode, I talked with Michal Strehovsky about what bflat and flattened.net are, and how he created these amazing experiments. We double back to Michal’s previous appearance on the show (back in episode 47) in order to give a little back story as to how he got bflat to where it is, and why he built it.
It’s worth remembering that bflat is an experiment of Michal’s own creation. Whilst he does work on the .NET team at Microsoft, these experiments are entirely his own creation and have absolutely no support from Microsoft or the .NET team. This is just one of his fun, open-source projects.
Along the way, we talk about the importance of knowing about what happens to your code when you hit compile and how knowing even a fraction of how your code runs on a .NET runtime can help you to write better code. We also talk about the importance of knowing what your dependencies actually are, and how vital it is to understand what they are actually doing by reading the source code - i.e. the Unix philosophy
Remember folks: don’t use bflat for production or anything other than playing around with and seeing what you can do. It’s not meant for anything other than allowing .NET developers to play around with, or to allow non-.NET developers to get into .NET. Please don’t use it to create anything important or anything related to production.
Then again, this is just a fun conversation about how Michal managed to boot his computer directly into a snake game that he wrote in .NET… without an operating system. Pretty cool, huh?
So let’s sit back, open up a terminal, type in dotnet new podcast and let the show begin.
Jamie
So Michal, thank you very much for being on the show again.
It’s been like three years since you were on the show, so my goodness, that’s a long time and you’ve been working on a lot of stuff in the background, so hopefully we can talk about some of that stuff.
But firstly, welcome to the show again and thank you for spending your time with us.
Michal
Yes, thank you. Thank you for inviting me back. I’m happy to be on the show again. Hopefully we can have an interesting chat about what I have been up to over the past three years.
Jamie
Oh, sure, totally.
So, like, last time you were on we talked about your experimentation with building the smallest possible .NET app and it was impressive when you did it, but what you’ve done so far now is even more impressive. I just want to give the listeners a bit of a tease about that. But before we get on to that, I was wondering could you give the listeners a bit of perhaps an elevator pitch, a bit of a description of the kind of things that you’re building and the things that you’re working on.
Michal
Sure. My name is Michal So. I am a developer on the .NET runtime team. I have been doing this for about ten years now, eleven. I work on the ahead-of-time compilation technologies in .NET - I work on the virtual machine. And in my spare time I like to take things that I worked on and kind of push them into spaces where we don’t push the officially. And those are always my personal projects, my personal takes on on these things. So that’s that’s like one of those things. I would like to underscore that I will be talking about a lot of things that I have been doing in my spare time. They are not official Microsoft projects, but I find them interesting and people like to hear about them.
Jamie
Sure. So yeah, I’ll echo what you’ve just said there. I’m talking to Michal, I’m not talking to a Microsoft employee today.
I’m talking to a person who is discussing things that he likes to futz around with in his own time. So I’m not going to hold you accountable. Like if you say to me, “hey, I’m doing this thing,” that is something you’re doing, not something Microsoft is doing. I need to make that very clear to myself and obviously to all the listeners. Right. We’ve never had any problems with people mixing up a guest with Microsoft. But what we’re saying is these are Michal’s experiments and thoughts and opinions, not those of Microsoft.
Michal
Yes. I think last time when I was on the chat, I said that I came here as a .NET enthusiast, not as a Microsoft employee and this time it will be the same.
Jamie
Yeah, that’s exactly it. That’s exactly it.
Excellent. I’m just really quickly looking up what we talked about last time because it was about hacking around with .NET to see what we could build. And there was I think it was a snake game. It was a winforms snake game, self contained game in C# in under 8 KB. Which was I mean, I remember reading the blog post that you’d put out - and I’ll put [a link to this in the show notes](Building a self-contained game in C# under 8 kilobytes) - it’s from January 3, 2020. And I’ve got to say I understood about 30% of what you’ve written just because there’s a lot of very complex stuff happening and you’re obviously substituting a lot of code for stuff that you can compile yourself rather than relying on the runtime and things like that.
So that was super interesting to read about. And I read it and I remember where I was. I was walking through the train station and I read this article whilst I was standing to get my ticket and I was like, “I’ve got to get in touch with this guy. I understand nothing in this article. I’ve got to get this guy to explain it.”
Michal
I can talk about it a little bit more again.
Basically that was an experiment where I was thinking about, “what would be the smallest possible thing that I can get running on a real computer that is still written in C#?” One thing about C# that a lot of people know about is that C# is compiled for a virtual machine. So if you write your C# and you compile your app, the output of the compilation is, is an executable that will only run on a virtual machine. It it will not run on on like a real hardware, real CPU. But when you look at the executable, what is inside there are two things. Like one thing is description of types and methods that are inside. Like what are the names of the types, what are the names of the methods, what is the signature of the method? And then there is the code and the core is in the form of instructions for a CPU that doesn’t physically exist. It is an abstract CPU that was designed 25 years ago. Since then there has been a lot of evolution in terms of hardware and kind of when this file format was invented, Arm 64 didn’t exist for example. But still, when you write an app in C# you can run it on an Arm 64 CPU, even though the file format doesn’t even know what Arm 64 is.
The reason why this works is that there is always a virtual machine that will take this executable and it will make it so that the code in it can run. There are many strategies how we can make the code inside run. We can either interpret the instructions so go one instruction after another and kind of perform the operations - the operations will be in the form of add two integers together or call a method or draw an exception and you could just interpret it. The other option would be to compile it to native code just in time. Just in time means that we will start executing a method, let’s say your Main. And at the time when we need to run Main, we will look at all the instructions inside main and we will convert them to the native instructions of the computer that is currently executing the program.
So you can take the same binary with the same like C# output and run it on Arm 64 CPU or x64 CPU or Arm CPU. And it will work the same way. You can make it run on Linux or Windows, and it will still work the same because there is always the VM that kind of does this translation from the abstract terms to the physical terms. And the last option, how we can make this core runnable is to compile it ahead-of-time. And the ahead-of-time compilation is basically a second compilation after the source compilation step, where we take the output of the Roslyn compiler, or could be F# compiler, or could be VB compiler, or like any other .NET language - they all compile for the same virtual machine. And we take those files and we go over all the methods or all the interesting parts of it, and we generate native code as a single step. And the output of this compilation will be another executable. And that executable will be native. It will be specifically for some CPU architecture, specifically for some operating system, and it will only be runnable on that combination.
So I started with this when I was doing my 8 KB make experiment. And basically I wrote the game and then I started looking like, “what are the things that I can take out of the game and make it still runable on a computer?”
So I started how everyone would start. I just wrote some C#, compiled it to the virtual machine file format, and published it as an executable. So when you publish your app as an executable, you will include the runtime and you will include all the other dependencies. It will also be spat up into a folder and then you can run it. And if I remember correctly, by default, this was like a 60 megabyte executable because it includes the entire runtime, it includes all the base class libraries. Like even though I only wrote a snake game, it still had the entire networking stack, it still had LiNQ expressions, it still had all those things. It was big.
Then I started playing around in the supported territories. The supported territories were things like Trimming. So in .NET Core Three, I think we added support for Trimming, which is a step that you can opt into when publishing your app that will analyze what your app depends on, and it will try to remove things that are not necessary. So when I do this publishing with Trimming on this snake app, it will remove things that the app is not using. So if it’s not using LiNQ expressions, that DLL will go away. If it’s not using FTP clients, FTP client will go away. So with this I got an executable that was maybe 13 megs. I don’t remember exactly what those numbers were, but I think now around the .NET 7 time frame, it would be around 13 megs with Trimming enabled.
And then I started at the time looking into unsupported territories. There was an experimental project for ahead-of-time compilation. It was called Karate, and I compiled the app with Karate. Now that .NET 7 shipped, what was in Karate. We took many, many technologies that were built as part of the Karate project and we made them as part of .NET 7. So there is a new publishing option called Publish AOT that lets you compile your app ahead-of-time with trimming. And with this, the app gets even smaller because we could trim even more pieces. If you publish your app with regular trimming, it will trim out pieces of the framework that are not used. But the Runtime itself cannot be trimmed because runtime is like one kind of one DLL that has the JIT, it has the garbage collector, it has many components that are necessary to support any core runtime. Those components are support for reflection, support for attaching a debugger, support for attaching profilers, all those things. And this comes with some size. This sets how small a .NET app can be in general, because you always need the Runtime to run your app.
When you publish as native AoT, most of the Runtime is actually gone because we compile the whole app ahead-of-time and we no longer need to do a translation at Runtime from the abstract virtual machine terms to kind of terms where we are running the code. So the runtime itself is much smaller and the base penalty for just writing .NET code is basically the size of the garbage collector and some things on top of that. So just by publishing the app as aoT, I could get the app even, even smaller.
And at that time, I think it was around three megs or four megs as we are working on .NET 7 and .NET 8, this will go even more smaller. And then I decided to go completely crazy and I decided to also get rid of the Garbage collector and all the other things. So I slowly got rid of pretty much everything that was in the app. And in the end, I ended up with the eight kilobyte executable that really didn’t have much in it except for the app code itself.
Jamie
That’s amazing. Yeah. Obviously as an in the weeds developer who uses .NET on a daily basis, I don’t think I’ve ever had a burning desire to look into what I can strip out. But yeah, it is amazing what you can do if you want.
Okay, so if I wanted to write a Snake game in C++, which will compile down to native code, it would be not that much bigger. I’m guessing it would be around about maybe 500 KB, depending on the graphics libraries that I pull in to be able to do it, or if I do it in the console or whatever. But 8 KB for something that runs is just that’s mad to me. We’re in like we’re in the the territory of of what Go can do in my opinion, at that point. But I also fully appreciate the amount of effort that has gone into how do we get to here?
And again, we’re talking about these are the experiments that Michal has done himself. Not anything that Microsoft supports, right? But just the ability to get to that stage is amazing to me. It’s fantastic that that is possible if you are willing to put the effort in. And I guess I wonder whether you do know, you said they’re about removing a bunch of you’re doing ahead-of-time compilation and then removing parts of the runtime and stuff that you’re not using.
What happens if your app uses say, reflection and reflection ain’t there? Does it literally just crash out and go, “system not found exception or something”?
Michal
So, for the experiment that I was doing, I actually wrote my own core library, so I defined my own class called Standard Object. So if I tried to use something like reflection, I would get normal C# compiler error that says, “this class doesn’t exist.” So this would be an okay experience if I wanted to use a piece of the runtime that doesn’t exist, so for example I would like to try and throw an exception that would probably be a compilation failure somewhere a little bit later after the C# is done compiling and we are trying to compile it natively and it would just not work, right?
So this is definitely unsupported territories because working around all of these failure modes requires relying on pieces of the runtime that are not documented and that we want to be able to change because they represent the underpinning of the runtime. So kind of I know how to work around them in a very specific version of the native AoT compiler that I used. But those hacks would not translate to the latest version of the AoT compiler because it’s just not something that is supported or that the compiler is designed to do, because we want to have kind of layering and boundaries that are well defined. And a lot of these boundaries are not well defined because they are really implementation details on top of the runtime.
Jamie
Sure. And that makes sense, right?
I’m going to use the wrong words, but I suppose in the position that you’re in working on that team in that area, you have a knowledge of how that particular part of the ahead-of-time compiler works. So you know that, “hey, if I poke at this bit, this API exists and it’s not documented because,” like you said, “because we don’t want to expose this, because people will start using it and we need to change it perhaps quite often.” And again, you’re not speaking on behalf of Microsoft, right? You’re speaking on behalf of yourself as a person who’s poked around and said, “hey, what can I do here?”
And that makes total sense because I can imagine. In fact, I think I remember around the .NET Core three timeframe, I can remember. I want to say it was Damien Edwards yeah. And he said, “hey, we’re playing around with to see if we can do ahead-of-time compilation, but we’ve disabled ahead-of-time compilation removal of certain things. Because what if your app uses reflection and the compiler misses that? Then your app is going to fail at runtime. We can’t have that, right?” And I totally understand why that decision was made because you can’t do that. An app needs to be runnable right.
From anyone’s perspective, regardless of whether you’re the developer who wrote it, someone who worked on the tools, on the runtime, whatever, it needs to be able to run. Right? And if a tool that I use, I’m not going to say who built it, but if a tool that I use causes the thing that I need it to output to not work, then that’s a bad thing. That’s a bad tool. So I totally understand why there are parts of this that are not as well documented or not documented at all and why you’re able to sort of dig around and find those bits.
I would probably say that for the average developer writing CRUD apps, the standard sort of ASP .NET “load a thing from the database, show it on screen, accept some input and put it back in the database,” you don’t have to worry about any of this, right? Because the tooling will do everything it can to make that experience as good as possible as long as you’re giving it good enough code.
But yeah, I really like that you’ve gone, “hey, I wonder if…” because then we take that experiment and we jump forward three years, right, and we’ve got this wonderful thing that you’ve been working on. I know it as flattened.net, but in our communications before we hit record, you were mentioning BFlat, which is an absolutely wonderful like for those of you don’t know musical theory, right? You may know a bit about computer programming languages. B was a programming language. Then C came because it was one up of the B programming language. Then C++ came around because it was one better than C. The C# came around. And for those of you don’t know musical notation, if you want to go back a few steps between C and B - there isn’t a note there, but let’s say that there is. We want to call it Bflat, right? It feels maybe that was an unintentional decision, maybe you’re just combining like binary anBFlattened, I don’t know. But let’s talk a little bit about that.
Michal
Since we are talking about the name. I originally want to call this BFlat - as D not B because then it would be an harmonic to C#. In musical theory, harmonic note is basically the same note. It’s just expressed differently. So we are saying that C goes half a step up and then D goes half a step down. But there is a language called B and this has nothing to do with D, so I just went with BFlat. I wanted to use Flat because I like the Flat term, so that was the name.
But let’s maybe rewind and talk about what BFlat actually is. So BFlat is ahead-of-time compiler for C#. That’s the shortest way how to express it. When you want to write piece of code in a programming language, you often times start with creating a file, just like with the CS extension, and you put some code in there and then you would like to compile it. If you use a language like C or C++ or even Java, they always come with a command line version of the compiler that you can run and it will produce an output that you can run. For .NET we don’t have a way to invoke the compiler like that anymore. When you look at .NET framework, .NET framework came with csc.exe that you can run from the command line. It will work. Now we still have some csc.exe, but to actually invoke it you have to create a command line that specifies where are all the references, because it’s .NET no longer built in. Like there is a compiler, but it doesn’t know which version of the runtime to use. So whenever you want to run csc.exe, it’s a bit of a pain. So most people, rightfully so, use them as built to drag the compilation.
But since I like to go kind of low level with many things, I wanted to be able to use C# compiler on its own. And since I work on ahead-of-time compilation technologies, I also wanted a C# compiler that would create natively compiled executables. So a C# compiler that wouldn’t make the executables for the abstract machine anymore, but would generate executable that you can directly run. And since we have many different CPU architectures and many different operating systems, I also wanted a C# compiler that would generate these natively for any of the supported platforms.
So what I did as my personal experiment, I took the things that we ship as part of .NET that are part of the SDK. So I took the C# compiler, which is shipped as a NuGet package that you can actually use from your own app. So kind of if you want to compile a piece of C#, you can just reference the Microsoft.CodeAnalysis C# NuGet package, and that’s the Roslyn compiler. You can give it like a text string with your C# program and it can compile it into IL. So I took that, I also took the ahead-of-time compiler pieces that are part of the SDK, and I created a new compiler that just takes these two pieces and makes an exe that you can run. It doesn’t exist in this form in the SDK because the SDK has two tools, they run subsequently after one another. So first we compile into Il, then we write into disk, then we kind of pick it up and the we compile to native. I decided I just want one tool that can do all of this, so I did that.
Another thing that is necessary besides kind of compiling ahead-of-time is to include all the dependencies that are needed like the framework and any other OS dependencies. So I kind of put everything else into the package and I came up with a small package, relatively small, 70 meg package that you can just download. It doesn’t require .NET SDK, it doesn’t require anything and you can use it to compile C# to native code for Windows or Linux x64 or Arm 64. So that’s what I started with.
Then I decided to fold my previous experiments into it. The previous experiments were with the 8 kilobyte C# Snake Game. Since I already had a compiler that could compile things, I also kind of created a smaller version of the base class libraries that people could opt into if the want. Maybe it’s even wrong to call it base class libraries because it has like System.Object, System.String, maybe some Span<T> it doesn’t really have much; like it doesn’t even have a List<T> or object.ToString() or anything like that. But the interesting thing about it is that it lets you build those experiments that I did three years ago in a much easier way. You no longer have to kind of dig into the SDK, the official .NET SDK, and find some exe there and pass some undocumented switches and kind of invoke it in some weird way and get broken all the time because it’s not a stable API surface. So instead you can get this compiler that has it as an officially supported thing and hopefully it will not be breaking all the time and you can just compile your small app as small native executable.
And since I already had that, I decided to add one more thing. As an experiment I took the snake game and I made it run on bare metal hardware. When I say “bare metal hardware,” I mean make it run on something that doesn’t even have an OS. And if you look at modern computers, when the computer first boots up, it starts in UEFI environment, which is like a pre operating system environment that already knows how to access the hardware. Usually it’s used just to bootstrap the operating system so it’s kind of just enough to be able to load the first part of the operating system and then the operating system takes over. With BFlat you can create executables that can run in this environment. So I decided that hey, why not? We should be able to also check with this. So as one of the sample projects for BFlat I have the Snake game that you can boot into and just run on hardware with on an OS.
A Request To You All
If you’re enjoying this show, would you mind sharing it with a colleague? Check your podcatcher for a link to show notes, which has an embedded player within it and a transcription and all that stuff, and share that link with them. I’d really appreciate it if you could indeed share the show.
But if you’d like other ways to support it, you could:
- Leave a rating or review on your podcatcher of choice
- Head over to dotnetcore.show/review for ways to do that
- Consider buying the show a coffee
- The BuyMeACoffee link is available on each episode’s show notes page
- This is a one-off financial support option
- Become a patron
- This is a monthly subscription-based financial support option
- And a link to that is included on each episode’s show notes page as well
I would love it if you would share the show with a friend or colleague or leave a rating or review. The other options are completely up to you, and are not required at all to continue enjoying the show.
Anyway, let’s get back to it.
Jamie
That’s fantastic. It makes me think of… I don’t think you are of as much of a vintage as I am, Michal. But I remember back in the day booting into my old ZX spectrum or whatever and having to boot you into a prompt, and then you have to put the floppy disk in or whatever, and load the game from there. And the game would basically be the operating system itself. That’s the wrong way to put it. But there wasn’t really an operating system, right? It was almost like you were booting into a bootloader and saying, “load from floppy disk,” and then your game would run. So it makes me think of those days and being able to run something that was written in C# that uses, I’ll say, a variant of .NET because obviously I will come to that in a minute, right? “Is it running actual .NET that’s been stripped down or is it running a variant that you’ve sort of put together?” Come to that in a moment. But being able to write something in C#, which is this wonderful sort of high level language that is brilliantly fluent and you can write it in a procedural way, an object-oriented way or a functional way and just write whatever you want and have it run bare metal, pre-operating system is just… that to me is - I’m going to use the UK English term - “bonkers” is what that is.
Michal
Yeah, I would definitely not call this .NET. I’m very careful to only call it C#; and even C#. It’s a small subset of C#. But when you look at C#, the language has both low level constructs and high level constructs on. One side you have things like async and dynamic keyboard, and then on the other side you have function pointers or you have the unsafe keyword with pointers. It allows you to write low level code when you need it. But of course you should mostly stick to the higher level things.
For [the] experiments that I’m doing, I like to use the low level pieces and then really C# is not very different from C in that sense. What we allow in the limited version of the C# is not very different from C, but it’s still a little bit more ergonomic C because you can use pattern matching. Like pattern matching is very powerful and C doesn’t have it, C# has it. So kind of you can, you can still use a lot of these ergonomic features when you restrict yourself to the C subset of C#.
Jamie
Brilliant.
The crazy thing to me is that by looking around in the repo, right, it looks like it’s written in C#. I would have expected a low level tool to be C, C++, Go, Rust. Nut this is, like you said, the low level C#. What the heck?
Okay, so a lot of people who haven’t done computer science or programming language design, what you tend to get is you will build a compiler. When you first write a compiler, you may write it in a completely different language to the target language that you’re compiling, right? So the C compiler was originally written in Assembly, but then a couple of versions in you have the C compiler compile the source code for the C compiler. Right? And so what I’m leaning towards here is, are you using C#’s compiler to compile BFlat or are you at the point where BFlat can compile itself?
Michal
That’s a good question. So the build of the BFlat compiler is a two step build. So in the first step it uses the official .NET SDK to compile a version of BFlat that runs on top of the kind of officially supported official runtime. And then the build executes BFlat again to compile BFlat with BFlat. And those are the binaries that are shipped. The reason why the build is done this way is because I have customizations in BFlat that allow it to cross compile and I wouldn’t be able to do the cross compilation so easily with the official tools. The problem is really that if you want to compile something natively, the cross compilation with the .NET SDK doesn’t work so well as it works if you are not compiling natively. If you would like to publish with a VM your app, you basically just type dotnet publish, and you say which operating system you would like to use, and which architecture. And what the SDK will do is it will download the version of the VM that matches the operating system, and architecture that you want, and then it will add those files that are abstract not for a concrete machine. And that’s it. So kind of basically you have a copy, and copy and just that creates the executable you need to run.
But if you want to do native compilation, you need extra files like you need the platform SDK. So you would have to install Windows SDK if you want to build for Windows. Or you would need to get a Linux version of the linker from somewhere and the Linux version of libc from somewhere, and be able to kind of link the executable against the version of the libc that you want to target.
And the official SDK for .NET doesn’t know how to do this because it’s really a hassle to kind of collect all the dependencies because I don’t even know if there is like a Windows SDK that you can run on Linux. There are the open source versions of those, but I don’t know if there is something that is officially Microsoft supported that we could ship as part of the .NET runtime. So for BFlat, since it’s my personal project, I can just cut corners. So I can just copy some files from an Ubuntu installation, copy, copy some files from some other open source project and put everything into a zip. And it’s much easier for me to do this as a personal project versus if Microsoft wanted to ship something like this, then there are questions like, “okay, how do we build it reproducibly? Like what about security fixes?” If someone asks me about security policy of BFlat the I just tell them, “you can build it from source if you need.” So it’s much easier to do these things as a personal project and experiment with it and see if it’s useful like what people think.
Jamie
Sure, as you quite rightly said from Microsoft’s point of view, and again I’ll say this just again, it’s really important that we make this point very clear that this is a project that Michal is hacking around with in his own time. It’s got nothing to do with Microsoft, right? It is his project.
But I can totally understand where the Microsoft teams are coming from where they’re like," how do we ensure security, how do we ensure backwards compatibility, how do we ensure that the libraries we’re including are the correct versions and maybe we’re doing Sha 256 hashes on them and reproducible builds and things like that." Totally understand that, absolutely. But again, from a minimum viable product of a personal project that you’re sort of I don’t want to say hacking together, but like you’re putting together yourself, it makes perfect sense to just sort of as long as the licenses are permissible. Just copy this file from here, copy this file from there. Because it makes sense to do that, right?
I have a bunch of personal projects in no way on the scale of BFlat, but that require the presence of certain open source packages and they’re all using the same license. So what I do for my own personal project which is built on my machine, sits on my local server, my local build server that I run myself and host myself. I have just a copy of the dependencies that it requires because they are pre compiled so [files] or DLLs or whatever, right? And I’m happy with having looked through the code for that particular dependency and building it myself and being happy with right. That is the version that is sitting with this application.
And I feel like that’s, like you said, it feels similar to what you’re doing at the moment where it’s like, “let’s do it this way because this way makes sense,” right? If you are concerned about it, please go read through the source code, which I would recommend all developers do anyway. If you’re relying on an open source project and you’re pulling that code into your project, taking a direct dependency on it, you should know what that code is doing, right? You don’t have to be deep in the weeds with precisely how it works, but have like a 10,000 foot overview of how the actual code is doing what it’s doing. And not just from a sanity perspective of, “I don’t quite get why this isn’t working,” but as, “a you are shipping this and it will run on real hardware and you don’t know what it’s doing.” So as a side note, folks always take some time to read through the code that is available for your dependencies.
Michal
It’s a nice segue into another topic that I am experimenting with, BFlat.
So in [the] .NET world it’s common to take some dependency on something from a NuGet package and it’s some kind of binary blob that is just part of the app and people don’t think about the source code that comes with it. And often times someone would like to tweak something how it works in the NuGet or they want to do a bug fix and they have to find the repo that hosts it and they have to submit a pull request and hope that the developer will create a new version of the NuGet package. Or they have to kind of put their NuGet package somewhere and then download it from there.
One of the things that people keep asking about for BFlat is, “how do you get NuGet packages in BFlat?” One thing with BFlat is that it’s really just a compiler. It’s not a build system, it doesn’t have a MSBuild. So kind of the usual button developer way where you add the reference to NuGet package doesn’t work with BFlat you would have to basically download the NuGet manually unzip it and then add it as a reference, which definitely works, but it’s all the hassle.
So the other approach that I am kind of encouraging with BFlat is to look more at sources. So if someone wants to have a dependency on a third party library, they would submodule the git repo that has the dependency and they would build it themselves with BFlat and add it as a dependency there. So it’s like one of the reasons why BFlat will never be successful with .NET developers because it’s like super useful to be able to use .NET and super useful to be able to use MSBuild. I am kind of targeting BFlat more to developers that are not .NET developers that just would like to try out C#, but they don’t want to learn MSBuild, they want to do things their own way, like maybe they want to use make files. All of this would work really well with BFlat. And the thing that I am hoping that it would encourage more is for people to take ownership of the third party dependencies that they have to kind of treat it as their own source code to look at how it’s implemented to do fixes in it when they need to do fixes and offer it as an upstream, but not be blocked if it doesn’t get upstreamed. So kind of encourage more potentially forking things, having your own private fork, not kind of looking at dependencies as something that is a black box that either works or it doesn’t work, but kind of treat it more as their own code.
Jamie
Sure. And that makes perfect sense, right? Because if you don’t have an MSBuild like situation where you can say, go get this package from the Internet to download it, then extract it and do whatever it is that I do with a NuGet package. I’ve never really looked into how the whole tooling packages that kind of stuff up, but even though I’ve built my own NuGet packages, I’ve never looked at how they’re included. But rather than having to build all of that, actually saying to the developer, “hey, if you want to take a dependency on this thing, then you take a dependency on it in the Unix way, you go and get the source code, pull it into your repository and build it as part of your build.”
That totally makes sense to me because then like you say, you’re fostering this idea of developers taking, I want to say taking ownership of those dependencies. I don’t know whether that’s the correct term to use, but taking ownership and like I said, like you said, like I said, reading through it and understanding what it is you’re actually pulling into your code base, I think that really fits well with what your goal of this personal project is. And again, I’ll say it again one more time, this isn’t a Microsoft project, this is a Michal project he’s working on in his own time. It’s a personal project. It’s not maybe not meant to be for enterprisey stuff.
Don’t use it for your ASP .NET Core multibillion dollar application because just don’t, because you’re going to stress him out and he’s not going to sleep very well.
But bringing those developers who are maybe used to in Python, I can type into the CLI Python and then the name of my file, and it will run that file. I mean, it’s probably interpreting, it’s probably not doing compilation, but I can do that in JavaScript. I can just give node a JavaScript file and it runs right, like you were saying earlier on, it’s C, C++. I can do cpp which is the GNU C++ compiler, and just give it a file. As long as it’s got a main entry point, it doesn’t matter what that file is called. Right?
Whereas something that I don’t think either of us have explicitly said, but I think maybe you did in that last part, is that if you want to build C# .NET apps, you need to have a csproj file that tells the compiler where to go to get all of your dependencies and which versions of what software and what version of the language and all that kind of stuff. So it needs all of that. But actually, if all you’re doing is writing the equivalent of Hello World to understand how to do Hello, World in net, you don’t need all of the extra cruft around it. In fact, the C# and the .NET teams have proven this over the last few years by creating that sort of minimal APIs is the wrong word, global usings is the wrong word. But this idea of removing the cruft, removing the toil and dropping Hello World in C# from eleven lines of, “what the heck is all of this?” Down to one line, which is `Console.WriteLine(“Hello World”);. Right?
And so I feel like personally, BFlat has this ability to turn C# almost into a scripting language, and that’s probably the wrong way to put it. That is Jamie’s interpretation, not Michal’s goal here. But my interpretation is that it can potentially turn C# into a scripting language if you’re using BFlat, because you can just I’m just going to type a bunch of code. I’m going to wrap it correctly in namespaces and classes and put a main in there. And I can just go BFlat.
My file run and it’ll build me an executable that I can just double click. Or I can call from the command line, hey, there’s my code. Right.
So in that case, then, you’ve already sort of hinted at it, but who is BFlat aimed at? I mean, you said yourself it’s not aimed at sort of enterprise .NET developers who are making it’s not aimed, presumably at .NET developers, it’s not aimed at mobile app developers, it’s not aimed at, presumably, video game developers, it’s not aimed at the ASP .NET Core developer, who is the target audience for BFlat.
Michal
The target audience is mostly me. I brought the tool for myself.
I would like people to try BFlat if they just want to try the C# language and they don’t want MSBuild, they just want to use make files. There is like a subculture of developers who like to have full control and full understanding of what the tool is doing. They don’t want any background processes to keep running after they finish compiling. They don’t want the tool to make internet connections to find if all the NuGets are up to date. They want to know exactly what’s going on. And with BFlat it’s really just a compiler. It will not make any network request like it will compile and then it’s kind of done. So it could be interesting for those kinds of people. It’s definitely a very small minority. I don’t expect BFlat to ever become anything mainstream. It’s really just to experiment things. I think the valuable part of PBFlat is also the minimal core library.
It’s a thing that caught interest in people when I originally did my 8 KB Snake game experiment, and afterwards I did like a Hello World on raw hardware. Then someone from the community came and they basically connected those two things together and they made the Snake app run on bare metal hardware. Then someone else came and they decided, “oh, I would like to build an operating system in C#,” and use the native IoT compiler for this. And I was just talking in the BFlat issue tracker with someone who decided to take this operating system that someone wrote using native AoT and they decided, “oh, I would like to build it with BFlat. And the actually managed to get it running with BFlat as well.”
And it’s a little bit nicer because the operating system right now that someone has in C# they need to hack all the targets file that ship with .NET. So kind of they have to go into program files of .NET and they have to change things so that they can compile their operating system with it because it’s absolutely unsupported. With BFlat you don’t have to hack those targets files because it exposes a mode to compile for no core library. So it is potentially useful for people who would like to experiment this way.
The operating system project, I will send you a link to it. It’s really interesting. It actually has a graphical user interface. It even runs Doom that there is a YouTube video of like C# that runs Doom. It’s pretty interesting.
Jamie
Cool. I wonder the I’ll make sure to put the link into - and all of the links that we talk about today are going to be in the show notes so please don’t worry about, “oh wow, I’d love to know about what that operating system is and stuff like that.” So don’t worry about that folks there’ll be links in the show notes.
But I wonder so this is just me conjecturing as we wrap up, right? I wonder whether so I know that Scott Hanselman wrote an operating system in C# and .NET. I wonder whether that one would run with BFlat and what you would have to do to get that running. That’s just a thought experiment, folks. Iif you’re listening in, because we’re recording this way ahead-of-time, we’re recording this in April. So it could be that someone somehow magically heard what I just said and tried it out. Or it could be Michal may be about to drop the knowledge bomb that it is Scott Hanselman’s operating system. I don’t know, but it’d be interesting to find out whether that one could run on using BFlat too.
Michal
Yeah, it wouldn’t surprise me if it worked. I don’t know about that operating system project, but I know that multiple people tried, but only one project got to that stage where it has a nice GUI and you can use mouse and you can move Windows when it runs Doom.
Jamie
Cool. Well, that’s a great place to leave this conversation, I think, because yes, it sounds like BFlat is one of these wonderful experiment. It’s a wonderful experimentation thing that people should be checking out and seeing whether the what they can run on it. Right? Because I think it would be useful just to see, “oh cool, I can create a tic-tac-toe or whatever game, or I can create a messaging platform or something.” Don’t create a messaging platform, that’s way too complicated. But just for fun. Right, see what you can do with BFlat, but don’t get upset if it doesn’t do what you want it to do. It sounds like it’s a lot of fun to work on and it’s a lot of fun to find out what it can do.
But I guess as we wrap up Mikael, what are some of the are there any websites you’d recommend people check out? I mean, I know there’s flattened .NET that people can check out for details on BFlat itself. Are you open to people reaching out over Twitter or socials, things like that, to ask questions and maybe someone’s gone, “hey, this is a great project, I’d love to pick your brains about it.” Are you okay with people reaching out and saying, “hey, Michal, I’ve got a question?”
Michal
Yeah, I’m definitely open to question. So there will probably be a link to my Twitter. There will be a link to flatten net. From flatten .NET you get to the repo itself. The BFlat repo is actually split into two parts. One has the compiler that puts everything together. So that’s the thing that kind of invokes the C# compiler and then it invokes the native AoT compiler, and then the linker and everything else. And then there is another repo that is a fork of the official .NET runtime repo that has some customizations that I need for BFlat that don’t exist in the mainstream .NET Runtime. The customizations are pretty limited, but still I needed a separate repo and that’s where the assembly parts are. The default repo doesn’t have any assembly, but the runtime repo does.
Jamie
Cool, okay, well, I’ll make a point of putting all of those links into the show notes. That’s not a problem.
Yeah. What I’ve got to say, Michal, this has been another fantastic chat with you. I feel like I’m walking away with way more knowledge than I had when I first walked in. And I’m going to have to be trying out some of my silly projects with BFlat this afternoon, see what I can do with it and see what I can build with it. Like I said, I’m not going to build anything enterprisey with it, just some silly stuff, right? Because I’ve got a couple of, couple of apps that are just CLI apps, maybe I can just throw BFlat at it, pull all of its dependencies in as standard source control and get something that’s native AOT that will run perhaps faster, perhaps not, who can say, right?
Michal
Yeah. The thing about BFlat is that if you use the standard .NET Runtime libraries, it’s not really different from what you get with publish AoT [on the] official SDK. The extra value with BFlat is really be able just to run the compiler and the crossplat compilation part, which is something where I can cut shortcuts and ship it in like two days if it ever becomes something that is officially supported. And it will be probably a lot more work for Microsoft to do it in a way that kind of the enterprise the customers will be happy about.
Jamie
Cool, yeah. So please bear that in mind folks, that it’s not that it hasn’t been engineered from the ground up for enterprise stuff, it’s just that’s not the target audience. So if you’re doing stuff that is enterprisey, perhaps don’t try out BFlat.
Michal
Yeah, the BFlat is for hackers. Definitely it’s the compiler for hackers, it’s not a compiler for companies.
Jamie
Excellent. Well, what I’d like to say, Michal, like I said, thank you ever so much for this conversation. I’m sure that people are going to walk away from this with, “holy cow, look what I can do with my C# code!” So, yeah, thank you ever so much.
Michal
Thank you, thank you for inviting me.
Wrapping Up
That was my interview with Michal Strehovsky. Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at dotnetcore.show, and there will be a link directly to them in your podcatcher.
And don’t forget to spread the word, leave a rating or review on your podcatcher of choice - head over to dotnetcore.show/review for ways to do that - reach out via our contact page, and to come back next time for more .NET goodness.
I will see you again real soon. See you later folks.