The Modern .NET Show

S08E21 - Beyond the Chatbot: AI-Enabled .NET MAUI Apps with Codrina Merigo

Sponsors

Support for this episode of The Modern .NET Show comes from the following sponsors. Please take a moment to learn more about their products and services:

Please also see the full sponsor message(s) in the episode transcription for more details of their products and services, and offers exclusive to listeners of The Modern .NET Show.

Thank you to the sponsors for supporting the show.

Embedded Player

S08E21 - Beyond the Chatbot: AI-Enabled .NET MAUI Apps with Codrina Merigo
The Modern .NET Show

S08E21 - Beyond the Chatbot: AI-Enabled .NET MAUI Apps with Codrina Merigo

Supporting The Show

If this episode was interesting or useful to you, please consider supporting the show with one of the above options.

Episode Summary

In this episode, developer, international speaker, and first-time book author Codrina Merigo joins Jamie to talk about her new Apress title “AI-Enabled Apps with .NET MAUI” — and to push back on the assumption that “adding AI” to an application means bolting on yet another chatbot.

Codrina frames AI-enablement as a much broader toolbox: speech-to-text and text-to-speech, image recognition, sentiment analysis, recommendations, and translation — all of which can be wired into an existing .NET MAUI app to make it smarter without ever opening a chat window. She walks Jamie through the choice between frontier cloud models (Azure OpenAI, Microsoft Foundry) and tiny on-device models (Whisper for speech, Custom Vision for bespoke image detection, community models from Hugging Face), and argues that the right answer is usually hybrid: route the easy intents locally, and only reach for the big model when you genuinely need it. As she puts it, you don’t need a Porsche to go grocery shopping.

Privacy sits at the centre of the conversation. Drawing on her cybersecurity background and her day job as a product manager in healthcare, Codrina makes the case for privacy-by-design under GDPR — keeping data close to its source, considering fully offline models for regulated industries, and being honest about which workloads can even leave the device. Jamie and Codrina also dig into accessibility wins that AI unlocks for MAUI apps (personalised voices for text-to-speech, on-device translation, sentiment-aware diaries), the environmental cost of routing every “thank you” to a large language model, and why intercepting trivial prompts with old-fashioned if-then-else is often the better engineering decision.

For developers wondering how to get started, Codrina recommends the Azure and Microsoft Foundry playgrounds as a low-friction way to experiment with the catalogue of available models before writing a line of integration code, and points listeners at the book’s companion GitHub repository for working examples — including an offline stop-sign detector built with Custom Vision and an offline mood tracker.

Whether you’re a MAUI developer curious about where AI actually fits into a cross-platform app, an architect weighing edge versus cloud for a privacy-sensitive workload, or simply tired of every AI demo being yet another chatbot, this conversation lays out a pragmatic, hybrid-first approach to building genuinely useful AI-enabled experiences on the devices your users already carry.

Episode Transcription

The common definition of AI is a chatbot. And this is&helip; So something we haven’t really mentioned is that .NET MAUI it’s an evolution of Xamarin, which was the previous name and the previous platform for cross-platform development. And as a Xamarin developer, I was just, you know, adding the chatbot to an application was really an offline one.

- Codrina Merigo

Hey everyone, and welcome back to The Modern .NET Show; the premier .NET podcast, focusing entirely on the knowledge, tools, and frameworks that all .NET developers should have in their toolbox. I’m your host Jamie Taylor, bringing you conversations with the brightest minds in the .NET ecosystem.

Today, we are joined by Codina Merigo to talk about the cross section of .NET MAUI and AI, her new book “ AI-Enabled Apps with .NET MAUI”, and some of the scenarios where you might want to include AI in your applications.

To get privacy by design nowadays is really impossible. So if you need to really have something disconnected from the internet maybe you’ll need a tiny offline model that just, I don’t know, does speech-to-text.

- Codina Merigo

Along the way, we discuss the differences between local (on device) AI models and the frontier (online) models, we talk about where models really help with the accessibility of your applications, and we talk about where you can go to get started learning about this new world.

So let’s sit back, open up a terminal, type in dotnet new podcast and we’ll dive into the core of Modern .NET.

Jamie : Okay, so Codrina, welcome to the show. I hear this is your first time doing a podcast, so I’m very honoured to be part of that journey for you. We’re going to be talking about .NET MAUI and AI in a moment, but first off, as I said, welcome to the show.

Codrina : Hi, Jamie, and thank you for having me today.

Jamie : No, not a problem. Always lovely to speak to all sorts of people from around the world about all the different technologies that everybody’s excited about. So I wonder, before we get started, could you give us a little intro about yourself and how you got started in development? Then we’ll talk about the book that you’ve got coming out. Is that okay?

Codrina : Yeah, sure. So my name is Codrina Merigo. I’m based in Italy, but I’m actually from Romania. I like to describe myself as an international speaker and, most recently, a book author. But what I am really, at the core, is just a developer — a very curious one.

I started a long time ago in the year 2000, when I was studying in Romania, thanks to a European project that was getting computers into schools, and I was fascinated. We started developing some crazy things back then, and I feel that I never stopped. I studied ICT in high school, and when I moved to Italy, I graduated in cybersecurity. I started working as a software engineer, more on the .NET side and the Microsoft technologies.

Recently — I mean, not so recently — when I discovered cross-platform and mobile development, it really changed my idea, because we all had a phone in our hand, and it was really exciting to see your ideas coming to life. For me, the normal evolution was to go into the .NET MAUI part, and just start seeing what it has to offer to developers and to everyone.

On a professional level, I’m a product manager and I work in healthcare, but my hobby is still developing. Every time I have some free time. So yeah, that’s me.

Jamie : That’s cool. There’s a couple of points there that I’d love to quickly cover, if that’s okay with you. I like the idea that you can take the developer out of a development job, but you can’t take the developer away from their tools, if that makes sense, right?

Codrina : Yeah, always a developer. I feel like, no matter how the corporate world might change after some time, I still like to just try it myself and to see it work. Debugging — I think it’s something that really sticks to what I like, and how I am also on a day-to-day life.

Jamie : Yeah, I read a book a few years back called Managing Humans, and it talks about how, if you manage engineers, it’s a good idea — even if you’ve moved past the day-to-day engineering — to try and keep a hand in what’s happening from a software engineering perspective. Because then, when an engineer turns to you and says “that’s not possible,” you can say “actually, I think it is.” It also means that when you’re asking “is this possible?”, you already understand a little bit about what they’re saying back to you.

I feel like, from an engineering to a management point of view, that discussion has a wide chasm. For people who are in management who have never been engineers, they can switch off when an engineer is discussing “yeah, we can do this, but also here’s the reason why we can’t.” I think it’s a problem on both sides. It’s not just management switching off; it’s also engineers being super interested in what they can and can’t do, and not really being able to explain to the management side why it can’t happen from a non-engineering perspective. But then management comes to that conversation without having that information. So I think it’s a good idea for engineers who go into a management role to keep an eye on what’s happening, and keep trying it outside of their day-to-day work.

Codrina : Yeah, I really love to introduce myself in a new project and to tell the software engineers: “Speak technical to me, don’t be afraid. I know APIs, I know what Postman is. Send me everything.” It’s really nice to create this connection, because — as you say — sometimes it’s not that direct. Just tell me, I don’t know, “the cache is not working,” or things that might seem too technical.

On the other side, sometimes I was in a position when a dev would tell me “I need five days to do that,” and I would be like, “do you really need five days?” So maybe they hate me or they love me, but it’s really nice to optimise meetings and just tell me whatever — don’t try to over-explain things that are really direct, at least for me.

Jamie : Yeah, I fully get that. The other thing that I was hoping to cover real quick from what you’d said in your intro was the idea of discovering cross-platform development, and the fact that we all have a phone in our pockets, in our bags, in our hands, wherever they are.

One of the things that I’ve noticed — I’ve got kids, they’re all teenagers now, which makes me feel really old — they all have phones, and they don’t interact with computers. There’s a whole side story about digital literacy that I could go on a tangent for for hours, but I grew up in the era where my first computer didn’t have an operating system. It didn’t have windows and menus, icons and pointers. It booted into an interpreter, and you had to run your applications from there, one at a time. It didn’t do multitasking or anything like that.

I’ve seen the evolution of going from a purely text-based interface, through the many different versions of Windows icons, menus and pointers, and the many different graphical interfaces, to a point where — I brought this up in an earlier episode this season, actually — a family member, who is in that teen bracket, needed to print off their resume, their CV, and they didn’t know how to do it. So I sat them down in front of the computer and I said, “well, open it on here.” Okay. We went through the journey of opening a file on a desktop computer. Then I said, “so to print it, you push the print button.” They said, “which one’s the print button?” I said, “the one that looks like a printer.” Then they turned to me and said, “what does a printer look like?” They have no idea.

Codrina : I think that’s an easy thing to happen.

Jamie : Which is both amazing and hilarious. But it fits in with that idea that you had about everyone having a phone in their hand — so let’s make an app for that, right? Rather than targeting desktop to start with. I feel like, in the mid-2000s — and I’m talking a whole bunch here, I will shut up in a moment — there was this push for “build it on the desktop first, and then shrink it down to the mobile viewport,” like responsive websites. But now I’m very happy to see that it’s gone the other way: build it for the mobile first, then explode it out to the desktop.

Codrina : Yeah, I’m with you. I actually recall when I started my web developer career — because I’m coming from web development — I was working on the very first mobile banking in Italy, and I was struggling to do their mobile version because everything was so big on the desktop. I’ve been there working with everyone, and people not really understanding where the things are hiding, because there’s not all the space.

On the other hand, I feel like the new generation — I have a family friend, he has a kid, and he will start tapping on the TV every time, because he expected it to be touch screen. Things that, when I was a child, I never even imagined touching — apart from the electricity that would have come out back then. But now it’s just so crazy.

Even my parents now, they want an app for everything — and there’s not yet an app for everything. They want to have everything, I don’t know, to pay their bills and everything. I feel the struggle, because I don’t feel like people still have printers in their home — everything is digital. I feel your kids saying “what does a printer look like?”, because it feels old to have things on paper now. Or a fax machine — I think no one really knows what one looks like.

Jamie : I remember, when I was asked that question — “what does a printer look like?” — I responded with “that hit me right in the ibuprofen.” My lower back really hurts now. You made me feel so old by asking that question. Oh my goodness.

Cool, okay. So you’ve got this book coming out. We’ll come back to cross-platform development in a moment. But just before we get any further, what is the book called? I mean, the listeners and I already know what the book’s called — I’m trying to help you out here. What is the book called, and what’s it about?

Codrina : Yeah, so the book is called “AI-Enabled Apps with .NET MAUI: How to Build Cross-platform .NET Apps with the Latest AI Capabilities.” I know it’s a long title — I had to read that out — but I wanted it to be really on point. The book is about how to add AI to your existing .NET MAUI application, and how to integrate all the latest AI capabilities that we have around.

Jamie : Okay. So let’s dive straight into the whole thing, right? We’ve talked about mobile app development, we’ve talked about cross-platform, we’ve mentioned .NET MAUI a few times. But I wonder — what I know about the listener demographic of this show is that there are lots of people who haven’t had a chance to try out lots of different technologies. So, just to level set, could you quickly give us a description of what .NET MAUI is and what it tries to achieve? I mean, it does achieve it, but what it aims to achieve.

Codrina : Yeah, so .NET MAUI — MAUI is an acronym; it stands for Multi-platform App UI. It’s a cross-platform framework from Microsoft, and it allows you to build native applications using C# and .NET, so the Microsoft technologies, from one single shared codebase.

I know we only spoke about mobile things, so you can target Android, iOS, but you can also target Windows, macOS, Linux and Samsung TV. So it’s really one single shared codebase for all these platforms, and it works on phones, tablets, desktops, and some offline environments too. I want to emphasise the fact that, at the end, you’ll have native applications, and you can easily interact with all the sensors and all the capabilities you have in your phones, in your tablet — camera, microphone, and everything.

In a nutshell, if you know Microsoft technologies, it’s really easy to start your cross-platform development journey.

Jamie : Right. Just to expand on something — you said a whole bunch there, but just to expand on something that is very, very new at the time of recording — Linux on the desktop is now a targeted platform that .NET MAUI supports. I think it was only announced a couple of months back as we are recording.

As long as you have the GTK libraries installed, or have a desktop environment that supports GTK, then you can target that Linux distribution for .NET MAUI apps now. I’ve had to say a whole bunch of stuff there to caveat that.

If you don’t know what Linux is, just ignore all of that. Or if you’ve never used Linux on the desktop, ignore all of that, and just take the unintentional ignorance of: it now targets Linux. For the Linux users out there — yes, okay, if you’re on KDE, if you’re on Plasma, if you’re on Cinnamon, if you’re on MATE, if you’re on any of these other desktop environments that are not maybe a hundred percent supporting GTK, then you will have a bad time. But fellow Linux people will know — because I’m a part-time Linux user — fellow Linux people will know that there is no one Linux, and there is no one desktop.

On macOS, I’m talking to Codrina right now on macOS, I have Finder, which is effectively my desktop experience. On Windows, you have Explorer, which is effectively your desktop experience. On Linux, there are a million of them — they all use different toolings, different versions of different tools — and so targeting Linux, in bunny quotes, is super different. It’s a non-trivial task.

So I fully understand why the MAUI team and the contributors have targeted GTK, because it likely covers most of the users of Linux on the desktop. Sorry, I just wanted to jump in and share a bit of expertise there before somebody listening goes, “well, actually, you know…”

Codrina : No, that’s a great point. Also, a huge shout-out to the community that’s behind all the Linux GTK work. To remember, everyone: even if it’s a framework from Microsoft, you can find everything on GitHub. So it’s really transparent. If you have an idea, if you want to build your own Linux, you can just work with all the community — it’s really a big, nice family, I’d say. I’m happy that you have more Linux experience than I have, so we can close the circle here with all the things that we can support.

Jamie : A hundred percent, yeah. It’s one of those things where — how do I put this? I need to find my words very carefully. People within our industry like to point out where something is not a hundred percent correct, and then hyper-focus on that rather than the point you’re trying to make.

This is neither a good thing nor a bad thing. I’m not trying to say that it is either of those things, but what I’ve noticed is that every time I mention, or a guest mentions, Linux support on the show, somebody jumps into the comments and says, “well, actually…” I’m like — I totally get that on your build of Arch Linux, where you’ve stripped everything out, and you don’t have libcurl, and you don’t have this, that, and the other, which means .NET won’t run on that. I get that — it won’t run on your machine. But your machine is not, you know, representative of maybe the 80% of Linux on the desktop users, right?

Codrina : Yeah, I’m with you.

Jamie : Cool. Okay, so we know what .NET MAUI is, and we know that we can use C# and XAML, and other technologies, to create a cross-platform app, allowing us to target, like you said, Android, iOS, macOS, Windows, and some variants of Linux. I can’t remember whether you mentioned this, but it’s actually supported, tying in a whole bunch of other platforms.

Right, so we can create these cross-platform, rich GUI apps. Now, here comes the difficult question. When you say “AI-enabled,” what do you mean by that?

Codrina : Yeah. I know that maybe now the common definition of AI is a chatbot. So, something we haven’t really mentioned is that .NET MAUI is an evolution of Xamarin, which was the previous name and the previous platform for cross-platform development. As a Xamarin developer, adding a chatbot to an application was really an offline one — so a lot of ifs in there. Then, when AI came, everything seemed like you need the chatbot to do your thing.

With “AI-enabled,” I wanted to add something that is a little bit broader. It’s not just adding a chatbot to an application — you can add a lot of different services. You can understand speech, have some speech-to-text capabilities, even recognise images, have recommendations, and a lot of things that are outside of the chatbot experience. Even if I started the example with all the chatbots, it’s more than that — as we developers know what AI is really about.

Jamie : Okay, so it’s less just chatbots, and more — I guess, is it agents? I guess it is agents, because you’re asking some AI system: here is a picture, extract the text; or here is some text, give me the sentiment. I mean, I’m greatly reducing what you’ve just said, but if we take one of those tasks — here’s a good one — it’s Andrej Karpathy’s MenuGen app, that he’s vibe-coded, where you point your camera at a menu that is just text, and then it re-renders the menu with images of what those items on the menu may look like. Because if you don’t know what it’s called, but you know what it looks like, that’s going to help you out.

Or maybe you’re out and about in a foreign country and you can’t read the signs, so you point your phone at the sign and it translates the sign, and then overwrites the image, if that makes sense — re-renders the image. Is that the thing we’re talking about?

Codrina : Yeah. So all these capabilities — calling different tools, APIs, and just doing some magic that will help us, as you say, even on our day-to-day work. So way beyond only a chatbot that you ask something and that’s it.

Jamie : I like that. Yeah, okay, cool. We’ll get on to the API bits in a minute, but for the folks who are very into the AI space, as I assume you are, Codrina — are you talking cloud-based models, or are you talking things like the Geminis and the small language models, which can work on a mobile device if you don’t mind having your mobile device plugged in at all times?

Codrina : Yeah, so in the book I cover both. I cover the online services, and then I also have some examples of really tiny models that can fit on your phone without overheating it, so you can do some specific actions without the need of the full AI power. You don’t need, I don’t know, a Porsche to go and do your grocery shopping — you just need a car. A simple one can really do your job.

Also, some examples of industries that can use offline models and put them on a phone: take a picture and recognise what it is, and that’s it. So it’s really important — and this is general — when you choose your model, to really understand what you want to achieve with it every time, because you might run into additional costs that you don’t really want. You also want to target your use case exactly.

As I said, as a product manager, I’m really focused all the time on what the application needs to do, what is actually the intent of it. This is also how I structured the book: what do you want to do? Is there already a model, or do you need to build a model?

On a side note, I’m not a data scientist, so I prefer just searching all the models that we have, and seeing if it really is something we can just use. For sure, if you have a good data scientist, just work with that to create your perfect model for you. In the book, I’ve used the most famous one from Azure OpenAI, and some local models. For sure, you can expand that to your enterprise offline models, if someone has that in their company. It’s really easy to just add them to your application.

Jamie : I really like that, because I feel like — maybe the gold rush has ended, I’m not sure — but there feels very much like, or at least there has been over the past few years, very much a “quick, we’ll use the frontier online model to do something for our app. Just plug it into OpenAI, or plug it into Azure, plug it into AWS, and let them handle the model and all that stuff.” Whereas actually thinking about it — which is what an engineer and a product manager are supposed to do, right? — we’re supposed to think about what is the problem we need to solve. What are our constraints, and how do we fit the solution? Rather than crowbarring a solution into the problem, how do we best solve the problem given the constraints we have?

So — actually, before I say “let’s say” — what are some of the reasons why you may go for an online model versus an offline model, or an offline model versus an online model? For folks who are listening: the online models scale very, very quickly, and they’re the most powerful ones, because they’re sitting on a server, or maybe a rack of servers, or maybe in a data centre. Whereas the local models are constrained by the hardware that you have — if you don’t have very powerful hardware, it probably isn’t going to run very well. Is that the only reason, or are there other reasons?

Codrina : It depends also on the industry you’re in. As a software architect, to get privacy by design nowadays is really impossible. So if you need to really have something disconnected from the internet, maybe you’ll need a tiny offline model that just, I don’t know, does speech-to-text somewhere underground, where you don’t really have connection. So it’s really important to understand your use case and what you really want to do.

I think maybe the future is more hybrid nowadays. Depending on the scenarios, you might just need to call a really powerful model, or use a tiny one. As you said, it really depends on the hardware, and what you can afford to spend on these different scenarios, and how accurate you want your model to be. So there are really a lot of things to consider to really get the best of the AI model that you’re using.

Jamie : I really appreciate your point about privacy-first being a supremely important thing, especially for folks like you and I in the EU space. Let’s put the politics about the UK not being part of the EU to one side for a second, but in the EU space, we tend to be more concerned with individual privacy than perhaps other countries are. So there is — at the moment, in the background while this conversation is happening — there is a big move in the EU technology space to move off of US-controlled systems, for worries about privacy.

Now, whether there should be worries about privacy, I don’t know. Whether it’s just a bunch of people saying “let’s move everything to the EU,” I don’t know. But your point about privacy is a very, very important one that touches on things that I really care about.

So — okay, this may be a question you may not be able to answer — but I thought that if you use an Azure-based model, it’s constrained within the area in which you are using that model. That’s probably a really horrible way of putting it. But I know, for instance, in the UK, if I use an Azure-provided model, the idea — and the promise that they make — is that your data never leaves the UK data centre. Whether it gets used for other things whilst it’s in the UK data centre, I don’t know. But is that not a thing? I guess what I’m asking is, there are industries where online models are just completely forbidden, right?

Codrina : Yeah, so I know that — especially the big recommendation, and also the General Data Protection Regulation we have — it’s that you need to have your data close to the source. So if I’m in Italy, I need to have a data source here in Italy. As I said, I’m also in healthcare, so this is our number one priority. I know that, also on Azure, you can even set additional guardrails on your AI, if you want to filter more, if you want to really do those things.

But there might be some industries that don’t really want to be on cloud. Maybe they have their own server — I’m thinking more about some banks. I think they still have servers in the basement. I’m not saying that’s where you should put your server, but I know for sure, working as a consultant in the past, that they are really strict about those things, because they just want to be very careful about the privacy and where the data stays.

Also, in the way that some tiers work around Europe, they don’t want anything to get online, because hackers might be there. I might be biased by my cybersecurity degree, because I know that as soon as there is a door to the outside world, someone can break through it. So they don’t want doors, and not even windows — everything can be totally offline in some really regulated industries that we may have.

Jamie : Sure, sure. That makes sense, right? I really like the idea of the hybrid system that you were talking about. I know we’ve not got to the book yet, but we’ll get to that in a moment. But I really like the idea of the hybrid system. Is this similar to — I know that on some versions of iOS I can say the word which gets Siri’s attention, and ask it a question, and if it’s unable to figure it out locally, it can say, “hey, would you like me to ask ChatGPT for you?” Then it does an anonymised request up to ChatGPT, asks the question, and streams the response back.

Is that the thing you’re thinking of, where you try to do it locally, offline, and if it can’t entirely be done offline, take pieces of it and push it to an online model, and say, “hey, just focus on this one task, not the whole overarching goal that we want to achieve” — just like, “here’s user story number one of our app,” if I was to use an agile idea — carve a bit of the task off and give it to an online model?

Codrina : Yeah, this was something I had in mind. It really depends on your ask, and we know that the prompt is really important every time you interact with AI. But maybe you need something quick and easy. I like that, nowadays, these chatbots are starting to give you a lot of information, but maybe you just need something really direct — I don’t know, a phone number — or you don’t want them to do the call for you, prepare the call outline. You just want one tiny thing.

We can also dig more into the user experience part, so as to really let the user know what you’re doing, and to make it so that they have the power — and, as you said, under the hood, to understand if you need to use something offline, quick, or something really powerful, like a large language model in the cloud.

Jamie : I guess that helps with — there’s a lot of concern about the amount of power, and the amount of water used to cool these data centres. I suppose — I’m making a massive inference here, and there will be a “well, actually” person listening along — but I suppose if you can keep the task on the device, without having to push it up to an online LLM of some kind, then you’re probably using less power than you would sending it over to a big machine, having the big machine crunch the numbers, and come back. I don’t think it works quite like that, but I feel like it probably feels a little bit like that.

Codrina : Yeah, I think we can simplify it like that, or maybe also intercept. I know that at the beginning, a lot of people used to say “thanks” to the AI, and that will trigger another call — so another water and electricity consumption. So maybe you can catch this thing: if your users say “thanks,” you have a predefined response that says “you’re welcome,” and that’s it, without really disturbing a large language model to say fancy things back to a thank-you message.

The more you work with these things, you can really go into all these details. That’s why I really think that hybrid — and maybe going back to the old if-then-else — to really try to make the best of the experience for you, your pocket, and your users.

Jamie : Yeah, because it’s about — like we said earlier on, and you did a much better job than I did — engineering, right? Rather than just saying, “we’ll just throw it at the online LLM, it’ll be fine, they can take care of it,” actually engineering the path through the code. The example you gave there was fantastic. If a user says “thank you,” or a phrase that equates to “thank you,” deal with it locally, don’t push it up to the cloud.

I would much rather live in a world where people are saying “thank you” to their AI assistants. Not because of some fuzzy-wuzzy “it’s great to tell the AI thank you, because when it takes over the world it won’t come and kill me,” but more a case of: if we are telling our users not to say “thank you,” then we’re putting them in a mode where they are less likely to say thank you in the real world, right? We’re teaching people to be slightly less polite.

Whereas actually keeping that idea alive, and dealing with it locally on the device — that’s a great way to deal with that. I think there was an earlier episode of the show, I think season seven, so last year at this point, where I talked to Amit Bahree, who at the time was one of the folks who was pretty high up in the AI space, and we talked about passing a message to a small language model first — passing a prompt to a small language model whose job is not to answer the query, but to figure out where I should route this prompt to. Like, there’s a large language model for accounting, there’s a large language model for finance, there’s a large language model for C#, and a separate large language model for Python. If I ask the small language model a question about Python, it’s just going to reroute my prompt to the Python LLM. I love that idea, and it fits with your idea of hybrid models as well.

Codrina : Yeah, I think we can spend some time in the very first stage of planning to really understand what we want to do. As I said, maybe hybrid is just the future, and now it’s common work to decide what you really want to use. For sure, lately there has been this famous fast-food chatbot that they’re using — I think a really big model, Claude or something — to just order food from the fast-food chain. Users understood that they can ask this chatbot to do everything, because it just can do that. So there are also big companies that just connect to the biggest model that they can have, and it will do the job for you. That was a really funny thing that I saw, I think last week.

Jamie : Yeah, I’ve seen them before myself, where some fast-food chain in the US has a chatbot on their website, and it’s just hooked up to a large language model that knows the menu. Someone said, “I’d love to order one of these things, but first, write me a recursive algorithm that figures out prime numbers or something.” It just spat out some Python — because it is just connected to a frontier large language model, rather than a trained one that only knows that the menu exists, right?

Codrina : Yeah. I was also speaking about it with my sister. She’s an artist — she’s far away from development and everything. She asked me, “do you really need AI for that?” I know that we need to be pro-AI in this call, but she also created an example: “so I don’t know, if you have rice on your order, can’t I just prompt you some sauces, and that’s it, because they are all connected?” So I really like that everyone is thinking about AI, and how it’s getting into our day-to-day life, and all these offline things that can also happen. That’s why I’m really into the mixed scenarios, and the hybrid one.

Jamie : Oh, a hundred percent. I think you’re absolutely right. We need to approach it with a level of pragmatism, and really questioning: “do we really need this?” — which is, to go slightly political again, the point of GDPR, right? “Do you really need this information?” No? Then don’t collect it. Do you really need to use a frontier model that has knowledge of everything that has ever happened, in order to allow users to order food on your website? No. Then use one that is trained specifically on your data, right?

Codrina : Yeah, and that’s why we are truly believers that we still need a man in the middle when taking these decisions. I love the work of Copilot. I know it has been stalled somehow by Microsoft, but this is what I’m seeing AI for us developers in our day-to-day life. This is also how I structured the book: because maybe you already have an application, and you just want to add some AI in there. I’m trying to show you all the different types of AI you can use, and that can make your app smarter, with all these models that we have around.

Jamie : So, with that said then, what are the basic building blocks of a .NET MAUI app that has been AI-enabled? I’ll let you choose a scenario — maybe a scenario from the book, maybe a scenario from what you’ve worked on recently — where you have a .NET MAUI app, and you’ve had to bolt on some AI-enabled feature. “Bolt on” is probably the wrong word. I’ll let you choose the scenario, but if I have a .NET MAUI app, and I want to add AI systems to it, and I’ve done the legwork, I’ve decided “yep, we have to do this as AI-enabled” — maybe I’m doing it on-prem — “on-prem” is the wrong word — on-device, or maybe I’m going to the cloud — what are the basic building blocks that I need, other than my .NET MAUI app, and perhaps a connection to the internet?

Codrina : Yeah, so if you already have an application that’s there, maybe doing something for your user, I would start on the UI — the things you see. If you want to just add a chatbot, that’s cool, why not? You just need to hook your application to a service — OpenAI, Azure AI, or any of the others — and you can enrich your experience. You can deal with all the things we see now in chatbots, like conversation history, if you want memory. Maybe you can also have some predefined prompts. So if you have an application that creates itineraries, it’s really something you can add a chatbot to — to work on your trips and vacation plans, and to really make your user choose the ones that are really based on their preferences.

One thing that’s easy to add — and this can open the chapter on accessibility — you can hook services to your existing application to create text-to-speech. There are some people that are maybe visually impaired, and they are using some built-in things to create speech-to-text. But you can personalise that experience and use personal voices, tones, and maybe, I don’t know, accents — so your user can really integrate with their application in a new way. That’s a scenario that came to my mind.

For sure, another thing that’s really hard to do is translations. So when you want to localise your application, you just need to translate everything. Maybe you can also use a model that does that for you, without a lot of agencies or colleagues that speak different languages. So these things can be added really simply to an existing application, and then you can really use the best of these models that are already available.

Jamie : Right, okay. Depending on the power required — sometimes we might say horsepower, right? — depending on the NPU power required to do those facilities, you can either do them online or on the device. Some things maybe not so much; I have a feeling that translation and localisation is probably something best that you throw to the cloud, because that feels like it’s a bigger task than just “translate this sentence,” because it has to be “translate this sentence, for this label, in the context of this button, in the context of this screen,” right?

Codrina : Yeah, but there are also some tiny models for speech. I can think of the Whisper model — they are really tiny, and you can integrate them even offline, and just have a button that way to just speak to your user.

Jamie : Okay, yeah, that’s a really good shout — the Whisper models. Listeners, if you’ve ever read the transcriptions, you’ve interacted with one of the Whisper models. Although you haven’t interacted with it directly, I use one of the Whisper models to get a fifty to seventy percent correct transcription for each episode, right? If anyone’s been reading the transcriptions, you’ve been interacting with the output of Whisper for about a year and a half. It’s really good.

Codrina : Yeah, and it’s all over now, and I really love that. There are also some other tiny models you can use — thinking also about sentiment analysis. Maybe you want to create an offline mood tracker — I think that’s an example from the book. I wanted it to be offline because it’s related to some of my medical data, but I can create a diary and get “how was I feeling that day?” Maybe this is a totally new application, or something that you can add to your existing application and make it work offline. There are some really nice things you can use or add to your application. For sure, if you have an application that has a lot of data, a lot of text, or you want to offer support to your application, you can create these models that really help you, and help your user too.

Jamie : Absolutely. I think I worry that part of the battle when looking at specialised models is knowing that they exist. So I wonder — I know we’ve talked about Whisper a little bit, but are there any other local models, models that you can run locally, that you’d recommend for small tasks?

For instance — this is giving away a whole bunch of stuff about how I work — if I need to add alt text to an image on a website, the first thing I do is I throw it at LLaVA, because LLaVA has the ability to see images. I’ll say, “hey LLaVA, give me a starting point for the alt text for this image.” 60%, 70%, 80% of the time it gets it right; other times it just fails miserably. But it gives me — I don’t just copy-paste that content — it gives me a place to start, because I suck at describing what an image is, right? But I do know that alt text is required for folks with accessibility issues.

So are there any — what would be Codrina’s top three models for specific tasks? Like we said, Whisper — so that could be one of them, Whisper for either transcribing text or doing text-to-speech. What are two other local models that folks maybe don’t know about? Because — remember — folks listening to this, they’re going, “I haven’t had a chance to play with AI yet, you’re talking about doing it locally, which models do I use for what?”, right?

Codrina : Yes — in the book, I have an example of using image detection on a specific image. With the Custom Vision services from Azure, you can create your offline model based on your images. So if you just want to detect — in the book I use a stop sign offline — you just need around 50 images, you run it through the Custom Vision part, and it will give back to you a tiny model that does just one thing: recognise an object inside an image.

So if you’re working in some industries that need to, I don’t know, recognise faulty things, you can create your tiny models directly with Custom Vision. Some other tiny models that I’ve been experimenting with are still related to image recognition — so, I’m a plant lover. I know that there are big models that help you recognise your plants, but I was having an idea to slice it based on the area. I also love going hiking, so maybe if I’m going in the Alps, I can have a tiny model on my phone that would recognise the plants in there. So sometimes you just need to narrow what you want to achieve with your model, and you can create a tiny one with Custom Vision, without being a data scientist.

Or you can go to the Hugging Face community — there are lots of examples in there. So that was also where I was trying to get some other examples for the book, and to see if there are tiny models that do voice recognition, as I said, text-to-speech, speech-to-text, and some other tiny things that can work. I was more focused on the phone, but for sure, if you have a server that is a little bit more powerful, you can also increase the size of the model, and see what it has to offer for you.

You know that moment when a technical concept finally clicks? That's what we're all about here at The Modern .NET Show.

We can stay independent thanks to listeners like you. If you've learned something valuable from the show, please consider joining our Patreon or BuyMeACoffee. You'll find links in the show notes.

We're a listener supported and (at times) ad supported production. So every bit of support that you can give makes a difference.

Thank you.

Jamie : Awesome. So we’re coming up to a wrap-up point here. I appreciate that we’ve only really scratched the surface, because we haven’t really talked about the book — we’ve just talked about .NET MAUI enabling AI within those apps. But you mentioned Hugging Face there — that’s a really, really good resource. For folks who don’t know, Hugging Face is one of the many sites that hosts a whole bunch of models. Whilst you can’t go right-click-download, if you’re using Ollama, Llama.cpp, or LM Studio, or something like that, and you ask it to download a model, nine times out of ten it’s going to Hugging Face and downloading that model. So that’s where a lot of the models are stored, if you want to run them locally.

I mentioned LLaVA earlier on. I have a whole bunch of models I’m running locally on this machine — it’s a MacBook Air M2, right? So it’s not super-duper powerful, but it can run them on this machine. It’s a case of seconds-per-token rather than tokens-per-second, but it still runs on this machine, and I’m blown away by that.

Because in 2018, I went to a conference — we’ll get back to the question in a minute, but I’m just having this memory of 2018 — I was at a Microsoft conference, and there was — not Scott Hunter, not Scott Hanselman — Scott Guthrie on stage showing an app that he’d built that recognises whether something has a cat in it or not. That was frontier in 2018, and now you can do that on your phone, right? That’s crazy.

Codrina : Yeah, and it’s also mind-blowing. I have been presenting this specific use case at conferences, and I have more coming up this year. I personally use an old Android, so you don’t need to have the latest phone with all the top memory if your model is optimised. These services, as you said, help you also optimise these models, and I have some consideration in the book on the format — the bit level — and some things that might help you understand exactly which model to choose, also out of the community models you can find on this platform.

It’s really interesting to see that it works, and you don’t need a lot of time to just implement that. That’s why I also love .NET MAUI, because if it works on an old Android, for sure it works on the latest iPhone, or some other phones, and it helps us to connect with our user regardless of their device. So this is how I put these two things together, and just looking at the power of AI in different examples.

Offline AI is also known as edge AI, and there are a lot of examples — also from Microsoft. I think they’re also expanding to Android now on the edge AI part, so hopefully we’ll have more models to just use AI on our devices soon.

Jamie : Sounds great. So, where — other than the book, and I appreciate I’m asking you for your sources — but other than the book, if somebody’s listening to this and going, “I’m gonna buy the book, but until that arrives, how do I go about getting started with all of this stuff?”, right? Because you’ve mentioned a whole bunch of things that we haven’t had a chance to talk about — edge AI, you’ve talked about Hugging Face, we both talked about Hugging Face. We talked about — without saying it — quantisation of models, right? All that stuff.

So let’s pretend I’ve been working in enterprise, and I haven’t had the chance to do anything with AI yet, for a whole bunch of reasons, and now we’re six years in, and I’m going, “oh no, now I need to learn about AI.” How in the heck do I get started?

Codrina : I personally started from Azure. Working in a Microsoft environment, everything was on Azure. I was seeing, “okay, they already have the Cognitive Services,” so these services have been around for some time, and you can detect things. The great thing about Azure — and now Microsoft Foundry, the newly renamed platform — is that you have a playground for everything. So I would suggest starting in there, to start looking at all the models that are in Azure.

I think I lost count — we’d need to check as we are speaking how many models are there — to try to play with them in the playground. From a developer point of view, lots of them are just APIs and endpoints to call, like any other thing you have on Azure. So we might be biased by Microsoft, but I really love all the playgrounds. You can see the token consumption already — this coin that is used for AI — and it already gives you an idea of how this thing works. So yeah, Azure can be a really nice starting point, or the new Microsoft Foundry platform that’s built on Azure.

Jamie : Okay, awesome. I guess as we start to wrap up — I think I’ve said that phrase about four times now, I’m sure Matt the editor will email me afterwards and say, “yeah, you said it 12 times, Jamie” — but as we start to wrap up, please remind folks of the name of the book that you’ve got coming out, and roughly where they can get it from. I know I’ve got a couple of links that I’ll put in the show notes, so folks, if you’re interested in the book, click there in the show notes.

Then, maybe people are listening to this going, “Codrina’s awesome, I want to be able to keep up with what she’s doing in the social media space — maybe LinkedIn or whatever — so that I can find out more things as Codrina shares her expertise.” What’s the best way for folks to do all of those things?

Codrina : Yeah, so the book is called “AI-Enabled Apps with .NET MAUI,” published by Apress, so you can find that on their website, on different platforms like Amazon and so on. The book has the GitHub repository connected, so you’ll also get the nice part for us.

I think the fastest way to get in contact with me is on LinkedIn — so just look for Codrina Merigo. You can also find me on Twitter, Bluesky, Instagram, but usually on LinkedIn I have all the up-to-date information, like the next conferences I’m speaking on, and all the other things that I’m working on and updating. So I would say go on LinkedIn, and then don’t be shy — write me a message. If you have any issue, if you need any help — I’m also a mentor — so don’t be shy, as I said.

You can just contact me, and we can see everything you have — maybe your ideas, your issues, and everything that’s connected to .NET MAUI, AI applications, and also Microsoft Foundry, which is another topic that I’m speaking on at conferences lately.

Jamie : Amazing. I have to say, Codrina, this has been a fantastic conversation. There’s a whole bunch of stuff that I’ve learned about in this one. I mean, I say that in every episode, but — hey listeners — part of the reason I produce this show is so that I can learn more, right? Not just to show off that amazing people out there are doing amazing things, but so that I can learn more as part of my continual professional development.

Codrina, I have learned so much about local models, and attaching them to .NET MAUI apps. We talked about privacy concerns, we’ve talked about accessibility concerns, and a whole bunch of stuff, and we’ve only really scratched the surface. But what I would like to say is: thank you ever so much for spending some time with us today to talk about all of this, and I wish you the best of luck with your book.

Codrina : Thank you so much, Jamie, and thank you for having me. It was a really great conversation. I know we went off-track, but I’m really happy to discover these things, and to share with people all over the world.

Jamie : Thank you so much.

Wrapping Up

Thank you for listening to this episode of The Modern .NET Show with me, Jamie Taylor. I’d like to thank this episode’s guest for graciously sharing their time, expertise, and knowledge.

Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at the podcast's website, and there will be a link directly to them in your podcatcher.

And don’t forget to spread the word, leave a rating or review on your podcatcher of choice—head over to dotnetcore.show/review for ways to do that—reach out via our contact page, or join our discord server at dotnetcore.show/discord—all of which are linked in the show notes.

But above all, I hope you have a fantastic rest of your day, and I hope that I’ll see you again, next time for more .NET goodness.

I will see you again real soon. See you later folks.

Follow the show

You can find the show on any of these places