The Modern .NET Show

S07E10 - Unlocking the Power of AI: Jim Bennett on Pieces for Developers

Sponsors

Support for this episode of The Modern .NET Show comes from the following sponsors. Please take a moment to learn more about their products and services:

Please also see the full sponsor message(s) in the episode transcription for more details of their products and services, and offers exclusive to listeners of The Modern .NET Show.

Thank you to the sponsors for supporting the show.

Embedded Player

S07E10 - Unlocking the Power of AI: Jim Bennett on Pieces for Developers
The Modern .NET Show

S07E10 - Unlocking the Power of AI: Jim Bennett on Pieces for Developers

Supporting The Show

If this episode was interesting or useful to you, please consider supporting the show with one of the above options.

Episode Summary

Jim Bennett returns to the podcast after a six-year hiatus. Jim delves into his extensive history with developer relations, primarily through his involvement with Xamarin, a tool that significantly impacted his career. Jim fondly recalls the community’s camaraderie formed around Xamarin, detailing how it propelled his transition into developer relations, leading him to speak at events, authoring a book, and eventually working for Microsoft as a Xamarin advocate. And Jim emphasizes the importance of community in tech development; how developers don’t merely purchase tools—they seek solutions that address their problems. He discusses the ways in which sharing knowledge and experiences fosters a vibrant developer community, facilitating connections and learning opportunities.

Shifting focus, discuss Jim discusses his current work with Pieces, an innovative tool designed to enhance productivity through long-term memory and context retention. He explains Pieces as a solution to the overwhelming amount of information developers must handle in their daily workflows by minimizing context switching between tasks and tools. The software captures developer activities across various platforms—including communication tools and IDEs—and accumulates context to assist in making coding more efficient.

Jim elaborates on how Pieces integrates with different development environments and supports various large language models (LLMs) for customizability. This feature enables users to switch seamlessly between models based on their specific context and needs. The discussion emphasizes how this functionality equips developers to find and utilize relevant information without compromising their workflow, optimizing the coding process. Jim contrasts this with traditional methods of using tools like GitHub Copilot, which hinder productivity by requiring users to navigate away from their primary tasks.

The conversation also embraces the significance of security and privacy in using AI within development environments. Jim outlines the importance of on-device processing, especially in corporate settings with strict AI governance policies. He stresses how Pieces prioritizes privacy by ensuring that sensitive information remains local while still providing the capabilities of generative AI.

In closing, Jim encourages listeners to explore Pieces and engage with the community through Discord and social media channels, reminding them of upcoming appearances at key tech conferences, including talks demonstrating the powerful integration of Pieces into various development workflows. The discussion wraps up with an acknowledgment of the vital role that open communication and transparency play in fostering trust within the tech community.

Episode Transcription

So we’re dealing with code bases that are getting bigger and bigger every day. You know, those million line code bases, two million line code bases is not unusual. We are being pushed to do more. I remember when I was working at Microsoft a couple of years ago, Satya Nadella, CEO, his favourite phrase was, "you have to do more with less." But yeah, so Satya was big on this idea of do more with less. And this has kind of resonated across the industry as a whole.

- Jim Bennett

Welcome friends to The Modern .NET Show; the premier .NET podcast, focusing entirely on the knowledge, tools, and frameworks that all .NET developers should have in their toolbox. We are the go-to podcast for .NET developers worldwide, and I am your host: Jamie “GaProgMan” Taylor.

In this episode, Jim Bennett returns to the show after a six-year absence. In his previous appearance, Jim introduced us to Xamarin, but in the time since that appearance, Xamarin has been sunsetted. So it felt natural to start our conversation about the wonderful outpouring of support over on X with the hashtag #XamarinGaveMe.

The main topic of conversation is about Generative AI, Large Language Models, and how the new startup, Pieces, can help developers to keep the context of what they are working on at the front of their mind at all times; both when they are online and off.

If I said to you, "which character from the Pixar movie Up are you?" The answer is going to be, “Doug,” we’re all Doug. We’re all like, “squirrel!"

So if we have to go from our IDE to a browser to ask a question, we’re reading email on the way. We are looking at chat tools on the way. Oh, we’re in a browser. "Oh, I’ve got a notification on Blue Sky. I’m going to have a look at that." We are context switching. We are distracted. We are drinking coffee. We are losing our productivity.

- Jim Bennett

This episode follows on, very nicely, from our conversation with Amit Bahree back in season six episode 23. In that episode, Amit helped to lay the ground work for understanding how Generative AI works. If you find yourself not quite understanding some of the LLM-specific points in this episode, please consider giving Amit’s episode a look—there will be a link in your player.

Anyway, without further ado, let’s sit back, open up a terminal, type in dotnet new podcast and we’ll dive into the core of Modern .NET.

Jamie : [0:00] So, Jim, welcome back to the show. It has been six years and about 15 attempts to get this going today because my software keeps learning us down. So I apologize for that.

Jim : [0:10] All good. All, good. I really hope you managed to record some of the previous conversation. There were some gold moments there. But yes, for all you folks listening, this has been a mission. We’re like 40 minutes into the recording and technology has been fighting with us every step of the way.

Jamie : [0:26] Absolutely

Jim : [0:27] But It’s great to be back on the show. I know it’s been you know six years since I was last on. It’s great to be back on the show think I mentioned to you earlier as well the fact that you’re still going after six years, and I certainly wasn’t the first guest, is a testament as to how great this show is.

Jamie : [0:40] Thank you very much. Thank you very much. It’s all down to the guests. It’s what I keep saying to people, is it’s not me. I just sit here and ask questions, and learn loads. It’s y’all the guests that make this show what it is. So you know we’re we’re recording this November 27th, 2024, we’re right around the corner from Thanksgiving in the US, we’re both Brits. But I’m thankful to you, and I’m thankful to all the guests for making the show what it is.

Jim : [1:05] Yeah. So I’m thankful for you for kind of giving me this opportunity to come here and, well we’ve been laughing for the past 40 minutes as we fight with technology and it’s just it’s so much fun. Really appreciate you. Thank you.

Jamie : [1:15] No worries, I appreciate you.

Jamie : [1:18] So, yeah. You were last on the show talking about Xamarin, episode 20 back in the day. And we were doing a little bit of a talk in the previous version of this recording. We were talking a little bit about Xamarin and the wonderful #XamarinGaveMe when Xamarin was sunsetted earlier this year, now that everything is .NET MAUI was hoping you could maybe let the folks know a little bit more about your history with dev, and into dev rel and stuff, via Xamarin if that’s okay.

Jim : [1:50] Yeah, yeah. Of course mean so that #XamarinGaveMe that was the only time I think Twitter’s ever put–sorry has ever put a tear in my eye. It was a load of Xamarin community folks just reminiscing about what we’ve all achieved thanks to to Xamarin. And really Xamarin gave me a career in developer relations. The very first meetup I ever spoke at was the .NET Xamarin meetup in Birmingham in the UK, run by Dave Evans. And it’s because of Xamarin.

Jim : [2:22] I’d actually, I’ve been working for many, many years doing development, a lot of front-end development, working in banks, sick of the banking culture, wanted to take some time out and convert from working from a taking organization to a giving organization. So I focused a lot on community and I wanted a technology to learn, to contribute to community to. And Xamarin was that technology. It was just, I could take my C-sharp skills. I’ve been a C-sharp dev for years. I could build mobile apps for iOS and Android. I was a big Mac user and it was just phenomenal. So just be able to be a part of that community was brilliant.

Jim : [2:54] I think we all came together over a love of what Xamarin could have achieved and a shared experience of just the bugs along the way. You know, it was a startup, it was trying to do great things; had a few issues, and we all bonded together over this, “oh my god! This Xamarin thing is terrible,” and i think the reason we got so passionate about it because we love the technology. It’s kind of this love and hate being two sides of the same coin type idea.

Jim : [3:19] But yeah Xamarin really got me to where I am now. I started off with meetup groups conferences. I wrote a book on it. I started working for Microsoft as a Xamarin advocate. And as I was saying to you earlier, my office is, I don’t want to say a shrine to Xamarin, because that might make me sound a bit weirder than I actually am. I mean, it’s a shrine to Star Wars Lego, I don’t deny that. But I have Xamarin memorabilia in my office that just reminds me of those days. I’ve got the cover of the book I wrote on my wall. I have a Xamagon. I was in, Microsoft flew me to Vietnam to speak at a Xamarin Experts Day and they had these big Xamagons, which are like these hexagons with the Xamarin X logo on them. I have one of those up on my wall. I have all the Xamarin monkeys.

Jim : [4:02] So the folks who invented Xamarin were obsessed with monkeys. You know they seem to have named all their companies of corruptions of monkey names. I mean Xamarin comes from the Tamarin monkey. They start off with Ximian from the Simian monkey. And so I’ve got a whole of Xamarin monkeys on my shelf. I’ve got my Xamarin University Xamagon on my shelf. I’ve got my Xamarin MVP mug up on my shelf. So just having that memorabilia around just keeps reminding me constantly of the community that I was a part of that got me where I am, and just grounds me in the fact that everything we do in the community, we are doing to support those people who are trying to build with technology. And it’s like, “how can I help you with this amazing technology? "

Jim : [4:45] So it’s, yeah, seeing that outpouring of love was very, very moving. Seeing the names of folks I hadn’t spoken to for years just appearing saying , “yeah. Xamarin gave me this.” And a lot of it was, “Xamarin gave me friends,” “Xamarin gave me community.” It was so much about the people not, “oh. Xamarin allowed me to buy a lamborghini.” You know it was all about the people the connections and just the love for the technology.

Jamie : [5:07] Yeah. I think that’s always the most important thing, right. I know you work in developer relations, but it is the most important thing right. To have that community, that space where people who are interested in a technology can get together and go, “oh, did you know you can do this? Holy cow, I didn’t know you can do this too. Oh my goodness. " And sharing those ideas, right? Because we don’t do business with businesses. We do business and help people with other people, right? We collaborate with people. We work with people. We sell to people. We buy from people. So having that community is super important.

Jim : [5:44] Yeah. And the thing is, people don’t buy something because you sell it to them. They buy something because it’s helping to solve a problem, especially developers. We are a very fickle bunch.

Jim : [5:54] You cannot sell to a developer. Traditional marketing doesn’t work with developers. Instead, you need to show us the benefit of something, geek out with us about it, and we’ll get excited and do it. I was literally yesterday working on a blog post about the top five prompts that I use in Pieces, the app that the company I work for at the moment. And it’s all about, “hey, I’ve discovered this cool thing that I can do with it. And I want to share it with you.” I’m not trying to sell you that, “oh, if you buy this tool, you can do X.” It’s like, “check out this really cool thing that I discovered. This is so cool. I use this every day. I want to share it with you. " I think that’s really what helps these communities grow is just everyone’s passion about the cool thing.

Jim : [6:35] And we love geeking out with each other. It’s what brings us together. You put three developers in a room and they’ll all start bitching about JavaScript. It’s just, you know, I mean, literally we were doing this earlier, we were saying we should write the podcasting tool in C Sharp.

Jim : [6:49] But we have these shared experiences and we have this excitement about technology. And so if we can bring people together to get excited about things, that’s how we grow communities, that’s how we grow technical communities, that’s how essentially these companies like Xamarin, Pieces, who I work for now, and all these other companies can sell product, is by getting you together and getting you excited about what the tool can achieve for you. And the Xamarin community did that so so well

Jamie : [7:16] 100%. 100%.

Jamie : [7:18] I have to say I came to Xamarin super late I’d heard about it when it was a thing but I’d only started using it on a real project just after the the Microsoft acquisition; because a friend of mine who was on the team I was joining was saying, “wait. I’ve literally just paid for Xamarin university, and now I don’t need to,” and all this kind of stuff. And I’m like, “oh well. Hey, look, this is how it works right. We can’t we can’t change what it is it, is what it is.” So got to use Xamarin for free and she had to pay for it

Jim : [7:55] Yeah there was a lot of frustration, a lot of people applying for refunds because they paid like thousand dollars a year because I did that. I paid for the Xamarin Professional version, and then yeah, it got bought by Microsoft and, you get it all for free now.

Jamie : [8:10] Yeah, yeah. And that just means that everybody wins right. Everybody wins. It’s like with when they release new updates to, you know we’ve just had .NET 9 being released. So if folks want to use that on an app, hey you get all of the free stuff, all of those innovations. With all the different things to do with .NET 9, ASP .NET Core 9, Blazer 9 I guess, EF Core 9, C# whichever version we’re on now. You get all of that for free right and that is really cool.

Jim : [8:39] Oh yeah. Yes. Saying about really cool, I’ve been very much behind on the on the .NET Aspire train. You know i’d heard the name but i didn’t really understand what it was. And i was at Microsoft Ignite last week in Chicago, and was lucky enough to spend 20 minutes geeking out with Jeff Fritz. So Jeff Fritz, if you haven’t heard of him, he’s a dotnet guy does a whole lot of live streams. He runs his… got this live coders group, absolutely fantastic chap. And he was geeking out with me on .NET Aspire, just demoing what it can do. And the whole thing just clicked. It’s like, “wow. Having this kind of orchestration layer for all the features of my app, in one place, in code. No yaml files, it’s all C# code can write unit tests for it,” was just phenomenal. And just to kind of see this stuff, and as you say it’s free: you just install a NuGet package and it’s free and away you go.

Jamie : [9:30] Yep. Or, indeed, if you don’t want to use it, you don’t install the NuGet package. That’s the other thing right.

Jim : [9:35] Yes. Yeah you’ve got that flexibility over what you have to do. It’s a very unopinionated toolset, .NET. It’s, “hey this is our references for how we think you should do it but you do.” Very unopinionated.

Jamie : [9:49] Yeah. I feel like this is the worst segue ever but speaking of unopinionated let’s talk about Pieces, and LLMs and SLMs, and running things locally, and things like that. That was the world’s worst segue, I have to say.

Jim : [10:06] It’s not really, because Pieces is a very unopinionated tool. It’s like a lot of good developer tools. It’s there to help you and assist you, but not make you do it. It’s all about how it’s a tool that you choose to use in the way that you choose to use it. So, you know, I don’t know that that was a terrible segue.

Jamie : [10:26] I was very kind of you to say.

Jim : [10:28] So Pieces, we describe it as long-term memory for your developer work stream. So the idea is we all know what it’s like as a developer. We have way, way too much to deal with. So we’re dealing with code bases that are getting bigger and bigger every day. You know, those million line code bases, two million line code bases is not unusual.

Jim : [10:47] We are being pushed to do more. I remember when I was working at Microsoft a couple of years ago, Satya Nadella, CEO, his favourite phrase was, “you have to do more with less.” But yeah, so Satya was big on this idea of do more with less. And this has kind of resonated across the industry as a whole. There’s a lot of expectation on us to keep delivering new features. And the problem is with that, there is so much more communication that’s happening. There’s so much more information that we need to deal with.

Jim : [11:14] So it’s not just, “I’ve got a code base, I have to write some code.” I have tickets where I’m discussing the code that I’m writing, and obviously thinking about it in stand-up meetings, I have constant conversations. And they have all these new frameworks. It’s JavaScript, it’s a Monday, there must be a new framework, that kind of thing. There’s all these new tools, like I was saying just now about .NET Aspire. All these new things are happening. And so we’re constantly researching, we’re collaborating with people, and we’re writing code. And the problem is getting all this information in one place becomes hard, because when you try to access this information, you have to context switch. You come in the morning, fire up Visual Studio Code, about to write some code. What am I actually doing? So I jumped to my chat tool.

Jim : [11:59] And in my chat tool, I have the conversation with the PM to say, this is the ticket we need to work on. And then I have 10 other conversations with people, 20 other group chats. And so I start reading those because there’s that kind of expectation. I mean, in the chat tool, I should be reading these. Before you know it, it’s 10 AM. I’m two cups of coffee down. I haven’t written a single line of code. I still don’t know what I’m supposed to be doing today, but I’ve had a lot of conversations in chat tools.

Jim : [12:25] And so having this constant context switching really can reduce your productivity. And so kind of the goal with Pieces is, “how can we keep you where you are most productive, allow you to gather all the information that you need in one place, and then converse with that information through the power of the LLM of your choice?”

Jim : [12:45] So the kind of components we have is the ability to add files and folders of code, any file, any folder of code into your, your discussion. You have this discussion anywhere where you are by having IDE extensions for all the IDEs, browser extensions, a desktop app, a CLI. So wherever you are, you kind of have these conversations. And then we have this ability to extract context from everything that you’re doing across your machine. So we look at all the active windows. We pull information from there. We process it locally, privately, securely, encrypts on device. And then when you interact with the LLM, you can ask questions across all of this range of information.

Jim : [13:22] Sounds like kind of a lot. So I think the best way to describe it is kind of my favourite example is, yeah, I can be chatting with my PM on a Friday night. PM says, “can you look at this ticket? " I go and read the ticket, reading through it. Go home for the weekend. Don’t think about work. Build some Star Wars Lego. Come in on Monday morning. Fire up Visual Studio Code. From there, I can launch Pieces, the copilot inside of Visual Studio Code, and literally say, “what is the ticket I need to look at?” And Pieces can then answer that question because it’s got the context of everything that’s happening across my machine. It can say, “oh, you need to look at this particular ticket.” “Okay, can you give me more information on the ticket? " And then it could maybe summarize everything that I read in the ticket, you know? And that’s great because sometimes in tickets, you have 20, 30 people potentially discussing it, so many different comments, and then all that’s really relevant is the last few comments. So I don’t have to wade through that. Pieces can kind pick up this summary from everything that’s read.

Jim : [14:19] And then I can say, you know what, I’ve got this project open, but I did something similar in another project over here. Let me add that other project into my Pieces discussion and say, “hey, I did something similar in this project, how’d I do it?” And then have Pieces find the code for me.

Jim : [14:34] Or I could say, “you know what, this is something new, let me add my current project, how do I do this? " And then Pieces can take all that context from the discussion, the issue, and then across the entire code base, and come back and go, “yeah, this is what you need to do. " And it can do it with absolutely massive code bases. It can do it with a huge amount of context. And it can do it using, as I said, the LLM of your choice. So if you’re the kind of person who finds that GPT-4 works better for you, you can use that. If you’re the kind of person who finds that Claude works better for you, you can use that. If you’ve got an enormous project and you really worry about context, you can use Gemini.

Jim : [15:09] Or if you’re in an environment where you have to think about security and privacy, or you’re on a plane, you’re in the middle of nowhere, you’ve got no Wi-Fi, you can use an on-device model as well. So I can literally unplug my computer from the Wi-Fi, i can say use Llama, or Microsoft Phi, or Mistral; so many different on device models. And as long as I’ve got the model downloaded I can just say to Pieces, “hey how do I implement this issue?” And I get that answer where I am in my IDE.

Jamie : [15:36] Right, right. That last part of being about being completely disconnected is absolutely amazing. Because I’ve been using Ollama recently to run a whole bunch of models locally and just experimenting with them. “Hey what can I do with this? How can I do that ? How do I communicate with it and build a whole bunch of apps that have this abstraction layer of…” because I’ve intentionally put the abstraction layer in, for similar reasons right. If I just want to do local stuff, go through the abstraction layer, maybe an interface, talk to Ollama. And then I can swap it out for maybe GPT, maybe Claude , maybe whatever right. Through this other thing supply an API key, and things like that.

Jamie : [16:19] So that being able to do it on device, for me, is brilliant because, whilst I don’t have the world’s most powerful supercomputer–you know we’re recording this only a few weeks after the M4 Mini has been released, which i think power for performance is probably the world’s most powerful computer you can buy for the cost because it is tiny four watts of just idle power and look at what it can do. Anyway, I’ve gone way over top, way past the topic there.

Jamie : [16:49] What’s great about my Mac M2 Air is I can do Ollama locally. It’s not the fastest at doing it, but I can still do it. Which means that, like you said, if I’m disconnected, if I’m on a train, if I’m on a plane if I’m just sitting at home and just doing something that i don’t want to use all my tokens for one of the AI services, I can just do it locally. So being able to do stuff locally super interesting to me.

Jim : [17:13] Especially when we think about AI governance. So AI governance is a massive thing that CTOs, CIOs, CSO’s, CISOs, all those, all those kind of folks with C’s in their name, they are thinking about AI governance and they’re thinking around intellectual property. “What are my folks allowed to do? What are they not allowed to do?” And a lot of that comes down to kind of the, the AI choice.

Jim : [17:36] And I was actually chatting with a couple of large organizations at a conference the other day. So one was a massive financial services company, based in the US, another is a global, technology company. And they’re saying that their AI governance rules allow them to use on-device models only. They’re not allowed to use Chat GPT. They’re not allowed to use Claude. They can only use on-device models. And this is literally mandated at a company level. So how can you use a lot of these really flashy IDEs with AI capabilities where they’re using a model somewhere in the cloud that your AI governance does not give you permission to use?

Jim : [18:16] Well easy. You just use Pieces, and then you choose a local model, and you’re compliant with those rules. So AI govern is something we’re thinking about a lot in everything we. Do.

Jamie : [18:27] Yeah, yeah. And that makes sense right, because if you’re sending intellectual property up to the cloud–to one of the clouds or any of the cloud services–unless you’ve really read through all of that documentation, and read through it again just before you sent it, because who knows whether they’re going to change the license or not they could change it on a dime, you don’t know what they’re going to do with that information if they’re going to do anything with it. I know that Apple has some deal at the moment with Open AI for their on-device Apple Intelligence stuff: if Apple Intelligence can’t figure it out, it will offer to send it up to Chat GPT. And they’re saying that they’ve got a bunch of security things in place, but I don’t know what it… I’m just going to assume that Apple are telling me the truth. I don’t know. And so, yeah, from a governance point of view, from an intellectual property point of view, sending it up to the cloud to be processed. Yes. Okay. It might be faster, but you don’t know what’s happening with that data.

Jim : [19:22] Yes. And obviously there’s some situations where it’s corporate IP. If that did get used to train a model: bad, not the end of the world. B ut then you have things like government data. You know if you’re actually working for a government or health data , suddenly that feels a bit more unpleasant nd so just having these rules around it really helps you to focus on the data that you’re dealing with, potential for client data. And so if you can have a tool that complies with the rules, this is absolutely fantastic for you because you get the benefit without having to basically risk your job.

Jim : [20:00] You know, it’s kind of got to the point where most developers are using ChatGPT at some point for the work that they’re doing. And if you have a group of developers, you know, I think we were talking earlier about if developers love to geek out on things, so you’re hanging out with your developer mates and they’re going, “oh, ChatGPT does this. ChatGPT does this. Claude helped me do this. " And you’re going, “I’m not allowed to use that. " You’re going to feel left out. And so with the tool like Pieces, you can say, “yeah, of course. Hey, CTO, this is purely on device. Look, we can even put a firewall in to make sure that this app is not actually talking back to Pieces, apart from the initial login account management type thing. " Bang, I want to use this all on device. And hey, presto, you’re compliant with the rules. Yeah.

Jamie : [20:46] Yeah. And like I said, that’s the big thing for me because, uh, you know, I, I, I do a lot of, um, short lived projects, um, and jump between projects a lot and jump between different client and customers a lot. And so making sure that that information doesn’t leave the computer that I’m doing it on super important. Like it, it may not even enter into a lot of people’s minds, just how important that really is. And so i guess that’s one of the things, I guess, that separates Pieces from say GitHub Copilot, right . Because i don’t know what’s happening with Copilot. I open some code, Copilot sits…GitHub Copilot sits there and says, “cool can answer your questions about this code base it’s definitely going somewhere because it’s not running on my machine. So where’s it going?

Jamie : [21:35] So like the idea of being able to do that locally, because that’s super important.

Jamie : [21:39] Yeah. Obviously, you have to have a reasonable machine to do this. If you’ve got an older PC without any form of acceleration, then you’re not going to have the best experience. All the Apple Silicon devices works beautifully. And obviously, if you’ve got a device that’s got a good GPU there, then yes, you will get that benefit. But the thing is this is what’s starting to happen. We’re starting to get this trend where most machines these days are thinking about AI. If you buy a brand new machine, unless it’s the really cheap ones that they buy for schools, most of these machines nowadays have got some form of AI acceleration built in. They understand that’s the need. With Microsoft pushing like Copilot Plus, Intel with their Core NPU devices, we think about the rise of Qualcomm Snapdragon ARM devices with NPUs, and obviously all the Apple Silicon with the NPUs built in. It’s the future. Every single machine you buy going forward is going to have AI acceleration.

Jim : [22:41] Where we went from a chip that was just your CPU and then a separate graphics card to on-device graphics, to now you’ve got powerful GPUs on silicon with the processor. We’re going to get powerful NPUs on processor as well. And then these abstraction layers like Ollama and all these other different abstraction layers, they’re going to slowly build out all these capabilities. There’s going to be…it’s just going to work. You know, whatever device you buy with whatever GPU, NPU it’s just going to work without acceleration because that’s the place we’re getting to. You know, whether it’s going through OpenVino on Intel, whether it’s going through Cuda on Nvidia, whether it’s going through metal on Apple Silicon, it’s just going to work. And everything’s going to be faster, and faster, and faster. And so you know, yes, right now we have customers saying “yeah, this on-device models are too slow because my hardware is cheap,” but we’re kind of that start of the curve thing. You know, in three, four, five years time, it’s going to be much less of an issue because the number of people who have a machine that can’t support on-device models is going to be drastically lower.

Jamie : [23:48] Yeah absolutely. And I would always assume that, devs tend to want the super powerful machine anyway, So I would assume that a lot of devs will be running machines that are way powerful and able to do local large language models or small language models stuff. So would assume that a lot of devs are able to… I would make the…again I’m saying assume too much right, and we know what happens when you make an assumption. But what I’m assuming is most of the people in a business will likely have those cheaper machines, so then the devs and the designers and the people who need the virtual horsepower will probably have machines already that can do this right. Like I said I’ve got a Mac2 Air and I could do all this.

Jim : [24:39] Yeah, yeah. And I think obviously with Windows desktop machines you wonder if there’ll be less of that, because if you’re a dev and you’re saying, “I need this really fancy GPU on my machine,” there’s going to be, “what’s the business case for it? You know are you trying to play games on it?” But with a lot of people working hybrid it’s all about laptops, and a lot of laptops just have this built in . It feels like if you’re building a desktop and you ask for like this you know an Nvidia GPU you’ll get pushback, but if you buy a developer laptop because it’s powerful it’s already got the GPU on it; so your machine is likely to have that capability already.

Jim : [25:16] And then as well if it comes from the AI governance perspective, you know, “comes down from up top you have to use local models.” “okay cool. This is AI governance. This device here conforms to what my AI governance recommendations. So I want that.” And so that’s going to trickle down. But, “yes, to enforce this we have to give you this hardware.” And honestly really powerful machines are relatively cheap compared to the cost of a developer. “We’re going to spend five thousand dollars on a machine for a two hundred thousand dollar a year developer.” Nah. You know it’s a percentage point really, when you think of it like that.

Jamie : [25:50] Yeah absolutely absolutely.

Jamie : [25:51] And so I guess, we’ve been talking a little bit about how the infrastructure around it works. So we talked a little bit about Pieces itself, and how it will help me to keep this context, and keep the idea of how I’m doing work so I’m not constantly context switching. So how does that, without giving away the secret sauce I guess, how does that all work? We’ve talked a lot about local language models recently just those two just now and obviously we’ve both said, “Hey, you can use Claude, you can use GPT, you can use whatever.” So how does all of that sort of tie together? If I’m coming to Pieces brand new, what’s my experience and what am I going to be doing to set it all up?

Jim : [26:31] So once you turn it on, it just sits there and works. So Pieces has got kind of two components. One component we call Pieces OS. It’s not really an operating system, but it’s kind of an OS level tool. It just sits there in the background. And then we have the front ends that you use to interact with that, whether that’s our desktop app, whether it’s an extension inside Visual Studio Code, Visual Studio, your JetBrains IDE of choice, Sublime, NeoVim, Obsidian, your browser. You know, wherever you are, we’ve got a kind of extension that talks to Pieces OS. And what Pieces OS is doing is it’s running OCR across the active windows on your machine.

Jim : [27:09] So we’re trying to think what is relevant to the work that you’re doing? It’s designed to run on your developer machine. This is not something I’ll be putting on my 11-year-old’s gaming laptop. This is a developer tool. We capture developer-level context. We filter out API keys, personal developer information, things like that. And then we process this fully locally. None of the stuff we capture leaves your machine. And then we do our magic secret sauce over it. And then we store it in a vector database, essentially, just to kind of overly simplify it. We store all in a vector database.

Jim : [27:43] And then when you’re asking a question of Pieces, if you add other contexts, like a file of code or a folder of code, same idea. We store it in essentially a vector database. And then when you ask the questions, we run relevancy across every information that’s there. So if you say you want to use the long-term memory, which is the captured context, you turn that on. If you say files, folders, you turn that on. And then we will then take the question, use our proprietary on-device models to work out what is relevant to that question. Then all that gets packaged up and then sent to the LLM of your choice.

Jim : [28:16] So anything we capture on device will only ever leave your machine if you ask a question of a cloud-based LLM and in that case the only thing we send is what we think is relevant to that conversation. So if I ask a question, you know, “I was chatting with Tsavo, our CEO. about an issue the other day. Can you describe the issue?” What it’s going to do is going to it’s going to just take the conversation I had with Tsavo, the details from there. If that’s got a URL in it, I’ll take that URL, use that to look up what it read from that URL, and then just pull that information. And that’s all that’s going to be sent to the LLM along with the question I’m asking. It’s not going to send conversations that I’m having with other people. It’s not going to send unrelated issues. It’s not going to send code. It’s not going to send documentation, nothing like that. It just sends what is relevant up to the LLM of choice.

Jim : [29:14] And obviously, you know, this is capturing what’s on your machine. And there are times when you’re doing things on your machine that you wouldn’t necessarily want captured. So if you’re like an engineering manager, if you’re writing up performance reviews and you might want a certain amount of privacy. So for that, just turn it off. You know, or I’ve got an hour meeting, could be capturing notes during this meeting, I don’t want those captured, turn it off for an hour and then it kind of kicks back in. So there’s a lot of magic there. We’ve got some super smart AI and machine learning folks, you know, a lot of folks have literally could have been studying this very recently, so they’re up to date with some of the latest technologies and they’re kind of throwing their very academic expertise into building these out. So there’s a lot of really cool stuff we’re doing in terms of the relevancy to pick the right information for you. So a lot of magic there, but the, the oversimplified version is you think about a vector database having kind of semantic search, we kind of pile it in there, do a semantic search to pull out what we think is relevant, send that in the system prompt to the LLM.

Jamie : [30:13] Right. Right. And that totally makes sense. And it feels like it would be… how do I put it? I can’t think of the best way to put it without trying to sell Pieces to the people who are listening because obviously it’s not my not my job to do it.

Jamie : [30:29] But like being able to pull that context out, like I’ve been in meetings where–I literally take handwritten notes right for the people who are listening in I’ve got one of those Remarkable Paper Pros and I’ve been scribbling down notes as Jim and I have been talking. And that’s so that I can remind myself, “hey, tell Mark the editor to do this bit,” or, “this bit is really important, " or you know, “this is really cool. I want to check this bit out.” Like you know you said there was a Neovim extension; I’m like, “cool that’s on my list of things to learn next year,” is Neovim; just because I want to figure out: can I do a IDE-less IDE? Like just a bare bones text editor know that it is way more than that and also way less than that at the same time, but I know nothing about Neovim. So I’ve probably just upset loads of people. But, like, I’ve written that down because that’s really cool. But I guess if Pieces was running, I could have said to Pieces, “hey what was that thing that Jim said about Neovim?” And it’d be like, “oh yeah. This, that, and the other,” maybe it’s not capturing the audio but…

Jim : [31:32] If you had captions, it would capture that. We are looking down the line around video and audio capture as well. That’s obviously a lot more work and a lot more questions around privacy there, because obviously for this conversation, of course capture the audio. But if we were having a lot of meetings, you wouldn’t necessarily want to capture that.

Jim : [31:55] There’s a few privacy questions there. It’s not a hard problem to solve. You think about things like Whisper that can do transcription straight away. So you just need to feed the system audio into Whisper and then that becomes raw text that just gets fed in. But as we think about things like privacy around this, we’ve been very privacy-focused from day one. That’s why we do all the long-term memory processing on device. It’s why we offer local models. And so when we approach things like video, we need to make sure we’re being very mindful of privacy with everything that we do.

Jim : [32:29] And yeah, there are just some people who like to have Star Wars on the background while they’re working. And we don’t want to be capturing that. So how do we make sure we have the kind of allowlists, denylists in place so that we’re not capturing that? I mean, I listen to Lo-Fi Girl. I don’t know if you’ve ever seen that, the YouTube channel. and you know some of the there’s a Lo-Fi Girl that’s just been released based off a video game that I like: Life Is Strange Double Exposure, and that’s got talking going on. And I don’t want to jump into Pieces and have it mentioned, “oh yeah. Something about one of the characters in that game because i guess mentioned the Lo-Fi Girl.” Yeah so there’s a lot of kind of considerations around how we do that in the most effective way.

Jim : [33:10] So this it’s coming yeah we’ve publicly announced on this on our roadmap that is something we are looking at. A great example would be you’re watching a YouTube video and, “I want to learn this technology, watching a YouTube video.” We’re already capturing any text that appears, so if someone puts a screenshot code up, we’re capturing that, but then how can you ask deeper questions of, “based off the video I was just watching, how can I implement that in this code base?” And then once we’ve got that transcription in place, all that will just fall out because we’ve got these capabilities already built into Pieces.

Jim : [33:43] And so it’s definitely something that we want to do. We’re just making sure that we build it out right. And we are a startup. We are a small number of people. So there’s a million things we can do. It’s prioritization for those.

Jamie : [33:58] That makes sense, right? we were talking earlier on about um if a single like a solo developer is working on something open source they tend to uh implement the whiz bang this is an awesome feature i want rather than thinking about usability and privacy and things like that whereas you guys it sounds like you’re going the other way like we want to implement this whiz bang feature but how do we get there safely and securely and making sure that people’s stuff is uh locked away or how do we not include these things like you were saying right you don’t want to have um lo-fi goal included in your transcript of the meeting you were having or of the work that you were doing it totally makes sense

Jim : [34:37] Yeah. Yeah mean we’ve… I think it’s been really, really powerful for us to have this real deep focus on privacy and security right from day one. So there is a similarity between some of the things we’re doing and what Microsoft Recall, for example, does; but ours is very much focused on the developer, ours is very much focused on that security and privacy from day one , and we have a large user base who are using this and they’re very happy about it. And they ask us questions about privacy, and we explain everything very transparently. Whereas when Microsoft Recall was released, er Windows Recall, there was a very strong pushback ; because in its early versions all the screenshots were available, you know, they put out, “here’s an insider’s build for you to try it.” It didn’t have that the right level of security baked in, and they’ve had to kind of rewind on that, and kind of go back to the drawing board and really rethink about how they’re going to promote this. I mean, it was interesting that at Ignite this year, Windows Recall wasn’t mentioned at all. Nobody talked about it.

Jim : [35:39] They talked about Copilots having long-term memory, which is a term that we’ve been using, and it’s nice to see Satya using it as well because it emphasizes how important long-term memory is to developers. Yeah, there was a lot of talk about how copilots need the right context to work, but they didn’t focus on Windows Recall, simply because of the all the bad press they got because they didn’t seem to think about security and privacy right from day one.

Jim : [36:04] So it’s great that we do this, and we are 100% transparent about everything we do. You know, if there’s any large organizations who want to use Pieces and they want to really dive into how all that stuff works like yeah we’ll stick our CEO on a call with you, and we will dig into it. And you will, yeah, we can show you everything. Because with something like this we can’t hide it. If we hide it, people get worried about it, they think it’s insecure. If we’re honest about everything we do there’s more trust there.

Jamie : [36:32] Yeah You go from “who are these?” to being the trustworthy the trusted/trustworthy supplier right. You can then say, “no trust this because they’ve been open and they’ve talked about x, y, and z.” That’s not to say that Microsoft weren’t open. I feel like perhaps they were a little bit too open by saying, “hey, this is the screenshots folder. This is where it lives.” Which was… I get why they did it, but obviously then I’m then worrying, “wait, if I have this on a machine and someone gets onto my machine. A ll right, they’ve got to get onto the machine. But if they get onto the machine, they can see that folder. Not great.” Or suboptimal, I will say, but that’s my own personal opinion. And I’m not saying either, neither of us work for Microsoft. We’re just two people having a chat about some stuff.

Jim : [37:21] Yeah. Oh yeah. And we do have the ability to save the screenshots as a debug flag, because obviously we use that internally for testing, but we don’t save screenshots. If you wanted to see what was being captured, then yes, we could turn it on for you. You could see the screenshots out of a research perspective, but we don’t save screenshots. It’s all done. We just capture it in memory, run OCR over it, and we use the OS level OCR as well, so that that way nothing really leaves the OS. It’s not like we can’t send it to us to process it. It’s all done on device. It’s all very, very secure and private. And yeah, we don’t want to have this lying around. The context we save into the vector database is encrypted.

Jim : [38:03] So, yes, you could probably hack us and get our encryption keys and decrypt it. But if you’re doing that level of stuff anyway, then there’s probably easier ways you can capture the information. It’s like the old joke, the old comment that it’s really easy to get someone’s password. Yes, you could try hacking them or you could literally go up to them and point a weapon at them and say, “give me your password. " Nothing is infallible. So you’ve got to make it difficult enough. And I think we have got to the point where if somebody can get this information, off your device and unencrypt it, you’ve got bigger problems than just having this information encrypted inside Pieces.

Jamie : [38:44] Absolutely absolutely and so i guess so is is Pieces we said earlier on you said you can open up your chat app say it’s teams it’s like whatever you’d be having a conversation we can pull that in two Pieces and that can be added to our uh context i think or a long-term memory uh might be the better better phrase um is it just developer apps or like is it just my id just my communications tools or like so i make lots of notes about stuff i do a lot of work in i’m not promoting them at all but it’s just my favourite tool i do a lot of work in obsidian can i pull that in too or is it just like quite literally just my id just my communications and

Jim : [39:28] Literally any text you have on screen. If you’re watching a YouTube video and they put code up on the video, that will be captured. So anything you have on screen, just in your active windows; because we want to make sure it’s relevant to what you’re doing. And so the active windows you’re in is what’s relevant. The information, if you are reading some documentation in your browser, we’re going to capture what you’re reading we’re not going to capture the entire page. Because if you haven’t read that it’s probably not relevant to what you’re doing. If you need to add the entire page as context you can you can say, “look here’s a web page I want to add.” But the idea is we’re capturing across everything you do what is relevant.

Jim : [40:03] Now your question about, “is this kind of developer focused?” It is very developer focused. All our system prompts, all of our relevancy is focused on developers. So if you’re another kind of knowledge worker and you ask questions about it, the response will be, “hey I’m here to help you as a developer. I can’t really understand that,” because we’ve really optimized all of our relevancy, all of our system prompts, and all of our LLM guidance to focus on developers. Now that’s not to say down the line Pieces… the official name is Pieces for Developers, that’s not to say that there won’t be Pieces for Designers, Pieces for Excel Users, Pieces for Marketers, you know whatever. down the line. But obviously that would be a very different product set, very different guidance, the LLMs, very different relevancy. At the moment, it’s all about focusing on the user as development.

Jim : [40:49] And you’re saying about Obsidian, we also have an Obsidian extension. So if you’re in Obsidian taking notes, you can jump into the Pieces co-pilot in Obsidian and have your conversations there.

Jim : [41:00] I do want to add as well, there’s one thing I love that’s really cool, is because we have this centralized Pieces OS that manages everything, your conversations go with you. So if you’re in VS Code and you’re coding away and you start a conversation. If you jump into Obsidian to check some notes you’ve got in Obsidian and you bring up the co-pilot, you can carry on that same conversation in Obsidian if you want to. Or you can start a new conversation in fresh context. But it means as I’m zipping between tools, I’m writing some code, I go in the browser, I ask the co-pilot questions in the browser, I add some code into it, carry on my questions, jump into VS Code, carry on that same conversation. Drag the code that’s generated into VS Code, and so on and so on and so on.

Jim : [41:38] So that’s really cool. And another really cool thing, slightly related. is the fact you can switch models mid-conversation. So I could be having a long deep conversation with Claude, find, “oh my internet’s gone down,” or “you know what. I’m about to board a plane.” I just change the model mid-conversation to say llama, and I can carry on that same conversation. Or if I’m finding that Claude has not given me good results, I can flip to GPT-4 and carry on that same conversation, or Gemini and carrying the same conversation. So having that ability to flip mid chat is so so powerful use it all the time if I’m moving around places. Yeah i do it, I’m devrel; fly to events. I get on the plane, the wi-fi can be spotty; so it’s working over this part of the country, I’m chatting away with Claude, the the wi-fi just craps out. No problem. Flip to Llama. Carry on my conversation. Very good feature.

Jamie : [42:34] Nice. Because that’s something that I feel, like, for a lot of folks who haven’t used in-app generative AI “speak to your your model, your whatever,” they may not have actually come across this yet I keep mentioning GitHub Copilot just because that’s one that i know folks will know, So like you’re in VS Code, you’re in Visual Studio, you’ve got GitHub Copilot running. You can type away, if you’re like you said, if your connection to the internet goes down, you can’t do GitHub Copilot conversations. Or let’s say you’re in Visual Studio Code and you’re doing some part of your app, and you’re doing something. Then you close that, and open up Visual Studio, that conversation may not come with you right. Because you’ve opened a new project, you’ve opened a new instance.

Jamie : [43:18] So I like that being able to bring the conversation with you as you move around, because a lot of us maybe don’t realize that we spend time jumping between apps. Like you say, we do a lot of context switching throughout our daily lives, right? A lot of us, a lot of devs like to think, “well, I’m paid to write code.” Well, actually, you’re paid to solve problems, not to write code. And sometimes the solving of the problem is jump on a call and talk to someone. Sometimes solving the problem is go read some documentation. Sometimes solving the problem is, “look, we need you to write this document or this email, just do it.” And sometimes, all right, most of the time, the solving the problem is sit in your IDE and write some code. So having that conversation move with you and being able to switch the models as you go is so, so good.

Jamie : [44:08] I obviously, I haven’t done the experimentation that obviously, you have with all of the different models. But like when I’ve been playing with the local models that I have on my computer, so on the computer I’m talking to you now I’ve got: Pih3, Mistral, Llama3.2, and a whole bunch of others. Like there’s, is it quop-quon-something like that [Qwon-Coder2.5]? So I’ve got a whole bunch of models that are supposed to be really good because they’ve been trained on code-only, rather than human language. Because a large language model is just a model of language, and code is language.

Jamie : [44:41] And so being able to switch between those based on where you are, or at a glance…like Llama3.2, from my experience, isn’t great at writing brilliant code but the other models, that are specifically trained on code, are way better. And so, like found that I can’t say to Llama3.2, when I’m practising my Japanese writing, I can’t say to it, “hey Llama3 help me with this. This is what I’ve written, this is what i want to say, What did i get wrong?” I’m not saying, “give me the right sentence.” I’m saying, “what did I get wrong?” And from my experience, my experimentation, Llama 3 kind of goes, “I don’t know dude. Good luck.”

Jamie : [45:16] Whereas, I could pull a different model that’s been trained on English and Japanese translation, and it can tell me how… I mean, we’re talking outside of work stuff, but the point is the same right. You being able to swap those models to get the expert knowledge. Like you would in an office environment right. You’re working with some people and you’re talking, “I’m talking to Jim, Jim knows a whole bunch of stuff about Pieces and a whole bunch of stuff about Xamarin. I’m going to go talk to Jim if I’m having trouble with Pieces or Xamarin. " But if I’m having trouble, we talked earlier a little bit about .NET Aspire and, I can come to you or I can maybe go and see if David Fowler is available, or someone, or like Maddy Montaquila is available or someone like that. Because they know way more about Aspire than maybe you do o r maybe I do, you know. Or Blazor, right. I can go see, see Steve Sanderson, right. I pulled a specific model that will help me with that, with that particular problem.

Jamie : [46:11] I love that idea because then, again, it’s all running on my machine choosing the model I want to run. I’m not on the whims of, “which version of ChatGPT are we on now? We’re on 4o, or maybe we’re on 4i, or maybe we’re on x, " whatever. And because they’re trained differently, they will they will give me different responses right or completions i guess.

Jamie : [46:33] And the other thing to remember for people who don’t know a lot about LLMs: your completion is based on a probability of whether this is what you want. So if you’re using an online one, you’re at the whims of what the online model “thinks” is the correct response based on its own view of the probability of what you want. Whereas, if you’re running the model locally, you’ve got all of that context you’ve got all of that stuff; that can make a better, in my opinion, a better decision based on which completion do you want–and that’s the responses right. So yeah that lights up a whole part of my brain where I’m like, “why am i not doing this yet?”

Jim : [47:11] And what is also very cool is, a little kind of sneaky peek at our future roadmap, is we are looking around the idea of bring-your-own-model. So this is, yeah no dates on this yet. But this is something we’ve been talking to a lot of folks about. And part of this is going to be bring-your-own-model local, but also bring-your-own-model in the cloud. So the kind of scenario would be: if you’ve got an internal model, maybe you deployed it to, I don’t know, Azure AI Service that you have trained on your company’s code base, and maybe you’re doing weekly training cycles on it to have the latest code in there. You’ve got the power of GPT-4, but trained on your codebase. What if you could bring that into Pieces?

Jim : [47:54] Or if you have a local model requirement, again, you train a local model, but you just train it on one codebase, and then you bring that into Pieces. What if you could do that? And that is something we are working on. We’re actually looking for companies who’ve got this AI governance that says, “I can only use this model,” to work for them to make sure we get the experience as good as possible. But yes we are looking at this kind of bring-your-own-model.

Jim : [48:18] So from a governance perspective you can use GPT-4 just deployed to your Azure AI Service, AWS Bedrock. you know, whatever whatever whatever. Or you can custom train models, as long as they are language models that we can integrate with. So just kind of a sneak peek at the things that we are starting to talk to a lot of companies about and it’s on our roadmap.

Jim : [48:40] And this opens up a huge amount of possibilities because, yeah, if you’ve got a model that’s fine-tuned, say every week, across your corporate code base, and you’ve got, I don’t know, one of those massive microservice architectures where you have a thousand microservices across the whole company: “I need to bring in user information into my service, what can I use to do this?” The model knows, the model has been trained, and you’re bringing that into your IDE; into the place you’re writing code and suddenly just that discoverability across your entire code base becomes so much more.

Jim : [49:13] As long as, for example, all of your code is in, for all your microservices, all your corporate code is in, say, your company GitHub account, you could fine-tune the model just across, “everything in my company GitHub account uses to fine-tune my model.” You bring that model into Pieces, and just that discoverability across all the projects becomes so much more powerful. Because large companies are really struggling with, “we have all this code, we don’t know what it does, we don’t know what’s there.” And you end up with 10 teams, all building functionally the same piece of code because they don’t know about the existence of all the other teams.

Jamie : [49:47] Yep.

Jim : [49:47] And so if you can put an LLM over, the top of that, bring that into Pieces, you’re in VS code, “I need to interact with the user service, what service should I use?” Bang. There’s your answer that could save you weeks of conversations and meetings trying to find trying to find that question. So yeah, sneak peek, we are working on this. And if there’s anyone out there listening who goes, “oh yes. I want that” Please get in touch with us, we would love to work with some design partners to make sure we can really nail the experience for that.

A Request To You All

If you're enjoying this show, would you mind sharing it with a colleague? Check your podcatcher for a link to show notes, which has an embedded player within it and a transcription and all that stuff, and share that link with them. I'd really appreciate it if you could indeed share the show.

But if you'd like other ways to support it, you could:

  • Leave a rating or review on your podcatcher of choice
  • Consider buying the show a coffee
    • The BuyMeACoffee link is available on each episode's show notes page
    • This is a one-off financial support option
  • Become a patron
    • This is a monthly subscription-based financial support option
    • And a link to that is included on each episode's show notes page as well

I would love it if you would share the show with a friend or colleague or leave a rating or review. The other options are completely up to you, and are not required at all to continue enjoying the show.

Anyway, let's get back to it.

Jamie : [50:13] Right. Because I can see that being the future both from a code perspective but also from like “I am Acme and I want the Acme business model that describes how we operate, all of our policies, all of our everything.” And then I can say, I can pull up the Acme model and say, “hey, I need to book some time off. Who do I need to speak to? Or how do I do that?” Or, you know, “this has happened. How do I deal with that?” And I feel like that’s probably the next evolution for sort of business services to be able to actually say, you know, “how do I…”

Jamie : [50:48] And I’m sure there are–I never claimed to have thought of anything new myself–I’m sure that already exists. I’m sure there’s hundreds of companies already doing that. But like having that specific model that is trained to a single problem domain, in this instance, like you were saying, “here is all of the code that we have ever written, as a company for all of our services. Which library do I need?” There you go, “Which library do I need to do RGB to HEX conversion that we’ve written, that… maybe I don’t want to pull some public code, because maybe I have to do a full security analysis on that. Because I’m pulling in some unknown third party code. But I know someone, in another department, has written some code. Instead of going and finding that person, because maybe they’re on vacation, I can go, “hey, code base LLM. What is the code to convert from this to this?” or, like you were saying, “how do I add some data to the user service?” or all that kind of stuff.

Jamie : [51:46] That feels very, very, very helpful very targeted. Because like you said right…and even if you don’t include shadow IT, just like, “we have 12 different development teams,” like that in itself, to me, I can’t even think of how you manage one development team; so if you’ve got 10 , 12, 20 different teams, all doing different things, how is that manageable? AI, that’s how.

Jim : [52:12] Yes. Yeah. I mean, there was a company we were chatting to, kind of unrelated conversations, but basically they’re saying they have 10,000 developers. And you think 10,000, and this isn’t a huge company. I mean, I hate to think how many developers Microsoft has got, you know, I think maybe 50,000 developers, if not more. And they’re all working on these different systems and there’s coding standards, there’s tools to access other tools, to access other tools.

Jim : [52:36] The only way you can deal with that amount of unstructured data is AI. And so if you could reduce that worry of, “what the hell are all these people doing? I don’t understand all this code! I need something…” you just put an AI over it. And then bring that into the tools that developers are using every single day. We don’t have to context switch any more. It kind of goes right back to that very first example I gave of, “I’m in VS Code, what’s the bug I should be looking at? What’s the issue I should be looking at?” “We need to integrate with the user microservice.” “What is the user microservice? " “Oh, it’s in this piece of code here.” “What’s the API for it? " “Here it is, here. "

Jim : [53:08] And you haven’t left your IDE. You haven’t had to have a meeting. You haven’t, you know, you can even say, “who was the last person who contributed to this code?” Okay, bang, it was this person. Right. So if I have questions, I now know exactly who to go to. I’m not doing the ask, ask, ask. I mean, I used to work in the banking sector and, “oh, I need to interface with the system over here. I’ve got an example code, but I’m missing some data. Who do I need to speak to?” So I speak to this person, who tells me to speak to this person, tells me to speak to this person, but this person doesn’t work for the company any more. So I find out who was this person’s manager, who’s the replacement. And you end up spending a week literally trying, going round and round in circles to find the one person who can say, “the field is called this.”

Jim : [53:49] Imagine how much more productive I could be if I could just say, “what is the field for this?” and it gives me the answer and I carry on. Do more with less. I can keep doing more. I can keep delivering more. So all this comes back to the same idea of we want to leave you focused in what you’re doing, bring all the information that you need into one place, and then allow you to be productive and build the code. And as with any AI tool, it is a copilot, not an autopilot. So it writes the code that it can write so you can write the code that only you can write.

Jamie : [54:19] Yeah yeah. Because that’s something that I really appreciate about the way that Microsoft are, sort of, branding their stuff: it’s copilot. It’s giving you the information. You are the pilot flying the plane, the copilot is helping you figure out all of the information. Because, I don’t know if anyone listen–I know that Cliff will have done, because he is a pilot. Anyone listening, if you’ve ever been in the, I want to say cockpit, I don’t know if that’s still the term, but the front bit of the plane where the pilot sits the copilot sits. There’s like a million different dials, and gauges, and buttons, and lights, and switches, and that’s why there is a copilot; because you literally cannot read it all. And so the copilot will give you information, you can then make the decision, to say, “actually this code isn’t actually what I want,” or “that’s not quite right. I’m going to try this,” or “I’m going to change it to this.” Fantastic love that.

Jim : [55:14] Yeah. And the other thing as well is, you compare like a copilot in your IDE to going to like ChatGPT for example. Yeah the copilot is in, like you say, in the in that office at the front of the plane with the big windows. They’re not hanging out in the back drinking, coffee. And I kind of equate: if you have to go to ChatGPT or another website to ask questions, that’s like the pilot having to leave the front of the plane, walk to the back of the plane and then ask the co-pilot, who’s hanging out drinking coffee, a question.

Jim : [55:43] No. The copilot should be there right next to you. Which is why I love the fact the Pieces copilot is with you across all of your tools. Doesn’t matter what you’re in, the copilot is sitting next to you, looking at all the knobs and dials, looking at the flashy lights, whatever; and is there to help you you don’t have to leave what you’re doing and go somewhere else. And again because, let’s be honest, we are easily distracted.

Jamie : [56:03] Yeah.

Jim : [56:04] If I said to you, “which character from the Pixar movie Up are you?” The answer is going to be, “Doug,” we’re all Doug. We’re all like, “squirrel!”

Jim : [56:11] You know, easy distracting. So if we have to go from our IDE to a browser to ask a question, we’re reading email on the way. We are looking at chat tools on the way. Oh, we’re in a browser. “Oh, I’ve got a notification on Blue Sky. I’m going to have a look at that.” We are context switching. We are distracted. We are drinking coffee. We are losing our productivity. Same as if the pilot’s going to go to the back of the plane, someone’s going to ask the pilot a question on the way. You know, they’re going to be distracted. So having that copilot with you really helps to keep you in place, avoid those distractions. you know you’d be a lot more productive and then maybe schedule half an hour at the end of the day to catch up with all your chat messages when you finish writing code. So…

Jamie : [56:46] Awesome. I really do like that. I like having that copilot as close to where I’m working as possible. It makes sense right. You don’t want to walk to the back of the plane. If you’re writing the code, you don’t need to be walking to the other side of the building to find something out physically, literally, metaphorically, whatever. You don’t need that . Excellent.

Jamie : [57:07] So I wonder, we’re running out the time. So because I know you’ve got other things to do, and I don’t want to be pressing upon you too much; but wonder, just real quick, is there like a elevator pitch that you can maybe think of.. if there is one specifically for .NET devs, they may not be. But like or just for devs like, “Get Pieces because

Jim : [57:30] Even for .NET devs. For any any developer, get Pieces because with the power of his long-term memory and the ability to bring any context you want into a conversation, you can interact with the LLM of your choice, where you are, get the information you need based off the information that you want to provide and not be distracted, not lose out on your focus because you’re context switching. So it doesn’t matter what you’re doing, the work you’re doing in collaboration tools, research in your browser, the code that you’re writing, you can bring all that context into one conversation where you are. And it’s free.

Jamie : [58:03] Even better

Jim : [58:05] We are currently burning through VC funding. So Pieces is currently free, we’re not looking to charge at least for another three, four, five months, ifnot more. So it’s currently free, with free access to all the LLMs so….

Jamie : [58:18] Awesome.

Jim : [58:19] You know we could… Free99, as some of the folks in the team like to say. It doesn’t get any cheaper than that so…

Jamie : [58:25] That’s… yeah that’s genuinely awesome, too. Because, like, so we were saying earlier on about: if folks are um perhaps concerned about how it all works they can get in touch, and get a demo, and talk to maybe yourself, maybe whomever about that. And maybe they could include their CISOs, their C-suite people, and people who are deeply concerned about the governance, and where the information is going, and how it’s processed. They could get those people involved in that, so that’s even better right. Because, like I said, currently if I’m using a public copilot, a public LLM that is hosted on the cloud, unless I’ve read the EULA just before I send the prompt, they could have changed it, and if I’ve not fully understood it, I’m… what am i doing with this information? That’s awesome.

Jim : [59:16] I said we want to be that transparent. So yeah, any companies that are thinking about Pieces we will gladly put our Chief AI Officer or our CEO down in a call with the CIO, CISO, whoever is relevant, and we’ll just transparently go through all the details with you. Because we’ve been trying to be as transparent as possible right from the start around everything that we’re doing.

Jamie : [59:37] Amazing, amazing. Okay then, so let’s say folks are listening in, they want to learn more about Pieces, they want to learn a little bit more about you, maybe read the blog posts, see the videos and stuff that you’re putting out there. Where’s the best place for folks to go for all of that?

Jim : [59:50] So our main website is Pieces.app. So that’s the place to go. I assume we get a link to that in the show notes. You can download from there. That’s got links to all our documentation, all our blogs. We have a discord server as well, which is a really good place to reach out and chat Pieces.app/discord will take you directly there. And then in there you can ask any question you want and we can kind of connect you with the right people.

Jamie : [1:00:13] Amazing.

Jim : [1:00:14] And we’re all over social @getPieces. So on X we’re getPieces, on Blue Sky we’re getPiece, so on YouTube I think we getPieces as well. So again, that’s all links from Pieces.app. So yeah, come and hang out with us on social. We run a lot of live Discord events, we have some fun events going, so if you want to have your GitHub profile roasted by our team with the help of Pieces, we do things like that, we’re doing some city game nights, and plus we have some great conversations going on the discord as well we talk about Pieces, AI in general other kind of relevant topics. So come hang out on discord it’s a fun place to be.

Jamie : [1:00:48] Awesome, awesome. I will ask this, we’ll delete from the recording if you want: when are Pieces going to come and do some conferences here in the UK? Because I’ve seen all of y’all’s colleagues are all in the US.

Jim : [1:01:00] So not conferences. I will be doing a Pieces talk at Dotnet North.

Jamie : [1:01:07] Ooh!

Jim : [1:01:08] So I’m flying in, because i live in the west coast of the US right now, I’m flying in for NDC London. So it hasn’t been put up on the meetup site yet, but on the Tuesday of the last week of January, which is Tuesday the 28th, I will be at Dotnet North and I’ll be demonstrating how you can build a C# copilot, a Star Wars themed copilot. Because where Pieces has Pieces OS, the background service, that has a REST API, and we have SDKs. So I’ll be demoing how you can build your own copilot using Pieces as the LLM abstraction layer; and kind of just digging into how copilots work.

Jim : [1:01:42] So I’ll be doing that. And then I’m hopefully going to be giving another talk at the Microsoft Reactor in London that week as well; probably comparing large language models and small language models. I’m still trying to get that set up at the moment, my contact there was bumped into him into Chicago at Microsoft Ignite, last week, and he was taking some time off to see Chicago. So when he gets back into the office, I’ll be arranging that. So that’ll be at some point that last week in in January as well. I’m at NDC; I’m not talking about Pieces at NDC. I’m talking about something completely unrelated… dark side monitoring, in fact. But, obviously, I’ll be, yeah I’ll be around to discuss Pieces if anyone’s at NDC London as well.

Jamie : [1:02:20] Amazing, amazing. Well Jim, it’s always a pleasure to chatting with you know. I know we’re pushing right up against the when you need to be doing other things, because you’re a very busy person and I appreciate the time that everyone puts in to be on the show. So thank you very much for talking with, us I really appreciate it.

Jim : [1:02:37] Thank you for having me. It’s always a pleasure to hang out with you, it really is. I love being on the show.

Jamie : [1:02:41] Thank you very much Jim

Wrapping Up

Thank you for listening to this episode of The Modern .NET Show with me, Jamie Taylor. I’d like to thank this episode’s guest for graciously sharing their time, expertise, and knowledge.

Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at the podcast's website, and there will be a link directly to them in your podcatcher.

And don’t forget to spread the word, leave a rating or review on your podcatcher of choice—head over to dotnetcore.show/review for ways to do that—reach out via our contact page, or join our discord server at dotnetcore.show/discord—all of which are linked in the show notes.

But above all, I hope you have a fantastic rest of your day, and I hope that I’ll see you again, next time for more .NET goodness.

I will see you again real soon. See you later folks.

Follow the show

You can find the show on any of these places