Episode 90 - Clean Architecture and Subcutaneous Testing with Matthew Jeorrett

Sponsors

Support for this episode of The Modern .NET Show comes from the following sponsors. Please take a moment to learn more about their products and services:

• The Shrimp and Crits Podcast - Shrimp and Crits is an actual play podcast with a Southern twist. We play Monster of the Week by Michael Sands. If you like the sound of swampy monster mayhem, gators gone shopping, and magical fairy mischief, you’ll be right at home in the pan-handle town of Gullacochica, Florida.

Please also see the full sponsor message(s) in the episode transcription for more details of their products and services, and offers exclusive to listeners of The Modern .NET Show.

Thank you to the sponsors for supporting the show.

Embedded Player

The .NET Core Podcast

Episode 90 - Clean Architecture and Subcutaneous Testing with Matthew Jeorrett

Supporting The Show

If this episode was interesting or useful to you, please consider supporting the show with one of the above options.

Episode Transcription

Hello everyone and welcome to THE .NET Core Podcast. An award-winning podcast where we reach into the core of the .NET technology stack and, with the help of the .NET community, present you with the information that you need in order to grok the many moving parts of one of the biggest cross-platform, multi-application frameworks on the planet.

I am your host, Jamie “GaProgMan” Taylor. In this episode, I talked with Matthew Jeorrett about the Clean Architecture, Subcutaneous Testing, and how he utilitsed both to build the My Booking Hub web application in a clean, vertical, extensible manner.

Along the way we discuss the many positives of mentoring and sharing your experience with other developers; regardless of where they are on their journey. We also talk about how it is not possible to know it all, and by listening to those who are sharing their experiences, you can adapt and learn.

So let’s sit back, open up a terminal, type in dotnet new podcast and let the show begin.

The following is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository

Jamie

Thank you ever so much Matthew for spending some of your evening with me, I really appreciate it. You know, the nights are drawing in everybody wants to spend time indoors. We’re all spending time indoors anyway. But we all want to spend more time away in those, preferably away from the computer because that’s healthier. So I really appreciate you spending some time to talk to me today. So thank you ever so much.

Matthew

No problem at all. It’s so good to be on your show.

Jamie

And welcome to the show as well, my goodness.

Okay, so I’m not going to dilly dally too much. We talked earlier about clean architecture in subcutaneous testing, so we’ll maybe cover those a little bit. There’s also some new stuff you want to talk about as well. So why don’t we start with a little bit of an intro to yourself? Is that all right?

Matthew

Yeah, sure. So um, I guess I’m not a lifelong software developer, I studied architecture - and that’s like building architecture, not software architecture. And I worked in property for a number of years, about six or seven years. And then through CodeClan, I kind of got into software development, they taught me just enough to be dangerous. And then I got my first job as a startup, working with Blockchain technology, actually. So that was a bit of a baptism of fire, and worked there for three years. And then I moved on to Clear Sky Logic where I’m today working as a tech lead and lead developer on their booking platform, MyBookingHub. I’m five years in, and I’m loving every day of it.

Jamie

I can’t imagine being a journeymen developer, at the beginning of your career, jumping straight into blockchain. That that must have been, like you say a baptism of fire. Like, that could not have been easy.

Matthew

No, it was coming out of CodeClan, they obviously teach in certain technologies. So Java, and React were the kind of two frameworks and languages that they used, and then moving into more-like services the built in .NET, and C#, and Angular. So as well as the blockchain, everything else was different as well. So it was, it was quite challenging, but ton of support and learned quickly. So it was good.

Jamie

I love to hear about people getting support when they start new, new jobs; new jobs, or new technology or a new project, it’s always super important to do that. Because you don’t know the domain. And then like you say, you don’t know that you maybe don’t know so much about the framework, or you don’t know so much about the front end, And most people want to hear it that I’m going to shout out to but I do want to shout out to those those devs, who will take the time to sit with someone and say, “Oh, it’s cool. Don’t worry about it. Let’s just talk through it,” you know, because that’s what you need. I’ve been on this journey of Software Craftsmanship, and after having read the Pragmatic Programmer about how to be less, I guess, enthusiastic; no, not enthusiastic it’s the wrong word: Evangelical, I suppose. I’m worried about that term, but you get the idea, right? When people say, “no, React is the best!” “No, Angular is the best!” And I’m like, “whatever fits the bill. You two, can stand here and scream and shout about React does this and Angular does that. Or we can get the work done. Because we’re not being paid to scream and shout about one technology over the other, we were being paid to solve the problem. The CEO doesn’t care which technology is written in, they just care that when they hit the button, they somehow get money out. That’s literally it, you know, they are so far removed, all they care is they get the money, right? So let’s not worry about it too much.” So yeah, I really appreciate that you were given that support. That’s one of the most important things, especially someone who’s towards the beginning of their career, and someone who is, like you said, working in a new technology new domain. So that’s wonderful to hear.

Matthew

Yeah, 100%. And even experienced developers, you might be, you might know the stack, well, the technical aspect, but you won’t know the domain. So when you join a team, you won’t know the product, you won’t know the domain. And there’s, there’s a ton of learning to be done there. And there is no substitute for just sitting down beside someone and talking through it; no matter documentation, or well structured code or clean code can substitute just sitting down with someone.

Jamie

Absolutely. And that’s why the writing of the code is maybe 35-40% of the skills required. the rest of the time, you know, cuz you’re going to be doing even less of that when you actually do the work. Most of the time, you’re planning, you’re debugging, you’re reading, you’re working with other people. And it’s that ability to communicate and the empathy to be able to sit with someone else and say, “Look, I don’t qute get this can you explain it?” or to be on the other side when somebody sits with you and says, “I don’t go get this can you explain it?” Because both sides of that conversation are incredibly difficult to manage, especially if you’re new to the business. Because there’s this feeling that, “we have to know everything we have to know at all.” And guess what? You’re never going to know at all. There’s a person who’s pretty big in the .NET space called Scott Hanselman and he’s been doing it for 20-30 however many years, and he says, “I don’t know it all. I haven’t got a clue.” There’s a whole blog post about how when he was sitting with two juniors, and they were figuring out how to draw an X for close modal button. And one of them was like, “well, we need to pull in all of Font Awesome.” And the other person was like, “No, we need to pull in all of this library.” And these other people kept coming over and giving lots of examples. And he went, “or, or I could just type an X,” right. And the the idea behind the example that he gave there was that, like, there were so many millions of different ways to solve a problem, you’re never going to know how to do with all the technology that isn’t meant to be a negative, that’s supposed to be let’s celebrate that as a positive, right? Because there are so many ways you can experiment and explore. And you’ll meet people who have completely different paths through their development journey, and you’ll be able to go, “oh cool. So tell me about what’s Go on about? oh, what’s Python? Oh, cool. What’s, I know enough about Python, for instance, to know that it’s a machine learning thing. But what’s that all about? Java? I know, that’s quite similar to C#. A lot of people like to say that it’s not, but it is quite similar to C#. Let’s talk about that. Let’s figure this out.” Because it’s only by the cross pollination of those ideas that will ever get anywhere, right?

Matthew

Yeah. 100%? Yeah, it’s something I’ve been thinking about recently, is this idea that talking about the different ideas, it’s a really important step in solving problems is that you, you cast the net wide and see, “well, okay, it feels like this is a good way to solve the problem. But let’s think about what the other options are. And then we’ll pick the best one”, rather than getting tied to, “oh, let’s do the quickest way that we can.” And part of that is getting different people on board to have those different perspectives. Because I find it incredibly difficult myself, if I’m solving a problem, once you’ve got that initial idea of, “we could solve it this way,” to get away from that and go, “actually, what other options out there.” So I think that’s where collaboration, one of the ways collaboration is super important is that diversity of ideas, before you settle on how you’re going to do something.

Jamie

Absolutely. And I feel like you hit upon something there, there’s a real skill involved in, like you say, “I have this idea for how to solve the problem. But I need to be able to step away from that and try and look at it from a 10,000 foot view.” Or maybe you want to, say six months from now view or something, because it may solve the problem now, and that’s great. But then in six months time, when all of the requirements change, does it still solve the problem?

And another question, does it matter? Right, because in six months time, it might be a completely new code base written by completely new engineers, you know, we have to, I feel like in the the world in which we live in: the the code, the speed, agility, if you’ll pardon the expression, because I’m worried now that people who are not fans of Agile are going to enough I don’t mean, as in Agile, the process, just the agility, the the speed and direction in which - the velocity - in which we make our software, it changes so quickly.

You can literally be on a project now and in six months time, it may indeed be a completely different codebase. Or it may be exactly the same codebase but with completely different modules, right. And so I think that being able to make that decision of it solves the problem now fantastic. But can it be extensible enough, in six months time, but then also, like I said, does it matter whether it’s extensible or not? Because will it be replaced in six months time? It may be that the module, the thing you’re working on, “is get it sorted now, and they get it out the door, and we will incrementally, you know, accept that tech debt, but then we’ll replace it in two, three months time, when we get a better handle on the problem.” Because sometimes, even the people who wrote the specifications, they don’t know the real problem. Sometimes even the end, the client, the end customer, the end user, they may not fully understand the problem. And so it all has to, like you say it’s all a collaboration, it’s working with the other people on the team, working with the managers working with the specification people, the architects, the users; that’s what it’s all about, get everybody involved, and just keep those ideas flowing. Because otherwise ou know, you’re going to sort of stagnate, I suppose.

Matthew

Yeah.

Jamie

You’ll get to a position where all you know how to do is build .NET stuff with Angular or Java stuff with React or you tie yourself down to a handful of technologies. And then you’re ended up having to sort of shoehorn a solution into the problem, if that makes sense.

Matthew

Yeah, I think software is a funny combination of transient, cheap; like, software is cheap to write and cheap to run. But at the same time, it’s got a degree of permanence. If you’ve got a successful system, then a lot of what you write will be permanent for a while. And so and it will be around for years to come and people will have to deal with it. And so, understanding when something doesn’t really matter, or when it does is super important. And I guess that’s kind of where having a strong architecture in place that allows you to fit things within a framework that is flexible to change is really important.

Jamie

I agree completely. I suppose that’s where loads of different ideas around architecture have come in and where you we’ve got like microservices, versus monoliths versus functions versus containers versus this than the other. And it’s how do you how do you figure those things out? Right? It’s, it’s for the, for the, for the smarter people than me to answer that question. I haven’t got a clue. Like you say, pick the thing that solves it right now. And hopefully you can build it in such a way that people can come in and add things to it, right?

Matthew

Yeah, is when looking at the kind of landscape of architectures and writing out there, I think, one thing you start to do as you become a bit more experienced is that you look at it and you find the common themes amongst it. And it’s not about clean architecture versus hexagonal architecture versus whatever it’s about the themes about decoupling, and flexibility. And you, it’s sometimes easier to pick a thing and hang your hat on and say we’re doing that. But ultimately, you’ll you’ll, it’ll be your own version of that, like everyone does agile, but they do their own flavor of agile me, they don’t do this ritual, but they do this one differently. But But the important thing is that you kind of understand the principles, and they’re expressed in different concrete implementations. But ultimately, you’re put together, compose your own version of that, and hopefully come up with something that works.

Jamie

And that’s what it goes back to that collaboration thing, right? If so, I’m going to pick on Niger, right, I’m gonna pick on Nigel, and I’m gonna say that things with Agile Scrum, rad, extreme programming, that kind of thing. They’re not meant to be rigid sets of you must do this, you must do this, you must do this, is take the things that work for your team, take the things that work for your project, take the things that help you to deliver that thing on time. And use those and then take the things that help you to react in such a way that you can change, change direction, you can pivot, you can reprioritize, you know, that’s all it is. It’s not, it’s not, it’s not necessarily, here are the the 50 steps to agile, making loads of money, this is a great way to put it. But you know, it’s not a case of these are the 50 steps of follow these steps, and you are guaranteed to always release your software on time. It’s a case of pick number one, four, and eight and see how that goes. And if that doesn’t go well. So what number four for number five, that doesn’t go well swap it for number six, is it’s all mix and match. Let’s just figure it out.

Matthew

Yeah, I think, I think a really powerful idea in software and his principles. So principles are about not this is the right answer all the time. But it’s a tried and tested thing and 95% of the time, it’s right. And it’s like a direction that you travel in. So Agile has got its principles. And like the top level, you’ve got people have a process. And that’s like a fundamental, exactly what you’re saying. It doesn’t matter what process you follows, it’s about making the people effective. And the process supports that. And on the technical side with architectures, it’s about the principles guide, how you build the system, and, and consistency over correctness is something I come come across so many times recently, it’s about well, we’ve got this approach, it might not be 100% of the best approach. But actually, it’s proven. It’s tried, tested and proven. And so unless you’ve got a really good reason, let’s let’s keep it consistent and heading this direction, so that we don’t have to solve kind of transient the problem vote, there’s 100 ways to solve this, which one should be pick it so well, this is the way we’ve chosen versus the strategy here. And let’s focus on the real difficult problems, like the novel things that we have to do.

Jamie

Totally. And the scary thing is right, that if you think about it, our industry has really only been around for around 70 years. You know, I’m not going to count, although they were they were complete pioneers, I’m not going to count pre World War Two, because that was a machine was built to solve one problem. The idea of the stored program computer came about in the 50s. So we’re 70 odd years into this industry. We are right at the front of figuring it all out, right. I’ve got friends who are, you know, they’re sort of senior, like senior because they are old enough to be senior, you know, and they’re like, We I’ve one of my friends want to see these 50s And he’s like, I’ve been working in software for half of its life, not half of my life, half of its life. We are at a point where we can have people who are that age and talk about that. So what I’m trying to get at is we are still in the sort of figuring everything out stage. We’re still at the very early stage of if we were a physical engineering, we’d probably still be you know, Roman era times, you know, we are right back in the beginning.

Matthew

I don’t know if it’s unique to software, but part of software’s ability is that we solve our own problems. So like tooling, and IDs, and C, ICD, they’re all software problems that are helping to build software. And it’s kind of a feedback loop. And it makes it really interesting area to work in, you can use your own skill set to help you do your job with your own skill set better. It’s kind of I find that really interesting. So I guess in mechanical engineering, you make tools to help you make the products and that kind of idea through cool.

Jamie

Yeah, I love the idea of like, dogfooding, you build the software, and you use that software to build the next bit of software. It’s absolutely brilliant, because there’s no other, like better test for whether your software is good enough, then to say, Hey, I’ve used my software. And this is what I’ve been able to do with it, you know?

Matthew

Yeah. So first that if it doesn’t have users, and if you’ve got a genuine need for that software, then you’ve got that user right on top. Yeah,

Jamie

absolutely. And you become your own feedback. You know, you sit there and you spot the pain points, is brilliant. And maybe that’s, that’s a good excuse for developers to build things in their own time, perhaps I don’t know. Because then you get the idea of I’m the developer, but I’m also the user, you get that, because a lot of I’ve heard people refer to users with a silent L at the beginning. Because it’s like, they have no empathy for the user. And I’m like, you don’t get it. These are the people who will be paying your wages, right? They’re not paying it directly. But they are paying to use the software that you are building. And so you need to treat them with respect and treat them with empathy. And when they come to you with a problem. You have to just sit there and really think about how you going to interact with them. If if you think it’s a salt simple problem. And it is really a case of pepper Kak problem exists between JIRA and keyboard. Just don’t say that just say, Well, okay, here’s, here’s what I think, obviously, the system isn’t designed that well, because the process we’ve designed is you do this and this. But obviously, it doesn’t look like that. So let me take that on board. Let me change the system, make it work better. We’ve always got room to time, we will get onto the topic in a moment, I promise. There’s a there’s a person, I forgotten the surname, but he’s, he runs the YouTube channel, Linus tech tips. And he has recently started a challenge where he would use a Linux based operating system for a month as his daily driver. And he got into this situation where he done something wrong on his computer. And he done loads of Google in dried out 15 million commands without reading what the command does, and typing this one command in. And it completely messes up your system. And the lesson to take from that is not or what a silly user, the lesson to take from that is, how do we get the software to stop you from getting into that position? And when you’re in that position? How do we write the software such that it’s easy to get back out of that position? Not Oh, look at him, he’s done something really silly. And he’s done it wrong. It’s like, no, no. How do we stop that from happening? You know, how do we put things in place to help him out if he gets stuck? And that I think is, well, good software design is all about

Matthew

the really difficult bit of software engineering, in my opinion, I think a lot of people would agree as it’s not writing the code to make the thing do the thing. It’s understanding what that thing needs to do. And that that’s the conversation with the user and bridging that gap between what what’s possible. And everything’s possible given time, but what’s possible within budget, and what the user needs, and that that’s the difficult part in software development. And like you see, software is so ubiquitous, now users use it, and it’s your granny using it, it’s a bus driver using it as people who are not not technical have no interest in being technical. And so the art there is getting software that is self evident in its use. And it is obvious to and there is a set of rules, but they’re they’re kind of shared and developed over time as more software systems get built about the common language of software. And that’s going to evolve over time as more people become familiar with software and that the the current generation have grown up with software. And so they’re much more familiar with even simple things like a floppy disk means a Save icon, or maybe it doesn’t in the future, because that’s, that’s already way out of date. But, but that common language of design and making design self evident is really important part. And that’s not a software developers problem. That’s the designers problem.

Jamie

But then I wonder whether it’s the software developer, who has to work alongside the designer to help support them in making that happen, right, because it’s pointless having the floppy disk icon, which means save and when you click it, it doesn’t save, right?

Matthew

I didn’t mean to draw. I hate drawing hard lines here. It’s not like Yeah, but the writers because the designers, again are non technical. Their expertise is understanding user behavior and user experience and so Even if they kind of take up that top top half of the relationship between a non technical user and an API, there’s still a gap there to be bridged. And I think developers lead developers that their, their real value in the really difficult part of the job is bridging that gap and making sure so think about building the right thing, and then building it right building the right thing is the hard bit and then building it right. In my experience, one, if you’ve got a solid understanding of the right thing, it’s easy, but

Jamie

totally. So we’ve talked a lot so far about, you know, design and how you design code and stuff. And I’ve said a few times about jail and things like that. I think you said, clean architecture and hexagonal, we said earlier on, right, you take the bits that work for you. Do you have your own personal description of clean architecture, hexagonal architecture, whatever that is, I’ve heard the phrase ports and adapters, I’ve had all of these kinds of things. Do you have your own sort of description of how you apply them?

Matthew

Yeah, so I have a lot of, I’m in debt to Jason Taylor, who’s a developer who does a lot of evangelism about how to write software that I think they run a consultancy in Australia. And he, he kind of his videos introduced me to this idea of clean architecture. And so it’s a common idea. It’s not his idea, but he’s got a certain take on it. And it’s, he’s got quite a few repos, but one of them is showing how to set up a .NET core project with an angular front end using clean architecture. And so I kind of came across that a couple of years ago. And that’s really shaped my kind of, as a relative still consider myself relatively new developer later, a solid way of architecting, an API, back end service. And that’s the first kind of approach that I’ve come across, I’ve really tried. And we’ve we built my brain from the ground up using clean architecture principles based based on that. And and it’s, it’s really worked for me and think, like, I see that clean architecture, it means a bunch of things to different people. And and we’re not doing a strict version of that. It’s not even what Jason Taylor kind of has, it’s, it’s about picking the principles, I think, for clean architecture. The principle that is the kind of main one for me is that you look at the business domain, and it’s a domain driven architecture, I guess, as a family, is this idea that the business logic, that kind of crunchy behind the organization to the business is the most important thing. And it’s also really difficult and complex in and of itself. So that’s before you start tying it to the outside world through email servers, or data storage, or a front end, it’s about isolating that and seeing right, this is the important bit, let’s get that it saw attention and focus and put it in in a decoupled environment where it doesn’t have to know about these things. And then it exposes interfaces to say, right, I need to know how to save a record, I need to load a record, send an email, and look up an external API, and then the layers outside of that then then satisfy those interfaces. And that’s kind of how I think about the layered architecture. But it’s not, it’s I think, clean architecture, to me is not really fundamental difference from a lot of them. It’s just a kind of a certain grouping of the common concepts around.

Jamie

Sure. It’s like you said earlier on, right you, it’s a collection of principles that all sort of work together in a, it’s a formula, right? Or a recipe, right? You want to make a cake, you don’t go out and figure out how to make a cake, you get some eggs, you get flour, you get some sugar, get a bowl, you mix it, of course, unless you’re you know, you’re have any intolerances. You then just mix it all together. You stick it in an oven, hopefully in a different bowl. In a more oven specific bowl, several, you know, 45 minutes later, once the once the ovens up to temperature, you get a kick out of it. It’s a difficult bit then is what are the other ingredients? Well, okay, so we were a business that makes chocolate cakes. Okay, so how does the chocolate cake work? It’s these things and maybe some cocoa and maybe some vanilla. I’m envisioning a core business logic or lump that does the I take the ingredients, I put them in the oven. But I’m also envisioning like an interface that says, Give me ingredients. Right? And then it’s up to the thing that communicates with that business logic to say, Oh, well, eggs, milk, flour, and cocoa and vanilla. And I don’t know why I’ve got cake on the brain. Maybe maybe it’s just because I’m hungry. I don’t know. But that that’s how I mean is that is that an analogy that works are my way off base.

Matthew

Yeah, I think yeah, the agree. That’s yeah, I’m just getting my head right but the ingredients are the the inputs that you need in order to Do your business operation and their clean architecture? For me the domain is that is making the cake. And the interfaces are like where do I get the flour? From? Where do I get the vanilla extract from? Where’s the oven to put things in? And the business logic is about? In what order? Do you do those things? And its sole concern is that it doesn’t need to know by other manufacturers are temperatures. Well, it might take it How far can you take the analogy, but it might know what temperature but it just sets the temperature in the oven stays at that temperature?

Jamie

Or perhaps there’s an interface that says set the temperature to this? Yeah. But you see now and I’m thinking do we extend it? And my business logic is now no longer I bake cakes? I bake good, right? So my business logic is, give me some ingredients. Give me a temperature. And I’ll make it for you. Give me instructions about when to rotate it or thinking souffle, right? Don’t Don’t make loud noises near it. I don’t know whether it’s actually thing. I just know it from cartoons. So how do we get to that stage? So do I start from the outside and work my way in? Or if we’re seeing the business logic is in the middle? The domain, I suppose is in the middle? And we need to provide the inputs? Am I working outside in? Or am I working inside out? Or am I neither? Oh, brother, how do you do it? I suppose that’s probably the best, the best way to think about it.

Matthew

It’s one of the things that it depends what you’re doing. So I think domain that, for me, like I said, it’s a domain driven architecture. And that that includes the title, the the domain drives the architecture. So in a pure kind of hermetically sealed universe of architectural patterns, you’d say, right, let’s start with the domain. And then we’d build the application layer outside of that and build out the way but in reality, that kind of real world project, I used it for my book and hub, I was doing a rewrite of, of the platform. And so we already had the bits of code that connected to the email server, loaded data from the database, did the business logic and it’s just more about splitting them out and fitting them in their cubby holes, I think of architectural patterns as cubby holes rather than starting from scratch and seeing what’s the business logic here. Because a lot of the problems had already been solved, it was just about and put it rewriting and creating a clean structure using the existing code. So we didn’t start with the domain. But imagine if you were starting a new project, that might be how you start and domain driven design is something I’ve read a little bit about, it seems a great way of doing that and teasing out the business problems from from from that kind of inner core first. Because ultimately, I think we’re systems, that’s the purpose. So that should drive what you’re doing. As it’s funny, I often think about systems from a top down where ultimately, you’ve got a button that a user clicks on. And they don’t care what happens when they click that button as long as they get the expected result. So that’s one way of looking at it. But also, from a technical point of view, I guess. It’s actually you’ve got software system there. And you need to think about what it what it’s actually doing on a technical level.

Jamie

Sure. I love to pose questions to junior devs, or anyone who comes to Me, for any kind of mentoring, I say, imagine you have a button that plays an mp3. Do I as a user need to understand how the mp3 starts playing? Do I need to know how the mp3 split into time sequences and then fed to the audio driver? Than the audio driver? For whatever system I’m using will then quantize that turn that into a soundwave? No, I don’t I just need to know when I hit play, it starts playing. But then what do I need to know if if the file can’t be found? Or the file can’t be read? Do I see a horrible gray yellow screen of death and see, you know, this file, this line of code failed because that line of code failed because that line of code failed. All the ways here friendly, friendly message that says couldn’t find the file. Over sorry, little sad face? You know, we, you know, do I need to have a toggle to turn on the developer message do I do? Do I want technical users to use the system? All of these questions I feel you’re into like the design of a system that we don’t really think about. We don’t think about these questions because they’re sort of hidden away. Don’t worry about it. We’ll we’ll have exception pages whilst we’re locally debugging and we’ll kind of turn them off in production and it doesn’t really matter. And I feel like some of those questions get hidden away because like you say, the abstraction of I push the button, something happens. And magic happens, right? But then actually, it’s who needs to decide where that magic lives.

Matthew

And also, it’s never as simple as that think that’s a recurring problem is that you’ve explained to even the simple thing plug in mp3. Really, really simple from a UX point of view. But then there’s a lot of edge cases and the debt like software, you can easily write something that follows the happy path. It’s about the the edge cases and the unhappy paths and how you handle them gracefully. Is such a large part of solid, robust, usable systems.

Jamie

So if I’m understanding what you’ve said about clean architecture, and hexagonal and box adapters, and all that kind of stuff, if I’m understanding how you have implemented it so far, I could theoretically, take a chunk of the business logic that is designed with this sort of clean architecture, or to adapt to see things, take that out of the system and have that almost as I’m not going to say micro service, because I don’t want to hear all of the buzzword bingo. We’ve said crypto is a blockchain, we’re gonna say micro services. There we go. That’s the SEO for this episode done. I’m presuming that you can take this one core chunk of the system that deals with one processor, I suppose. And take that and put that in a service by itself is that it feels like it lends itself to it is that the case? Or have you not much experience in that,

Matthew

that doesn’t quite fit with how I would think about breaking down the system. Because looking at it as a monolithic system. And I guess, with clean architecture, you could, if you were looking at each service, it would be in my mind, I had not thought about services very off the cuff. But each each service would would have the entirety of the clean architecture. So thinking about microservices got, let’s say a REST API. So it would have all the layers of micro clean architecture within that. So it would have the domain logic crunchy logic, the application layer on the infrastructure layer, and, and you would break it down kind of vertically, rather than horizontally. And I think that that’s, that’s kind of maybe something that’s quite particular to the way that Jason Taylor’s implemented, clean, clean architecture is that it’s this idea of the vertical, so So like a wedge of the pie. So you’ve got your, your business operation, then you’ve got the application layer that supports that business operation and hooks up to the infrastructure, and then you’ve got your endpoint. And one of the ideas I really like about the way Jason Taylor has done it is that you kind of isolate the top that the, the external facing part of that, so that like the application layer is the messy bit where you tie everything together. But he vertically slices that as well and says write for the, for the Create booking endpoint, we’ve got one discrete piece of code that handles are in the application layer. And it’s got its own details, and its own class that represents the inputs and outputs. And if you if you come into a system that we think about as you come into a system, you need to make a change in the Create booking endpoint, you can go into that folder, and you can look at those files, and you can play around with them. And you know, the limit of your damage is going to be within create creating a booking operation, you’re not going to have to worry about affecting the update user or the Delete booking appointment like and so he kind of makes that relationship of all the parts in the, in the slice of the pie, really closely related so that you can see what what you need to affect and what the I think in software, I was thinking about, what’s the what’s the limit of the damage I can do here, if I get it wrong, how bad can I get it wrong, and if your scope of your changes within an endpoint, and that kind of makes you feel comfortable that you don’t need to test everything else as well, when you’re doing some regression testing, you’ll be able to retest the correct booking endpoint, and you don’t have to worry about anything else.

Jamie

I like that in the Pragmatic Programmer, they talk about this idea of tracer bullets. And I dislike the term because it’s rather violent. But the idea is that when a gun fires, there, the rounds fire so fast that you can’t see them. But every three or four rounds, there’s one that is lit up, or maybe it’s got phosphorus on it, so that you can see where it’s going. It’s a traceable, it does no damage when it hits whatever, or it does less damage when it’s whatever it’s adding. But it’s bright enough that you can see it. So you’re not just firing wildly. Again, I don’t like this analogy. So bit too violent. But there you go. And but the reason that they say this is because they say regardless of whether you’re working from the the interface of your software down, and I don’t mean like public interface, I, Jamie or whatever me like, you know, like it may be is the API or maybe it’s the user interface, whether you’re working from the screen to the silicon, or silicon to the screen. They they recommend implementing one feature, front to back or back to front all the way through to start with. And this has got nothing to do with clean architecture not to do with domain driven design. But that way you’re staying focused, right and a fits well with what you’re saying about if I work in this one specific area today and only make this change here. I don’t need to don’t necessarily or shouldn’t have to test everything else with my regression tests. Because I know I haven’t touched it. Those files are all the same that classes are all the same. The the services, the contracts, everything is still the same. So I don’t need to touch anything. I feel like all I’m doing is echoing what you’re saying. But What I’m what I’m pointing out is that, again, it’s all down to principles, right? This principle applies outside I think of clean architecture domain design that guy they or domain, first design, domain driven design, whatever you want to call it, boss adapters, can you see I’m doing the SEO thing. But it’s outside of that, right? Because if you are focused on one specific area, let’s say you’re, you’ve, let’s go out of the world of software development, right? You are a mechanical engineer, and a car has come in to be fixed, or you know that all that needs to happen is you need to change a tire. Right? If all you do is change that tire, you don’t need to test whether you’ve broken the engine, or whether you’ve somehow wiped out all of the transmission fluid, or whether you’ve somehow shorted the radio, you all you need to do is test that that tire works. And that is feels like it’s a perfect example. Maybe not, maybe I’m being too too big headed by saying it’s a perfect example. But it feels like an example of what you were saying about that wedge, right? My Businesses, I’ve got to change the tire, I don’t need to bother checking that the airbags don’t apply.

Matthew

Yeah, I understand. I think that the idea that when you’re building software you work in verticals, is, is really a good one in and it’s kind of fits with agile, like, what’s the measure of progress in Agile is working software. So unless you’ve got that entire vertical working, you’ve got nothing effectively. And it’s something that we think about quite a lot is spiking, and getting that kind of just making it work from from from the silicon to the screen, like you see, just just hacking it together in team you can get that traceability whether that path working. And yeah, I think that I’ve never thought about that before. Yeah, that fits really well with the way that the project set up is you’ve got all that code in the application layer, kind of in one place, you don’t have to worry about a shared, like a repository layer, for instance, you just have to write the repository method that exactly matches what you’re doing in that one endpoint. Yeah, that’s really, really good point.

Jamie

Thanks. I also like to think of, the less code that I have to write, the less code I’m going to get wrong or be have to debug. Right? So going with what you said, there, if I’m writing to a repository for my core piece of logic, I don’t if I’m only having to implement one method for that repository interface, maybe it’s the get widget method, right? I don’t need to write a test for my save widget method. I just need to write a bunch of tests for my get widget method, which means that when it doesn’t return a widget, I know that something is wrong, I don’t have to worry about oh, well, maybe it’s not saving correctly. So very silly example, because there’s obviously going to be it’s going to be rather obvious to most people that when when you’re getting something, it’s not necessarily the save method, or maybe it is, but you can then you can then isolate that and build your tests build your system around it. Right.

Matthew

Exactly, yeah. And also, when you’re building a method, a new repository method, for example. And you’re saying, right, get booking, I need to get booking method, where you’re going to use that where you’re using that will will vary, you think, Oh, it’s just to get booking method. But if you’re getting a list of bookings, paginated lists and you’re getting like 500 bookings at a time, you don’t want to pull every single piece of every column out of the database. And you don’t want to join it until the price ledger and the the customer and customer addresses, you just want the absolute minimal data summary for that. Whereas if you’re doing like get full booking by ID, you’re going to join it on top records and return that. So the idea, I guess, of separating these endpoints into their own handlers with their own details, and their own repository method, if you like, is that you don’t have to predict how things are going to be used and shoehorn different use cases into what you’re writing, you can say, well, at this point in time, I’m writing this endpoint, which needs this exact data. And, and that’s good enough for now. And that’s it solves trying to generalize things too early in software is quite a common problem. And the rule of threes, I think, is the the kind of antidote to that, as you say, well don’t don’t combine code, before you’ve used it a third time. And then you might have a really good idea of when there is common behavior. And this is real common behavior, not things that just look like they do the same thing. And that’s something we’ve started to do in my book in habits. It’s getting to be a mature system now that we’ve recognized, well, actually, there’s there’s a number of different endpoints that all have their own implementation of this thing. But now we’re seeing well actually, maybe we need to pull them out into common queries column because we’re using Entity Framework. So the idea of repository layer doesn’t really fit so we’re calling them queries. And then and we’re doing that so it’s that’s about with the architecture, you know, like tying yourself to this ideal. Everything lives in that one handler that’s how the architecture is evolving. I’d say well, actually, in reality, we’ve got commonalities. And and there’s good reason to pull them together now because there’s a bunch of complexity there. And and we need to unit test it. And if you haven’t seen complexity, multiple places, unit testing, it doesn’t really make sense. So we’re now kind of maturing and make it really putting our own footprint or how, how do we do clean architecture, because that’s not something that’s in Jason Taylor’s example project. It’s not, it’s that’s the point at which you really start to progress and make the part on your own.

Jamie

There’s a really interesting point you made there when you said, when you read loads of interesting points, but there’s one that really stuck out to me, which was, he said, you know, wait until you’ve had to do something three times. And there is there is a good argument for dry, don’t repeat yourself. But there’s also an argument for quite literally the opposite, wet, write everything twice. There is a great argument for that. And, you know, Donald Knuth said that, I think in the 70s, he said, the root of all evil, is premature optimization. And he’s absolutely right. Because, you know, if you spot some code that you think, might be used in multiple places, but you haven’t, you’ve only written it once. It’s not, you’re doing that single wench, but you’re like, Oh, I think this chunk in the wedge might be used in multiple places. I’m tempted to make it generic and stuff, futzing with it now, to make it to extend it to let it be generic to either be used by everything else. Well, you’ve got scope creep there. And it’s no longer is no longer fixing against that fixated against that single wedge. And then you’re adding complexity, because as much as so generics are great, right. So in you know, in C#, you can do of t, right. So, so link is a great example of this, select these things where it matches this pattern. And they’re all over this type of object or whatever. That’s fantastic. But it’s also rather complex. It’s not, it’s not horrendously complex, because it takes you know, once you’ve gotten the idea of how it works, is brilliant. But also having to write that and then having to test that can be horrendously complex. And so taking some time out to actually think, Do I really need to make this generic? I think is a really good idea, or do I really need to worry about this? If I have to copy paste this one method twice? Is that the worst thing in the world? Right? Because maybe these these two copies of the method handle things in such as a totally different way, that is not worth the extra engineering effort, like you say, to throw in some code in there to do type safety and to checking in to make sure we’re okay. So it works if you’re passing a string, and it works if you’re passing a double. But if I pass in an int, it everything blows up. Is it worth making it so that if you pass in a string or a double it works, and throwing that check in for the for the INT? Or is it worth just saying look, we’re at the we’re still in the sort of discovery phase doesn’t matter. He sends it in down there and it blows up. That’s your own fault. Or we just wrap that up and log it and we just tell the user something went wrong. We’ll get back to you when we can here’s a GUID for the problem, right? Is it worth that time? Who can say right? And like you said, it’s all about the principles. It’s all about figuring out, is it worth it? What’s the cost benefit analysis? Right?

Matthew

Yeah, I think that’s a great point that you don’t want to build these things until you need them. And the temptation when you’re writing a horizontal layer is that you kind of build out that before you really understand what it needs to be. And so yeah, building the the vertical, you just focus on exactly what you’re trying to do. And the DRI principle, yeah, I think it’s, in a lot of people say this is most misunderstood principle. Because it’s not about saying, don’t write code that does the same thing twice. It’s about saying, don’t write code that has the same reason to exist twice. And things can apparently do the same thing. But they had different reasons to exist. If you combine them and say, we’ll do that in one place. Then down the line, you realize, oh, actually, they’ve got different reasons to exist. So they need to start doing things differently. Then you started coupling things and you’ve got you’ve got problems that come with that. Yeah.

Jamie

Absolutely.

Sponsor Message

Narrator

Hey, sorry to interrupt your favorite podcast but I’m here to tell you about shrimp and crits an actual play podcast with a southern twist.

My name is Ian and I am the keeper for this show as we play Monster of the Week by Michael Sands. If you like the sound of swampy monster mayhem, gators gone shopping and magical fairy mischief you will be right at home in the remote Panhandle town of Gullacochica, Florida, where spooky danger has begun to wash ashore.

Shrimp and Crits is the story of Sarah Pain, the mundane

Sarah Pain

All I’m asking for his answers. That’s all I’m looking for is the truth

Narrator

Ari Green, the searcher

Ari Green

You know the proclamations of the fae? I suggest you follow them from now on

Narrator

And Ray Ray, the most mundane monstreess you will ever meet.

Ray Ray

“Mr. Zeus, I’m a big fan. I knew you were, I knew you were real,” And Ray Ray’s just like bowing in front of this swan

Narrator

As they fumble their way through protecting their skeptical town from mysterious evils.

We release new episodes every other Monday on the pod catcher of your choice. I hope to see you soon in sunny Gullacochica.

Find out more at linktr.we/ShrimpandCrits or check the show notes for a link.

Jamie

Using the the cake example from earlier on, right. You might be tempted to write a thing that takes in a, maybe you have an interface that is I clickable, right? Or I bakeable right? And that is or I can’t think of the right word, but let’s say your eye kitchenware, there we go right, and you have an open class that implements eye kitchenware, but you also have a microwave class that implements kitchen where you whenever you are very rarely going to put something into the microwave or switch it on for 45 minutes and walk away. But most of the time when you’re baking, you will likely do that with something in an oven. Switch the oven on get its temperature, put something Then close the door, walk away, come back 45 minutes later yumminess has happened. But if you do that with a microwave, I don’t know whether you’ve tried this, you’re likely going to end up with a exploded microwave. And so building in the complexity into the AI kitchen, where to handle Am I an oven? Or am I a microwave? You’ve immediately like he said, You’ve immediately coupled those two classes together. Because the the microwave now needs to know about the oven, or indeed the consumer of both of those classes, needs to know whether it is a microwave or whether it is an oven, or whether it’s a kettle or whether it’s a cup, because a kitchen, where is an item that you find in the kitchen, right?

Matthew

Yeah, this is now a given spot spot the pattern and principle but it’s in the solid principles, but interface segregation, like that’s exactly, exactly why you don’t do that you don’t put things together that don’t belong together. Because the consumers? Well, I guess there’s there’s different reasons. So if it feels like the same thing that the consumers will, will then have to sort of know the implementer will then have to do things that it doesn’t want to do, just to satisfy that interface. And if you separate things out, and I guess the thing in the clean up tech world that Jason Taylor is kind of project putting, not having a repository, you don’t have an interface, you’re not satisfying an interface, you’re just writing some code that does a thing. And you’re not trying to make it at all reusable, you’re solving that one problem in that one place.

Jamie

Absolutely. I’m all for writing as little code as possible. I think I said this earlier, honestly, the fewer lines of code that I write, the fewer books that are going to be. And that’s not meant to say our Jamie’s the worst at writing code. It’s the same for everyone, right, the fewer lines of code we write, and I don’t mean like make it really complex and have multiple link statements on the same line. I mean, like, there is a reason why the principle of no more than x number of lines of code in our method exists. And that’s because you are making that method do one thing. It’s what I call the the Charles Emerson Winchester effect. For the people who are old enough to have ever watched mash on TV, you will get this the very first time you meet one of the characters called Charles Emerson Winchester, the 30 says, gentlemen, I do one thing at a time, I do it very well. And then I move on. And that’s the whole point of making methods that are very small, it’s not to be really clever. And to use lots of different angle brackets in different places. It’s so that the method does one thing, and it does it perfectly, or as close to perfectly as you can get for a given value of perfect using a .NET example string.is null or empty, doesn’t do anything other than tell you whether the string you passed in is no or is empty. It doesn’t go to the database doesn’t go over the network. It doesn’t do anything to do with numbers, it literally doesn’t tell you how long the string is, it just tells you is this string null? Or is it empty. And the same whether with string is null or whitespace. All it does is it tells you whether it’s not or whether it’s got any kind of spacing characters in and that’s it, it does nothing else. And that’s where this idea of making things smaller and writing small methods comes from. Because the fewer lines of code, that there are a few books that are going

Matthew

to be and making it easy to understand grok ability. I don’t know if that’s a word. But if you could look at something, it was more than 1010 lines, you get bored by the end by the end of the method. Yeah, yeah. And looking looking down the line, something I think about from the star is another reason to segregate your interfaces is if you’ve got a really simple operation, then when you do come talk to Mizer, then you could you could optimize that easily. Because you’re doing one thing, if you’ve got a complex method, then you can’t you can’t easily find like make efficiency gains there. Because you’ve got all number of inputs, your inputs could be anything and you have to deal with all those cases. So an example from from My Booking Hub, for example, in the in the law service layers, we’ve got a service that that that takes the range of postcodes and finds out the travel time between them. And the mapping service that we hook into, will take a matrix or take lots of start by sources they call start points and lots of destinations and give you the whole matrix of all the journeys between them. But I kind of realized that if we want to optimize that’s quite a really complex thing, talk to me. So maybe we simplify it, you see, you could only have one start and lots of destinations, or one destination and lots of starts. And and that actually worked for the way we use mapping within the system. And so we wrote the interface that wraps the mapping service to have that interface. And then down the line. When we come to optimizer, it’s a much easier problem. You’ve only got a rule of values or common values, not a matrix to deal with.

Jamie

Sure, knowing that you have The service and you have this this business entity that needs to know the answer. It allowed you to make that that chord and they allowed you to cover. But it’s going to be no good to do this with a matrix, we have to do it with a roar column, as you said, one start multiple stops, or one stop and multiple starts. But because I suppose because you knew about the domain, you would know that there is perhaps no need to do multiple styles and multiple stops. Because the domain logic, the business logic of the rules that you are trying to implement in the code, saying that, hey, in our business, we don’t have multiple starts and multiple stops, one of one or multiple of the other, or rather, one of one and one or one or more of the other. You start somewhere, and you may stop on the runway, or you may stop a bunch of ways. It’s, it makes sense, because then you’re not, you didn’t then spend time figuring out to represent that matrix and how to optimize for that matrix. Because there was no need to, because the business said, we don’t need that. And I think that’s, that’s one of the hardest things that developers have to sort of learn as they sort of go through their journey. It doesn’t matter where you are on that journey. I, you know, I am convinced that no matter where you are on your your educational journey, in software, development, software, engineering, coding, programming, whatever you want to call it, we can all learn something every single day. And I think one of the most important lessons is from one of the Alice in Wonderland books. And the king said, start at the beginning, come to the end, and then stop. And that’s what you need to do. You need to write enough code to solve the problem. And then stop writing code, pencils down tools down, and then figure out can we change this? Do we need to change it? Does it solve the problem not to use an example from my own past, I once wrote an API to allow me to sort of deal with my video game collection or MoMA, pretty big video game player, pretty avid collector, and they hooked into several different services to get lots of information. And they started with, okay, so I need an API that can do create a record, read or record, update a record, delete a record for games, uses, ratings, images. And now what I needed to do was store a game in the database, where I didn’t need to do all of this fluffy stuff around it, create a record, don’t write fantastic, that’s my, that’s my section of the code. That’s my traceable. I don’t, but I didn’t, I sat there and wrote out a huge API. Afternoons were wasted just building up, you know, controllers and actions, not getting anything done. Because there was there were still be interfaces. And it was just like, yeah, poke at this, it will throw an exception, but I know the API exists, exactly build a user interface. And there’s nothing wrong with setting the screen going down to the silicon. And there’s nothing wrong with starting the silicon and coming up to the screen. But you need to, like you said, you need to find the core rules of the business thing you’re trying to implement, and then take tackle them one at a time. Because then you’re not you, you may even find you can lean on no code, low code. Why not?

Matthew

Yeah, I think we think about as developers, the temptation is to solve interesting problems. But actually, you need to solve the real problem that the customer has. And sometimes they’re not interesting. And if your code feels simple, and you’ve solved the problem, then you’ve done an excellent job and move on. Don’t wonder what complexity you might add.

Jamie

Yep, I forget the engineers name. But there was an interview, they listened to on the dev SEC ops, one of the dev SEC ops podcasts. And this engineer said that they were talking to one of their junior engineers, and they were saying, we need to build this release, we need to put loads of feature flags into the into the app, because we may want to turn this on. And the senior engineer turned around and said, We haven’t even built that feature yet. Why do we need a feature flag for it? What are you doing? You’re you’re adding code to load feature flags. For features that don’t exist yet. You’re adding complexity before it’s required.

Matthew

That is something that I quite recently have been bumping up against a few times is writing codes, in a pull request, the unit of code that most of us working, that is not really to the thing that you’re solving. It’s looking down the line. And so we’ll need to do that later. And I think is a cliff edge, it looks like there’s nothing in the news. And it it feels I always say what’s the what’s the harm that can be done in that. But so many times I see that you write some code and you think, Oh, we’re going to do that. And then that feature gets cut and you’re not doing that anymore. And then you’ve got this date code that’s just confusing in the system and say, Well, why does that exist? It’s not called anywhere. And it’s quite a strict rule. I’ve very few kind of hard rules thing. What I have one hard rule. But but that is one that I’m pretty far more on is that it’s not actually solving the problem we’re solving right now. It doesn’t get into the wheelbase.

Jamie

Absolutely. You need to think like a rocket engineer. If I’m going to be build a rocket that is going to go to the moon, I need to only put the things on the rocket that will get it to the moon, and only the things that it will need when it gets to them. Because I have a physical constraint of how bulky and how heavy this thing can be, the heavier it is, the more fuel I need to burn to get off the ground. And if I’m burning more fuel, that means a I’m producing loads more greenhouse gases, if that’s one of your problems, and be it’s costing a shedload more, because guess what, you got to buy more fuel. And then if the thing crashes, and I don’t mean like a software crash, like it bends over and explodes all of that money and all of that time was wasted.

Matthew

Yeah, I think on writing more code, I think the principle I’ve heard it before is that all code is bad code. And it sounds it sounds like a trite kind of trivial thing. But I think it is a good, I think it’s got a lot of truth in it. And the more the more code you write, the more like, if you can avoid writing code, and unfixing with the caveat that making things terse and short, like fewer characters is not more code. It’s, it’s something more subtle than that, then you should yeah. And it’s, I think people, people have a tendency by myself included. So now I can write good code, I can write clean code. But now, all code is bad code. And if you don’t have to write code to solve a problem, then don’t under the low code, low code options, like they’re going to become more prevalent as we why write custom software to why why re invent the wheel if you don’t have to, and it is not appetizing to a software developer to use a pre built solution. But but it’s sometimes the right thing to do.

Jamie

Yeah, I mean, I can count on one hand, maybe to the projects that I’ve worked on the last 10 years, they’re not read something from the database, display to the user, that the user make some change to it, transform it slightly, and put it back in the database. That is 90% of the bread and butter of stuff that I do. And and it will likely be a maybe not 100%, maybe not even 90%, maybe 70%, maybe you’re the lucky ones who work on super interesting things. But I will guarantee that a large percentage of what we all do as developers is get the user to type something in, transform it, store it in the database, pull it back out of the database, read transform it to the user, it’s forms over data is, is important for a reason the majority of businesses work like you will fall into areas where it’ll be slightly more interesting than that. But guess what the day has got to go somewhere is going in the database. And it can’t just go in as it is it’s got to be transformed before it goes in. So you still working on that forms over a year?

Matthew

Yeah, the kind of flipside of that is that you always think, oh, you should be able to generalize these problems, and why aren’t there more kind of general solutions, but when you dive into specific business requirements, that complexity kind of pollutes that and you end up thinking, sometimes you have to write custom solutions. And and it’s about finding that level, like, where you can, what you can write custom versus what you can integrate with existing solution. And that tension between what if you pick an existing solution, it’s not gonna, it’s not gonna be 100%. Perfect for your, for your requirements, probably. But it’s 90%. Perfect. And if you can use that thing and make do with not having that 10%, then there’s a massive benefit of not having written the 9%. But, and quite often you think, Oh, well, doesn’t do this one, one cool feature that I need. And so I’m not gonna use I’m gonna write the whole thing myself. Yeah, it’s not very economical.

Jamie

Yeah, I feel like a lot of I see this more with junior developers, I feel like I’m picking on junior developers now. But I see this more with junior developers of like, you say, it doesn’t solve a large enough percentage of my problems. Therefore, I’ll do it all myself. And I’m like, Okay, write this cryptographic library that does TLS, or some kind of H Mac that does some wonderful funky maths and you get a number out, that doesn’t do everything you want. So you’re going to go write your own. Even PhDs don’t write their own crypto for a reason.

Matthew

Having worked in and like blockchain security software that that is one of the golden rules of cryptography is you don’t write your own. And that’s not for a wasted effort thing. That is security and peer review. You It’s just impossible to write write software in an isolated room that is proven to be good. You have to get out in the wild and test it so we need to focus effort around the the open source and the show software for doing things that are super critical, such as cryptography.

Jamie

Sure. We talked a little our pre discussion so for people who are just listening in, you know, if you want to be on the show, do get in contact and we’ll Have a call, and we’ll discuss what it is you want to talk about. You mentioned in passing this thing called subcutaneous testing. I was wondering if you could discuss that a little bit. And then when when we’re done with that, we’ll talk about your product, My Booking Hub. Let’s talk about that as well afterwards, right. So I know that subcutaneous means under the skin. And I know the testing means you’re testing. So I’m testing under my skin. What’s it all about?

Matthew

Yeah, so this is another idea, I’ll use a softer terminal and steal that. Jason Taylor basically has a sister repository to the clean architecture, one that’s clean testing. And in his videos, he talks about this idea of clean, clean, sorry, subcutaneous testing. And I don’t think it says idea, but he, he is where I found out about it. And it’s this. The idea is that you’ve got the testing pyramid, which I think a lot of people be familiar with, you’ve got your unit testing at the bottom, which are small, isolated tests that are really reliable, easy to write. But don’t tell you don’t give you confidence that the system works as a whole. And then up at the top, you’ve got an end to end tests, I guess, at the very tip, and they’re about flexing the whole system, including database and even the UI. And they are super hard to write flaky, hard to maintain. But they give you a really high confidence, if they’re passing, then your system is holding together and working as expected. And I guess the the idea of subcutaneous testing is you kind of chop on the top of that pyramid and saying, Well, if we can reduce some of that flakiness, which is from from the kind of user interface, if you like, and duck under that, and then start our test at the point that we’ve got a bit more of a level playing field, then it’s a bit of a sweet spot between assurance that the system’s working as a whole, and by maintainability, and ease to write. So the subcutaneous tests and, and clean up clean testing reports about an API so so your end to end is like a rest, rest request going into the system. It’s not a user interface. And so the subcutaneous test kind of in ASP. NET Core, if you imagine you duck under the request handling, and the authentication and validation, and you got to the point where you’ve got your DTO that’s been passed in and D serialized. And then you test everything else down to the database and back. And so that that was kind of how we set out testing. In My Booking Hub, we don’t have end to end test, but subcutaneous tests. And it’s a really powerful idea that gets you off the ground quicker, but we’re actually moving thinking of moving away from them now, because the drawback is that you’re talking under the authentication and validation. And so you can’t test those things. And so the question is, do you test those things separately? Or do you test them kind of in isolation, but my feeling is that if you’ve got unit tests around authentication and validation, that’s great, you know that the code that does that is working, but you don’t have an assurance, like an automated assurance that that is working in the production system. So although it was an interesting path to go down, I think on reflection, end to end tests probably are preferable. And I think we’ll we’ll move towards that going forward.

Jamie

Sure. What I’m taking from that is something I already knew, which is testing is hard. It’s not easy. And that’s not just test driven development or development driven testing. I think it’s called characterization tests, or behavior tests or whatever, right? Getting that balance is really hard. Because you could get, you know, you said earlier on about, you know, you those end to end tests right at the very tip of the of the of the pyramid, they’re very flaky, but they test everything. Assuming that you’re asserting something. You could say that about any test, right? But I’ve literally seen enterprise level code bases where a where they focused on code coverage, right? It’s got to be code coverage got to be 75% or higher, or 90%, or higher. And my personal yardstick is 60%. Going over 60%, you’re wasting your time. Because at that point, you’re testing framework code, and you’re testing the placement of braces, right? Maybe I’m just cynical, I don’t know. But what I found is, when when there’s this is a difficult balance to strike up against actually running an actual test that asserts a bunch of stuff and proves something correct. And hitting this impossible metric. And what I found is, when teams where they’re edging more towards the impossible metric, they never assert anything. I’ve seen tests that if you throw an assertion in he would fail, but because it touches all of the code, doesn’t matter. Hey, we’ve achieved the goal. Well done, lads, we can go home. We’ve achieved the goal of 100% unit Test coverage. But then like he says, like, at what point do we go from a unit test into an integration test? And at what point does it go from an integration test? To, you know, maybe like you said, a subcutaneous. And then for subcutaneous, how do we go from subcutaneous to end to end? And then like, you said, these off testing that authentication, you may have a unit test that tests if I give you a bad username or password or known bad combination, it doesn’t let me in. Does that unit test also cover? If I set up a user with these permissions and I send this request in? Is that a unit test? Is that an integration test? Sometimes it’s difficult to find where on that pyramid, your test sits? I think the only way to, in my opinion, the only way to sort of answer that is to work for your tests as part of the specification. So like, if you have a user story, let’s say using user stories, and he says, As a user, I could push a button and it will play an mp3. Okay, right. Let’s write the test cases into this piece of work. You know, user presses button, there is an mp3. You can even go granular and say there are speakers attached, the volume is turned up, the person can hear, right? What about accessibility issues? Does the test pass? If the person is deaf? are we testing whether they can hear it or whether it is like, that’s me being really, really, really over the top and specific? You may think a stupid example. But where in that list of questions, do we stop? And do we cast that as fully tested or not? If we can say that when you push the button and plays the song, but the person can’t hear it? Well? Is it because they are hard of hearing? Is it because the real speakers put in is because because the application can crash and be playing in that there is data streaming to the audio driver? But the other driver is working? Is that a pass? Or is that a fail is that an end to end test is a silly example, because very few people who work in sort of web dev are gonna have to deal with that kind of granular level of activity. But I find that sometimes it’s better to have a silly example. Rather than spend hours thinking of, because I come up with all these examples off top of my head. I don’t know what I’m saying.

Matthew

Yeah, there’s so many interesting points. So one, the the enterprise people who were writing tested in lotteries, or anything, I don’t know if they’re doing that deliberately or not. But I mentioned earlier, like, I’ve got one golden rule. And it is like, an absolute hard rule I try not to break ever, is I always see a test failing before I. At some point, even if you write the test with the code, I always break it so that I can see it failing and check that it’s actually doing something and preferably doing like failing in the expected way. Because there’s so many ways that you can rig rig tests, a classic one in n unit, as you forget to put the test attribute on it, and then it just doesn’t run as a test and you won’t get an x. But if you’re not sharp for Eagle, ID, you’ll you won’t see that the test is missing. So yeah. And tying, tying together the different layers of testing. So the end to end versus the integration versus unit testing, I think something that I really struggled with, and I’m just starting to shape in my brain now is that, I think of it like unit tests are the kind of a mass around the feature, that really obviously they’re really rich, and they they kind of flex the edge cases, and they’re kind of they’re very specific to that one feature. And then the integration test, you’re kind of sending a probe down, and making sure you hit something within that unit test that group of unit tests. And that that kind of ties the two together and that’s kind of weird mental image maybe. But if you’ve got an education, your unit tests that says maybe your your passwords can’t have a, an app symbol in it, but it can have an exclamation mark, if you write an integration test, that tests that the Add symbol isn’t that feels unexpectedly then you know that your complex logic is there. And if you know it’s there, then it’s been tested by your unit tests because you can’t write your integration tests every single permutation and, and it’s think thinking about how you concretely tie up your integration tests with your unit tests.

Jamie

Absolutely. I agree. Legacy it’s a it’s a complex problem to solve. And on the on the example that I gave about test without assertions. I’ve also seen entire test classes that are commented out. Like it says public class test whatever squiggly bracket squiggly brackets and everything in between it commented out because there is a test for it, you know, I’ve seen or I have seen all these uses, not to write tests and to write tests.

Matthew

Okay, it’s something I’ve heard about, but not looked into myself as that prevents, like a rigged test. I don’t know if you’ve come across this, but it’s mutation testing. And the idea is that the runtime that the package that does this, I think it goes into actual source code and flips the conditions within your source code, and then checks that the tests fail. And then that way, it gives you a high assurance that your tests are actually doing something you can you can automate. That seems like a powerful idea. I don’t think it’s quite like, taken off. But there’s libraries that will do it for .NET.

Jamie

Yeah, I’ve looked into a few is that cost benefit analysis, right? You need to figure out whether it’s worth investing that extra engineering time to do that test. If it’s for something, I suppose if it’s some for something in your core business logic, that is, if this breaks, we lose a million dollars. Yeah, you want to do all the testing, you want to do mutation testing, end to end integration, and all this kind of stuff. But if it’s for when, I don’t know, when the user selects, I want my profile picture to have a yellow background. It’s probably not as important, right? It is important to that user, but to the overall business, not so much.

Matthew

Yeah. And I think the way that I work around that risk is that like, say, it’s my golden rule, I never, I try not to commit tests that I haven’t seen feeling for the right reason. And if you do that, when you write the test, then there’s there’s all risk that your tests are not working.

Jamie

Okay. So to tie all of this discussion together, we were talking, again, in this pre interview call, about how you’d used all of these techniques, and loads more and all of these principles and loads more to build a project you’re working on called My Booking Hub, let’s just sort of quickly talk about how they all sort of fit together when you were building that what my booking of is, and just discuss that for a few minutes. So I guess first off, what is My Booking Hub? And then secondly, you talked about all of these different architecture patterns, and they’ll never destined buttons, did you use all of them? Or was it more a case of, hey, I know about this thing, but we’re not going to use it because it’s not relevant.

Matthew

Yeah. So so My Booking Hub this is kind of, I’ve got a book out there, isn’t it? So picking up is our booking platform that we’ve built. And it’s there’s lots of booking platforms out there, but kind of USP is for mobile services. And so it’s about people who deliver services to customers at their home or their office, or wherever that location might be. And what we do is we use mapping software, to understand where the customer wants an appointment, and where the service providers currently have appointments booked and where their home base is. And then we offer the customer slots that they can have their desired appointment on, based on that. And we do that in an efficient way, which is kind of a win win win, where the customer feels like they’re doing something good for the environment, the service providers got a dense diary, and they’ve got lots of appointments, and also they’re driving less miles. So like, say, that’s good, that’s good for good for the planet. We kind of build that in partner with with one of our customers and a core validating service, fresh, fresh core of our team. And we’re looking to roll it out to to other service providers. So electric charge point and solves Believe it or not one of our kind of potential markets. Yeah, so it’s, there’s a web website in the film, the show notes. So if people want to check us out, and you can, you can click there and find out about or get in touch. And on the second point, so how was My Booking Hub? What what does it use in terms of all these ideas we’ve talked about? I think the key one was the clean architecture, kind of structure, project structure. So I kind of borrowed a lot like I say, from, from Jason Taylor, is clean architecture and set up the kind of three layers which he he describes the domain layer, which has zero dependencies, and it’s this crunchy business logic we talked about that is complex and important in itself, the infrastructure layer talks to the external world. So it connects to the database, it sends emails, handles, HTTP over rest. And then the application there fills in the gaps or hooks up the domain to infrastructure. And so that was that was kind of bedrock that we decided on very early on in My Booking Hub, and stood the test of time, when we’ve had new features come in to be developed. I’ve never had too much difficulty and understanding how it breaks down into those parts of the application into the architecture. For instance, the the algorithm that kind of crunchy but it does the clever appointment suggesting and one of the problems with the existing system, when we rewrote it was that was quite strongly integrated with the database and so to test that you had to hook up to a real database to write any tests against it. But with clean architecture, it became obvious that the Making it zero dependencies is, is what we need to do. And that would go in the domain. And so by making the algorithm zero dependency makes it super easy to unit test, you can just fire in models at it without having to worry about database connection and give us high coverage in that. We’ve tested all the edge cases and the complexity of that. So that was that was a really important decision. And I think it’s allowed us to understand going forward when we’ve continued to make changes to the algorithm and improve it.

Jamie

Excellent. Okay. With all that said, if you were to give some advice to someone who wanted to create, either create from scratch or rewrite a system, using some of the techniques we’ve discussed today, the different types of architecture, whether it’s clean executor, monolithic microservices, whatever, and all the different types of testing we’ve talked about, and all the different things that you’ve talked about with my booking up, do you have maybe two or three pieces of advice you’d give to someone who’s looking to do that to sort of start from scratch? And are some of those pieces of advice? Go watch this video, go read this book? Or is it more a case of eight? Just make sure you get the fundamentals? Right before you start was? What are some of your pieces of advice?

Matthew

Yeah, I think going and reading the Well, looking at the report from Jason Taylor, and he’s got many videos online, I think, can’t stress how excellent a resource that is. And so that’d be Advice number one. And I think the only other bit of advice I would give is, consistency over correctness. Once you’ve picked an approach, try and stick to that and don’t, don’t get lost in all, we can make this better. And we can do this thing differently. Try and set your principles early on, and then stick to them. And then concentrate on building a system using using that architecture, those principles, it’s very attractive to think, two months after you you’ve started your project and new validation library comes out or you feel you find there’s a better way to do validation. And so you start writing validation in a different way. And then people come into your project to see the same thing been done different ways. And it’s really hard to understand what’s going on. And I think something I keep on coming back to a lot recently is consistency over correctness, it doesn’t matter if it’s the best way of doing it. But if it works, and it’s the same as everything else, then people can understand that and it saves the hassle of just looking at code and thinking, I can’t understand this, I’m going to have to sit and go through this and load it all into my memory before I can start to do anything for it, the codes laid out in unexpected way. And it follows patterns. It’s so important.

Jamie

Sure, and it saves on context switching as well. All right. So if there’s a chunk of code, if there’s one of these pie slices, that does things a certain way. And then you have to you finish the work on that pie slice, maybe you’re implementing a feature or fixing a bug, and you jump to another pie slice to do some other work. And it is it is the same code base is talking to the same business logic areas, filling in the same gaps where the application area is still got those connections to the outside, or the same things are in place. But everything works completely differently. Maybe you’d like to say you’re doing the validation differently, or maybe it’s a different type of authentication. Or maybe it’s a different database for no real reason other than it’s a different database. Not like I’m not getting at, like, you know, all this bits in in ASP .NET is identity, because that’s a good fit for this project. And this bit is in what I mean just like a completely different, right, this bit of the code talks to Oracle, because the person who wrote it had an Oracle license and is safe, we’re doing Oracle, right. The problem there is like you say you’ve got the context, which when I come to work on that second chunk of code, I have to a, get an Oracle license, be installed loads of software, and then C before I can even do that, I need to understand what the codes actually doing. So you’re absolutely right, I think I think consistency is more important. Because then you’ve got almost like a common language in the team. You know, the the whole idea of domain driven design is that you, you write the I’m probably going to mangle this I’m trying to sort of reduce it a lot. You write the dough, the program, and the logics which fits the domain of the problem and uses that common design language we talked about earlier. And then if you are doing things the exact same way, almost like coding standards, using the same libraries using the same sort of setup, then all the only thing you’re switching on is the part of the problem that you’re solving. Not Oh, and this bits written in VB or this bits written in Go all this bit is written in, you know, there is scope for that in other projects. But if you’re working on a single project that that is, and I’m not trying to say that My Booking Hub it was a small project, but if you’re working on a single project that has this pie slice idea, there’s a vertical first circles get vertical, the horizontal aroma and always have to my entire life, we’ve got this vertical design idea, this pie slice idea, then you would expect jumping between the pie slices, a lot of it to be not the same, but very similarly put together, right?

Matthew

Yeah. And it’s on a really fine, fine grained level as well. It’s not just about different languages and different database tools. But, but but different naming and different structure linting on the code, I think, linting people, people will laugh. And so it’s linting, like, focus on the important promise. But I think linting can actually make a code base much more reasonable, because anything that’s different in a code base catches your eye. So in JavaScript, like semicolons are optional. But if half the files have like, or some of the lines don’t have semicolons, and some do, that catches your eye, and it doesn’t matter in 99% of the day, it catches your eye, and it distracts you from what does matter. And so I think tight coding standards are important. And developers, like maybe think it’s part of their identity is how they read code. But I think it’s one of these things, it’s really about the team, doesn’t matter how you do it. A lot of the time, it’s about having that shared way of doing it. It should be a consensus. But then consistency over correctness, you do it that way. You don’t you don’t waver from that for all, for all good reason.

Jamie

Sure, it goes back to the idea of what works for the team, right? For instance, friend of the show, Zach, he says, I’ve had my last argument of linting, I’ve had my last argument over where the braces should go, and where semi colons go. For context, he used to be a JavaScript developer, he’s now moved to Python and things like he’s had his last argument to do with that, because the linter will take care of it for him. He’s got his ID set up. So he hits Ctrl. S, and the the, the coding standards are applied for him. So he doesn’t have to remember. So if he’s, if his personal, I am writing it in the flow, coding standards are that he doesn’t put semicolons on, he knows that when he hits Ctrl S, or Ctrl, you know, bang Q w or whatever, because he’s a vim user, he knows that when he exits out of that file and saves it, the linter will step in and do the bits that, you know, put the semicolons and rearrange air, but the brackets here and put this out, and then the other, and then he doesn’t have to think about it. He’s done the hard part of translating his thought into the syntax for the language. And then the tool comes along and sort of pretty fires it. I don’t want to I don’t want to reduce that. But he puts it into the right coding standards for that, that that team that he’s working alongside of. And I think that’s a it’s a very important thing to mention. I think things like editor config and linters are so important. And, you know, most of the IDS hook into these things and will do it all for you. And so I think is a great idea. I think you’re right, I think consistency, not just like you say not just the libraries we use the way we write it, but actually how the code looks when it’s loaded into your IDE. When you’re looking at the characters. That’s as important to like it.

Matthew

You say prettify, but that’s actually the name of the tool, I call them to prettify the name of the ticket was pretty funny on save, we actually put that in the front end recently. So that so that we get that function is it’s just modern tools like devs, doing their tooling for devs. And it’s all beneficial.

Jamie

Yeah, excellent. I like it, because then it’s similar to oh, gosh, what’s it called, like our Babel and things like that. So you can write whatever version of JavaScript you want. And it will get transpiled down to a version that you know, you’re targeting ie So you ready with the error syntax? I’ll take care of the difficult bit of doing all of that wiring up for you. So you don’t have to worry about it. And yeah, it’s the tools, if the entire team writes with the arrow syntax, but yeah, but then you have to target IE, you’re going to have problems. But if you have a tool that sits in between sort of halfway, fantastic, you’ve got that consistency, it doesn’t matter if it’s correct or not. You’ve got that consistency. I like it. So, Matthew, right, as we come towards the wrapping up, how can people get in touch with you? What was the the URL for My Booking Hub again? And do you have any sort of top resources you recommend? I know, you said a lot. You’d say go watch our Jason Taylor’s videos and read some of his posts. Are there any others? You know, like anything? Just like, let’s help spread some knowledge, you know?

Matthew

Yeah, for sure. So I don’t actually it’s one of the things that local renter a website, which people can contact me by LinkedIn, find me message and be more than happy to, to elaborate on any of the things we raised or just hook up with people. Yeah. Jason Taylor. His GitHub handle is Jason Taylor Dev, slash clean architecture. Hopefully that’ll be in the show notes as well. thing a completely different style of resource but one I always go back to is this website called the morning brew by Chris Alcock and it’s just a really good distillation of things that be happening. Kind of .NET, Microsoft focused with a little bit of React and Angular. And every weekday he posts around 10 articles that just give you a flavor of what the most recent things are. And anyone in the office who’s like, oh, they need to, firstly, .NET is getting released this week. How do they know that? One way you can do that is just who can, the more improved,

Jamie

I have to say I used to use that quite a lot. I read on the train into work, and be on the phone, read through a few articles, bookmark a few get into work where you still work. As soon as we hit slack, right? Guys, you got to read these three articles, because they were amazing. And none of it is related to what you’re doing. But hey, check these out. This is where the the industry is going. Or, hey, you know, there’s this cool new feature coming to Xamarin or whatever. It’s a it’s a wonderful resource. It really is. So yeah, I can I can recommend that too.

Matthew

Yeah, I’ll give you a beta of the links in the show notes.

Jamie

Oh, absolutely. Yeah, everything we talked about will be in the show notes. So people don’t have to make this joke about. So you don’t have to if you’re driving into work, he doesn’t dive over the dashboard to hit the pause button. So you remember the time code for when you send it or anything like that. Everything is in the show notes. So don’t worry about that. But yes, I guess I guess Matthew, all that really remains to say is to thank you ever so much for being on the show. And and thank you for sharing your knowledge. And I hope that lots of people go check out my book and go and get in touch with you on LinkedIn.

Matthew

Thanks very much, Jimmy. It’s it’s been a pleasure always always like talking tech.

Jamie

Excellent. Excellent. Well, thank you ever so much.

The above is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository

Wrapping Up

That was my interview with Matthew Jeorrett about the Clean Architecture and Subcutaneous Testing. Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at dotnetcore.show, and there will be a link directly to them in your podcatcher.

And don’t forget to spread the word, leave a rating or review on your podcatcher of choice - head over to dotnetcore.show/subscribe for ways to do that - reach out via out contact page, and to come back next time for more .NET goodness.

I will see you again real soon. See you later folks.

Useful Links

• CodeClan
• Clear Sky Logic
• My Booking Hub
• Jason Taylor
• Jason Taylor’s GitHub Clean Architecture repo
• Matthew on LinkedIn
• The Morning Brew
• The Shrimp and Crits Podcast