Episode 103 - Software Architecture with Paul Michaels

So first thing I’d like to say, Paul is thank you ever so much for being on the show again, because this is second appearance. My goodness, you’re part of a very small elite group of people. I think there’s only five or six people who’ve been on the show twice. And I think there’s only two who’ve been on the show three times. So you know, you’re geting there.

Thanks for having me. Yeah. Do I get a gold watch or something? I’ve been on like, five or six times.

I mean, maybe it might be a plastic one. But yeah, we can organise that.

Oh, my goodness. Okay. Yeah. For for the folks who’ve maybe not heard your previous episode, or are new to you and the experience that is Paul, I was wondering, would you mind giving us a really quick sort of overview, maybe an elevator pitch about, you know, the kind of work you do and all that kind of stuff?

Yeah, for sure. And so I’m head of development company called Music Magpie at the minute. And we, you might have heard of the company, they sort of, in the what’s known as the recommerce space. So they, they sort of buy and sell stuff. Various things, CDs, DVDs, phones, whatever you’ve got. And I’m also a Microsoft MVP. As are you, recently. I also in my spare time, run an online group called mini hack, which is a sort of an online hackathon inside of two hours, which you’ve kindly come and judged for us on on occasion. And I also write a blog when I’m not doing that. So that’s pretty much me in about five sentences.

Fantastic. You’re a fellow “who cares about free time in person” then?

Exactly.

I do remember the the mini hack that I was that I judged, it was amazing. One of the teams took like a sound file, converted it to a bunch of numbers, then converted those numbers into Japanese. And then back again and got the original, almost the original sound file out. It was amazing.

Yeah, we have. So I come up with these ideas that I think we’ll come up with a thing to get people to do X. And then we give it to them. And then nobody ever does x. So they always come up with something that’s different, but in most cases, better than I kind of anticipated, because it’s just the weird and wonderful things. And because it’s like it, you know, it’s not, it’s not like, you know, if you get paid to sort of develop software, then somebody comes up to you. And they say, “well, I want you to write this software, and it’s gotta,” I don’t know, sell widgets, or whatever. So you’ve got to write software that sells widgets, at mini hack you haven’t gotta do that. The brief is as vague as we can make it. At the end of it. If you just come up with something that’s just a laugh or a joke or whatever, everyone just laughs and jokes. It’s not you know, nobody cares if you’re gonna don’t make it; and half the time people do not win because of just like the funniest thing.

Yeah, like I say is that it was an absolute joy to watch because just listening to these teams talking about, “oh, we’ll do this and we’ll do that and then,” wasn’t the one of the teams did something with colour on that mini hack I was involved with? Where they like the change the sound into colour, and then? I can’t remember. But it was all amazing. Was what it was.

Yeah, I think there was some very strange, very strange things. And yeah, every time it says this, the same idea. So I think we’ve been trying for a while to get people to do. Because I had decided I was going to use like security. So I thought we would do OWASP top 10. It took two or three attempts to get somebody to actually do security every other time it was like they were just kind of circumvent it and do stuff that was just fun. So in the end, we’re gonna like, right, you’ve got to actually do something that’s on the OWASP, top 10. And even then, we ended it with some very strange things. We had a guy that was judging it was been on the board or something that the OWASP board and he sort of one of the team produced a SQL injection attack, because we said, you know, write something that’s vulnerable, and he was like our kudos for, for actually doing that. Because these days, it is not easy to write something that SQL injection vulnerable because you basically fight in the software from every, every level. So yeah, it’s a it’s interesting. It’s interesting to run an interesting to watch.

Excellent. So yeah, I mean, my personal experience, I think all the listeners should check it out, because it is tonnes of fun. And like I said, all I did was watch. But yeah. And it is, like you said, it’s, it’s getting people to exercise those creative muscles to just like, solve a silly problem in a silly way or whatever. It’s brilliant. I like it.

So yeah, let’s talk - you see, you mentioned earlier on, you got this new book out. Let’s talk about this new book, right? Because last time you were on the show, you had a new book. This time, you got another new book. Yoiu’re like author extraordinaire, over here, let’s talk, let’s talk about this. You know all the words and arrange them in a pleasing manner.

So when I was a kid, I always wanted to be an author, or I always wanted to write fiction books. So maybe one day I’ll write a fiction book. But yeah, so when I suppose the last couple of places that I’ve worked at has made me think quite a lot about architecture. And specifically with I mean, I liked writing that first book, but one of the things I didn’t like was the fact that it was when when you’re writing about software that doesn’t exist yet. And at the time, it was like, I think .NET Core 3.1 didn’t exist at the time I was writing it. That is really frustrating because you write a chapter and then you finish the chapter, and Microsoft have changed it. Like not just changed it a bit, everything has changed, like right from the ground book. So you basically got to delete you chapter and start again.

So I liked writing the book. I didn’t like that, because it was frustrating. So when I said if a write something else, and I’m going to do a, I’m going to do like a topic that doesn’t have a time and date associated with it. So I thought I’ll try and write someone architecture. And I had a look around, and I didn’t see anything where there was a sort of a book that gave you some principles, and then gave you some examples about how to use them. So pitched it to Apress, and they said, “Yeah, we’re up for that.” And then and then sort of started the journey. And it took, I think, they were saying, “can you sort of get it written in six months?” And then when I finished laughing, it ended up taking around about 18 months in the end. But they were quite patient with me.

Awesome. Yeah. Yeah. Like, “can you write a book in six months?” I’m like, “dude, I can’t even write a JIRA ticket in six months, what you’re talking about?”

No one can write a JIRA ticket. In any amount of time.

No, of course. So let’s talk a little bit about like, the stuff in the book then. Right? Because like, like you said, like, software architecture itself is like, it’s not really it’s not really pinned to a certain date or time or technology. Right. So, I mean, well, so first of all, what’s the technology in the book? Is it is it .NET? I guess it is, right?

Yeah, it is. Because I’m not much of a - what’s the term polyglot? I’m not much of a polyglot. So I kind of stick to the couple of languages that I’m familiar with. And given that one of the languages that I’m vaguely familiar with the JavaScript, then obviously, I’m going to stay at stay away from that, you’ll get a lot of complaints from me saying that I can’t, I can’t deal with non static languages anymore. I’m too old.

The way I see it is, if it’s not for you, it’s not for you, you know. I’m not one of these who’s like, “you must write it in this language, or I’m gonna, you know,” and I try to, you know, there are other shows, shall we say, who spend a lot of time making fun of other languages. And I’m like, “where does that get us? It doesn’t get us anywhere. And secondly, all of you jokes are trite because you’ve already told them before,” you know what I mean? So like, let’s just move on, right? If somebody wants to write something in Python, or JavaScript or go or rust or whatever, then let them you know. What’s the problem.

Yeah, I agree.

But anyway, so that yeah, I stuck to .NET because it’s what I know. And, yeah, so it’s written in that. The interesting that - I think we were talking about this before we started recording - is thatiIf you stand still long enough in IT, then everything just kind of comes around. So it’s quite interesting with software architecture, because the sort of stuff that was considered in some cases used, you know, in the 70s. And 80s. Has come around again, and, and is, is now relevant again, because the the reason that it wasn’t relevant at the time is because of things that don’t, don’t matter anymore.

So if you take something like event sourcing to the concept of you store everything that happens to a system since its inception. But if you went back, like, I don’t know, 30 or 40 years and suggested that to one of the guys programming away in sort of, you know, C, or assembler, or COBOL, or whatever it was at the time, and his database and his hard drive that’s got like three meg[a bytes], and that’s like the most that money can buy. Then you said, “we’re going to store everything, and we’re not going to normalise anything, we’re just going to store all the data. Since the time the system started.” They go, “okay, and now he’s paying for it, you know, where are you getting these, these millions and billions of pounds or whatever? You know, where are you going to store all this in some kind of massive warehouse with hard drives.” But, obviously, these days harddrive space is incredibly cheap. And so that, that conversation changes, but it’s the same. It’s the same idea of it, nothing’s really changed. It’s just that the sort of landscape that you’re in is changed.

So it’s interesting to kind of think about other stuff that was previously kind of discounted, because of the constraints that existed, those constraints have gone out. So we’re in a different landscape, and you can go back. And so you can write an event source system, and you can store every event has ever happened to our system since its inception. And it’s, yeah, it’s a fascinating, fascinating place to be. Because, you know, when you do that, you end up with a whole new set of problems. And it gives you things like that give you a lot, but it’s kind of a theme that I kept on coming back to when I was investigating architecture that, you know, there’s no free lunch. So every time you every time you come up with some kind of solution, whatever it is, you pay for it. And in some cases, you pay quite high price. So you know, if you want to go down one of these routes, and then make sure you know what the price you pay in is. Because it can be prohibitively expensive in other ways.

Yeah, I do like the… gaining that breadth of knowledge of different systems gives you the idea, the experience to actually go, “this is not right for us. This is right for us,” right? Because, like you said, you know, the standard sort of CRUD app that everyone built, right, “put something in the database; update the thing that’s in the database; create a new thing in the database; get me the thing from the database; and now delete it,” right? Well, you know, normally we’d use something like SQL with an SQL style database, maybe SQL Server, maybe Maria dB, maybe Postgres or whatever. And you store normalised data - that’s data is partitioned across tables - you know, and it has keys everywhere. And that’s how it relates, like relational data. But sometimes that’s not the right thing. And sometimes, not storing all of these events, is not the right thing, if that makes us sometimes storing every single event that has ever happened to the system make sense. Like, maybe not for an e-commerce website, perhaps or maybe not for it to do, right, storing every single event that happens in it to do probably not the greatest idea. But maybe e-commerce is great for this. Maybe recommerce is great for this, right? Because you need to know who did what and when, and how it changed the system, right?

Yes. I suppose the the the thing is that what you just said is absolutely right, that it’s the keyis not to over engineering, I suppose. So, you know, if if your to do solution, and that’s the thing you need. So, you know, in our business, we provide solutions to problems, right. So if your problem is, “I need a way of listing the tasks that I’ve gotta complete in a day.” And the the quickest and simplest answer to that is, “here’s an A4 sheet of paper and a pen.” Then that cost you like I don’t know, three or four quid and you’re good to go. And the alternative is you spend two or three weeks writing and app that then you’ve got the problems that an app’s got: you’ve got to pay for it to actually be produced or, you know, maybe you’ve got to pay financially, maybe you got pay in time, whatever it is.

If the fastest thing is is a manual process, then go with the manual process, because half the time, the manual process kind of works. And I mean, one of the exercises that that is sort of, I think it’s quite useful to go through is, when you’re considering what it is, you know, what, what requirements the customer’s got whoever that may be. If you sort of go through and say, “well, how would I do this manually? What would it take? And at what point does that become infeasible to kind of continue to do it manually?” So you know, if you’ve got to do app, you know here is an a4 sheet of paper? And what is it take you right now? Two minutes, okay, right, good. But we’ve got whatever it is 500,000 People who all want to write to do up Okay, now, how long does it take? You know, it takes me three years, okay, then you probably need a computer. That’s, I think that’s a, an interesting method, a good a good process to follow to sort of go through and sort of think, you know, maybe thinking through in your head, or maybe actually sort of talk to a customer about what what would you do if you had to solve this problem manually.

And for things like I mean, you know, just going back to event sourcing. Event sourcing is a manual, you know, we started as a manual system, and then accounting predates IT by thousands of years. And accounting is an event sourcing systen: you write down, “this is what I sold, “on a piece of paper, and then you might keep a balance, but you write down each individual transaction. So yeah, it’s a I think that’s, that’s quite a useful way of thinking about things. And not over engineering things is a bit of an art, I suppose.

Oh totally. I was talking to someone just today actually, about the fact that, you know, you can do conference driven development, right. Where you go to like - as we record right now, the date of recording NDC is happening, right - And it will be the first day of the talks are happening today, I guess, because they do like a couple of days of courses to begin with. And you can legitimately make a load of money by doing conference driven development. And by that, I mean, you pay 3000 pounds to go to a conference, you furiously make loads of notes on things like CQRS, and domain driven design. And these are all great things. But then you shoehorn them into a solution that then takes three months longer than it should do. I’m not saying CQRS is bad. I’m not saying DDD is bad, they’re actually quite good. And they solve very important problems. But sometimes, like you say, the best solution is, “here’s a piece of paper, here’s a pen,” or “have you tried a spreadsheet?” or, you know, let’s just store all the data as a JSON blob in the database.

Yeah. And I suppose that that’s the thing. And I think if, with a lot of these problems, you know, if you need an event source system, you’re probably going to know it. Like, it’s going to be something that as you start thinking it through, it’s going to be kind of, you know, these these sort of things, you know. The same with sort of microservices and so on, which are another one that are quite popular a minute, if you if you need a micro service architecture, then you’re going to be feeling the pain, and you’ll know that you need it to solve our pain. And, you know if you’re not feeling the pain, if you can - I can’t remember, I think it was Dylan Beattie usI saw do a talk once and he said, “every system architecture is box-box-cylinder.” If your system works as box-box-cylinder, do it. It’s so much easier. It’s just you know, most of the time now, certainly with .NET, but I imagine that most modern things: File -> New and you’ve got your two boxes already. And you don’t have to write a line of code. Just so you kno, 20 minutes of mouse click in and you’ve got your box-box-cylinder. there. And if that works, do that. I mean, there’s no point in writing a microservice architecture that’s going to take you months and months and you know, all the problems that that brings with it not that you know, certain points they’re not worth addressing. But you know, most people don’t have them problems.

Absolutely. I mean, I’m very much a case of solve the immediate problem. Because then you’ll find out that the problem isn’t the problem is something else. Right? I’ve often said that the customer, client user, whoever doesn’t know what they want to the given what they don’t want. And I feel like too much work can be spent on engineering, the most elegant possible solution using all of the principles - that people take to be laws, but principles. You know likw, “you must do it like this.” Well, actually, it’s more of a suggestion, you know. I’m not saying SOLID is wrong. I’m not saying CUPID is better. I’m not saying don’t program to interfaces, but sometimes, you know, it’s like. Which will solve the problem faster? Spending weeks - or rather, maybe - spending days building interfaces to interfaces, to interfaces on the off chance that you swap something out, or using new to glue your system together because you need to get out the door today? Which is going to solve the problem, right?

Yeah. And, I mean, obviously, there’s, there’s considerations around that. And I suppose it’s a bit different if you if you sort of messing about any bedroom, as opposed to, you know, actually going and getting paid for doing some of these things. But, I mean, I saw a sort of talk recently, that sort of similar sort of vein on they were, I think it was ThoughtWorks, they were saying that using SPA frameworks, by default, is something that you should be very wary of doing. So, you know, if a server side rendered, ASP NET MVC app works, use that. Don’t be messing about with React unless you need React for a reason. I like React, it’s a, it’s one of it’s a nice, it’s a nice client side framework. But it again, it’s not, it’s not free to use it, it comes with comes with complexities and and you sort of introduce a load of problems that you’ve then got a solve. And at some point, I think it’s worth backing off and going, “oh hang on. Have I actually created this problem for myself? Did I need to?”

Yeah, I suppose the same could be said about, you know, using ASP .NET, or using Django in Python, or using whatever go uses, right? You need to weigh up the pros and cons, right? You need to think, “how long is it legitimately going to take me to build this? Not back of the envelope style conversation. But like, how long will it take for me to build this system? And how much investment does that represent? And then how much like support, right?” Because like, I think it’s amazing that people use like brand new technology as soon as it comes out to build a real life product. But then, like, what do you do when it changes?

With JavaScript you get that, like three or four libraries a day isn’t it. It’s crazy.

Just in the time, we’ve been talking, there’s been a thousand new JavaScript libraries created then destroyed.

But yeah, now that we’ve finished sort of making fun of all the other architects out there, we’re making things a little bit too complex. What I really like about event sourcing, right, is the idea that you can literally rewind through time, right? Because it represents all of the changes that were made to the system since day zero, you can actually, like you can like an accountant can write, they can look back through their log of what has changed. And they can say, “ah, right, I can see where I’ve made the mistake. I carried a one here or I’ve put too many decimal points in,” or something. And then you just fix that one thing. And then magically, everything else sort of flutters into existence and everything sort of just fixes itself. But what other… so, is the book just event sourcing? Or are there other things going on in there?

No. So, I mean, I’ve covered a few other basically, it’s a kind of, it’s a list of problems, you know, we just we just talked about the to do problem, which is a well known and well solved problem - everybody needs to do and everybody’s written one. But it covers, I’m sort of interested in some of the other sort of ways of solving problems. So I think what I’ve done each each chapter is I’ve said, “well, here’s a specific,” I’m going to use quotes here, but nobody can see outside of you. So for anyone listening up just on the universal sign for inverted commas. Basically, it’s a problem that for every chapter, so I’ve made up a problem for every chapter.

And one of the interesting ones was a sort of travel app, if you like. Because one of the things that I was interested in was how travel agents - or anyone who wants to sort of deal with third parties, but in a single transaction - kind of manages it. And so I investigated the idea of a sort of multi step or multi phase distributed transaction. So effectively. You know, the idea if you’ve ever been on Expedia, or you’ve been on these kinds of companies, where you’ll say, “I want to whatever it is a hotel, a plane, a car,” and it goes off and it goes, “okay, I booked your hotel, plane, and the car.” it goes, I can’t get the hotel plane card at this time, what it doesn’t do is it don’t come back and go, “I’ve got a car, but you can’t have the hotel or the plane.” Because obviously, you know, there’s no point in that. And it’s interesting, when you look at the sort of way to do that in technology, it’s very difficult. Because obviously, if you’re, you know, transaction on a database, if you’re on your local database, you can, you can have an ACID transaction, because you can say, “I’m going to sort of stage this entire set of changes to the database, and then either the go through it or don’t go through.” Obviously, you can’t do that if you’ve got to go to one system and say, “okay, make this change,” and then go to another system and say, “make this change,” and the second systems says, “well, I can’t make that change.” Now what do you do? Because you’re not like, you can, you can go back. So I sort of investigate ways that you can, you can get around that. I mean, essentially, this is this, the ways that you get around it, you either you either persuade the people in who you’re interfacing with, to, to allow you to sort of stage the transaction in a sort of traditional distributed transaction. Or you basically book it and then cancel it afterwards.

But yeah, I found I found the whole thing interesting. And the solution that I came up with, I ended up using Service Bus as a transaction coordinator. Which sort of, I don’t know what, I don’t know how useful that would be in a sort of real life scenario. But it was pretty cool to write the proof of concept.

I like it, you see that seems like it’s a non trivial, like you say, there’s a non trivial problem to solve, right? You got these three separate services, and you can book or buy or purchase or whatever, something from each of them. But each one of those requires the other one to go through. And that is not something you can like, it’s not something you could do in ACID, right? You can’t. It’s not even like it’s sharding the same database, because you’re talking to different ones, right? It’s just like, part of me is going, “how in the heck would I do that?”

You get a similar sort of product. So when you go to like a sort of what you’d call an event driven architecture I suppose or microservice, however you want to sort of phrase it. It’s a similar sort of process. And so if you’re storing your state across multiple points in your system, and you’ve got to deal with the fact that at some point, you’re going to have a difference. And, you know, you may say the system is going to eventually catch up and sort of be be consistent across. But you know, you might not be a guarantee that and so you’ve got to come up with a strategy of well, how do you deal with that? How would you deal with the fact that, you know, this database is x and the other database is y? Is it x or is it y? And how do I know which, which, you know - in some cases, it might not matter. In fact, in most cases, it probably doesn’t. But if you’re writing a system to manage a nuclear power plant or sending a rocket to the moon or something, then it probably doesn’t matter really, really badly.

Yeah. And I think that’s that’s the difference right? There is a - and I don’t wish to upset any travel agents who are listening or people who are planning on going on holiday. This is almost like a scale of importance, right? If you are writing In an event driven system for a nuclear power plant, or for an aeroplane or for a rocket going to the moon or something, then consistency really, really matters. But I feel like there must be some like acceptable tolerance for travel agents or whatever. And again, I’m not I’m not making fun of those folks are the people who build those systems. I’m just like, you have to have some kind of tolerance, right? Otherwise, you get to spend your life pulling was left of your hair out going, “why is it that it just doesn’t work?”

Yeah, it is fairly tolerant. And I suspect that there’s there’s very few nuclear power plants that are distributed systems. I think most of them probably were in in VB6 or something.

Maybe.

I was just gonna say I’m sure I heard. I can’t remember where I heard it. But so when in America there was an I don’t think it was a nuclear power plant. I think it might have been a water treatment plant or something. And they got hacked. And the way that they got hacked was they had one of the control machines set on with a LogMeIn or something like that, and a weak password. And the only way they spotted it was they saw the mouse moving across the screen.

Oh right.

I can’t remember where I heard about it. But it might have been .NET Rocks or something like that. But yeah, it was. It just just surprised me a little bit. But apparently, it was a - I don’t know why, because I don’t really understand about water treatment plants. But there’s some chemicals you can put in a water that would poison the supply chain. And these are on dials and this machine. So you know it is apparently a dangerous problem that they had. But yeah, just reminded me when we’re talking about like.

So what you’re saying is working from home is not always the best solution.

Well, not if the keys to the kingdom are at a machine in the office, and the only way of stopping people breaking in is watching it.

Goodness. I’ve been, I’ve been on a real security kick for the last like five, six years just learning about like app security, development, security. And there’s this wonderful story, if you allow me to sort of go off on this little tangent for a little minute.

Terry Pratchett wrote in one of his - I think it’s a book called slip of the keyboard, which is a collection of some of his sort of newspaper articles from before he was an author, when he was a journalist. And he said, The great thing about nuclear power plants is they’re incredibly secure. Right up until you hire John, the electrician, who needs to run a wire from this room into the next room. And rather than running a wire out the door, around the corner, and back into the other room, he drills a hole through the wall that passes the wire through; not realising that that wall is there to protect any radiation getting out of that room. It’s wonderful like allegory for any kind of security ever. Right? You can build the most secure system possible. But the problem is that if you build it securely, then it’s inconvenient for the users of the system. If it’s a software system, if it’s a building, if it’s whatever, right. The more secure is, though, the less convenient they will be. Right?

Think of like every time you go into a building, and you’ve got to, like maybe you’ve got a fob or a card, you want to bip against a sensor, or you got to type in a pin number to open each door. Right? If you forget your fob or your PIN number, that’s horribly inconvenient, right? Because then you’ve got to explain to your boss, “you got to let me in. I forgotten my fob,” or whatever. So then people invent ways of getting around it. People will hold the door open, people put a brick in the way of the door so it doesn’t close. Right? Because they’re sick and tired of hearing about Dave, the engineer who keeps forgetting his password or whatever.

And yeah, so some of the world’s greatest security has always been beaten by some of the almost like the most low low level of not even a hack. It’s just like, This just makes it easier for me to do this, right.

Yep. I might my cehck that out by Terry Pratchett. I need to I need to re-read some Terry Pratchett actually.

Yeah. Well, I’m a big fan, and I don’t keep it quiet. I’m a big fan. But yeah, slip of the keyboard that says journalism stuff. And that’s where that story exists. I forget which story it is, but it’s one of them and it’s really good. But yeah.

Okay, so we’ve talked about distributed transactions. We’ve talked about event sourcing. Which leads us slightly to event driven architecture, right? So, like, I’m used to, let’s say, I’m used to doing simple CRUD apps, and I’ve got maybe a monolith is an nTier monolith. I click a button it goes, you know, passes a message to a service; service talks to the database; message goes into the database, we’re done. Right? Give me this thing out of it. It goes, user interface; service; database; interface. Yeah, brilliant not a problem. That’s not event driven is it? Because like event driven is like, I want a thing; go get me a thing, and I’ll happily wait to hear that you got thing, right.

Yeah, essentially. So what would… the problem that I think you’re trying to solve with, when you sort of implement something like that is to try to make something horizontally scalable. So if you’ve got your sort of CRUD app that you’ve just described, that’s probably fine. And if you’ve got, if you’ve got 10, 15, 100 people using that, it’s probably fine. When you get into massive scale, and then you’ve got to have a way of, essentially, because obviously, you’re your bottleneck, there is your database, or it’s your service, it probably that probably ends up being your database. At some point, there’s, there’s about one of them things that are bottleneck. So if you can replicate the service and the database that people are using, then you can obviously increase the number of people using it.

So basically, the way that people tend to do that, or the way that that has been done, is that when when logic travels through a system, you just bung a message somewhere, and you say, the next thing that needs to happen is this, you know, this, this next thing or, you know, probably do it the other way, you probably say I’ve just done X, I’ve just done y, and then the next thing kind of picks it up itself. But yeah the advantage you get is sort of scalability, the obvious disadvantage that you get there is complexity. So it’s a massive trade off that you pay, you know if you’re going from a simple single database and a simple service that calls that database, and you go to a system where you’ve got several databases and many services and sort of Message Broker in the middle. You’ve just increased the complexity of the system by a hundredfold. And slowed it down a little bit probably, because obviously, it’s a lot faster to just deal with a single message and send back the data somebody has asked for instead of sort of sending it around a distributed system and then return it.

So yeah, I mean, again, it’s a fascinating architectural sort of mechanism. But and, you know, in some cases it can be so you don’t have to obviously with with these things, you don’t have to take everything, you don’t have to sort of read the various, you know, books by the people who sort of suggest these things and then take everything that you say. But what you can do is you can you can pick and choose. So you might decide that actually, what I need is not sort of horizontally scaled system, what I actually need is some way to stop my existing system from crashing. So maybe all I need is a message queue. Or, you know, maybe I just need to slow people down. So maybe you can - I think Cloudflare did like a way like a waiting room now. In fact, I saw, I actually went to Grand National this year, and when I tried to get onto my my bookie site after a bit it started showing me the Cloudflare waiting room thing to say you can’t get in. [But] if that solves your problem, and then that’s probably a hell of a lot cheaper than messing about with the introducing a Message Broker and so on. And I suppose it’s starting to sound like I’m talking down on these architectures and saying they’re not necessary and they are necessary, but they’re not always necessary for everything.

That’s the key thing. Yeah. Yeah. It’s like, what’s the thing that Hanselman always says, “when all you’ve got a hammer, everything looks like a nail.” And so, if you only ever do event driven architectures, then all you’ll ever built, like you’ll make to do the event driven architecture, right? You’ll have to do app which talks to a message queue, which has a dead letter queue, and an active and an un-active queue and, like a sharded database. And, you know, have you ever seen there’s a Java repo, and it’s brilliant. It’s enterprise HelloWorld. And it’s like, 50,000 lines of code, just to print HelloWorld on screen. Like, that’s the kind of thing I mean, right? You over engineer your application, but then there are times when that fits.

Yeah, and there’s times when it’s definitely necessary.

I suppose the one of the things that I suppose it’s, it’s worth mentioning is the sort of concept of a minimum viable product and what that means. And I suppose the thing about that is, if you’ve got a set of requirements, and these are the things that you need, in order to run your business, or whatever it is, That’s what you would call your sort of minimum viable product. Once you’ve satisfied all of those criteria, what you’re saying is, you can now sort of, essentially run your business or do your thing, based on the system that you’ve got. So anything you do on top of that, is not necessary for you to run your business. So there is a question of why are you doing it? Because every time you change - and, you know, again, this is something that I suppose people forget - every time you change a piece of code, that’s risky, right? I call it “code churn”; it’s risky. So it doesn’t matter how good a programmer you are, you’re going to make a mistake at some point. So every time you change a line of code, by definition, you’re changing the behaviour of the system. And unless you, you know, like some absolute, god of programming, and even then, in front of probably mostly then, you’re gonna make a mistake, you’re going to not see something, you’re going to not realise something. And at some point, something’s going to break.

So once you’ve got your MVP, if you if you are still changing your system, then you should consider what the benefit of that is versus the risk, because there is a risk to break in things that are currently working. So yeah, if you’re just changing it for the sake of it then certainly, if you’re ripping out a framework and putting another framework in, and at the end of it, the functionality is going to be exactly the same, because it’s a newer framework, or whatever it is, or then, yeah, you’ve got to ask yourself, why are you doing that? And then who’s paying for it?

Yeah, definitely. And that all goes down to that wonderful skill that we definitely, definitely do all have right of requirements gathering. Because if you don’t know what the requirements are, then you don’t know whether you’ve satisfied them, right?

Yeah, exactly.

I feel like that’s a skill that like devs, and BAs, and clients and everyone needs to have: what do I actually need? Or indeed, what are they actually need, right? And that’s a non trivial task, right? Because it all revolves around communication and empathy, right. And I think those are the two hardest things to get right in development. Because, you know, there’s that stereotype of, we’re all nerds, and we all hate people. And we all like sitting in our little cubicles, put the hoodie up to block out the world, put the soundproof headphones on and just sit there and just hack away. But, you know, we’re a lot more complex that - well, I’d like to think that we’re a lot more complex than that. But whether we are or not, we still have to deal with other people, right? And being able to communicate and get, what is it that you want me to do, right? At the end of the day, that is what we, you know, we solve the problem, but we need to know what the problem is to solve.

Yeah, and finding that problem is quite often hard because you get this system where you, you ask somebody what is it the problem that you’ve got, and they start, that you know, typically you’ll get a effectively a solution so I suppose it’s that sort of the same thing is when you go to the doctor and it like, “what’s wrong with you?” “I don’t know what’s wrong with me that’s why I’m here.”

“My hurts when I do this.” “Well, don’t do that, then.”

But yeah, but it’s kind of human nature. And so if you, if you’ve got a problem, you kind of try and solve it, and then then you extend that. And it might be that you’re actually solving a problem that you don’t have, or you’ve identified a problem has been a problem that doesn’t exist. And that’s a really, you know, we have job titles that people’s job is specifically to sort of tease that out, you know; business analyst or whatever to try and work out exactly what, what it is that that somebody wants, and whether or not what they’re asking for make sense for a problem we’ve got. I mean, it’s the faster horses thing in it. So Henry Ford said, “if you would have asked my customers what they want, they’d have told me they wanted faster horses.”

Yep. Yep. You know, and that’s why I always start, if I’m dealing with, you know, a new client, new customer, a new user, I say to them, “go build me a spreadsheet that does what you want it to do.” Like, it’s almost like I’m asking them to prove that there’s a problem. But it’s more a case of show me how it works, right? Show me the process, because we can both talk spreadsheet, right. So go build me a spreadsheet. And then we can talk spreadsheet together about you know, the charts that you want and the data you need to take in. And the special formula that you don’t realise you’re applying the like, you’ll say to me as magic numbers and to me as magic numbers, but do you really mean something. And so build me that spreadsheet or walk me through a physical like you said it right? At what point does this physical activity need to be computed? Walk me through the physical activity of you writing your to do list or you’re writing a report or whatever, walk me through that. And then I’ll be able to extract from the things that I can digitise or that I can programmatically do for you. And then we’ll both be at at that better stage. I feel like this is a little bit… I mentioned earlier on and it wasn’t to make fun of it or anything. But I think that’s what DDD really solves. It’s like, “talk to me in your language. And I’ll talk to you in your language, or we’ll just sort it out, right?” Because I often find that it’s the it’s speaking in metaphors and acronyms and things like that, that just, that’s where the requirements get muddy. I think, at least in my opinion, anyway, maybe I’m hanging around with the wrong kind of customers. I don’t know.

Yeah, I used to work in a place that did sort of gaming gambling software, then. Yeah, a lot of that. A lot of that stuff was it takes a while to get your head around it. I mean, there’s there’s bet types that, you know. I’m sort of a seasoned gambler, if you’re like, I do a bit of gambling, and I often, you know, most of my life but yeah, there’s that type like, you know, the you can obviously the bookies want you to be at a bet on somebody. It’s a foregone conclusion. So don’t read about football rarely, but you know, if like Manchester United are playing like Yeovil town, or something, they have various types of handicaps. They have European handicap, and Asian handicap and all this and the rules are really, really complicated and trying to sort of translate that into into, well, basically first into English because it’s like, the it’s a series of like, basically complicated maths. And then translate into code is, yeah quite a interesting task. So yeah, I see what you mean about the sort of ubiquitous language.

Yeah. And I feel like that, that’ll help. Like, my personal opinion, is DDD and ubiquitous language is a wonderful thing. I feel like it’s more important than most of the principles like SOLID and stuff. I’ve made fun of SOLD in this episode, but I genuinely feel like the way we deal with humans is probably more important, or slightly more important than the way that we do the minutiae of the individual lines of code. Because like I’m being super vague, I’m saying, like I’m saying um, I’m saying kind of, I’m saying that a lot, right. Whereas, if you wanted to try and take what I’ve said and digitise it, you can’t, because I’m fumbling around and not finding the right words. But if we’re speaking the same ubiquitous language, you’re speaking the language of the business. It’s not easy to start with, but it starts to become super trivial. Instead of talking about null reference exceptions, and packages that are missing, and build pipelines that are failing, you start to communicate with the client or your user or whoever, in the language that they understand: it’s broke. I need to go fix it. Right? They don’t care about how it’s broke, they just care that is broke.

Exactly. It’s broke, make it less broke.

Yeah right. And that’s me greatly reducing BDD down to like, the smallest thing, right? It’s so much more than that. But that’s like the core nugget. I think that’s what’s so important about like requirements gathering, I like the idea that, you said that for each section, or each chapter of the book, there’s a problem. And, I mean, you have to do the vanilla ice, but there’s a problem and you’ve got to go solve it, right? I may have just been spending 50 minutes moving all the way up to that pun, I do apologise.

But like, you know, are these the ways to solve those problems, you know, event driven architecture, event sourcing and distributed transactions? Or are these just, you’ve thought of these problems, and they could be solved with these different architectures? Like are there other architectures in the book? Are you just focusing on these?

Yes, what I’ve done is basically add a list of potential sort of avenues that that you could go down. And obviously, that’s not an exhaustive list, because there’s hundreds.

So I’ll start off with how would you solve this manually? Just to try and do the requirements gathering bit, you know, what, what exactly as it were trying to do? And how would you do it? And then I go,“okay, well, we could use this technology, or we could use that technology, and how would that work?” Let’s talk that through a bit. And a couple of times, obviously, intentionally, I kinda go down to a dead end, to sort of discuss a piece of technology. I thought, you know, I want to get this in. So would this work in this situation? Well, it probably wouldn’t. But let’s talk it through and see why it wouldn’t. But would part of it work? Or could it be adapted? And so on and so forth? So yeah, I mean, that’s basically the idea to sort of just play it through.

In the same way, as you know, you would have worked with some pretty talented then solutions architects in sort of recent years, and, you know, sort of the way that they would analyse and that, suppose that that job title gives you a bit of space, I suppose. Because if you’re the average software engineer, and somebody says, “okay, here’s a system, build a system,” what you’re essentially doing is you’re architecting the system as your going. If you think about that, in sort of building a house scenario, which I come back to quite frequently, it’d be essentially delivering a load of bricks to a brickie and going, “okay, start start building the house, but while you’re building it, you know, where are the windows gonna be? Decide that and, and work out where the door is, and the the, what he called the lentils, all that kind of stuff, you know, what was loadbearing?” I don’t know much about architecture, like physical architectures on a lot of this. I’m just saying words.

But the point is that everyone would think that was ridiculous. But don’t necessarily think it’s ridiculous when you’re dealing with a software engineer. So quite often, you’ll give a software engineer - and in some cases, quite a junior software engineer - a system to write and theyll, kind of, you know, just start writing code, just start typing. And you’ll get something, in the same way as if he gave a brickie a big pile of bricks and say, “build me a house,” you’ll get something might not be the thing you want.

No, I agree, I agree. You need to, you do need to sit and plan out, right? Because like, exactly like you said, if you’re building a house, you don’t just start laying bricks and just let the chips fall where they may. You have to plan it out. You have to figure out how you’re going to get like the gas piped in; the water piped in; where the electricity is going to come from. And that’s even before you start architecting, the actual house, right? It’s got all of these resources coming in. And then you can start - I’m greatly reducing physical architecture here - you can start then drawing pictures of what you want the house to look like, and translating that into a technical drawing. And without that technical drawing, I very much doubt that there are many construction companies will come anywhere near you. They won’t go near you unless you’ve got an actual architect; got a technical diagram; we’ve got all everything written out to spec, we’ve got a budget, we’ve got all that stuff. And like you say, you know, some engineers are just like, “great. You want me to build a product, right? I’ve already done File New Project, right? Whilst you’ve been saying, We’ve picked you for this new project, what am I building?” You’re building that? Brilliant. I’ve already started, you know what I mean?

Yeah, and I suppose the thing is, it doesn’t unlike, you know, obviously, I found this out because the room that I’m starting at them, and it was, was an extension, so we have to get a technical architect. Obviously, a builder will not turn up unless you’ve got plans, and so on and so forth. That’s not true in software. And I don’t think you necessarily need somebody whose job title is software architect, but you do need somebody who plays that role. If if not as a job title. And, you know, it might only be box-box-cylindar, you know, it’s worth at least that decision.

I’m a big fan of things like architectural decision records for that reason. Because, well, partly for that reason, partly because I’ve got the worst memory in the world. So you know, I might decide to do something in a project and by the time I finished the project, somebody says, “why don’t you do it like that?” and I’m like, “I can’t remember.” But if you write it down in an architectural decision record, then - I suppose for anyone who’s listening, who doesn’t know, an architectural decision record is basically, most of the time it’s just a markdown file in your solution. And you just write, “I have decided to do this, I could have done this other thing but I didn’t. The reason that I did this, and not the other thing was this.” And then you just date it, and save it in your project. And then the next person who comes along can see that you’ve sort of written down, you know, “I’ve explicitly thought about my decision to use Entity Framework over dapper,” or “I’ve explicitly thought about the fact that I’m using C# over Python,” or you know, whatever decisions you make while you’re writing your project, then you just basically write them down and keep them with your code so that people can sort of see that. And then if like me, you’ve got a memory like a sort of goldfish, then by the time it comes to somebody saying, “okay, why did you do this?” then even that question that needs to be asked, because it’s kind of self documented.

So I like that idea. Because there’s been so many projects over the last maybe two or three years that I’ve worked on, where I walk into the code base and go, “ooh! What the…” and before I even start thinking, “oh, my goodness, this is terrible code.” My immediate thought is right, “I need to try and figure out what the original developers were thinking when they did this. Not as a negative, what were they thinking more like, what paths did they go down? Why did they consider doing this? So then I can understand why it’s written this way.” And I like this idea of having this sort of living documentation with your app, where you saying, “look, I’ve thought about doing this way, I’ve thought about doing it that way. And this is the only way that really makes sense, given the constraints we have right now.” And I feel like that’s the important bit the, “right now,” bit.

Yeah. There’s no right and wrong to any of this. It’s like, you know, it’s not like, I’m gonna, I’m gonna say it’s not like maths, and I’m sure there’s somebody who understands maths better than I do, will correct me on this. But you know, in maths, there’s, there’s like one way of solving a problem. And so if you get two numbers, and you need to add them together - and I’m sure this isn’t true, when you get a bit more advanced maths, but you know, I’m going to say anyway. If you’ve got two numbers, you got two and three, and you’re adding them up, then that’s always going to be five. But that’s not true with software. Because it might be that there’s three or four different ways of doing something and they might be equally valid. What matters, I suppose is that you thought about it, not that you made the - I’m doing the inverted commas thing again 0- “right decision”. It just matters that you thought about, and most, you know, that I suppose the time that it is frustrating reading someone’s code is when you don’t you kind of think, well, that wasn’t even considered. If you can say it’s being considered and somebody wrote or I decided not to do it because of this reason. It’s kind of fair cop isn’t it. And when that the worst thing about what you’ve just said is when you look at our code, and you think oh, and then you do a git log and you think, oh is me.

Yep. Yeah. I’ve done that way too many times. Oh goodness, why did I write it this way? And that’s why I tell people like I tell juniors and mentees that the person you were yesterday, at least in an engineering context is a completely different person. You’re not just leaving comments for other people who work on the code, you’re leaving comments, or you’re leaving documentation for yourself, right? Because you tomorrow are going to be different to you today. Because tomorrow, you won’t have all of the context of why you’ve just made this decision right now. Right? You’ll just have oh, I made that decision yesterday. Or some idiot made this… oh, right. It was me.

Okay, so Paul, where’s the best place for people to find out more about you? And indeed for them to get the book? Is it you just go to Amazon and type in “Paul Michaels Software Architecture” and it comes up?

Yeah, pretty much.

So to find out about me, I’m on Twitter @paul_michaels. I blog at pmichaels.net. The book in question is called “Software Architecture by Example” and I hope that if you type that into Amazon, it would come up in the top two results but who knows? I’m guessing you got some shownotes with this, or I’ll send you a link to that. And you can link to it. Yeah, that’s probably the best way of getting in touch with me. Either of them.

Cool. Okay. Awesome. Well, thanks for being on the show. Paul. I really appreciate you being on again. And best of luck with this new book. And, you know, maybe there’s another couple of books in you, I don’t know. You said at the beginning you’ve always wanted to be a fiction writer, maybe now’s your time to write a fictional tale of architecture gone wrong. Software architecture that is.

Yeah. Thanks for having me again. Yeah, I mean, I think I think I’ve said it last time, but I think this is probably the last technical book that I’ll write. But I did say that last time, so maybe it won’t be. But yeah, I would quite like to come back and talk to you about my best selling fiction novel, whnever I write it. But that might be in 10 years time.

I mean, why don’t you title your next one, “The Last Technical Book I Ever Write” by Paul Michaels. And then the one after that, “This Is Really The Last One, Honest,” and then you’ve got the beginnings of a trilogy, right?

Was at The Eagles that did the Hell Freezes Over Tour, because they said when will you get back together. And they said when hell freezes over, and then they go back together that the hell freezes over tour.

Or perhaps the Rolling Stones with their “we are retiring” tours. Like, all of them.

Every couple of years. Yeah.

Awesome. Well, like I said, Paul, thank you ever so much. It’s been great catching up with you.

Thank you