S05E09 1h 06mSeason 5

Episode Transcription
Hello everyone and welcome to THE .NET Core Podcast. An award-winning podcast where we reach into the core of the .NET technology stack and, with the help of the .NET community, present you with the information that you need in order to grok the many moving parts of one of the biggest cross-platform, multi-application frameworks on the planet.
I am your host, Jamie “GaProgMan” Taylor. In this episode, I talked with Patrick Smacchia about the nDepend project, why he and his team started working on it, and just how important it is to keep a handle on the dependencies that your application has.
Along the way we talked about code metrics, cyclomatic dependency, and ways to progress as a junior developer.
So let’s sit back, open up a terminal, type in dotnet new podcast and let the show begin.
The following is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository
Jamie
Patrick
Jamie
Patrick
Jamie
Patrick
I did a few large bank and there is a lot of IT in these guys and also some iOS speciaal I did a few large company and very recurring problem they had and they still have these guys they have very large legacy. Like for example by now .NET is more than 20 years old so some company have some 20 years old legacy to cope with; enormous with dozens of people, a lot of turnover, millions of line of code. And what I noticed is that they are a real problem understanding what’s happening, what was happening they are the real problem to making it evolve they have a lot of very large monolithic piece of code, some untested code. So I’m sure to the professional developer listening to me that I talk to them because this is how software is happening because nobody… no environment is really perfect you cannot control the turnover et cetera.
So in a world almost everyone has a problem with code quality and at the point came to me the idea to try to build a tool it was not the idea to make a professional tool at all it was just an open source project hobby which I called nDepend, which started in 2004 - in April 2004. And at first I was interested in Robert Cecil Martin code metyri - so for those who don’t know maybe you know him as Uncle Bob So he’s a famous figure in our industry that wrote very interesting books. So Robert Martin did created some interesting metrics about understanding how companies should interact especially he is also at the seed of SOLID principles. So all this was very interesting and what was really interesting is that you can compute metrics from the source code.
So at that time it was kind of innovative reflection and actually at first I used reflection, .NET reflection to get some information from the code. So it was just released as an open source project it was interesting for me to use it in my professional environment as a consultant; it was interesting to try to see where things could be improved. And I put it on the web and then I figured out that a lot of developers were in need of such tooling; and a lot of people were talking to me, “I would like this feature,” “I would like this feature,” et cetera, et cetera.
So came to me the idea to create a business out of it, mostly to be almost 100% on it. Because it was very interesting all this year because the thing is that you don’t cut that much when you are a junior you are learning a lot, you are fixing bugds et cetera, but you are not creating. So it was a bit frustrating for me it was no question that I would be happy, more happy doing my project the way I want. Exactly in the condition I want.
So nDepend started to be a professional tool in 2007, and since then, like, thousands of clients registered and the team grew and sorts of things are evolving. And what is extremely evolving a lot is .NET. So every tool vendor around like JetBrains people and others, we are working hard to cope with the Microsoft reason because for a long time the .NET Framework was slightly evolving. But suddenly I think in 2016-17 they start to move everything on the .NET Core platform and also the language team are doing amazing thing in creating new version of C# pretty every year. So I have to say it’s a bit hard to run after all this. It’s challenging but it’s also rewarding and it’s interesting for us because this is also a condition to make the net platform grow all this new stuff. Today I think we can say that C# is at the top of what you can expect in 2022 in terms of [an] object oriented language, and same for the .NET platform. So I don’t complain. I don’t complain. More and more projects are started in .NET which means also more user for us, which is very good. So yeah, this is a very great time to work hard on .NET and to be part of it and as the last almost 20 years I’m on it and maybe also when did you start .NET?
Jamie
Patrick
Jamie
Patrick
So concerning nDepend we were talking. It’s a static analyzer, so it’s a .NET static analyzer. And I have to say that something very big happened in the middle. I mean big in the static analysis. in.NET it’s the Roslyn compiler that reach ga in Visual Studio 2015.
Okay, so basically I assume everybody knows what is Roslyn. So basically it’s an open source compiler, for C# proposed by Microsoft. And they were smart enough to introduce in it the analysis part. So basically all the warning you get when you are compiling, they give you the possibility to generate your own warning. So by writing your own rule, which is of course excellent. So Microsoft proposes hundreds of rules and they are the tools with their own rules. And there is also the Resharpoer engine, which is very good. And then there is the nDepend engine, which is different, which is not based on Roslyn and hopefully for nDepend there is room for it because it does things differently. So basically nDepend is scanning your code so everything you give it to him it can be source code, DLL, PDB code coverage data, everything. So you try to analyze everything and create a snapshot view of your code base. And then the idea is that a rule is a query against the snapshot. Okay? So you can create query. There are like 200 queries by default.
So what’s interesting is that in this snapshot, contrary to Roslyn, you have very high level way to approach very high level to approach dependency. For example, if you want to know dependency between a type and a namespace or if you want to know also code coverage or so nDepend can compare to snapshot. For example, a query we run every day on nDepend for example, “is telling me the classes that were 100% tested, covered by test and that are not any more 100%,” which is not something that could be easy with Roslyn but it’s extremely easy with NDP.
This is the kind of stuff nDepend does very well and also all the finding all the entangling code: “tell me which monolithic part of where can I find all the monolithic stuff and also give me advices on what can I do to try to make it a bit more componentized.” I like this one to levelize things. I mean to have a top down approach and not everything not a big ball of mud but just a nice, directed graph. Okay, so did this kind of thing nDepend also for example, when you compare you want things like API breaking change for example. So tell me what used to be public that is not public anymore or tell me the interfaces that change it because maybe my user implementing it and then they’re going to be broken or all this stuff. Also some naming. So of course we have tons of naming conventions but it’s very easy with nDepend to create your own naming convention because actually what I did not say is that query is a C# inquiry because the model exposed by nDepend from the snapshot is of course a /NET API model. So you can query query it.
And also something that makes it different is that it’s very, very fast. Because nDepend is by no means better. But that was in is different because everything I said would take time to unwind. Because on a very large application with like 10,000 source files, maybe it will take a few minutes. And nDepend, it would take a few seconds because nDepend doesn’t store in his model. All the very tiny details you get in Roslyn.
So Roslyn is excellent at doing things like maybe this prereference will be new so maybe you should do something about it or maybe you’re going to fix that kind of exception here. So Rosen is excellent at finding bird or maybe this line is not needed anymore, et cetera. These things are not approached by nDepend but instead the other thing. So that’s why the tools are kind of complementary and actually we use it in our own shop bus approach with success. This is very interesting to be able to scan very large code base very quickly to not lose time. This is something extremely important when you’re developer to not lose time. This is why you need the best IDE - which can be discussed actually - this is why you need the best hardware; and recently I wrote a blog post about how to make your visual studio build faster. So that’s interesting because there are quite a few things that can be done there are also some extension to visualize your build. So all this to say that don’t lose your time, okay? If you want to have that kind of advice. I talked about nDepend earlier, quickly, in a matter of seconds and inside your IDE, then you can try nDepend and it can be very fast on it, but you won’t get all the tiny detail that you get with Resharper and Roslyn. I have to say they are very good because typically as a developer, you only work with one or a few subset of source code. So for those NGI, they are very good at this subset because this is the one you get in front of you. Why do they nDepend will get more 10,000ft view of problems.
Something also that makes it very very different is that there are two things there are technical debt estimation, okay, and there is a focus on new problem. So I think both are very interesting. So first, for the focus on the new problem, as I said earlier, the snapshot are compared nDepend. Compare it so it can run all the rules on it. And so it knows the issue that were introduced between the snapshot, the issues that were fixed and then now that were interesting because typically it’s when you are changing your code that you are introducing new problems and nDepend, live inside your IDE, will tell you about this very new problem. Because when you run a static analyzer or the large code base, you will get literally thousands of issue. Any Resharper, Roslyn, and nDepend, you will get thousands of issue. But typically, you will edit. Maybe in a day, maybe like 15. Source Five, you will introduce some new mistakes. And this is the one you should fix now before going back, before leaving work. This is the one to fix/adapt.
So this introduces a kind of priority in the issue and nDepend brings you that while other engine you open the source code. And it’s very good at showing you the problems in the actual source code, but you have no idea if the problem are new, so it’s hard to prioritize. So of course you can say as soon as I open the source fire we fix everything but this is not practicable so something needs to be prioritized - sorry for the accent.
And the second thing is about technical debt estimation. So technical debt is an analogy. It’s an analogy with the financial debt. So typically, we work hard. We want to quickly get the job done, but at the end of the day, you will get all this little problem could be monolithic, could be untested code, all this stuff. And this is what Fowler called technical debt because you introduce some debt in your code and as every debt, you need to be reimbursed. Because if it’s not reimbursed, which means you don’t care for code quality, you will end up with poor quality code. And the problem with poor quality code is that it’s hardly maintainable. So here we are in the maintainability side, and we all worked now in the past on fully maintainable code and we all know how such a nightmare it is to fix [a] bug, to add new feature here we are typically on the problem I mentioned I saw in very large corporation they all have or windows very large code base with tons of technical debt. So the idea is that technical debt, you can put a number on this debt, so you can put an estimation.
Because as everybody knows, if you cannot measure it, you cannot make a progress on it. So first you need to measure. So the right way to measure the technical debt is the human time to fix it. Okay. So it nDepoend attempts to put an estimation for each issue there are a few other tools that are doing it. What makes nDepend different is that because the rule is a C# LINQ query then the technical debt estimation is just a formula. You can put a formula in your LINQ query. So for example, I give it as an example you have a method or a class that is not 100% covered that you’d like to cover it 100%. So nDepend is going to have a look at the size of the class. It’s going to have a look at the complexity of the size in terms of
if, case, switch, et cetera else all this stuff. So he’s going to analyze all these steps which is called psychrometric complexity and he will look what is covered, what is not covered, he’s going to look at what are you calling? Because if you are calling interfaces this is easier to mock than if you are calling, for example, some implementation. So you’re going to look at everything and try to infer formal formula an amount of time you will have to put to write tests and that’s satisfied that several users are happy with this kind of estimation. For example, for the rule that you should test your code you can get an estimation like to test this class maybe you’re going to spend 3 hours or three days and then the manager can also do the math to try to compute an amount of spend in terms of Dollar, or Euro, or GDP. So there are conversion also in nDepend to help the manager. So this worked quite well and this is an interesting way to measure how close you are to perfect quality to the modern way or at least in terms of nDepend rules.So yeah, I think it was interesting to really compare what nDepend compared to Roslyn because everybody is working in Roslyn and not everybody is using nDepend. So I think you get a point of what makes it different.
A Request To You All
If you’re enjoying this show, would you mind sharing it with a colleague? Check your podcatcher for a link to show notes, which has an embedded player within it and a transcription and all that stuff, and share that link with them. I’d really appreciate it if you could indeed share the show.
But if you’d like other ways to support it, you could:
- Leave a rating or review on your podcatcher of choice
- Head over to dotnetcore.show/review for ways to do that
- Consider buying the show a coffee
- The BuyMeACoffee link is available on each episode’s show notes page
- This is a one-off financial support option
- Become a patron
- This is a monthly subscription-based financial support option
- And a link to that is included on each episode’s show notes page as well
I would love it if you would share the show with a friend or colleague or leave a rating or review. The other options are completely up to you, and are not required at all to continue enjoying the show.
Anyway, let’s get back to it.
Jamie
Because that’s a difficult thing. I always worry when I’m talking to clients, customers, users, whatever. When I say, “yeah, I can fix it, but it will take this long,” or when I say, “hey, I’ve done this. I’ve implemented this feature, but I’ve had to accumulate a bunch of tech debt along the way.” Because to non developers, tech debt is one of those esoteric things, right? Look, you’ve done it, right? If you haven’t done it properly, then why have you said you’ve done it? Whereas people who aren’t perhaps developers or haven’t come from that area, that direction, they maybe don’t know that sometimes you have to do things cut corners, I guess. Maybe cut corners is the wrong phrase, but you have to do things in a way in which you need to remember we’ve got to come back, we’ve got to fix this. Or like you said, maybe there’s some code that is just fragile, right?
Patrick
Jamie
Patrick
Okay, we have a UI project which most of dependency goes to the value WPF. But now that we are migrating the UI, that in two steps. First, the UI will work on WinForm and the WPF on the .NET 7 or 9 or whatever desktop. And second we’ll use a new technology, but for that we have to wait until Microsoft took a decision to adopt a technology. So we don’t know which technology yet. So this is a multiphase migration and then we can see, and everybody can see that the maintainability of the software is really coupled with the dependency. If you depend largely on this and suddenly you cannot depend on this anymore. For example, WinForm and WPF are not working on MacOS and you want to work to work on MacOS or even in the browser. But then this is where hey, the assumption I had about dependency earlier are not valid anymore.
So one advice about this migration we figure out that .NET Standard is actually very actually our main target. Our main target is to get as much gone. And I mean, we end up with 90% of con compatible against .NET standard 2.0, which means that there are a lot of new fancy classes like
DateOnly, TimeOnly or whatever. The new fancy classes in the nDepend premier, we can live with that or we can even embed it because they are open source. But what I mean is that we are restricting us on this because we don’t know exactly what the suture we build meant. We don’t know maybe if nDepend will run in a browser like five years from now or whatever.So now we choose that to have most of our code and when I say, “most of our code,” it’s also the third party code will depend on for example we are using monosyside to analyze the bytecon, the Ielcon and hopefully monoch side is compatible about the .NET Standard. So typically we want all our code to be compatible with .NET standard which is another way to see the clean code phenomenon which is another term coined still by Uncle Bob, the guy I mentioned at the beginning. So the idea is really look at your dependency. So what Uncle Bob say? It’s about the domain. Really your domain should depend on no infrastructure. So you don’t want database in your model, you don’t want UI, you don’t want network access or whatever, you don’t want all of this, you don’t want any infrastructure. So I see the idea of depending on the .NET Standard 2.0 is exactly the same, no infrastructure or almost no actually you have the older five system et cetera underneath standard, but then also the network. But it’s a way to really restrict your dependency in a way to be really maintainable in the future. Even though Microsoft has a very solid direction, seven, upcoming eight, nine, et cetera. I guess it’s here for good, it’s here to stay. But at a point in our business, in your business it would be interesting to add your project to run, I don’t know, I don’t know something here. It’s very interesting, important for us to control our dependency and this is what we’re actually doing now and we are preparing the future for that.
Jamie
Patrick
Jamie
Patrick
Jamie
Deserialize<T> Right? But what they didn’t realize, because it wasn’t really highlighted to people until Blazor became a thing, that NewtonSoft.Json is almost twelve megabytes of NuGet packages. Right?Patrick
Jamie
Patrick
Also some wide range of feature I didn’t mention yet is all the graphical aspect. It’s all the code visualization things because we talked about the dependency. But it’s interesting to visualize dependency. So there is really three ways to approach the dependency with nDepend. First, there is a dependency graph and I’m very proud of it. I mean, we worked very hard and I think I think is the top notch you can get in the industry by now in terms of dependency visualization.
It can, it can work for example on like the 15,000 classes of the .NET core. It can analyze and you can visualize it with zero slowdown on standard hardware. For example, just to compare with what you can get to some other tool. So even on very large code base it will let you visualize all your dependencies. It will also navigate. There is a lot of navigation things and search and you want to see the coupling between two components or you want to see a code graph. Like from this method I can reach, what can I reach from there? So all the dependency needs you can think of typically they are unlimited. Here what I Like is that we are really dog fooding a lot our model and we are really spending a lot of time in nDepend every day to analyze nDepend and do. Especially now that we are doing all this migration thing, this improvement things that we are doing or reorganizing our dependency. I can see how useful this kind of tool are there.
So you have the graph, you have a dependency matrix that historically was the right way for us to see the dependency because the reason is that dependency matrix is scaling. So for those that don’t know what is a dependency matrix, you know what is the matrix. There are some rows, there are some columns. So you get some component, say some classes namespace or assemblies method, whatever in rows and the same for column. And then you get the sale and interest sale is non empty. It means that there is a dependency. So the dependency can be from row to column, column to row or both or empty. So they are the four. So I allow you to see and at first we used a lot this way to realize con because it scales. Because the problem with the graph is that if you have say 1,000 plots typically on most graphs it will be totally unbeatable what we did with nDepend. And because the graph has been related two years ago during the COVID crisis actually. So it was a very productive lockdown. What happened is that we found a way to scale with the Graph. There are several ideas that you can get graph inside the box of larger graph so you can get nested graphs.
We are also a cornering shame. So we use several visual approach and typically you can visualize any large piece of code and you can go from top down, you can zoom in. So I think that was the innovation we brought two years ago that cannot be found in any other project to really scale with the graph. But now it means that you have very polished dependency metrics and a very polished graph. And the third approach to cope with dependency code query I mentioned earlier, like we can do some rules, but what’s interesting is that you can do anything with code query.
So for example, you can plot a metric, for example, you can plot the percentage coverage of a component. You can just write a little code query that returns color and then you can plot this color across time and then you get some trend which is another feature nDepend. But you can also query the dependency. You can ask for a class, tell me which class depends on me and also the class that depend on those one and also the class, et cetera. So you can construct a call graph. This is interesting because the graph and the matrix are generating the code query. So most of the time you won’t have to write the code query because they are generated for you. So you have the code query results, which is very optimized. Again, it can scale millions of line of code, et cetera because it depends the different approach. And you can work with the result and the graph at the same time, which is very useful when you are doing large scale refactoring or let’s say for example of a large monolithic code base. So it’s very useful to have all these tools working together and visualize things.
And also when you progress with the tool you can write your own code query. And for example which classes in this component is using the WPF because I don’t want any WPF anymore. And then tell me which methods or classes of the WPF are used and how the amount of usage, the amount of coupling, et cetera. You can write your own query to do that and then you can redefine I have a query, I have a result and my task will be done when the result will be zero.
Jamie
Patrick
Jamie
Patrick
FullCoverage attributes which mean that by design this class should be 100% covered. And then you can write a rule that says tell me which one target is full covered but not 100% covered. For example and this is very interesting because actually I didn’t talk about test much, which is another very important aspect when you’d want to do the right things.And at the very origin the technical test metaphor was all code not covered by test not tested. Because covered is not enough, it must be tested as well. Okay? It’s not about enough to be just at the very beginning, but then you can also put things like API breaking, change and all this kind of problem I mentioned earlier to put in your debt to extend it. But what’s interesting is that when I analyze the code base, the .NET code base fully covered, I can see that 70% of the debt is is from there, from from the few rules about poor code coverage. And when I look at my technical debt estimation, I can see nDepend smart enough to really see there you have your biggest problem is that your code is not covered.
Something I would like to mention: so I said about coverage and also about testing. So this is different thing. Coverage, it means that the line exercise and the line is tested when you make sure that the results obtained from the line is the results you want. Okay? That’s two different things. There is something I never saw anywhere in the industry but in our shop. It’s natural. So I would like to say that typically you get the assertion only in your test code, okay? Asset display is not new, et cetera. But few developers are using assertion in the code itself. And I think it’s a drag because in the code itself. This is really where you get the results. Of course, the results. It’s not like in the test where you get some input and you expect some input, but so many can be done with assertion in the code itself. So we wrote our own small framework where we get both our assertion in code and in test. It means that we have so many more assertion exercise when we are running all our tests. For 14,000 tests we have so many more assertion, millions. Literally, we are using nUnit, it’s I think 13 million or something like that, assertion exercise. And sometimes we can see the assertion in code are violated and sometimes it’s in test. But it’s not just in test. It’s really another way to make sure you could to find the bug early to avoid regression. This is not related to nDepend because independence is not an assertion framework or whatever. But this is something I don’t see enough in the industry.
And also, for example, another topic which is important, which is UI testing. It’s very hard to test the UI. Like I click this button and I get this text in the text box. It’s very hard to do. You have to have some framework, but it’s a lot of work and it’s toddy fragile. Suddenly the button becomes a regular button and then you’re stuck. But if you can pilot your UI and then exercise all the assertion inside your UI code, then you have a way to both cover your UI code and also exercise enough. Maybe not everything as you would get in click this button, et cetera.
But still we have a few tests like that. They are very long, they are like 37 tests. Because for example, for the graph we test, I have a small YouTube video. I can communicate you so you can share with listener. You can see all the graphs working alone, we don’t say anything and then we verify. So much fun in the UI without maybe not make sure that this pixel is gray and this one is white, or this text box. This is a different approach. But this works because when we introduce bug in our UI, which happens often because this is how the software works, it’s hard, but then it’s automatically detected this way and it means not much extra work for us to test our UI and to be confident that we didn’t introduce breaking change in it. This is just things I wanted to discuss because we are doing these things a bit differently and this works. And this saves us a lot of time and a lot of problem with some Is users.
Jamie
So I guess as we start to wrap up here, Patrick, because we’ve been talking for going on an hour, and I’m aware you said before we started recording, you had a bit of a cough going on. Let’s not carry on too far. I don’t want you to wake up tomorrow and not be able to talk because of me.
Patrick
Jamie
Patrick
Then you get the Build Machine edition. So the Build Machine edition, you put it in your CICD, then you can get reports. So the report, it can get some graph, it can get some technical debt estimation. It also contains all the issue things, all the issue rules are passed, et cetera. So this one works on macOS, Windows and Linux. Okay? So this was the first migration phase that happened actually last year. So we have all our non UI code already in .NET Standard and this is how we can run it on Windows upon .NET Framework, but also on Linux above the .NET Core. Okay?
And the third SKU is Azure DevOps So we have a full fledged hub. It’s not just a report, it’s much more interactive than a report. So if you are using Azure DevOps, you can also try this edition and see if it fits your needs.
So it’s really the way you want to use it. Do you want to use it in your IDE, in your Azure DevOps, in your CICD?
We are now working on GitHub action. Okay? So we’ll also have a GitHub action edition to be in GitHub. I’m not sure when this show will be released. So maybe at the time you listen to show, maybe the GitHub action is already there. So just have a check. So of course just go to nDepend.com. You can try all the SKU from the download page. Okay. For the Azure. DevOps, you need to go to the marketplace. Azure DevOps. There is also a free trial, full feature but time limited.
I think it’s interesting to just download it, scan your code and see what the tool can say. Work for a few days, scan again your code and compare it against the baseline and see the difference you will have. And especially in terms of issue and technical debt because as nDepend can say about the new issue and the fixed issue. Of course you can also compute the new amount of technical debt introduced or fixed. So you can measure if you go in the right direction or the wrong direction and you can plot trend, et cetera, as I said earlier.
So yeah, just try it. Promise. It’s very easy to install, it’s just XCOPY. I like XCOPY. I don’t see why since our users are developers, I don’t see why they would need to do something else that just zip and let go. So it’s very easy. As I said, the dependency are something very important for us. So the registered view table is very lightweight, like 13 megabytes. I know some flashlight, application weights, more than that so you get all this feature.
Also, this is really related to the minimalist approach I talked earlier. Like also do more with less software alike. It’s really important to control your dependency. It’s really important to control your amount of code.
Okay? When I see some redistributable like one gigabytes, I can’t understand that maybe for Visual Studio, but for some other projects I have had time to why the hell 1GB of my computer should be dedicated to this application that don’t do that much. I like for example the 4K demo. Some guys are doing extremely nice demo, only 4 KB. So I like this approach of being minimalist. And this is also related to all the performance things because typically if you are carving with your code to be small, there are high chance that you are carrying for your run time the amount of memory your application is consuming. Certainly it will be small as well because somehow it’s related. I like to care about the size and software because unfortunately we don’t get extra CPU cycle, we get extra cores nowadays, like I have 16 cores but I don’t have more than like four or 5 GHz. We won’t get more than 5 GHz. So I like the idea to be minimalist and do the best to code, like if you were in the don’t do that much if it’s not absolutely necessary.
This is close to what we were talking before.
Jamie
So I agree with you completely about let’s control our dependencies. Let’s make the simplest code possible because at the end of the day, whatever code I write, I have to support and me tomorrow is a different person to me today, right? So technically it’s another completely different person who has to support the code. So let’s do that.
Patrick
It goes very quickly, which means it’s good for our future. But yeah, it just works. If you follow all these advices, it really works.
Jamie
Patrick
Jamie
Patrick
The above is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository
Wrapping Up
That was my interview with Patrick Smacchia. Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at dotnetcore.show, and there will be a link directly to them in your podcatcher.
And don’t forget to spread the word, leave a rating or review on your podcatcher of choice - head over to dotnetcore.show/review for ways to do that - reach out via our contact page, and to come back next time for more .NET goodness.
I will see you again real soon. See you later folks.