The Modern .NET Show

Episode 107 - Unstructured Data With Kirk Marple

Sponsors

Support for this episode of The Modern .NET Show comes from the following sponsors. Please take a moment to learn more about their products and services:

Please also see the full sponsor message(s) in the episode transcription for more details of their products and services, and offers exclusive to listeners of The Modern .NET Show.

Thank you to the sponsors for supporting the show.

Embedded Player

Episode 107 - Unstructured Data With Kirk Marple
The .NET Core Podcast

Episode 107 - Unstructured Data With Kirk Marple

Supporting The Show

If this episode was interesting or useful to you, please consider supporting the show with one of the above options.

Episode Transcription

Hello everyone and welcome to THE .NET Core Podcast. An award-winning podcast where we reach into the core of the .NET technology stack and, with the help of the .NET community, present you with the information that you need in order to grok the many moving parts of one of the biggest cross-platform, multi-application frameworks on the planet.

I am your host, Jamie “GaProgMan” Taylor. In this episode, I talked with Kirk Marple about unstructured data, his new product Unstruk Data, and why he chose to use .NET to build something which makes heavy use of machine learning techniques.

Along the way, Kirk shares a whole bunch of lessons and experiences from his career which goes all way back to 1994 when he started at Microsoft. He shares a number of stories and advice about things like building minimal viable products, entrepreneurism, ideas, and whether your product needs to be cross-cloud.

So let’s sit back, open up a terminal, type in dotnet new podcast and let the show begin.

The following is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository

Jamie

So first thing I’d like to say Kirk is: thank you ever so much for taking some time to speak with me and the listeners. I say this to everyone, but I genuinely mean it: I’m really appreciative of anyone who is willing to spend some time to put up with me for, you know, 90 minutes, 60 minutes, whatever. But we’re also for sharing that knowledge, right? Because I feel like the current series - the current season, sorry - of the show, is all about, “let’s share knowledge,” you know. Because every single person has a lesson to share, regardless of where you were on your journey in development. I feel like if you’re right at the beginning, if you’re, you know, maybe you know, I’ve got a few friends that are in, like the middle management stage, a few friends that are right up in retirement stage. You know, wherever you are on that journey, there is a lesson for you. And I’m sure that you have a lesson to share. So, you know, I really greatly appreciate whenever anyone spends time on the show.

Kirk

Well, thanks so much. I mean, I’m, I love the show, I listen, very often. So it’s really a pleasure to be here.

Jamie

Thank you very much, Kirk. That’s very kind of you.

So I was wondering, before we get started, would you mind giving the listeners maybe an elevator pitch introduction to Kirk the man, the myth, legend? Is that right?

Kirk

Yeah, I’d love to. So, I mean, I was going to call myself a career software developer. I mean, I got started in the early days, I mean, was, I mean, doing some crazy like Fortran and different things like that. And my first job working for a government contractor, and then quickly moved into being a C C++ programmer for years and always been in the Windows stack. I mean, for better for worse and ended up. I mean, just yeah, go through management and being a lead and do my masters and CS, and then starting companies, and still a dev. I mean, I still consider myself a dev today, I’m actively, like, coding even as as being the CEO, but I wouldn’t really have it any other way.

Jamie

Excellent, excellent. I’m very much the same. I want to be slinging code, I want to be making something right. You show me an empty IDE, and I see all the potential. I am in no way an artist, but I feel like it might be similar to when a painter or a sculptor sees the blank canvas, you know, and you’re like, “wow, if I do this, and this and this, look at what I can create.” I mean…

Kirk

That creative side is I mean, just getting in that flow. And I mean, just being able to be like, “hey, I mean, I don’t know where I’m going to be in an hour or two, but I’ll figure it out on the way.” Like, it’s kind of, I always describe it as more like sculpting. I mean, you have this kind of raw thing, and you’re just hacking away and hacking away at it, then you’re getting down to that level of detail. And I love that part. i That’s why I would have a really hard time just not coding anymore.

Jamie

Yep, yep. And I think I read in a - I’ve been reading a lot of books in and around the development space recently. And one of them was a book called Manageing Humans, and the the author they’re talking about, “what if you get into that management space have to having been an engineer or a technical person, and specifically from like, a development background?” And he says, “don’t stop coding.” He says, “don’t do it every single day as your job, but maybe find an hour on a weekend just to write, you know, look into what React is or something like that,”

Kirk

Yup.

Jamie

So the idea that he’s positing is the by keeping an active interest in it, even if you’re no longer interested in doing it, you have that sort of connection. And you can have that empathy with your direct reports, right? So when they say, “it can’t be done,” or when they say, “it’s gonna take a week,” you’re able to then go, “you know, what, you’re the experts. I’ve seen some of the stuff you’ve had to do, and I appreciate it. It’s gonna take a week, I’ll go and explain that to whoever.”

Kirk

Yeah, no, it’s I mean, that’s the thing. I don’t consider myself a front end Dev, but I’ve done enough in the past. And it was, I mean, our team is doing react right now. And I was my, my front end experience was all ASP. Net, and core. But the concepts are pretty much the same. So I mean, when they can talk about, okay, it’s, it’s this or this, or I can talk component models, I can talk templating or whatever. And but I don’t get in the mix. I mean, on the back end, I’m way more opinionated. But it’s a bit yeah, I think it’s, I mean, you got to be a bit of a generalist even to, I mean, appreciate the bits and pieces, like I just set up CDNs last week. And it’s like, I mean, it’s not a normal part of my job, per se, but it’s like, you got to have a little bit of taste of everything. I mean, how Cloud Storage were found networks were like, and then even if you’re not doing the work directly, you can sort of I mean, manage and advise better so.

Jamie

Sure, and I personally feel like that’s where the difference between, say, a mid level developer and someone who is a senior or lead or anything like that is not just the amount of years of experience, but that that breadth of experience.

Kirk

Yeah.

Jamie

I’ve played around with machine learning. Or yeah, I’ve been, like you said, I’m setting up CDNs; I’m doing DNS records, because it’s always DNS, you know, all these kinds of things. Right. And that, to me is someone who has a, like, senior level or anything like that, because my view of that is that you can’t really, I feel like you can’t really lead a team of people until you’ve had enough experience to know where the potential pitfalls are, or indeed, to say, hey, you know, we’re picking on I was picking on React, right? Oh, well, we want to use React, well, actually, Vue might fit better, or, you know, maybe we don’t even need a SPA framework at all. Some projects you don’t, you know, and that’s just having the experience of poking at these things. Allows you to make that decision. Right?

Kirk

Exactly. I mean, even in things like authentication or things like that. And it’s like, you got to do the pain. And you got to go back to tools you’re familiar with, like, we use auth0 and stuff like that. Because you’re always in the weeds with something like that. And, and a lot of its configuration, and I’ve the company, we have I mean, it’s I brought over developers that I work with it a couple of companies, and we all get we’re like, “Okay, well, we know how to solve this, because we did it the last company,” or, “we know how to solve this. We did.” And but then there’s times when you want to investigate something new, like we’re doing stuff here that we’ve never done before. And so you kind of I just let them run with the things that I know, I don’t have to question. And then the other stuff, I’ll I’ll try and sort of think about a little bit more.

Jamie

Sure. And I think maybe that’s one of the things that people - when they get to a senior level say like someone in your position - you surround yourself with those experts, right? You surround the person, surround yourself with the person who’s like, “hey, this person knows machine learning,” “hey, this person really knows Azure,” “this person really knows Docker or whatever,” and you turn to them, right? Because they are the experts. That’s why you hired them. Right?

Kirk

And that’s what Yeah, I had, like, I had prototyped, and kind of built the back end of our product before starting the company just on my own. And then I went and said, “Okay, I mean, hire the best front end ads that I know, because we need a UI.” And it’s just an area that I’m not as good at, like I have a bit of an opinion. But I and they just cranked it out. And it’s just I mean, it’s solid, and it’s functional. And the bugs we have are nothing. I mean, they’re just usually visual, and I mean, little things. But I mean, it’s for a very small team. I mean, you got to have good people around you.

Jamie

I mean, every project has bugs, right?

I’m pretty sure even “Hello, world!” fails at some point during like, a nano second… or you don’t turn your computer on is bugged out, right.

Kirk

Yup.

Jamie

So we were saying there about surrounding yourselves with experts who know the different tools and techniques you need to solve the problems that you’re solving to build the software. And I was wondering, because you’ve mentioned a few times about this product that you’re building. You were saying just before we started recording, you’re about to hit 1.0, which is amazing. So congratulations on that. But what I will say is because of the pod, whatever they used to call it, the time-cast-pod-machine-wibbly-wobbliness, we are now talking in the past. Right? So for the listeners…

Kirk

Hopefully we hopefully will have shipped by then.

Jamie

Yeah, so I was wondering, can you give us a little bit of an introduction to the software, the package, the system that you are shipping? And then I’ll ask a little bit about the technology behind it, because I got a few questions about that. If that’s ok.

Kirk

Yeah, I mean, the concept is, and so my background was in media entertainment, I’d worked in like broadcast video and software and things like that. And so we had these tools called media asset management systems, that would be like you put your video in there, you could search, you could filter, you could do processing on it. And so I built a company around that. But this is kind of a another layer of an onion skin around that where it’s not just video, it’s audio, documents, 3d geometry, point clouds, even data files. And so I kind of look at it as like a really broad media management tool that has ML built in for enriching the dataset.

So we basically handle the data ingestion side automatically, and then automatically create a knowledge graph from all the data basically the metadata that’s in the files, as well as things we extract via ML; so that would be audio transcription, computer vision, NLP for documents, and then that goes back into the graph. And then the customers can do data discovery. So essentially search on top of it, but also analytics and visualization. So you can basically everything has a geospatial element and a time element to it. So they can say, “show me all the images flown by this kind of drone, and the last six months, in this geo fenced region, where I saw building.” You could do complex queries like that. So that’s what the UI does. It’s really I mean, it’s kind of like a sort of, if you’re familiar with like Tableau, or thoughtspot, or something like that, for the structured data world, we do that for the unstructured data world.

And then the other part is automation. So alerting. So we can say, “hey, when a tag has been applied to a file, hit me on Slack,” or something like that. And so we’ll get more complex with that in the future. So we just started, that’s a new feature now where we can hit a web hook or Slack and really to connect to the business workflows. And the concept, there will be like data triage. So they might be uploading or capturing a lot of data. And they want to be like, “oh, I saw water pooling. On my maintenance run,” we can automatically flag the manager in a Slack channel, saying, “oh, look, here’s all the history of water pooling,” we can give like a daily summary of things like that.

So really, it’s a pretty big swing. I mean, it’s a data platform for, we call it for real world assets, or real world places. And it can be like aerial surveying, drone surveying construction, really anything across the board. And we really treat it as like a Knowledge Hub for the enterprise, or the companies where, because the biggest thing is there a lot of companies doing this for maybe just imagery, but we’re actually cross referencing, like a Zoom meeting, and a document, and a 3d file and things like that into this knowledge graph, which is pretty unique with it.

Jamie

I like that. There’s a couple of things that I kind of want to cover about that. So you mentioned early on about media management systems. And in my head, I have this example that I know of that the BBC have. And it’s all human curated, right? It’s all people curated. So my understanding is that the BBC, obviously, they have a huge collection of, you know, a huge library of audio, video, whatever it is that they’ve ever recorded, ever made, all that kind of stuff. And my understanding is that they have a team of researchers who catalog “oh, in this footage, at this time period, there’s a shot of some, you know, maybe some swans or there’s a lion yearning or whatever.”

Kirk

Yup.

Jamie

So if you’re making something for the BBC, and you say, “I need some library footage, it needs to be a lion, and it needs to be yawning. And it needs to be sunset.” They’ll go, “right, we’ve got you, “and they’ll go over there and find that footage. Is that is that kind of what this is?

Kirk

Yeah, I mean, the product today, you could you could do video, I mean, we would extract the scenes, one the technical side, we’re using Azure Video indexer for that. And that could that gives me a ton of metadata back of I mean, it gives emotion in the speech, it gives object identification, it’s a really powerful tool. So we take that metadata and then basically create the graph from that metadata, that the analysis metadata. And yeah, I mean, that essentially, we could do that same kind of search. And what we often say is, “the customers want to find their interesting sort of tidbits in the data,” like, and that’s really where the ML comes into play. And that’s why we have a good partner program where we can plug in any ML models from different sources. So we’ve done partnerships with Robo Flo and assembly AI and primer.ai for different algorithms they have. So we’re not really in the business of creating the ML algorithms. But we’re plugging them in just like I used to plug codecs in, in the video world, and you just licensed them from different vendors. There’s a lot of commoditization happening in ML. So we just want to be able to plug all those pipelines in, and then take the results of that, and federate that back into the knowledge graph. Because, I mean there’s vertical companies that maybe they’re in vegetation or something like that. They do a lot of computer vision models for that. If we can get access to that is an API, we can easily plug it in. I mean, it could be like a day or two’s job on our side. And just boop, like it’s right in there. We had like connectors to the different pipelines. We have rules, like set up rules to say, “for an image that you see within this geospace, run this model on it,” and that’s where it gets super powerful and we don’t have to sort of, I mean, do everything at that point, but we can just have a really good partner program to work with.

Jamie

You see that this is - in my head, I’m starting to see like, how this could be good for, like the big companies, right? Some of the people that I’ve recently worked with have been multinational enterprises, right? And when you’re on a team of five people, and there are a million people hired by the company, the amount of data that they must be generating on a daily basis through meetings, through Slack messages, through maybe Confluence, or wikis, or something like that. And like, it’s all disperate, right? You only, like you only know about the data if you were either involved in creating it, or if you’re involved in maybe editing it right.

But I can imagine that taking an entire enterprise’s, like all of its data, and just saying, “right, okay, when was the meeting held to decide when this flagship product is going to ship? And then who do we need to disseminate that to?” I can imagine that being a, from the user’s perspective, a relatively simple sort of search to do, you know: “we were talking about Project X, it was roughly within these three months,” I’m really simplifying the the problem you’re trying to solve here, I do apologize. But like, you can then extract from that maybe sentiment from that you know, “everyone was really stressed because the the 2.0 was about to drop. But we had all these, all these bugs, and then maybe an external third party, we were hoping we could work with pulled out or whatever.” So you could I could see how you could pull all of this information; seemingly is unrelated, but is related, right.

Kirk

And the other interesting part, and part we I mean, really, I had an opinion about originally is the enrichment of that data. So we may, so we kind of map to, there’s this thing called schema.org. That’s kind of a standard. It’s what JSON-LD and Google uses for their knowledge graph. But it’s just a schema for people, places, things. So we map everything to that. And so we canonicalize that data. And then we could say, “oh, the ML found a person.” And sure, it’s just it’s a term, it just would have my name, but it would it would identify it. And then what we could do is recursively, enrich that. So we could say, “oh, we see a person, go call LinkedIn API, and enrich with their with that information.” And then like almost a spidering.

And what we want to do is, I mean, we do that automatically, for places today, we call Google Places API get like, we, if you put in a term will find the lat-long, and do some things like that. And but over time, when we want to make the system pluggable, we’re gonna be like, I’m I actually had investor asked me this is like, “could I put in my Zoom meetings with my founder companies, and it could find the terms of the companies and go look them up on CrunchBase, or in our CRM, or something like that.” I was like, like, “almost.” I mean, we’re like that far away. Like, I mean, we have almost all that built, we would just have to have a way to plug in API enrichment, data enrichment. And we kind of do it, you can hit a web hook today. And then with our graphQL API, you could post us back. But we I mean, we’re looking at some other ways to do that. But yeah, it’s our goal is really to make this completely pluggable. That it doesn’t have to be the kind of verticals that we’re maybe targeting right now. The vision is like, it’s a it’s an unstructured data platform for everything. And that is really, but you gotta you gotta bite off, like a little bit of it. At the start, so, ya know, it’s, it’s really fun stuff. I mean, I, I’ve been building a podcast discovery platform, kind of based on this technologies, it’s nights and weekends. And that’s what kind of drove a lot of this kind of data enrichment and spidering concept and things like that.

Jamie

Excellent. Well, what I’ve realized so far is we haven’t really even said what the name of the product is. So let’s just quickly say that, and then I’ve got questions about the technology used to build it is

Kirk

The company name is Unstruk Data. And it’s unstructured data platform, pretty simple. So we’re, I mean, it really is treating it more like a horizontal platform, like, I mean, like a Snowflake or something like that is really what we’re aiming at. We do have a really nice UI. I mean, that is a kind of a built in analytics tool on top of that, but I mean, it’s a basically a consumption, you pay by consumption, data platform. We’re selling it like that. And, but yeah, I mean, it’s right now it’s, it’s pretty much a I mean, the company and the product are pretty much the same thing. But I mean, we do anticipate maybe over time, we’ll have some other verticals that we can spin into and make some more focused products as well.

Jamie

Excellent. Okay. So, here’s the potential sticky problem, right? There’s nothing wrong with .NET I love .NET; that’s why I started the show, right?

Kirk

Yeah.

Jamie

But why .NET?

Kirk

I mean, my team will probably like If I say something different, but it’s I mean, it’s personal preference. I mean, that was, I know, I mean, they somewhere, I always get in sort of fun arguments with some members of our team about like JavaScript vs .NET Core, and C# and things like that. But, I mean, I’ve been a C# developer for 20 years. I mean, since .NET 1.0 beta. It’s what I’m comfortable with. And it’s what I built a couple hunderd thousand lines of code in for the last five years of my life, just nights and weekends. And so we had a really big codebase, I mean, on the backend. So that was one of the big reasons it just would have made no sense to try and rewrite it in anything.

But that said, I’m also very opinionated, that I think it is really a great back end platform development platform. And then it mean, I’m, I mean, I’ll put it up against other I mean, anything else out there really, I mean. And it’s reliable, I mean, the memory management’s gotten so much better, the performance is really good. It really is. And even just the NuGet package ecosystem is really solid and the open source ecosystem, like we’ve used, and contributed to a couple of packages, for like, metadata extraction, and different things like that.

And so I had built it in .NET Frameworks originally four point something five at the time, maybe or something. And then .NET Core just wasn’t ready yet. And so it was about five years ago, maybe. And then I started to keep it updated every once in a while look at it and go, “oh, it’s time to update yet?” And finally there was a point where they hit enough of it, that I was able to port the entire platform over to .NET Core. And since then, I mean, I spent, I think it was this past Christmas, I spent going up to .NET 6 maybe - or 5 or 6 whatever it was around then - and finally got everything migrated and just pushed over. And we’re all Azure functions. And I had written this originally using Service Fabric, but then realized, since I was paying for it myself, I didn’t like paying for a cluster sitting around that I only use on the weekends. And so that at some point a couple of years ago, I ported everything to functions. And it’s been great. I mean, we use durable functions, we use durable entities. We use Event Grid and Event Hub, as well. So we’re pretty much I mean, it’s a fully managed environment. And use Azure DevOps for CI CD. So it’s a I mean, I mean, I’m a solo back end developer on most of it. And so it’s, I mean, it’s an efficient system for us.

Jamie

Sure. That’s a couple of points that I’d like to make maybe a question or two, on just that. And that’s, I love that you picked .NET, because you were already productive in it. And I think I’ve been saying this a whole bunch to juniors, mentors, to people who are not even developers, you know, I’m a bunch of discord [servers]. And everybody’s talking about, “you should have a Mac, you should have a Windows, you should have a Linux,” and I jump in and I say, “look, everyone, I had my last argument about ‘you should have’ about 15 years ago, right?”

Kirk

Yup.

Jamie

“What you should do is use the tool that you are most productive in to solve the problem that you’re wanting to solve.” And if there is nothing in, like, let’s say Linux versus Windows versus Mac, right? Well, if you say in the creative industries, and you’re working alongside people who use Photoshop, you want to be Windows or Mac, because Photoshop is available for Windows or Mac. Yes, the GNU Image Manipulation Program is compatible with it. But there will be corner cases that you can’t do. Because other people can because they’ve got the right tools. If the tools aren’t there, don’t use it. Right. But use whatever you are most productive in. Because, you know, I’m sure you’ll you’ll appreciate this Kirk: the customer, the client, whoever you’re building the software for, doesn’t care what you’ve built in.

Kirk

Yeah. They care about reliability, speed, things like that. And, I mean, there’s these days, I mean, there’s not that much of a cost difference, I guess. I mean, you could I mean, even running, I mean, like Linux containers versus Windows there. I mean, there’s definitely it’s not a trivial difference. I mean, if you’re really at scale, I got that. I mean, we’re still running on Windows Azure Functions today. I haven’t done the work to flip it over to Linux ones, but I think it should work. Generally, it’s probably more like our code, I think should work. It’s probably more packages we use, maybe aren’t compatible, but we just haven’t done that. I mean, I think if we would get to the next level where we really start to scale, we would then go in it’s it would be optimization at that point. But I mean, for us, it’s it runs well. I mean, it’s, I mean it already since we’re on .NET 6, I mean, it’s one less thing of a headache. And I mean, knowing your customers wouldn’t know. In my video transcoding company I had was all .NET I mean based and COM DLLs for the codecs and things. So I mean, we had we were doing like COM Interop and things like that, but and back in the day, and most of the other tools out there were Linux based and I mean, or Mac based, but we were, I mean, had a functional company that broadcasters use, and it was all running on .NET. So I mean, it can be done.

Jamie

Yeah. And I think that, not to upset anyone at all. You’re all wonderful developers, I’m sure. And when I meet you all you’ll be able to prove it to me, and I will believe you. But like your code, and I’m doing this visually, right? So imagine you’ve got a stack of things, everyone, imagine maybe a ladder, right? Your code is right at the top, underneath your code is the framework, if the framework isn’t very good, your code isn’t going to be very good. The framework runs on top of the operating system, the operating system is no good, the framework is going to be no good, your code is going to be no good. Doesn’t matter how much optimization you do on your code. It’s still got those at least those two layers, then underneath that there’s the infrastructure, right, assuming you’re running on bare metal. And so if the infrastructure is no good than the operating system is… you can see what I mean, right? There’s multiple levels between you and the silicon.

Kirk

I mean, one of the things I learned having my last company is, I mean, just at the storage and networking layer. I mean, so much of the optimization and performance can really come I mean, we were doing I mean video transcoding is very IO heavy, and core heavy. I mean, we were running on like, 32-64 core systems where it was all on prem, like bare metal. And I mean, I would basically have to tune everything to really I mean, just make sure we are parallelizing, and multi threading, and how much RAM are we using? And we we dealt a lot with, like NASes and SANs, and all that storage and like even dealing with like, “oh, should we be on InfiniBand,” or like different things? And honestly changes, we made it that infrastructure layer, the code would just take advantage of it. And so the code was already optimized enough, but we really could tune how much like, “okay, well, what kind of read, were they running?” and this and that.

I think, I don’t know, maybe it’s an old fogy kind of thing to say. But I think a lot of developers today, just don’t even think about at that level, like performance, tuning and just getting down to the level of I mean, it works. I mean, oh, just spin up more like more servers, like to run it on. And they’re not doing like, I grew up as a Windows developer for years. And you’ve really had to tune and do perf testing and profiling. And I think a bit of that essence has gone away. I mean, in like, a lot of the newer programming frameworks and things like that. So I tend to really, I mean, look at a great, it’s what I’m doing this week is just going through the logs on Application Insights, and a really fine tooth comb looking for, I mean, duplicate calls we’re making, and I mean, how long each how many RUs we’re taking on CosmosDB? And because that’s where the meat is, because when you scale? How are we going to get to 100 customers running 1000s of files at once, if little chunks of code are taking five seconds, not 50 milliseconds. And so, but I don’t know, I, I’m probably weird that I really liked that kind of perf tuning and testing. But, uh, but it makes a huge difference. I mean, for as you as you scale out.

Jamie

Sure. I think I think perhaps the - what’s the word I’m looking for? It’s like, it’s almost like that tantalization of just like you said, “we’ll just scale, this doesn’t matter. We’ll add another couple of CPUs to it.” Because like, if you’re in the cloud, it’s just a sliding scale. As you slide the scale, you hit Apply, and it’s almost instantaneously there.

Kirk

And you can see the dollars flow out of your wallet.

I think it’s I mean, being an entrepreneur and at a smaller scale. I mean, yeah, like how much we I mean, our cost of goods matters. I mean, for our pricing model, if it costs us 25,000 a month just to run our product. I mean, that’s how many customers, we have to get to beat our cost of goods. And, I mean, we’re maybe like, $1,000 a month right now, just to have all the lights on. But that can scale up pretty far. I mean, because we get Cosmos DB auto scaling, and we have um, you know, the Event Hub scaling, and I mean, our baseline Cost of Goods really isn’t that high. And it’s honestly like, I mean, things like App Service and media services that we use. And those are some of the bigger flat rate costs really.

Jamie

And I guess all it takes - and this is something that comes up in a lot of the user groups that I go to - some of the user groups, I go to a non technical and I do definitely recommend that developers go to non technical user groups. So I go to use to attend a WordPress technical group, sorry, a WordPress user group for non technical users. And they used to have people come in and say, “okay, so the day that your website goes viral, because you know, anything can happen the day your website goes viral and your hosting costs go through the roof. What do we do?” Right? And that’s, you know, that’s I’m gonna use the bunny quotes here, I wish not to offend anyone that’s “just” for someone who has maybe a photography blog or a current affairs blog or something, right? So I can’t even imagine with a system like yours or any of your big competitors. I can’t even imagine, like, you know, your first PR goes out, everybody hits the website, you sign a million really big customers, and then all of a sudden, up go your operating guests, right?

Kirk

Like, it’s, I mean, because we’re at the point now, where we’re giving away basically 10 terabytes of data for with our baseline. So we just launched our Azure Marketplace this last week, and working with them to get co-sell ready and do all those sales things. But once we actually get the organic signups open, everybody who signs up could use up 10 terabytes storage and all that I mean. So we got to really, I mean, think about I mean, we need the customers just to play with it, and get the feedback. But I mean, we don’t we also don’t want to, like literally last night, I just implemented storage quota. And so we can at least make sure that it can’t run too wild. And so, you know, it’s an important part of it, too. But I think you got to, I mean, there’s always that like MVP kind of pragmatism of like, “okay, well, you don’t have to do everything day one, just do it later.” Maybe that’s another old school kind of thing. Like I kind of build for performance and scale earlier. I mean, anticipate it rather than refactoring it later. And there’s always things we were going to do better later. But I mean, our baseline I think, is actually really good. And it’s I mean, it hopefully will be once we get go live. But I think it’s you got to figure out how far on that scale, you really want to be. And, sure we could have shipped something nine months ago, but it would have been crap. And we really tried to put our best face forward. But it does take longer. And I mean, you got to find that right balance for your yourself and your company.


ZOOM Platform, no not the video conferencing app, ZOOM-Platform.com! The premier DRM-Free games portal.

Unlike directionless larger stores, ZOOM operates more like a boutique offering all-time classics such as Duke Nukem, exclusive hidden gems like Hardwar and innovative indies like REKKR: Sunken Land. ZOOM will even take requests and hunt games down for you.

When ZOOM signs titles, they make sure to get every release, version, language, you name it. That’s how ZOOM carves it’s niche. ZOOM hand-scans box art, manuals and other bonus content and even records soundtracks. On top of all this, ZOOM always makes a point of securing rights to obscure expansion packs and mods. When you buy the ZOOM Platform edition of a game, you know you are getting the most complete and definitive edition.

ZOOM’s staff and technology are best in class. In fact, when competitors can’t get a game running on modern hardware, they go directly to ZOOM. They’re some of ZOOM’s best customers!

ZOOM provides unparalleled 24/7 customer service and closeness with its user base. ZOOM’s management interacts directly with customers via Discord. They take their small business approach seriously. ZOOM-Platform.com, your friendly neighborhood DRM-Free games store!


Jamie

Oh, definitely, I was talking to my friend and co-host of tabs and spaces, Zac today, about, we were doing some pair programming and writing that he was writing his function, I was providing him little feedback. And he’s like, “this doesn’t seem like the right way to do it. But we’ll do it this way, accept that technical debt and come back to it later.” Because what we were building was an MVP. And I feel like in some instances, I mean, you were saying that you prefer to build, not necessarily for stability, because everyone hopes for that. You’re building for stability and performance. But with different, you know, it’s different per industry per target audience, right? You’re hoping that your MVP can be scalable enough that when he does go viral, when a million people hit it, and a ba-billion gigaflops of data or whatever it is that you measure all this data in, gets used, you know, you can you can account for that. Whereas in the MVP that Zac and I were building, it was okay to go, “well, you know, what, we’ll just doesn’t really… we’ll shave that yak later, you know, we’ll we’ll deal with that later, we’ll just we need to get out of the door first.” Because in our instance, not in yours, but in our instance, it’s like getting into the customers hands. And then if it becomes a problem, or rather, when it becomes a problem, we’ll deal with it. And I think that those two those two viewpoints are equally as valid. I feel like maybe yours is a little bit more valid, because you have more experience than me. Right?

Kirk

No, I think it’s it even is, I mean, there are places in our product that we are we do do that. Like there’s places where I’m like, “okay, we’ll implement like one of these, like one, eighth of the functionality, but there’s a, there’s a pattern now that we’ve created, that we know how to build the other seven.” And but it’s tricky when you’re building a platform, because it’s like, if people use a platform, and it’s terrible, or it doesn’t really work. I mean, you’re they’re never going to adopt it anyway. So you and then the scale and the performance, you got to figure out what your kind of KPIs are, like your key performance metrics are and say, “okay, those are things we really care about. And then things outside that box, we just cut.” And I mean, we were going to ship a couple extra integrations in this v1. And we’re like, look, we just don’t have enough testing time. There’s a couple things we just said look, fast follow will be you got to be able to like, find that line, know the of the pattern that you can add things back later really quickly. But I mean, for me, it’s a classic, like object oriented kind of pattern based programming that as long as I mean, we really try and make sure that patterns are tight. And then we can kind of put meat on the bone later.

Jamie

Sure, and that makes sense. Right? And until you have the product in your hands in someone else’s hands you don’t know you may not necessarily know which path so Other ones to focus on, right? And that’s where like analytics, and maybe user testing comes in everyone, like everyone talks about us. Yeah, the further away from the developers fingers that go gets, the more expensive it is to fix. But also, there are just some things you can’t test until you get it into the users hands. You know, we were, we were talking about it just before off air just earlier on today about how there are just some things you can’t you can’t there’s going to be some edge case, right? Someone’s going to upload some data you’ve never seen before. And then you go, Hmm, that was interesting. Now we need to go and fix this book.

Kirk

Well, I think we might have talked about before, but it’s there was a I was a month or two ago, I found this issue with durable functions, and Event Hub triggers and something like that. And it was basically just a really weird bug. And I thought it was on me, and ended up being this really special, weird use case. And it ended up I just had to rollback the Microsoft package to the previous version and went away? And how do you anticipate that in the schedule? Like I literally blew entire weekend, probably 20 hours of fighting this thing and trying to figure it out. And then I there’s actually a GitHub issue on it. And they’re still looking into it. I mean, it’s it’s a really strange use case in their engine that I don’t know how I hit, it seemed like I was doing really normal stuff. And, yeah, I mean, that’s the kind of stuff where it’s like, I mean, we were, I was trying to get all these features done, and then literally just got sidelined. And so you got to be able to, I mean, plan to have that buffer, and just be able to roll with those kinds of things are always gonna happen. And at the worst times, too, so.

Jamie

Yeah, definitely. Yeah. I feel like some of the problems that are now obviously, they’re on a much smaller scale, right. But some of the problems I’ve had in the past are when that dependency has flipped from point zero to 1.1. You know, when you’re like, Oh, my goodness, I mean, look at .NET Core 1.0, Project, JSON, everything’s gonna be JSON, don’t worry about it. And then 1.1, which I felt was a major step one. Well, they did I remember that, go back to CES brunch. And it’s like, that’s, that’s a huge step. What are you doing?

Kirk

I was crazy. I totally, I was writing. I was thinking about porting it, and then I like, I think I’d put it on ice for a little while, just because I was like, let’s let me let this just shake out. I mean, it’s just crazy. But yeah, I’d forgotten about that. It was really interesting. But no, it’s I mean, there’s definitely things I mean, yeah, there’s a couple of like, there’s like, probably two packages in our system right now that I can’t upgrade. It just like one of them just, it’s an open source thing that just breaks something with the cosmos dB. And I just looked into it for a while. And I’m like, I have no idea. Like, I’m just gonna leave it. There version. And but that’s tech that. I mean, you gotta you gotta go back and look at it someday. But it’s also maybe I have to fork it. Maybe I have to, like, like, actually get in there. And, and we’ve had to do that. I mean, we’ve had to fork a couple things. And like some, I was just using this JSON RPC library that hadn’t been touched in two years. And I was like, oh, like, it’s the only one out there that looks any good, but ended up having to fork it and clean it up a bit. So

Jamie

sure. And I feel like there’s there’s I don’t we put it there’s an expectation, I guess, on open source with certain larger enterprises, right? Look at look at what happened earlier this year with look for J. Right. Yeah. And, and it was a it was a horrible system. It was not no another system. Let me try that again. It was a horrible situation, right? And my heart went out to all the people who have to support all those systems. But the thing that I didn’t coin that didn’t quite sit well with me, but only because I’m not in that world of a huge enterprise that requires these fixes now, was, I’d seen on like Hacker News and things. People were receiving emails directly from companies. We owe this multinational massive company, you have created this open source project, we demand that you fix this issue. Right now we stop it and you fix it. Yeah, we’re not going to pay you just fix it. And I’m like, you’ve there’s got to be some give and take. Yeah. Yeah.

Kirk

I mean, it is a weird. I mean, I don’t know, this is sort of a soapbox. But the the idea of companies being built on open source software. And I still wonder, I mean, we are we are to, but companies that build like platforms. And I mean, like, there’s not really a monetization strategy. To go back to the people that wrote it originally. I mean, I could see if I mean, people getting VC funding for some new open source project. I mean, what about all the original contributors? I mean, I don’t know how I think that’s gonna shake out probably in coming years, and maybe through GitHub and things like that, I think are some great ways. But, I mean, yeah, I mean, I would love to, I mean, have a revenue stream going back to the open source folks that we use. I mean, it’s it’s when it’s people’s time, I mean, and it saved us our developers time, like, they should get paid in some way. So I mean, I think that’s going to But we may see that shakeout Moreover, over the coming years. So

Jamie

maybe so I mean, I feel like it’s definitely a problem for smarter people than me to figure out, because I’m just an idiot, right? I’m happy to admit that.

Kirk

Now, I mean, we’ve I mean, that’s something we’ve, I mean, we’ve been donated to a couple projects, just, uh, I mean, even like, I mean, ask for things to be fixed or new features and stuff like that. But I mean, it’s, it’s, it’s a good, like, I mean, if you can work with the developers that are on these projects, and, and then they’ll get stuff done. I mean, for you like, like, if you need a fix, like a lock for J fix or those kind of things. So it’s, you got to you got to have the community and relationship side of it as well.

Jamie

Sure, sure. I think I think he, I feel like it’s a bit too much to just expect those open source devs to just go, you know, what, I want to drop out, I want to stop working on the stuff that puts the food on my table, and fix your problem, you know, and I feel like that’s a bit too much to just expect them to do. That’s the classic.

Kirk

Oh, no, I was gonna say I was listening to another podcast, they were talking about how there were some open source projects. And then once they started getting picked up by enterprises, the enterprise were like, We need to pay you. Like we need support on this. And so I see that that’s, I mean, that’s kind of how Databricks and a lot of these companies got started. But it’s, I mean, as long as you have that monetization strategy of like, okay, cool. Now we can pay for support and pay for features and things like that. It makes a lot of sense. But it’s, you got to have that structure in place.

Jamie

Sure, sure. So let’s, you’d mentioned something way at the beginning of this, this recording, and I just want to go back to it. Because it’s piqued my interest, you said that you were, you had spent some time building a podcast discovery platform with the platform that you’ve built. And I was really, could you just go into that a little bit? Because obviously, as you can tell, I’m a bit quite interested in Yeah, you know,

Kirk

stuff, I still think it’s a great idea. I mean, obviously, I’d love to, I mean, hopefully, we can take the same technology and still do this in the future. But so I had a little side company that I’ve after I sold my last company, I started working on some different things, not a little LLC that I just basically wrote code in. And the concept of I was, I was trapped, I was commuting up from LA to San Francisco every, every week for like six months, and I started to get into podcasts more, probably listen to several hours of them, I was driving. And I started realize, like, there’s so much information in the podcast that just gets lost that I forget, like, I can’t take notes while I’m driving. And I mean, like somebody mentioned a company or an article or something like that. I mean, even here, I’ve named Rob probably 10 companies already. And I was like, why couldn’t you just use audio transcription and enemy extraction to pull that out of the podcast. And I could kind of build a search engine over all the podcast data. And so I started, that’s what got me down the Knowledge Graph route. And I mean, essentially, it’s the same tack that started this company. And it worked. I mean, but the cool part about it is it could then spider to different things. So you could say, I mean, mentioned an article in a show in shownotes, because I would go to the RSS feed, pull the show notes, they’ll go grab the HTML URL, the HTTP URL, I would go in and look at a webpage, do entity extraction on that. And I essentially built a web spider. And it was really cool where I mean, I could go in, like, find albums and episodes of podcasts and people, places, things, that whole concept. And so I have this idea of what I call it back. I’m old enough to remember what Xen was like back when we’re all doing it on paper and college at Stanford. So I started this project called Xen, where it’s basically like, you could put media into a zine and create this, sort of almost like a video editor. But it was a way that you could author once and publish many as a podcast, as a video as a blog post as a tweet. So it was kind of this I guess, you call it omni channel thing where you could sort of an end, it could be dynamic. So maybe you could, it was almost like a dynamic magazine, based on the data around, I mean, where you could kind of have a podcast that would be viewable automatically as a webpage, or with all the spidered links, or you could publish it as a video with more of a image sequence or different pieces clips, you could automatically pull it in. So I mean, it’s, that was my original thing. I still think it’s a cool idea. It’s, uh, it was hard to monetize. I mean, it gives, it’s like, if you’re not Spotify, or you’re not Apple, I wasn’t sure how to do it without having a customer client. Because you need something to render it or you have to find a standard. And so I thought I could render it as just an mp3, but you need like, closed captioning links like BTT or something like that for the links. synchronous link. So I looked into that, but it was something this is a nights and weekends project. So I mean, it was that this I was kind of like noodling on this for awhile, and then had the chance to start the company, it was like, I was take the tech, built it for an industrial commercial use case. But I mean, I could literally go back and use it for the same, that same project eventually, as well.

Jamie

So that’s what I love about people doing passion projects and things that they’re interested in, right? When people come to me and say, I want to get started in development, but I don’t know what to do. I said to unlock, okay, let’s put the decision of choosing a language and a framework to one side, tell me of a problem that you have, that you wish you could do faster. And let’s just build that, right? Because yes, like a lot of people, and I’m the worst for this, I have no side projects going on, because I’m terrible at coming up with ideas. So maybe I need to listen to this episode. But I feel like if you’re able to identify a thing that you’re interested in a hobby, or problem that you want to solve whatever, that kind of thing. They build a system to do that, well guess what? You’ve just built yourself a side project. And if you’ve if you found it useful, other people will, too, you know, that’s how like, that’s That’s literally how basic was created by Bill Gates. And Microsoft started.

Kirk

It’s, I mean, it’s so true. Like my other passion project, I mean, pre pre pandemic was live music, I mean, just go to shows and stuff like that. And I had, and part of the idea I had was, okay, you. I mean, there was, I was living in Chicago, there was this great bar across the street, they would have live music, like five days a week. And I was always like, well, these are small bands. Like, I don’t know if any of them are good. I don’t know if they’re genres. Good at all, or, I mean, it’s so it’s like, how do you have this concept of kind of seeing relationships of okay, well, what other venues have they played? Like, what bands have been on the bill with them? And even then, from I mean, maybe then going down and spidering? Or what articles have they been mentioned in like a pitchfork article or on a blog page or so like that. And coming up with those correlations, essentially, it’s almost like a personalization, or recommendation engine. But that whole knowledge graph based, so I was actually thinking of building. If you ever heard of the product, the app bands in town, where it can kind of tell you what Bands are playing locally, it was like a better bands in town was the concept. was more knowledge graph based and just more detailed. But it’s still a tough thing to monetize. Like, I think it’s one of those like, solution in search. I mean, I didn’t the problems there. But it’s like I was thinking of promotional sponsorships like it could I mean, advertising from a, I don’t know, alcohol, I mean company or something like that for a bar. I mean, those kind of things, I think there would be an angle there. But the tech was super interesting, where maybe you could be like driving on the street and be like, oh, did you know there’s a show next week, like at this place. And when I was living in LA, there was, it hit me this one time, there’s the Echo, which is a big concert hall down is just down the street, and somebody played there that I would have loved to have seen. And just in all the ways of Twitter and all these things that I somehow I missed it. And I’m like, alright, that’s done. Like how we have the technology, how could I have not been alerted that my like, enough knowledge around me, like of my Spotify playlists, my like, all Twitter feed, all that stuff should have known that I should have been interested in that. And so that was kind of the problem I was trying to solve.

Jamie

That’s, yeah. See? So there’s something you said, we’re getting a bit sidetracked. But I like for folks who may be just getting started maybe monetizing shouldn’t be like, my thing is, build it first. Or what was that phrase? Build it? And they will come? Is that the phrase? Right? Yeah. And and for for the folks that my, my level, right? We’re a little bit smaller scale. We come up with our little solo ideas, but build it first. And then like, my thing is, doesn’t have to be monetized, right? I appreciate it’s a lot of someone’s effort, and a lot of someone’s time, and it’s a passion project. But you don’t always have to try and get money. So

Kirk

that was really on me. I mean, I was looking to start a new company. And so I knew I knew I wanted to start a company again. And I knew this is kind of that sculpting area. I didn’t know where I was kind of getting to, but I was just learning, adding layers to this platform. I knew I could, and we can still do it with our platform. Like I don’t think we know where this is going to end up. I mean, we’re gonna, we’re gonna carve it and polish it and like, aim at a different directions. And I’m sure I mean, we’re gonna find a sweet spot. But it’s, I mean, at the start, you really don’t know where it’s going to end. And so you want to give yourself the ability to adjust later and not just kind of have one shot. But yeah, excuse me, I think I mean, I think the best projects are passion projects, where I mean people care more if it’s something We are really interested in I mean, this, I mean, the media, I’ve always been in this media software space my whole career. And so it just, I mean, that’s something that I love. And it’s I mean, it’s worked out for passionate and commercial sides.

Jamie

Sure, sure. There was there was something that we that we mentioned when. So, peeking behind the curtain inside a baseball talk. I hope that’s the right phrase. The we were, before we do a recording, we have like a discovery call and chat about what we’re going to chat about. So almost like very meta. And there was something that I think I think you brought up, and I sort of ran with in our notes, and that was, you’d said something about, yeah, we’re looking into maybe cross cloud, maybe we should, maybe we shouldn’t, how will that’s gonna work. And I just wanted to get your opinion of should people be cross cloud, right? I mean, we’re talking about a system that you’re building, that’s going to be, you know, huge, right? Those, you know, the mom and pop shop down the street that has a brochure website for their for their business, that they need to be cross cloud.

Kirk

I think it’s, I mean, it’s kind of a what value do you need to extract out of it? And why? I think, I mean, cross cloud is great for I mean, true resiliency, like if you can’t afford to have AWS go down and lose your business, or Azure go down. And I mean, but on the other hand, like if I mean, if you’re, there’s very, very few companies that probably cared about level, like, if AWS goes down, there’s a lot of companies go down, and people just kind of roll with it. I mean, because everybody’s down. And so you’re not really standing out in the crowd. And I think I mean, I think part of it is a probably a technology, like flavor that people I mean, people like there’s some people who are AWS people, and some people that are Azure people are GCP people. And I mean, even on my team, my front end, guys are more AWS guys. And like in but they’re so parallel. And sure little things here, and little things are better. And we always like needed each other about those things of like, what might be better on AWS, or this snap, but they’re really, really similar at the end of the day, probably 95 98%, the same. And I mean, cost structure, they have to maintain competitiveness for cost structure. So the only reason we would look or I mean, I think anybody from a mid level company and down, they probably should just pick what like, what they’re productive with. Maybe what and just pick a side. And but at the huge companies, then it’s really all about, I mean, you just got to stay up no matter what I mean, if you’re like PayPal, or somebody like that, or I mean stripe or somebody that they I’m sure they must have a crop. Maybe they don’t I mean, but I would assume they have across cloud. Something that I mean, they can make sure they stay up no matter what. So for us, it’s not something I mean, we are cross cloud, for my data ingest standpoint. So we can pull from any cloud, but our repository and the code is running only on Azure, we are looking at doing cross cloud from a like, oh, you may want to run on the AWS version of us. Because your data is on AWS, and maybe it’s faster to ingest. We’re I mean, we use so many managed services from Azure, today that that would actually be a pretty heavy lift, the code would generally pored over I mean, we wouldn’t have to change the code too much. But it’s all the stuff we talked to. And that would be where all the adapters to like cosmos dB, and this and that. And so. But I mean, we do have a thought of I mean, well, okay, what about on prem? Or what about a private cloud? And then that’s an idea of Oh, Kubernetes and K native for for function apps and stuff like that. But at this point, I mean, maybe we would do it in the future, if anything, we just should have started there. But I don’t think que native, at least my research I’ve done, I don’t think they’re as advanced and far enough along to be truly cross cloud. And I mean, Azure felt like, so you still have to pick lambda or Azure functions. And there’s just not enough of an ecosystem there. I think they’re getting there. I would love to use them. But if for us, it’s just the way the DevOps is integrated. And the way all of its integrated App Insights, and the ecosystem just gives us a huge benefit there.

Jamie

Sure, I think I think you made a really, really good point there about if your medium business down. Don’t worry about it. Right. Yeah. You know, there’s a couple of examples from the last, I want to say the last eight months at the time of recording, that really sort of highlight that, right. There was a time when Facebook had a problem with DNS. No going on Facebook. Oh, no. But also, that was a big problem. But the bigger problem was Facebook’s internal auth to get into the building used Facebook, right? Yeah, yeah. They have bigger problems than just you can’t log on to Facebook. Yeah,

Kirk

I think it’s I mean, there’s probably people out there that are like, Oh, that’s horrible. You got to solve that. And then I’m a little more pragmatic. I’m like, people live. I mean, unless it’s a live or die thing. I mean, people go get a cup of coffee while you’re while this DNS is down. And I mean, or, I mean, sure, there’s people losing money, there’s companies losing money and all that. But I mean, is that do you have to solve every problem in the world? And that’s where I think you really need to be pragmatic about what can we really handle and what can’t we I mean, like, like CDN, like, we just implemented a CDN in front of our storage. And for anybody out there, this great tool called Graph CDN, that’s a Graph QL CDN that we just started using. I think it’s built on top of fastly, but it’s super fast. And but that gives us it now a layer on top, that we can now say, Oh, look, I mean, here. I mean, this was something we cared about that we really need to have quick response times good caching. Now, are we multi like region yet? And all about like, like cosmos dB? Are we set up for geo replication? No, not yet. But those are areas that we can lean into more later. But for now, we cared more about performance. And so you just kind of to pick your pick your poison a little bit.

Jamie

Yeah, and I think, I feel like that fits really well with. So from my point of view, as someone who’s not building hugely scalable stuff like yourself, that geo that geo replication, is something that I can aspire to attain. But it’s not something that I desperately need, right? If a user has to wait that extra half a second to get their record from the database for their To Do app. It’s not the end of the world, right?

Kirk

When they make it, I mean, we use TerraForm for infrastructure as code. And I mean, it’s, for me, mostly, it’s just flipping another switch rewriting TerraForm or just going and clicking a couple buttons in the Azure portal. And you have I mean, and so I mean, there’s, I think it’s more, I always kind of go by the rule of okay, know what’s down the road, but don’t assume that you have to drive down the road. And I think it’s kind of the way I look at I mean, our staff, I always try to anticipate maybe a quarter ahead of what technology decisions we need to make. And if we need to make any, any changes and things like that. We don’t necessarily have to start writing code for it yet. And that’s why we have a great product team here that we do sort of talk through, okay, well, what’s coming next? How do we anticipate and then me more as the backend architect, and I can be like, Okay, I need to refactor this a little bit or create a different pattern. And as I’m doing other stuff, I can kind of anticipate what’s what’s coming next. And so, I mean, I just wrote this new kind of job based pattern for basically wrapping everything in a job that runs in a durable function, a durable entity, actually. And it’s just one of those things where it’s like, yeah, we’re not, we don’t use it for a million things yet. But I know this is just going to become like, the core of everything we do for all these processing things. And so we bit the bullet. And then and that’s when I hit that bug with terrible task. And it was a it was a huge project, I mean, of a lot of code and a week of heavy, heavy work. And I mean, but at the end of the day, now, it’s we’re set up for the future. And now we have a brand new powder, and it’s just super solid, and we can run everything through.

Jamie

I like it. I like it. I feel like I may have to pick your brains about the that gel based architecture. Yeah, for sure stage once you’ve once you’ve proved it, you know that? I don’t want to say prove that works, but prove the use case and, you know, prove that you can do other bits of the software with it. Because I’m a very big proponent of sort of like prototyping stuff. Can we do it another way, rather than just this way? Maybe in a feature branch, maybe on some other environment? Is it good? Brilliant? Okay, have we got the metrics to back that up? Brilliant, then, you know, we’ll go in that direction. I feel like it’s more of a scientific approach. So like, Let’s travel down this this path a little bit. If it proves to be no good, we can turn right back and come come back. And all we’ve done is just dropped the number of hypotheses we have from five data form. So now we’ve made progress. We’re progressing slowly towards the correct way to do it, right?

Kirk

Well, it’s like, yeah, this pattern with durable entities IDE used for one thing already, we have this feed concept. It’s kind of like a demon that sits back and checks if there’s anything new on s3. And so it just makes a request. And I was like, Oh, I could just reuse that kind of polling, that kind of pattern for sort of long running jobs that I mean, go are kicking off something and it like, we’re gonna use it for ml training. So I need something there to watch because they don’t have a web hook to watch and do that. And so then I started refactoring that, but then I started to realize our infrastructure was wrong, that you’re supposed to put each of those in a separate function for verbose performance and because they each need their own storage account is what they suggest. So that ended up being a refactoring of TerraForm and a refactoring of this and that, it’s you think you get a pattern where you look at it, it’s a little bit of code changes. And then you’re like, Oh, like this actually has all these other repercussions. But in the end of the day, it actually got us way better performance, because now we’ve partitioned off the different sort of sections of of our processing and a different function apps. And so they can scale independently. And so it’s been interesting. I mean, just to the progression of you don’t see what’s going to be coming next, and things kind of change underneath you. Yeah, but the good thing is, I mean, I, I’m always a big proponent of API extraction, or extraction, adaption or where you basically mean, our front end doesn’t care. If I change it to that pattern. I mean, they could talk to a Graph QL API. So we can really innovate on the back end, just I mean, as long as the performance is good, I mean, it’s you really can have those adaption layers and for your teams, you can kind of I mean, go at different paces, and things like that is something we also look forward.

Jamie

Yeah, I definitely agree with that any kind of binary API so they could cross your classes or, or rather, across your interfaces, or a rest G RPC, whatever API, the contract there doesn’t need to know about the details, right? It doesn’t need to know if it’s done. It doesn’t need to know it’s Java doesn’t need to know it’s on Kubernetes, on AWS or whatever. So yeah, I completely agree with you on that one. Excellent. So I’m sorry. Go ahead. No, no, no, go go. Well, I mean, what I was gonna say is, is let’s, so let’s remind the listeners about the product. And that is .NET. And, you know, at the time of recording you right around the corner from 1.0, hopefully.

Kirk

Yeah, I mean, Unstruk data platform, it’s actually up on Azure Marketplace. I mean, as of now, and we’ll be at a time recording. And so it’s a we have basically a starter plan for the rest of the year that you can subscribe to there. But also just Unstruk.com, our website, we have a contact form there for anybody that’s interested. But yeah, I mean, we’re super excited. I mean, we were working about a year, since we got seed funded. So it’s a I guess, about 13 months since since that time, we’re, and we had a version kind of early access version in cute like, October of last year, that we started getting feedback on and then realized, it’s there’s a lot to this product. And so we were like, like, we got to just like really get heads down for all of the the last couple of months. And there’s more stuff that’s really important. And that’s that classic MVP. Thing is, I mean, we thought like, we took a stab and said, look, I mean, let’s try and sell this part. And it was more of a search tool. There wasn’t the automation, there wasn’t the all the integrations, there weren’t in the UI was kind of it was a v1. I mean, it was like a point eight UI. And we really, I love the the new UI, I think is great. And it’s just crazy to think like I mean, how you find that line of what’s shippable is, is really a moving target over time. And so, and you have to be confident. I mean, you can’t you I mean, I do question like, Is this enough? Is it not enough? Is it? Do we lean into the right areas, but I mean, we’ve done enough customer discovery in the past years, and other companies I was at that, I think, I mean, you gotta have a gut feel about it. And I’m sure we’re gonna be wrong, but I want to be wrong in the right way. Like that we can, we can adapt. And I don’t think we’re that far off the mark from the early feedback we’ve gotten so but it’ll be I just can’t wait to have new feature suggestions coming through like our, like our product board portal. And that’s when it gets fun when people are like, that’s cool. But can you do X, Y and Z? Like that’s what I’m dying for right now. So

Jamie

ace, ace. So what about keeping up with with your banker? What’s the best way to do that? Yeah, on the socials, he opens LinkedIn.

Kirk

Mostly, I’m on LinkedIn every day, just I mean, reading and doing a lot of customer research. And I’m on Twitter. I mean, I mostly just post about, like baseball and like stuff like that. And but not not too interesting there. But yeah, I mean, the companies on Twitter, just at Unstruk. I’m at Kirk Marple. But LinkedIn is where I kind of do my main business social stuff, is the easiest way to get hold of me.

Jamie

Excellent. Well, Kay, it’s been wonderful to chat with you today. And I, I genuinely, I always say this, right. I always say to the end, that I have learned a ton, right. And I really have right. And I do learn a ton from every episode, but more so for this one, I think because there’s a lot of you’ve mentioned a lot of being pragmatic about what you’re building and not necess is a really choosing, not necessarily hitching to the right tech. But finding the tech that’s right for you, I think, if you don’t mind me saying that, right. That’s what I’ve taken from this. And then, and being able to find that almost any way you can find an idea for a product or a business or anything like that, almost anyway, you just have to look right. I loved your point there about discovering when, when your favorite band is in town, and how Spotify, Twitter, LinkedIn, all those social networks that are supposed to be tracking all this stuff? Yeah, like them?

Kirk

Yeah, I mean, it is. I mean, that’s, I’ve always been, I worked at a radio station DJ, when I was young. And it’s always been around music, that’s always been a passion. But it’s like, once you start to see a pain point, and then you can be like, “oh, I can apply all this stuff I know, to this problem.” And who knows? I mean, maybe someday, I’ll come back around to that. It’s an evergreen idea, really. But, um, but I think I mean, the pragmatism, I think, is, is a big and just, I mean, developer productivity, I think is an under kind of rated thing. Everybody’s like, “oh, you got to get PRs push, you got to do this, you gotta do that.” But, I mean, do does the team have the right tools? I mean, are that we’re always I mean, looking at different tooling. And like I was saying, I think before this, the recording of there’s a new QA tool we’re using, that we love called reflect. And the team doesn’t mean we’re using another competitor, and they’ve gotten a boost from it. And it’s just things like that, that if you find the right sweet spot, you can carve out I mean, a bunch of time in your day, but also been keeping an eye on I mean, there’s things like our CI CD workflow, I know could be faster. And so there’s, you just got to be aware and look at those productivity areas that are valuable.

Jamie

Excellent, excellent. Well, thank you ever so much, Kirk. I really appreciate coming on. And sharing your knowledge and experience with not just me, but with all the listeners, you know, so I really, I wish you the best of luck with Unstruk. And maybe we can we could talk about post version one when it comes out if you’d be interested.

Kirk

Yeah, I’d love to know. I mean, hopefully, we’ll see where it ends up later this year. And now we’re, I mean, super excited about it’s a I mean, it’s probably the most exciting time for a company to get a get a 1.0 out and it’s the hardest time probably do but but ya know, it’s, we’re looking forward to it.

Jamie

Ace. Well, thank you very much Kirk. I wish you I hope you have a wonderful rest of your day.

Kirk

Thank you so much. Appreciate it.

The above is a machine transcription, as such there may be subtle errors. If you would like to help to fix this transcription, please see this GitHub repository

Wrapping Up

That was my interview with Kirk Marple. Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at dotnetcore.show, and there will be a link directly to them in your podcatcher.

And don’t forget to spread the word, leave a rating or review on your podcatcher of choice - head over to dotnetcore.show/review for ways to do that - reach out via out contact page, and to come back next time for more .NET goodness.

I will see you again real soon. See you later folks.

Follow the show

You can find the show on any of these places