Transcript#
This transcript was generated automatically and may contain errors.
Hey there, welcome to the Posit Data Science Hangout. I'm Libby Heron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.
Can't wait to see you there. I would love to introduce our featured leader today, Brijesh Chejerla at Florida Blue. Brijesh, could you introduce yourself? Tell us a little bit about you and something you like to do for fun.
Hey, everyone. I'm Brijesh Chejerla. I am a data scientist at Blue Cross Blue Shield of Florida. I have a PhD in computer science. And by way of introduction, I'm an enthusiast in many things. Data science happens to be one of the conduits to help prosper my enthusiasm in those many things. Machine learning is at my heart, it's at my core. And I chose to be a data scientist way back in 2016, because after I graduated from my university with a PhD, I was like, what do I do with this? And I was thinking about where do I go next? Because I was pretty particular about what I would do and which field I would be in.
And then I realized being a data scientist, you are a plug and play model. You can be in different fields. You could be in sports, you can be in medicine, you could be in healthcare, you could be in construction, you could be whatever. And then they all kind of fit together. You do very similar stuff. But at the core of that, it's still the machine learning. So that's kind of the reason why I chose to be a data scientist. And yeah, I'm here. I graduated from being a data scientist to a machine learning engineer, I would say. But yeah, that's on the professional side of things. As for fun, I love to watch football, soccer for American audience. And I spend a lot of time, I would say, going through the analysis and all of those things, you know. So that's that's kind of my fun activity for the most part. Generally, I love to listen to podcasts. Most of these podcasts these days are football related for me, because I'm in that space right now. But then otherwise, so to speak, I just love to converse with people. You know, that's a fun activity for me.
Well, you are doing that today. And Rory in the chat, I can already see is wondering what team you support, Prajesh. Manchester United.
Manchester United. Any other ManU fans here? Man City fans unite. We have some other fans. Oh, no, we have some booing. Rory's a good choice. Rory's a good choice. James says cries in Tottenham.
Please mute everybody who is not a ManU fan.
Transition from academia to motorsports
Amazing. Well, Prajesh, I would love to start with something. I'm going to hop in before the Slido questions as a breach of protocol and say, I have to ask you about your stint in motorsports, because I know that your sort of like transition into industry from academia had a pit stop. Oh, that was an unintended pun. Had a pit stop in motorsports. Could you just give us a little bit of info on that? I truly didn't mean to do the pit stop pun. Well, that's pretty nice. So after I graduated, I was searching for jobs, obviously, you know, and then I just didn't want to work for the sake of working. I was pretty, I wanted to take, spending five, six years in academia, I really wanted to take a break. So I took my own time to find myself a job. And then this motorsports job came along as a shot in the arm on a scape. And I had no intention to even look at motorsports for that matter. I was a, I am still a Formula One fan. I still watch Formula One. But NASCAR was something that was way out of my radar.
So I got this job through a connection, as in one of my friends recommended me to this job. And then he connected me with the people who were looking for a data scientist to be hired. And that's how I started working at Hendrick Motorsports. So that was my first foray into motorsports, first foray into NASCAR, everything was very new. I was looking at jobs in the Bay Area. And then I had to move over to Charlotte, North Carolina. And that's how things worked out, you know.
Data science applied in sports
All right. We have some more sports questions. There's an anonymous one already that says, favorite example of a data science, of data science applied in football or sports in general.
So one thing that most people don't know is that the amount of data that you see on screen, on TV, is not the same kind of data that people get to work with behind the scenes. Because NASCAR is basically stock car racing, right? So not all of the teams have the same amount of funding. So NASCAR as an organization does not allow you to, or it does not give you the data, because they know that people who have, or teams who have more funding will have, they'll have some benefit over teams who do not have that kind of funding to kind of, you know, look at the data and do whatnot with that. So you're restricted with the data that you collect.
So you will have to be, you'll have to get very creative with what you have and solution for it, because the models that we built when I was working at Hendrick Motorsports were, it was basically, you know, to be able to predict when the driver should pit, you know, when should they go into the pit stop. And that is dependent on the fuel consumption, and it's dependent on the tire degradation. So we had to build models based on that. And then we had to try and see if that made sense from a machine learning standpoint, because if you're a data scientist, most of what you do is, let's say, regression. There are many engineers who work at, you know, the company, who are, who also have used regression and who also know about all of those things.
So your question would then be, do I just use a hacksaw, I mean, a saw to kind of, you know, break a twig, or do I actually do the bare minimum? So you'll have to be very creative about how you go about it. So again, for sports, you know, I know for a fact that, you know, football uses a lot of data science these days. There are people and there are teams that actually try and predict what the scores could be based on who's playing and the positions that they're playing and, you know, the recent trends and the form and so on and so forth. You know, how many yards they're able to cover, after how many minutes does their performance go down and so on and so forth.
And it so happened that for a Manchester United versus Manchester City derby game, there was an algorithm that predicted that City will win 2-1. And that's exactly what happened because of the goals ratio and all that. So it was very interesting. And I thought, you know, that would be something cool. So if you're into that kind of a thing, or if you're looking to get into that kind of a thing, you should know that, you know, the amount of data that you'd have would be very limited. And probably, even if you would say, well, let's build up this pipeline in order to get fetch all of this data, you still won't have that simply because, you know, it just doesn't either exist, or it's too difficult for you to go through all of the hoops in order to get that kind of data that you need as a data scientist.
And as data scientists, the one thing that we all keep saying is, oh, we need more data to be able to generalize a model to be able to make better predictions. And for motorsports, you have to be extremely accurate to the milliseconds level, right? And at that point of time, you have to think, am I overfitting this model because it's pretty accurate, or is it generalized enough? Because if it's generalized enough, then you may not be giving the race engineer or the race crew chief what he wants. But at the same time, from the data science level, you know, if you don't generalize, you're basically overfitting, which is not a good model. So you'll have to be very creative and you have to be very specific about how you want to go about those things.
And as data scientists, the one thing that we all keep saying is, oh, we need more data to be able to generalize a model to be able to make better predictions. And for motorsports, you have to be extremely accurate to the milliseconds level, right? And at that point of time, you have to think, am I overfitting this model because it's pretty accurate, or is it generalized enough?
That sounds like a tough problem, but like relatable across all industries in data science. Am I specific enough? Am I generalizable enough?
Well, we have lots of conversation going on in the chat about different people supporting different teams. Somebody wanted to call out that Man City is different than Man United. You're a Man United fan, right? We have some Man City fans. Everybody is very interested in the sports stuff. So I wonder if we could do one more sports question and then shift gears because I have another question for you. So the other question that was in there was also anonymous and it was, would you be able to provide some resources for people who are interested in machine learning and football? If you have any tips.
Yeah, I don't think, to my knowledge, I don't think there's anything that is resourceful enough for machine learning and football together. Because at the end of the day, you're applying a solution, a mathematical solution to an existing problem or to a real world use case, which has some data. So it's that connection, which is why I said during my introduction that I wanted to see if being a data scientist, I can plug and play. Can I go into finance? Can I go into more sports? Can I go into healthcare? Can I go into whatever? And will I be doing very similar stuff? Because then I would make myself the most useful. Because let's say if I get bored doing the same thing for three years, I'd probably jump and do the same thing.
So I wouldn't say look at it as a connected set of things. Just there are a, there is a ton of resource. There are a ton of, tons of resources out there for machine learning. I actually shared a book if anybody wants to go look at it for sports analytics. Yeah. I always say that, you know, keep to the basics of machine learning, keep to the basics of statistics that will serve you no matter what you do, no matter where you go.
And then, you know, you can apply that to anything. You know, you could, you could look at different types of regression. You'll see, depending on the data that you get, no matter how the data that what kind of data you have, if you're building models, it's one thing. Data scientists also do analysis analysis. You have to have very strong statistical background, right? So you have to develop that. You have to understand how to look at this. You have to look at, you have to look at design of experiments. You have to understand that you have to, you know, create hypothesis. You have to, you have to look at when it comes to the world of data, it's an elephant and everybody is looking at the trunk of an elephant or the leg of an elephant, kind of know that it's supposed to be an elephant. And when you draw it, it generalizes to an elephant, but it's not exactly the same elephant that everybody looks at. That's what data is. So I would just say approach the bottom up.
Approach the, your approach should be bottom up. Whereas, you know, your, your foundational machine learning, your foundational mathematics, your foundational statistics are very strong so that it can be applicable to any data that's given to you.
Picking an industry as a data scientist
I was going to say, you know, that is a fantastic lead in to our next question, oh, which is gone. Hold on a second. It disappeared from Slido. I think Rachel and I probably clicked something in the exact same moment. I think someone maybe with their question. Okay. So Rob Creel had asked a question. Rob, if you are available to ask it live, that would be amazing. I'm going to highlight it in Slido.
Yeah, thank you. So I share your perspective on the general applicability of data science and that being appealing, but like my background is in mathematics itself, just a master's in pure math. And like I, something I've struggled with has been like, how do you pick an industry? Like in my job applications, I've worried that I've been casting too wide an eye on the net. And I almost wish I had like one specific industry that was just my favorite and I can target. So I just love to hear about your experience. Did you sort of just somewhat accidentally land where you are or was it targeted and how did you choose your targeting?
No, great question, actually. So I don't think I did a good enough job trying to explain how I got to where I got to as part of my introduction. So I'll use that for now. So the way I started looking at it was two things. One is how much money am I getting? And two, how much of my personal ethics and morals do I have to forego for that kind of money, right? Those were my two parameters. So like any company that you go and work for, any industry that you go and work for, you will have to compromise on certain things because that's the, you know, that's how the world works. So I say, just pick on those two things and think, you know, what your financial status is and, you know, how strongly do you feel about certain things?
The reason I picked motorsports was because there were no ethical or moral dilemmas for that for me. And two, it was something that excited me, right? So that's kind of how I picked that off all of the other options that I had, because in Bay Area, you can actually go and work for different companies, you know, and some of the big companies and all that. And I just wasn't really sure if I wanted to get into that. Coming off a PhD, the first, I was extremely idealistic. So I would say, do not be idealistic. You know, if anything, don't be that. Be more pragmatic about where, you know, and where to work and what to do and stuff like that. So that's one thing that I would say, because the first three to six months, I struggled, because I thought, you're supposed to do it this way. And the business doesn't really care how you do it, as long as you get it done in a certain
time frame, and you give them similar outputs that you're expecting, you know. So that's another perspective that I would say, you know, you would have to develop or think about if, you know, you want to even develop for that matter. And then pick and choose a couple, like, for instance, healthcare. Healthcare is going to be a field where data science, machine learning, artificial intelligence, whatever, whatnot, you know, let's say, if you're even a programmer, for that matter, you're going to be in that space and find yourself good enough jobs for the most part, you know. And unless you're working for some seriously shady companies, I don't want to name them. Unless you're working for some seriously shady companies, for the most part, you know, you'll actually do a pretty good job, because most of what you do in IT or data science is technically be helping the member or, you know, the patient or whoever as well, you know.
I don't think it will be in your capacity to choose how the member is actually affected, because that's, you know, beyond you, because it's then to the outreach managers and how they use the models that you build and stuff like that. That's one thing that I'd say, you know, so be sure that no matter how hard you try, unless you are at a very high level, you can only influence it so much. So, you know, even the analysis that you do, you can do a fair and thorough analysis and stuff like that, but that will be the end of what you can actually influence.
Because I'm in healthcare, and I can give you that example right now. If you want to work for, let's say, a company like Spotify, you know, who does these recommendation based stuff, there's nothing wrong with that, you know, there are no moral or ethical obligations for you to think about very seriously, unless you seriously have some concentrations against it. So think about it from that perspective, and choose whoever gives you money, make sure that you get more money, I would say, no doubt about that. But at the same time, unless you get into it, and you start working you won't really be able to crystallize all of what you actually want to do, you know, but so get yourself into it. I got myself into it as a data scientist. I want, I grew into being an architect, then I was an admin, you know, I was a Posit admin for some time, for about six years.
I wanted to think if I were my own leader, if I were, there's a book called Think Like a CEO, right? So if you were on your own leader, and you as a developer, would you look up to your leader who has knowledge about most of these things, but then also has a vision, because as leaders, you expect to have a vision, or would you want to just look at yourself as just an independent contributor, where you're very happy doing just model building, and then, you know, take your paycheck home, and then you're happy with it. So think of those things, and then go both deep and across, you know, both vertical and across. Develop new perspectives, because once you become an architect, or once you not become, once you think like an architect, once you do system design, once you get into data engineering, your perspective about data science will change. I have done that. I have consciously trained myself to think like that. So I would encourage you to, you know, look into that perspective, but no matter what, pick a, you know, look at those two things, pick a job, get into it, and then explore your options, is what I'd say.
Develop new perspectives, because once you become an architect, or once you not become, once you think like an architect, once you do system design, once you get into data engineering, your perspective about data science will change.
Audience Q&A: ML techniques and deep learning
Fantastic. Okay, well, we have a question that is from Mark, and I'm going to put it in the chat here. Mark, if you are available to ask that live, that would be great. Let's see if we can find Mark. Can you hear me? Hey, yep. Yeah. Hi, Brijesh. My question is, like, in your role right now, like, what is the most commonly used ML technique that, you know, that answers a problem almost, like, you know, more often than it, than you expect? And my follow-up question to that is, like, where does deep learning and, you know, AI, LLMs come into play in your day-to-day?
Mark, you're, you're the same map of what's at Florida, right? Are you not? Yes, yes, I am. Yeah, like, yeah, yeah, we work together.
Mark, my colleague, my friends. So, so, yeah, I think if you ask me this question in 2017, the default answer that everybody knows is XGBoost. Throw anything at XGBoost, it'll do well, or well enough. But I would say start looking at, don't, don't look at using an algorithm as it is, you know. You have to look at your data, you have to understand your data, and then think of which algorithm use, you know, works best. So, you'll have to build your data in order to use your algorithm, right? So, there, the most commonly used one, again, is XGBoost because it works extremely well because of how robustly built it is, or what kind of, how, how much of a robust algorithm it is. But then if you think about it, right, it only goes to solve for so many things, and it's extremely rectangular in its format, there are limitations to that. And if you want to know the causality of why something is happening at a role level, and you want to say, okay, this particular person in these, you know, the top decile, or the top 1% of the people give me the reasons why, and I want to reach out to them x, y, and z.
If they happen to be more nuanced, then that's when you go into deep learning. Because then the way you structure your inputs to the model will change. So, you'll, you'll then start using transformer-based, or attention-based models, or you could probably develop something from LSTM-based solutions, or you could think of, you know, states-based solutions, and stuff like that. So, you'll have to start thinking of the problems, and see how you can structure the data in order to get the output that your business partner is looking for. So, if the business partner says, yeah, all of these members are fine, but then what about these reasons? If you tell, can you tell me why does this happen? If data points are correlated, or if data points are, okay, when I say data points, if certain events are related to certain other events, which are not to be looked at in isolation, you will then have to construct your data so that those two events are together, and the model understands that in context, and gives the output, at which point of time you would have to use deep learning. So, it's, it's about how the, when you talk to the business, and the business tells you these are my parameters that I want to look for, there is an ideal state where the business says, I want all of these things, but then there is a more practical state where you can get to an approximation of that ideal state. So, you'll have to pick and choose your algorithm based on that, is what I'd say. I hope that answers your question. I'm sure it does, and Mark, thank you so much for asking that fantastic question.
Tools, tech stack, and the Posit admin experience
Well, we had lots and lots of chatter going on in the chat about systems design, about architecture, adminning, and you mentioned that you were a Posit admin for many, many years, but you were also a practicing data scientist. I think that there are probably many people who have that same experience. So, I would love to ask you at our kind of halfway mark here, a tool-based question. So, what tools do your teams use? Are you in R? Are you in Python? Are you working? Are you deploying things in Connect? What is your experience as that sort of half-and-half admin and practitioner?
So, our tech stack for data science machine learning at Florida Blue is Posit Workbench, Posit Connect. So, we develop in Posit Workbench, we deploy in Posit Connect. I was a reluctant admin to begin with because I didn't know anything about it, honestly speaking, but then I'm glad I got into it because being an admin changes the way you look at how you want to deploy stuff, how you want to make your models available. So, that helps you architect your solutions better. That helps you become a better solutions architect for that matter. So, it's a connected set of events, you know, depending on what you want to do. Now, I think we had around 150 developers and about 800 to 1,000 consumers of the content. Whoa, whoa. Say that one more time. You had how many? 800 to 1,000 consumers of the content. That's the license that we have for it. And about 150 active developers and increasing. The last I checked was 150. I don't know if that's gone more.
That's a record. I think that's the highest I've heard. So, what I did was, and then, you know, we kind of used to work in silos and then people, we would get emails back and forth saying, okay, this is not working, that is not working, this is the thing, that is the thing, blah, blah, blah, blah. So, what we did was we just created a Teams channel and said, if you have any questions, post it here. And if somebody else also knows the answer to that, go ahead. So, what we did was we then have a set of packages that are internally developed to our own needs that we use. And those are specifically used on the Posit platform, you know, like, you know, to connect to databases, to deploy, and to do all of these things. So, that's kind of how it started. And then it's at full maturity right now. We are looking to move from on-prem, our solutions are basically on, sorry, our applications are basically on-prem. And we are now looking to move into cloud. So, the cloud that we're looking to move to, we'll now have to start thinking about how to set these things up, how will a session spawn for a person. So, it's not, because it's a server, right? Like Workbench and Connect are on the servers and multiple users use it. So, you'll have dependencies and all of those things. So, all of those things need to be figured out. You will then understand and appreciate how difficult it is and how important it is for you to look at authentication or user authentication and how that affects the way you work and stuff like that. I keep saying this all the time.
If I had a DevOps team, I as a developer would just want to wake up in the morning, make my coffee, sit, just turn on my computer and log in and then just get away, you know? So, when I was the admin, that's kind of what I wanted to provide to our users so that they have least amount of friction to do the stuff that they wanted to do. But at the same time, that was a lot of work for me outside of my data science day-to-day work. So, it's an interesting thing. So, if you ever want to get into admin, you have to learn Linux. You have to learn how that works. You have to understand how to solution some things from a security perspective, which is, again, a different difficult thing depending on the figure that you're in. So, in healthcare, security is paramount, right? So, you have to be extremely specific. Users don't, especially data scientists and analysts, they aren't necessarily software developers. So, you don't go through the STLC process all the time. So, you'll have to guide them through those things. So, that got me into the security. That got me into being a cyber secure software developer hack. And so, that's kind of how my journey grew. So, if you're interested in that, I would very strongly encourage that you get into that. It helps. It helps you become an all-around person for that matter.
Transitioning the admin role
Fantastic. Okay. Well, we will move on from admin questions in just one second, but I know that Rachel has one because she mentioned it earlier. Yeah, it seems like people want to write a book called The Reluctant Admin as well. But I was wondering, how did you so successfully transfer that IT admin ownership over, especially with so many people being reliant on the tools? Like, what was that transition like?
So, the transition period was not easy, honestly, if I'm being very honest. It's not easy, you know? So, Posit admin is a very niche-specific admin role. It's not your general systems admin. It's not your general Hadoop admin. It's not your general Linux admin. You need to have the knowledge of all three of them, and you need to know how Posit workbench works. You need to know how the config files are set up. You need to know how Posit is set up or is expected to be set up. You need to know all of those things, and then you need to know the connect side of things as well. And being a Posit admin also comes with, hey, I have these R-related questions. Can you help me? I have these Python-related questions. Can you help me? So, I'm not a R person. I've never tried to be an R person. I would just say I'm literate at R. So, any of my R questions, I kind of diverted to my teammates who are experts in R. Any of the Python-related questions, I handle. So, it's a very niche kind of role where you have to be extremely good at many things at the same time.
So, the new admins who have come in, they came from not this kind of background. They weren't R developers. They weren't Python developers. So, it took us a while. So, we kind of developed an internal mechanism where it made it easier for them to kind of know what sort of issues that come up. So, what we did was we have a Git issues and we created a dashboard of, you know, a Git board. And then, basically, if you're a new admin, you can search for an issue and that issue will have been resolved somewhere. And then, you know, if, let's say, I didn't exist in this company tomorrow, you can actually look at how that issue was resolved and then go about resolving that issue. That's kind of how we designed it and set it up. That's helped our new admins. We still continue to help new admins if, you know, there are Kerberos issues or something of that sort, right? And if they have any questions or whatever. And we still sometimes do get on the calls for, hey, we're going to set these things up in cloud. You know, what do we need to look at? Or, you know, do you have a few use cases that we need to look into and stuff like that? So, that's kind of how we evolved. It's taken about six months, I would say. It's not super easy. So, it's going to take time. Yeah. Thank you. It sounds like there's a great relationship, though, between data science and IT at the company.
So, at our company, we are in the IT wing. There are other data scientists across as well. So, there is a data science for different business areas. We were, I think, 10 years ago or whenever the enterprise data science team. So, we stuck in IT. That's where we've been.
Wow. Okay. We have so many questions in Slido. Let's see if we can spend the next 10 minutes getting through as many as we can. Makarand had asked a question in the Slido. And let's see if we can find Makarand to unmute. Yeah, I'm unmuted. Hi. I'm there.
There. Hey. Would you like to ask your question live?
Sure. Hi. This is Makarand here. I am in California Bay Area. You know, with AI, this question is pertinent to a lot of us. How do you think AI is going to impact data science roles? I personally see a huge impact already and can speak more to it. And then, you know, do you use AI as a productivity tool in your day-to-day? And if so, how?
AI's impact on data science
Yeah. Very pertinent question. So, yes, AI is going to impact not just data science. AI is going to impact even software development for that matter. AI is going to impact to a reasonable degree now architecture in the future. Probably, you know, that's where we're going to go. I think the only safe stream right now is data engineering. So, if you are a good data engineer and, you know, you're worth your salt, then you're safe for now is what I would say.
But as for data science, right, yeah, it's definitely going to affect because I have seen⦠I'm pretty sure most of us have seen that, you know, you throw a CSV, let's say, at the model and you ask it questions, it's going to generate X, Y, and Z. If you are a Pandas user, you know that Pandas AI is a package that you can actually load to just ask it questions, you know. So, it just depends on how it's being used. I don't⦠California is a different bubble in itself altogether. So, outside of California, all of the other companies are still trying to get up to speed with this. In many companies like ours, there are a lot of, you know, compliance and regulatory issues that we have to overcome before we start doing all of these things because of where the data sits and where it goes and what you can ask and what is quote-unquote touched by AI. CMS defines⦠CMS doesn't define it adequately, but it says touched by AI or use AI. So, now AI could be NLP, could be, you know, your language models, could be anything. So, if it's generated by AI, it's actually generated by the language model or is it just an extraction? It's not very adequately defined.
So, for now, it's okay, but then very soon, there's going to come a place where many of the jobs are going to be redundant because you can then do more with less. Now, but that also means that you're not necessarily going to get more productive. It means that you're going to have to work on a lot of things because the assumption is that, hey, you don't now have to write the code that you otherwise previously would have to. So, those assumptions are changing. I am seeing that in multiple organizations, you know, the whole idea of trying⦠For instance, we have a AI coding agent, like we have Windsurf, you know, this thing, you know, as part of the organization. So, many people use that. I use it to⦠So, I set up a few things so that I use it for code review, you know. It helps me review the code much faster because with all of the other things that I do, I won't have enough time to actually do enough, like, detailed code review. So, I ask it to do X, Y, and Z, and it comes back with whatever. But then I still rely on my own know-how to actually go and look at the logic that's actually written for the business. It's not just about the correctness. It's also about, is the business logic correct? Are you reading the right kind of columns? Are you reading the right kind of⦠This thing, are you doing the right kind of transmissions? Does it make⦠What⦠The transmission that you do, is it clinically relevant? All of those things, I don't think AI can do yet, you know.
But for most of the things, you can check for code correctness, you can check for, you know, you can write quickly tests and stuff like that. So, that's where I see it useful. But in terms of impact, in what we do from a exploratory data analysis, Exploratory Data Analysis. The good part is we can do that EDA, initial EDA much faster, but at the same time, there will come a point of time where all of this will become agentic and then it'll just run a model and do X, Y, and Z.
You have to think of a solution, put in your inputs in a certain way, and think of your outputs in a certain way, and then you can write code and stuff like that. But if you're expecting it to do the auto ML part, it's not there yet. Maybe it will be, maybe somebody will, you know, decide to build an agent solution out there, which does that, and that will change the game. I don't know. But yeah, it's definitely going to affect. But in order for you to hedge yourself against that risk, I would still say, be fundamentally thorough in your machine learning, deep learning, statistics, mathematics, whatever you have it, you know, and then coding is kind of out of our hands. You know, you tell people I'm a better coder than a coding agent right now. It's a hard thing to prove these days. But you can always still say that I'm a much better data scientist, or I'm a much better machine learning engineer than a coding agent or, you know, the AI can do right now. And you would still get away with that, and people will trust you.
But in order for you to hedge yourself against that risk, I would still say, be fundamentally thorough in your machine learning, deep learning, statistics, mathematics, whatever you have it, you know, and then coding is kind of out of our hands.
Awesome. Thanks, Rajesh. Thank you, Makarand, for that question. And I wanted to call out that there's a huge conversation happening in the chat, including Makarand talking about systems design versus ad hoc work, which is really, really interesting. Makarand, I'm going to have to like reach out so that we can talk more about this. Maybe you can come on the Hangout with us and talk about it as well, because it's a fantastic topic, like this balance between rigor and speed that we see in business, especially as academic thinkers, where the business is always going to need these like quick turnarounds. Anyway, everybody go look at that conversation and chime in, and we will head over to another question in Slido. Oh, go ahead, Rajesh.
System design and architecture
Before that, if I may, about the system design. So the way we look at it here is, you know, it's not just about data science models that you build. So we also build up the applications in our team, right? So that brings in itself its own level of system design that you'll have to think about. You'll have to understand what works, what doesn't work. That's another part where AI kind of falls short, because you are the one who has to think about you. There's practically no way for the AI to be cognizant of all of the existing stuff that's going on in your organization and how things are set up and all of those things. So your system design is basically your architecture at the end of the day. Any architect's fundamental thing is system design. So for you to be able to know that, you have to be very cognizant of that. So when you build your systems out, that's what I mean to say, you know, when you architect your solution, it's not just about which algorithm I use or how I input the data. It's about how many users are going to use it. How am I going to connect it to different things? How am I going to fetch from this database at whatever pace? Is it going to be given to me in a stream? Is it going to be given to me in a batch? Is it going to be given to me in a JSON format? Is it just going to be a row by row fetch? All of those things you'll have to think about when you build your model, because your model outputs are going to then be dependent on the speed at which your system integrates.
So system design is definitely something that's a must for most machine learning engineers. So yeah, be relevant. Fantastic. We definitely had some other systems design questions in there. The Slido is just so full of questions. We will never be able to get to all of them because we have 15 minutes left. But I would love to bring in Logan to ask his question live, because it definitely fits into the sort of vibe of today, generalizing our work experience to other things and being a plug and play mindset with your data science. So Logan, if you are available to ask yours live. Yeah. That would be. Hey, Logan. How's it going?
It's all right. I wanted to ask, I'm a little young in my career as a data scientist. So I wanted to ask, it seems like you've switched a lot of careers where data is vastly different. The systems that you're using are vastly different. And as data scientists, we're expected to know, be the experts on our systems and data. How do you quickly become an expert on the, when you walk into a new job, how do you quickly become the data expert?
Becoming a data expert in a new role
I don't think there's any such thing as quickly becoming a data expert. So the way I grew into my role is I, so if this is a funnel, I'd say here, if this is a funnel here is where data scientists usually operate. And I'd say, stop just being here. This is a local minima for you. Just don't be here. Which is why I say, if you think like a architect system design comes into the picture, you have to know where your data exists, what's your data pipeline. So depending on whichever company that you work for, try and get to know the data pipeline. If you don't have restrictions to know that data pipeline or to access that thing, that's a different issue. And it makes it much easier, but get to know that, where all of the data is flowing into, what kind of data does my company get in? If you want to call it a data lake, what flows into the data lake and outside of the data lake, what kind of pipelines are built to go into relational databases or NoSQL kind of setting, or am I just feeding into flat files or am I, if you work in industries where there are images or audio files and stuff like that.
And then you'll have to start thinking about, if I have to collect more features for whatever data science work that I'm doing, can I extract features only from this relational database or from NoSQL, or can I also go above and beyond and say, you know what, from this particular WAV file, I can actually transcribe this document, this audio file, and then do some NLP on top of that, or basically if you're using a language model to be able to summarize or to extract, can I get more data points out of that? Is this information available? That is kind of how you have to think, because you should never stop yourself in this here. You also have to think about a data engineer. You have to think about, okay, but if I have to connect these two different databases, how do I go about connecting them? Am I writing the most optimal SQL? And then once you connect different databases, you'll try and say, oh, these have these many other fields that I didn't know. You'll have to spend time visually looking at all of the different fields and see if it's useful to you or not. You'll have to be very curious and asking same question as to what does this mean? What does that mean? And then most importantly, I would say you'll have to think like the business or like your end user. If you were a recipient of the analysis that you were given, what would you want or what else do you want? How deep would you want to go? And so that's kind of how you have to switch your thinking.
So your data science, with all of the machine learning and the statistics background that you must have learned in your school, or even when you're in your job that you're learning, that you're being an architect or a system designer, and then being a vertical data engineer is fundamental to being a good machine learning stack. Then if you want to add yourself another vertical, which is DevOps, that's great. It's not necessarily going to build you with anything, but let's say if you want to become the CTO of a company, you will have that thought process where you say, I want you to do X, Y, and Z and think about this. I want to solution this and so on and so forth. So that's kind of how I would say grow into your career in that perspective and not just limit yourself to just being, I was given this data. I'm only going to look at these features and based on these features, here's the output. So like I said, go and look for data that you probably don't even think you need for right now. Just look at the data, like see what's available. And then you'll know that, okay, when the time comes, oh, I have this data over here, would that be useful? Or there is data over here, which is somewhat translatable to the feature that I'm looking for. Let me see and add that. That's kind of how I would go about it.
So your data science, with all of the machine learning and the statistics background that you must have learned in your school, or even when you're in your job that you're learning, that you're being an architect or a system designer, and then being a vertical data engineer is fundamental to being a good machine learning stack.
We have lots of voting for yes, think like the business. And also I wanted to point out Hubert's comment in the chat, which was that understanding of the domain knowledge is super important. And people tend to just kind of skip that because of the deadline pressure that they get, but understanding the data and the domain is absolutely invaluable. Okay. Let's head over to Slido one more time and we only have 10 minutes left, but hopefully we can get another question and then talk a little bit about career advice. So one that might be quick to answer, we had an anonymous question that said, I've been told that I need to have a portfolio and I need to have done a bunch of projects on GitHub to be a stronger job-seeking candidate. How true is that?
GitHub portfolios and job seeking
It is true to a certain extent, especially if you're in a tie break situation. So I've interviewed for people to be part of my team or to be part of other teams as well. That's my first thing that I look for is, do they have a GitHub profile? And what kind of work they've done? I'd like to see what kind of changes they've made over time, the comments that they do and stuff like that. But at the same time, it's more relevant if you are very early in your career. And let's say you're looking for a job and for whatever reason, you haven't been able to seek a job for six months. When you start on Jan 1st, let's say, nobody thinks that they'll have to wait until June or July to find a job. They would hope that they'd get a job sooner than later. But I just say, start on the side, building out projects
that you think you're interested, not to show somebody else that you can do X, Y, and Z. But if it's a project that you think that you're genuinely interested, automatically you will go very deep into that. And when you talk about that in your job interview, you will then tell them how you thought through the problem, how you thought through the solutions, how you thought through the system design, if there is a system designed to that, how you thought through the scale of problem, how you thought through the lack of data, if it's data that's publicly available, and how did you get that data? How did you transform that data?
All of those things will automatically happen. So even if it's just one project, the depth of the project is sufficient. And then you'll anyways have, depending on the team that you're interviewing with, the code that you work with, they'll ask you to write your code. So it's not a given that if you have a GitHub profile, they'll wave off the code interview part or something of that sort. You'll have a few more downy points here and there. But the way I look at interviewees is how they are able to think, especially in this day and age when you have coding agents and the need to be an exceptionally good coder is kind of reduced. It's all about the way you think about a solution and the way you come up with the solutions. Have you been able to put out a dashboard of sorts? And all of those things do matter. So think about it that way and build up a site as well.
Career advice
Fantastic. Okay. Well, seven minutes left. So maybe we can get in our last question that we like to ask, which is career advice. There's been a ton of career advice already in our call, but Prajesh, are there some things that you would share with people who are sort of like lower down the ladder than you or your peers? Like what helped you in your career? What would you share with others?
Like I said, when Logan was asking, don't put yourself in a local minimum. If that's what you want to do and you're happy in that space, that's fine. There's nothing wrong with that. There is no right or wrong things here. But if you are a curious person, if you want to grow up the ladder, even don't just put yourself in a local minimum. I have a bunch that I've kind of wanted to get into, but none of them are related to go to this particular website, this particular YouTube channel, or listen to this podcast and whatever. I think the first thing that you need to develop is perspective as a data scientist or a machine learning engineer or whatever. It's about developing new perspectives. You will have to spend your time and actively curate what perspectives that you develop and how that translates to what you do in your job.
I very consciously built myself into thinking like a data scientist. I was a researcher, so I used to think like a researcher. And then I quickly realized that just thinking like that wouldn't help because there are deadlines to reach. There are so many other things, not every job, not every problem is a research problem. So you'll start having to think about what is the simplest way I can get to the solution. If there is a simple way, do I even need to go and think about a more complex machine learning solution? Take it to the business. If the business is happy with that, okay, so be it.
But then you kind of get a buy-in because I think a lot of people's issues are we are not really given enough time to implement X, Y, and Z. Initially, you will never be given enough time, but then you get a buy-in after producing some results and then tell them, hey, usually this is not a good thing for X, Y, and Z reasons. You have to do all of these things, which usually takes three months instead of the two or one month that you asked for. And so you're kind of buying more territory based on how you go about it. And for that, you need to know the different verticals and the different horizontals and how they work. If somebody says scale up, I want you to be able to do this for a million documents. You have to make them understand that from a DevOps perspective, it's not just random scale up. It comes with certain costs. You won't be expected to see the same amount of time and accuracy because of certain things that you would probably see something in a thousand documents versus a million documents. And it changes the way systems are interconnected. It could potentially induce more failures. It would potentially induce more missed documents or whatever. So you'll have to talk to them about it.
Usually many people in the business are willing to listen to your things, but at the same time, keep it somewhat technical. So you have to learn to speak like the business and explain in English rather than going too technical about X, Y, and Z, which they may or may not necessarily fully understand. So go vertical and go horizontal at the same time if you want to really be exceptionally good at your job. That's my long and short of career advice.
But outside of that, I have a few that I'd like to go through. First and foremost, be kind. That's the first thing that you should do. Be kind to people. And as a data scientist or a machine learning engineer, you have to train yourself to be unbiased. What that means to say is you have to stop thinking of this works. I know I have so much experience, blah, blah, blah. Put all of that aside when you get into a meeting with someone. Hear them out. Don't talk over them and hear them out and understand what they're trying to say. Many times when we go and sit with the business is we try and get the requirements. The getting the requirements part is more like the business says these nine things are my requirements. Am I delivering on that? No. There's a lot of in between the lines. So try and understand what the business really wants. Sometimes the business doesn't really know what they want. So try and talk to them about it. Try and work through that thing. And then you can then get a better business requirements.
First and foremost, be kind. And as a data scientist or a machine learning engineer, you have to train yourself to be unbiased.
And then when you work with your business analyst or whoever is writing the business requirements, you can say, Hey, this is probably not going to be done in this phase because of these reasons, but we can do an approximation of this. So on and so forth. So it just depends on how you genuinely listen to people. If you're interested, be a mentor, becoming a mentor or becoming a teacher changes the perspective altogether in the way you actually go about your own job. Because then there's a difference between you understanding something and there's a difference between you explaining something. And then like Feynman or Einstein, either of them said, if you're not able to explain something adequately, you haven't understood it well enough.
So being a mentor to somebody who's a junior or not as much experience as you are always helps. And they bring in new perspectives that you probably never would have thought of. So that's something that you would have to definitely look into. And I think this is extremely important for data scientists, develop rapport with people and get them to invite you into meetings that you have no business being in. Just sit there and listen to what they are discussing, especially if it is business related, because you will then understand why business is asking for something rather than what they're specifically asking for. So just go there. I used to do this. I used to just say, Hey, can I just join here? And then I just, I used to sit there. I used to say, I used to say, can I be a fly in the wall? And they'd be like, okay, CC Brijesh. And I just go and listen. And I developed so many perspectives. That's kind of how I developed my know-how of some of the domain knowledge. Otherwise you won't get that domain knowledge. You will then hear more problems in the business than you think, Oh, you know what? I can actually solve for this. That gives you some more acceptance and buy-in as well in the company that you are working for.
Fantastic. Well, we have hit the top of the hour. So I have to jump in and stop you Brijesh, but please follow Brijesh on LinkedIn, where I think he might share more of this wisdom. We thank you for hanging out with us. I cannot wait to see you on Tuesday at the Data Science Lab where Hadley Wickham is going to be live coding the week's TidyTuesday dataset. And by the way, the TidyTuesday dataset for next week has not been set yet. So if you would like to submit an unhinged dataset for TidyTuesday for Hadley to live code, you can do that today. I think that the deadline is today. John Harmon, who does TidyTuesday, is going to choose the dataset tonight. So hop on that. Go check out the Call for Talks for PositConf 2026. And thank you so much for hanging out with us, Brijesh. This was wonderful. I had such a good time. Thank you, everybody.