Transcript#
This transcript was generated automatically and may contain errors.
Hey there, welcome to the Pozit Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.
Can't wait to see you there. I would love to introduce our featured leader this week, Jim Weiss, Chief Risk Officer, Commercial and Executive at Crum & Forster. Jim, I am so glad to have you here today. Could you introduce yourself? Tell us a little bit about what you do, and something you like to do for fun.
Yeah, hey everybody. Thanks for having me on. Good to be in the hot seat.
It's getting a little warm. Yeah, Jim, I'm a 44-year-old data science person in the U.S. property casualty insurance industry. I lead a small group of data scientists focused on ML and ML ops. They're all way better at data science than me, and also have responsibilities for monitoring emerging risk longer time frame issues. So that's kind of cool and fun. What I do for fun when I'm not doing all that stuff is like watching professional wrestling, love WWE, wearing an AEW hoodie right now. So if any professional wrestling fans in the house, holla.
All right, you're gonna have to let us know in the chat here if you are a professional wrestling fan, and if you know of any professional wrestling data sets, because that's always like the most fun thing we talk about in the chat, like data sets that have to do with our fun things that we do. Okay, Jim, I would love some background for all of us so that we can ask good questions about Crum & Forster specifically, and maybe a little bit of background on insurance.
So who is Crum & Forster? What do you do? What types of problems, does your team help them solve?
Sure, Crum & Forster, a couple of good books. I mean, anyone's actually gonna go read the books after this, but if you really want to, a couple of good books you might check out are the Once in Future CNF by our CEO, which is the least self-referential corporate biography you're ever gonna read. And also the Fairfax Way, which is Crum & Forster's parent company, which is a little bit more self-referential. But essentially, we're a specialty property casualty insurer, and we mostly serve commercial businesses. So if you buy insurance on like your home or your car, it's like that, but it's like for businesses. So if I have like a pizza shop, I need like workers not for my employees, and I need auto insurance for my vehicles and the people who are driving them, and I need to buy property insurance on the pizzeria, stuff like that.
And where data science factors in a lot of ways, but one example where my team often factors in is just trying to find a good, fair, equitable price point for each policyholder who applies for insurance with us. So for that pizzeria example, I want a price that basically aligns with their risk. So I might be looking at stuff like how much pizza do they make? How many customers do they have? Where are they located? How much foot traffic is around the pizzeria? Stuff like that. And we're typically using a regression model. So I told Libby before this, like love regression models. So feel free to challenge me on that.
Data types and problems in P&C insurance
Do we have any regression stands in the chat who want to talk about regression? We always do usually have some modeling questions, but that is a great context building question as well, is like what data types do you see? We all come from many, many different industries. I actually do have a background in PNC insurance, but I think most of us are from wildly different backgrounds. So what kind of data types would you see in your day-to-day or your team would see in your day-to-day? And what is an example of a problem that you feel like you've solved lately or that you've been working on lately? You can stay high level.
Sure. Like no problems ever truly solved permanently, right?
I guess to kind of take it back up a level to the data question, insurance is a super data rich industry. Historically, a lot of the data has been dark. So I guess kind of taking it to the world where I work and operate the US property casualty commercial insurance industry, a lot of our data we're getting starts in like just unstructured documents. So we're like getting PDFs, we're getting like spreadsheets, words with just information about what kind of insurance the person wants or the business wants, the attributes of the business, et cetera, pretty non-standardized. In the previous several century history of the industry, a lot of times that's just been keyed in by people. As Gen AI in particular has become more proliferated throughout the industry, Gen AI is doing more of that work, but ultimately coalesces more towards structured data sets, which have what policyholders paid for their insurance, some of their risk attributes, like the stuff I rattled off there.
And then most critically in a lot of cases, and for a lot of my team's projects, whether or not they had a claim and how much that claim cost. So types of projects I'll typically plug in type of problem I'm trying to solve is might be looking at a portfolio of insurance that hasn't been profitable for us for some reason or other, try to figure out why using data science techniques such as regression. And then if we can isolate factors that are contributing to unprofitable pockets of our portfolio of policyholders, then we can either make pricing adjustments, potentially decide to exit that market, or most ideally for everybody, identify things about their risk profile that they can adapt and evolve.
And maybe to give a specific example from one of the many, many types of insurance Carmen Forster sells, I would point to cyber insurance, because there might be unfavorable risk attributes in a policyholder applicant. They might have poor website security, but we can look at all the attributes of the website, build a model that figures out why a particular type of website or website hosting structure is propense to losses, and then either introduce some sort of discount or surcharge based on your hosting footprint, or alternatively recommend ways you could change your hosting footprint in order to make your website safer and reduce what we're going to have to pay out in insurance claims. That's wild. I had not thought about in this day and age, that being part of PNC property and casualty insurance, because digital stuff is kind of nebulous and non tangible, right? That's a whole special niche that I really hadn't thought existed.
Regression modeling poll and community discussion
So that's really interesting to hear about. I put a poll in the Slido, by the way, to ask everybody if they use regression modeling in their job. And at first, the votes were very no heavy. We are now tipped over into 75%. Yes, but we're still getting votes. So y'all go vote. Tell me if you use regression modeling in your job. I will tell you that I don't think I've ever used regression in a data job. I think I'm a no. I don't think I can vote because I'm the moderator of the Slido. But consider one more vote in there. Lots of other things. Tons and tons of descriptives. I've done lots of surveys. Libby, could we do another poll, whether or not people think I'm a dork because regression is my favorite thing ever?
Yes, no, maybe. Is Jim a dork because regression is his favorite thing? We can weigh in in the chat, though. Do you identify as a dork? If you like it, that's what I was going to say. Gregory says, we're all dorks here. I think you're in good company. Okay, so we have 39 votes in now, 40 votes in now, and we are still at 78%. Yes, using regression modeling in a job. And I just realized I did have a shiny dashboard that did use regression modeling. Okay. It's okay to use regression.
Yes. Yes. Brent says, we all like a good regression model.
Okay. I think we're on the yes train for the most part for the Hangout crew.
All right. I better hop back over to the question side. If there's any other polls that we want to give, let me know because we can pop another one in there and we can answer it. Was there a legit question, Libby, whether I identify as a dork?
No, that was just me. I'm asking the chat. Do we identify as dorks? I think that we all do. I'm sure you do. It would depend on the dictionary definition. I think there's three or four definitions. I'm not looking at the dictionary right now, but I think there's three or four definitions of dork. And there's one that's a playful expert in a topic. I would definitely say yes, but then an unlikable person who's epically focused on a topic, I would not identify with that one.
Okay. Not narrow-minded. Amelia says, I had no idea there were people out there not using regression. Well, data science is not one thing. And that's the best thing about it, in my eyes, is it's so broad and so interesting. And there are so many people that also really work on a lot of descriptive, like, here's just what's happening, right? And don't have a lot of need for correlating things or figuring out what might be happening in the future. We better hop over into Slido because Russ has a question for you that is very much on the topic of regression. Russ Conti, would you like to unmute and ask your question live?
The Tweedie distribution in insurance
What types of regression do you use? I've been working with, like, logistic regression the last couple of weeks. But what kinds of regression do you use?
Love the question. Thank you, Russ. And by the way, happy to talk 45 minutes about, like, why I love regression. I guess the specific type.
Use a little bit of logistic. I think the most popular loss distribution in the history of insurance would be called the Tweedie. Has anyone heard of the Tweedie distribution, named after Tweedy Bird, the Looney Tunes character? Yes, Donald. I guess, can Donald unmute and give us, like, a primer on the Tweedie, or shall I?
I was just going to say, Donald would absolutely love to, I bet. Donald, do you have your mic working today? Donald is, like, the king of very niche statistical things.
Donald. We're going to get Donald to unmute.
I'll give it a stab, but if Donald comes off mute.
No, no, sorry, he can't. Okay, that's good. We can't all unmute. But Russ did add a Wikipedia link to the Tweedie distribution in the chat for us, so we can all look at it.
It's really good for insurance problems, because when you're looking at insurance profitability, there's at least two dimensions driving them. One is whether or not you're going to have an insurance claim, whether or not your policyholder is going to have an insurance claim, and the other is whether or not your policyholder is going to have an insurance claim, which logistic regression might drive at. Or you could look at, like, a Poisson, which gets into, like, how many claims your policyholder is expected to have. But then oftentimes of, like, greater significance in our world is how much the claim is going to cost. So what makes Tweedie a beautiful thing is it kind of combines those into a single distribution with a parameter that decides how much weight each of those two things get. And you can either, like, grid search that or just judgmentally apply a selection. The most popular selection in the history of insurance is 1.7.
The most popular selection in the history of insurance is 1.7.
But at the end of that, you get kind of a scorecard of the different risk factors in your model and what they're influencing. The one downside, I would say, is the model coefficients don't really provide attribution into whether it's, like, the claim propensity or the claim costs that's driving your outcomes. OK, fantastic. Russ, thank you for asking that question.
Community poll: types of models at work
And I'm going to put another, I guess you could call it a poll up, but it's going to be open text. And it's going to be what types of models do you use at work?
And I hope that when these answers come out, you will be able to see them. Like, everybody else should be able to see them. And if not, don't worry, I will share them with you. Who asked what kind of models question?
Let me know if you can see it. It's just what types of models do you use at work? And who did the question come from, Libby?
From me. I'm just asking the chat. I'm asking the chat and everybody can answer the survey in Slido. I hope that it's working, y'all. Y'all need to, like, test this out and tell me, because this one, for some reason, it looks like it's not working. But we will hop back over to our questions and ask another one from our list here, which is an anonymous question that says, in your insurance modeling career, what is the most unusual or unexpected variable that has contributed to one of your models? I like that because it could be all kinds of things. We'll put you on the spot here.
Yeah, absolutely. And by the way, massive FOMO that I'm not in the Slido. Like, it's just, like, killing me that I can't answer these questions, Libby. I'm, like, foaming at the mouth to answer what types of models.
I'm going to give them to you. I'm going to ask them to you. Don't worry.
But I guess question, like, weirdest, definitely one came to mind, but I forget the exact verbiage of your question, Libby.
The most unusual or unexpected variable that has contributed to one of your models.
Cool. Definitely have a memorable example. I forget whether I planted this question or you're ambushing me, Libby, but either way, it's cool.
This is an anonymous question from Slido, so somebody else asked this.
Cool. I don't think I planted this question.
There are no plant questions. I promise you.
Cool. So essentially, in insurance models, especially ones that are used to determine rate, oftentimes they have to be vetted by regulators. And that's why the ability to, like, explain or at least speculate cause and effect is very useful. And of course, regression is a very helpful tool in helping you articulate potential cause and effect underlying your model predictions.
So I remember one model I had filed with a regulator with my prior employer.
Wasn't like a Jim Weiss production. There were a number of people involved in it. And used a ton of variables, like weather, geodemography, vehicle information from the area. Pretty much every external data asset under the sun was factoring into this model. And one in particular, the exact variable that landed in the regression model was called kurtosis of average daily humidity. And this was predictive of how likely a car was to crash. And we had a little bit of internal debate, like, should we file this one? What does this even really mean? So we eventually filed it in all 50 states. And I think we got approval in all 50 states or close to it, because there was strong statistical evidence surrounding this. But I just so happened to be at a conference. It was not PositConf, unfortunately. You should all go to PositConf. But it was not the conference. It was actually an actuarial conference. And one of the regulators who had actually reviewed this filing was at the conference. And they were mocking this variable as the dumbest variable they had ever seen in any of the hundreds of insurance filings they had seen over the years. So I thought, A, it better be known for something. So that was cool. But also, what is the chances that I would be at that conference when they were mocking my variable? I have to think that's pretty statistically improbable.
Well, what's funny is, as you were saying that, I was thinking, oh, man, what are the measures of kurtosis? And what granularity do they have? Because I remember kurtosis being something that I would think of as negative 2, not something that's 1.953 that would be really specific. If anybody doesn't know what kurtosis means, it is like when you're looking at a distribution, there is a shape where it's either super sharp and tall, right? Or maybe it's got these really flat tails, and it's like a lower hump in the middle, and then a longer tail out here. That shape is the kurtosis of a distribution. And there is a measure of it. But to me, that seems so, so vague and interesting. Okay.
Handling model differences and stakeholder communication
Fantastic question that ended up having a fantastic answer. So thank you, whoever asked that question. It was anonymous. I think it was fantastic. I wanted to see if anybody can see the answers to the free text stuff. Because Abigail says, I don't see any answers, but she can edit her response. So I will go through and read some of these.
I asked everybody, what types of models do you use at work? And some said linear and logistic, so still regression. We've got some chi-square, basic descriptive statistics. Let's see, state-space models, boosted regression trees, GLM, GAM. We've got some multilinear regression, some XGBoost, and some K-means. We've got negative binomial, which is what I have never used that before. So new to me, multinomial, logistic, and linear regression. Some mixed linear regression is pretty popular in here. Yeah, more mixed models, four-parameter logistic, hierarchical regression, more regression, time series forecasting. Oh, time series forecasting for intermittent demand called the Croston method. It's really niche. Oh, I don't know who you are, person who uses the Croston method, but tell us about it in the chat so we can learn.
OK, I hope that was illuminating. Let's head back over to some questions here. There is an anonymous question that says, how do you handle it when you see that the results are significantly different across models? You might have mentioned something like this before. I'm not sure. This is an anonymous question, so I don't have any more details for you. Yeah, great question. Results significantly different across models. By the way, I see a comment on the Weibull in the chat, and I have strong thoughts on the Weibull. But I guess for the anonymous question, I think you can look at the question in two ways. One is differences between your models over time, or the other would be differences between candidate models, both of which solve the problem equally well. Maybe the one I encounter more typically in practice is changes in our model over time. So I feel like I work for a big org, a couple thousand employees, but it's only 1%, something like that, of the US property casualty insurance industry revenue. So we're not huge, right? So the data sets I'm dealing with tend to be on the more thin side. And that's not a friend of model stability. So when we go from model version to model version, oftentimes we'll see, oh gosh, the dynamics in the model massively changed. And we have to figure out, hey, did we screw it up, or did the world change, or was it a spurious correlation? I saw someone mention spurious correlation in the chat. And it's usually a combination of those things.
But I guess from my perspective, the way we typically articulate it is our stakeholders. And the stakeholders hate it, right? They hate when they make a decision on a policyholder's price point, and then the model changes like two years later. And they're absolutely justified in hating it. So I think the best answer is, if we have a reasonable hypothesis for which of those three possibilities it is, world change, we screwed up, we overly anchored to a spurious correlation, just say that, right? Even if it means we screwed up and leaned too far into a finding. And if we don't know, we just say we don't know. But most of the model change considerations from my perspective that I deal with relate to stakeholder management.
But most of the model change considerations from my perspective that I deal with relate to stakeholder management.
I guess maybe to hit the other angle of the question in a nutshell and how we typically deal with it, if we have multiple candidate models that are giving us different views of the same problem, probably the most statistically valid approach to that would just be to ensemble them and come up with some combination that maximizes their respective contributions in an optimized way, bringing them all into the equation. We don't typically do that because it comes at the cost of complexity that would make it more inoperable for our stakeholders and increase the maintenance burden on me and my team. All right, great. Thank you so much for that question, question asker. You were anonymous, so I don't know who you were, but thank you.
Bayesian versus frequentist debate
We have a very hot topic question from Gregory. Gregory, are you available to ask your question live? Gregory Power for Isabella, who's helping me unmute. Yeah, sure. The heated question would be, are you team Bayesian or team Frequentist when it comes to your statistics? I may have interesting opinions on it. My eyesight is terrible. Okay, it's Bayesian. It's a statistical rethinking of Bayesian statistical textbook that I got recommended from the data science hangout group. It's a good one. I'm just curious about your opinion because I think both are valid in different contexts. I come from a Frequentist background, then I realized that there's something that I can do to help combat stuff in different fields. I'm just curious about what your thoughts are on Frequentist versus or a mix of the two or maybe some secret third thing that insurance does because insurance is wild.
Awesome question, Greg. Thank you. Totally feeling the heat. Can hardly take the heat. Maybe I'll start with secret third thing and then get back to the first two options presented. I think the secret third thing in insurance is making stuff up. And I don't say that in a bad way. Sometimes in insurance, you got to make stuff up. I'll give an example, AI risk. A lot of orgs, a lot of our policyholders are using AI, but we didn't really have gen AI in the world in a scalable way until two years ago. So, there's no data to analyze. You have to find similar data sets. You have to make a lot of assumptions. Sometimes you just have to make stuff up that's based on solid reason.
If we're kind of isolating on... Actually, let me correct the transcript. Jim Weiss does not make stuff up. Make stuff up is a colloquialism for making expert judgments based on past historical experience and analysis in a way that best addresses stakeholder needs. There we go. I was hoping you would do that just because I was like, Jim, this is on YouTube. You're going to be on YouTube saying you make stuff up. And if anybody snips the transcript to only show the portion where I say I make stuff up, I will be pursuing legal action against you.
And meanwhile, of the other two options, I would say I'm more spiritually Bayesian. In practice, I'm more of a frequentist. I forget if I mentioned I'm an actuary by background. And basically, the entire academic history of PNC actuarial science is kind of Bayesian in nature because it's anchored towards what's the present state of the world reflected in the way you've deployed your business and then developing incremental changes on top of that, which you have a prior. The prior is the current state of the world. With respect to actually, hey, all the answers to Libby's question about which models you use, am I doing an MCMC to get my answer to my underwriting profitability problem I'd be working on today? Absolutely not. A, because I don't have time to set that thing up, even as much as Posit Tools have democratized that. But B, because from my perspective, and prove me wrong here, argue it. Just say I'm a dork, bad definition because I don't do business this way. But setting up the prior in a way that's effective in the MCMC requires so much judgment and touch, from my perspective, that it essentially takes you back into that making stuff up category. And if I'm just essentially making stuff up, so to speak, then I'd rather do that in kind of a way that's conversational than trying to force that into a prior innovation approach. I guess, Greg, how did I do? Am I off the hot seat on your question? Okay.
I think that Greg is satisfied. I appreciate you. All right. I wanted to make sure everybody knows what MCMC means. It's Markov Chain Monte Carlo. So that's something that you could Google, if you would like to pop that in the chat. And then also wanted to make sure that you saw the link to Statistical Rethinking. That's the book that Greg mentioned. It is by Richard McElrath. I always say his last name, McElrath, but I think it's McElrath. He has all of his lectures on YouTube. There's a full series from 2023. He rerecorded them this year. I don't know if he's finished rerecording them, but there's a 2026 playlist that I linked in the chat. They're fantastic. Definitely go take a look at that because the book, if you're just reading it alone, can be a little bit dense to get through, although all of the language is super approachable. He has one more lecture to go on that 2026 playlist, it looks like Dylan's saying. The 2023 one is great. It's complete. He also has, if you happen to be a professor, he has a couple of other playlists of the 2026 ones that'll be like a beginner A and a beginner B, I think, where on his website, you can see he has like, okay, if you just want to add in parts of this textbook into your section of teaching, he helps you do that. So go check him out as a resource.
Fraud detection and automation
Enough of that, enough of teach your brain. Let's hop back over to Slido. And I see a question here that has a couple of answers from Gregory on it too. So Gregory, thanks for the replies on those. It's an anonymous question that says, healthcare organization is interested in identifying fraudulent claims. Currently, they're looking at Excel spreadsheets, same time, different person. Do you have any advice on a better way to guide them to automation? This one is a very niche question. So I'm glad that Gregory answered here. Gregory said, first idea would be to use any kind of database so that you could have more safeguards around your data. Good idea. And then libraries like pointblank are super helpful for validating data. pointblank is in both R and Python and is a fantastic package by Rich Iannone. Do you have any thoughts on this specifically, Jim, about better ways to guide people to automation?
Maybe, and I can kind of tell her a little bit to the fraud aspect as well, perhaps. So raise your hand if you like fraud. I didn't mean to raise my hand. To be clear, I was raising my hand to illustrate you should raise your hand if you like fraud. I do not personally like fraud. No. Okay. And the reason, like, nobody raised their hand, right? Okay. Nobody likes fraud, right? Because fraud drives up the costs of insurance and any service, really, for everybody. There might be like a very, very small pocket of society that says, hey, insurance fraud is good because insurers are these big evil corporations, which we're not. We're making the world a better place every day by providing valuable protections in people's greatest time of need, right? But there's a small pocket of society that thinks fraud might be justifiable against corporations. But that's not true because ultimately what the fraudsters are doing is they're taking money, putting it in their pocket, and then charging that money to everybody else because that cost ultimately gets borne back on all the other consumers of that service, whether that service is insurance or anything else.
Now, to solve problems like fraud, you need a lot of data. As data scientists, like, a lot of us think we can solve any problem. That's not true, right? Often, if given all the tools needed to solve a problem, we probably could. Oftentimes, we don't have all the tools. And for something like fraud, and getting back to the automation problem, like, yeah, Gen AI makes it light years easier to take a variety of disparate spreadsheet formats or PDF formats and reshape those into a single uniform format that then makes it a lot easier for you to latch on to and use as a jumping off point for your fraud model. But your fraud model is going to be based probably on like a half a percent or 1% sliver of the universe, whereas the fraudsters are exploiting the fact that nobody sees the whole universe. So the way the insurance industry has addressed this, and have only worked in one
industry, but my guess would be other industries go about it in a similar way, is coopetition, which is it essentially makes sense to partner with your competitors to solve a societal problem. So in our industry, there's an organization called Ferris Coop, it's also my prior employer, who maintains a massive data set with nearly 100% industry participation, where their data scientists do pattern recognition across like an entire industry data set. So sometimes the best way to solve the problem is to realize you're not the person to solve the problem.
So sometimes the best way to solve the problem is to realize you're not the person to solve the problem.
That's a very clear headed answer. All right. Thank you, anonymous asker, for that fantastic question. And then we're going to head over to Natasha, who has an asterisk, which means let me ask that for me. And I will. Natasha, you always ask great questions. Thank you. It is, what software do you use? Gregory says, I'm going to bet money it's SaaS. Gregory, it's not a bad guess, because I come from BNC insurance. And let me tell you, it is the land of SaaS, banking and insurance. All right, Gregory's shrugging like this. Jim, can you clue us into the tools that your team uses, be that language of choice, IDE of choice, data store of choice, overarching toolset of choice?
Tools and language of choice
I get so much shade for my toolset. I'm not going to get shade from this group, I'm guessing, but I get a lot of shade. I don't think so. We're all friends here. Love RStudio, greatest IDE in the history of human creation prior to Positron, which is now the greatest IDE. prior to Positron, which is now the greatest IDE in the history of human creation once we work out some of the bugs with Bailey and the team in our company deployment of Positron.
But I'd say, despite my love of R, and I think the reason I love R is A, the IDE, but B, the lack of finickiness in the syntax. I just want to jump off a ledge when I'm coding in Python because it's so temperamental. Vibe coding, of course, has self-temperamentalness for all languages to a very large extent, so this commentary is born largely two years ago. But the non-temperamental nature of R and the statistical focus, I think, have made it very good for my job.
The reason I get a lot of shade is because my team and I often use R, maybe not for what its core capability and strength is, but also for a lot of our ML ops, deploying our pipelines, deploying our applications, deploying our APIs, having a lot of agents running in the background all the time to do our work. And while I'd say it can do those jobs, but it can't necessarily do them in a way that complies with business partners' SLAs and achieves, like if you're doing a plumber API, for instance, that's not necessarily going to get the same response time as if you did the exact same thing in Flask, for example. So love R, get a lot of shade from my coworkers, some of it earned, some of it because they're haters.
Oh, we love everybody here. We have R users, Python users, SQL buffs, we have Excel people, I am, you know, I use spreadsheets every single day still. And we have some conversations happening in the chat that are all about RStudio and Positron. And someone says, oh my gosh, the chat is like jumping around like crazy. Someone says moving to Power BI, and that is like a big shift that I'm seeing from a lot of people is moving to Power BI from something else, from Tableau, from Looker, all kinds of things.
Any of Julia users? Or Power BI, I haven't seen any Julia yet. Okay. But I've seen a few, several Power BIs. So that is a trend we are seeing.
And then Annalise said, anyone still use SPSS? I shamelessly loved SPSS. I had to use it in all through undergrad. And like, I still kind of miss SPSS output. I'm looking at a regression output between like SPSS and R. I like SPSS better. Like, I think it's a better output. Don't hate me, y'all. Tom says, yeah, but trying to switch to R. David Under says, mostly in university research. Yeah, a lot of people use it in school. Okay. Okay. We will move on. Thank you so much for that fantastic question, Natasha, so we could talk about it. And thanks for all the love about RStudio and Positron. We obviously love them too.
Lolu's question on DAGs and instrumental variables
And Lolu, you asked a question and I would love you to ask it live if you're available. Can we find Lolu? I don't see. Not find Lolu. Maybe Lolu had to go. Although maybe Lolu has his name Oh, wait. Lolu, I see you. I see you. Jabril. I know that your name is not your nickname is Lolu, right? So I know that there's like two names. I just didn't know who it was. Let's get you to unmute. And I'll put your question in the chat too. Okay.
Thank you. Sorry about that. It was just switches sometimes between Lolu and Jupyter. They're both my names, but either or. I was just actually curious on your workflow. Do you ever form your hypothesis using directed acyclic graphs tags? And do you ever use instrumental variables in your models? And which one have you found is the most common instrumental variable used in the industry? I'd be lying if I said I was conversant in all the syntax. I'd probably have to go back to some of my college tech books to remind myself what an instrumental variable is. I guess, could you either explain instrumental variable to me?
Things like distance to a certain place. This kind of things. What else? I know there's some common ones that people use. Let me see. What's the next one? Distance earning power. Time. Drive time. Something that's correlated with one of your main variables. But not directly correlated. It's just there in the ether, but the hypothesis is just there.
Thank you for the education on instrumental variables. I'm definitely going to drop it into conversation now. I sound smart. We're all going to go Google it right now. And I guess, yeah, from my perspective, in general, highly regulated industry, getting buy-in from stakeholders who are ultimately the gatekeepers of whether or not your model insights make an impact on your bottom line. Explainability is a big plus. Plausible correlations to real world outcomes is a plus. So things like distance to fire station for property insurance absolutely rises to the top of the list for types of things we're looking at.
Auto insurance. We've spent a lot of time, my team and myself and our business partners, working on the commercial auto line of insurance. So you might buy personal auto insurance from State Farm or Progressive or Allstate or Lemonade, new entry to the marketplace. But businesses also need auto insurance. And on kind of an operating basis, not counting like ancillary sources of income, like their investment portfolios, that line of insurance in the US has been losing like $0.10 to $0.15 on the dollar every year on an industry level for like the past 10 or 15 years, which is insane. Nobody can have a product that loses $0.10 to $0.15 on the dollar on a year-over-year-over-year basis unless you're a Gen AI company, in which case it's perfectly cool.
So for commercial auto, we build a lot of models to try to predict and understand where our more and less profitable segments are and the type of variables we're looking at for stuff like that. By the way, no trade secret information being disclosed on this call. Just to be clear, this is common industry patterns for commercial auto line of insurance. Weight of the vehicle, where the vehicle's operated in a general geographic sense, like the traffic density of the area, like how far outside its typical operating area it goes, the driver's records and ages, how much insurance you're purchasing, the limit of insurance for physical damage coverage, how much is the vehicle worth. I think all of these things lend to real world explanations for why you would be less profitable for vehicles you're insuring that have these types.
When you get back to something like I mentioned earlier, the kurtosis of average daily humidity, a little bit tougher sell, right? And you can have really strong statistical evidence, which we did at my prior company when we developed that model and sought regulatory approval on that. But it makes the stakeholder buy-in a lot tougher because the real world causation isn't at the fingertips as much. Perfect. Jim, that was a fantastic answer. Lulu, that was a fantastic question. You took us in a great direction. Thank you so much.
Yes. I just wanted to get some clarity on the DAGs, if they ever use DAGs in hypothesis itself. I am also out of date on the syntax there. Could you elaborate a little bit more on DAGs? Directed acyclic graphs like drawing, confounders, direct impact, direct effect, co-confounders, things like that, before actually building the model. So you have the theory like, okay, this is how it should behave. But then eventually, if it doesn't behave that way, then you actually have to go back and retweak that, okay, it doesn't meet the theoretical hypothesis. Yeah. I was just wondering whether it's like industry standard or something that's just in academia. Probably somewhere in between, I guess, from a deployment perspective.
Bias management and confounding effects
So I actually wrote shameless plug. I think it's probably clear I'm a very shameless person, it's become clear. So I'll give a shameless plug consistent with my character. I wrote a paper on bias management in property casualty insurance. Either just Google Jim Weiss bias or go to my LinkedIn, click a link. And it looks at confounding effects between protected classes in society and insurance outcomes. There's an evolving body of legislation and regulation in the US that essentially challenges insurers, especially personalized insurers, private insurers, insurers who are selling to individuals. Less so commercial insurers like mine now, but I expect it to kind of work its way down the value chain eventually.
This evolving regulation and legislation challenges insurance companies to identify these potential correlations with protected classes and root them out of their models and business practices. So from that perspective, identifying potential confounding effects, and specifically ones that are deemed by regulation and legislation and just general morality to be problematic, is becoming critical for insurance data scientists' work.
That's kind of an outlier example, though, where we actually need to give specific, dedicated, attributed thought to things like confounding variables. In a more general sense, for regression models, how we're typically dealing, confounding effects is either... I would just rattle off typical approaches we take. We might put a control variable, dummy variable, in the model to siphon off some of that signal. We might just... General confounders we do want in the model, we'll look to not have multi-colonial variables, whether that's through selection or some sort of dimension reduction like PCA. And probably most common in our world, because we're stretched, right? So we're not going to do a PCA. We can't explain a PCA to our stakeholders. We'll probably just do a penalized regression, and that in and of itself will weed out some of the multi-coloniality for us.
Now, with respect to an ongoing basis and, hey, did we pick the right stuff? And do we have to take a remediation? From there, I think it just gets back into classic model life cycle management and just looking at your drift, looking if it's still maintaining its predictive power over time. And as soon as we see red flags in our leading indicators, getting back into that model and taking remediative action as soon as possible. Perfect. Thank you, Jim. Thank you, Lolu, for the questions. I will hop back over into Slido. And I also wanted to call out in the chat, there's some good conversations happening. Isabella, Bailey, and I all put your paper in there.
Oh, cool. Awesome. Shameless plug worked. And I apologize that I'm like 10 or 15 years out of date on my jargon. So thank you to the group for bearing with me. I do not think that you're out of sync with the jargon. I think it's okay. So if you are looking for his paper, we've triple linked it there. There was also a little conversation about how Lemonade had stopped supporting Illinois a few years back. And I thought it was worth saying that state regulations for insurance actually have an impact on whether or not an insurance company is going to cover a state. And there are some states with some that are very interesting. Massachusetts has some has some heavier regulations around insurance that makes companies pull out of them. Florida, North Carolina, New York, these are all states where like what I was an insurance agent who was licensed in almost all 50 states, but I was not licensed in Florida. I was not licensed in North Carolina. Those are like special places. So if you live in one of those places or a state where your insurance has to be different, or you don't have all the carriers, it's probably part of it. Jim, do you have any insight on that?
General nature of regulation and how carriers operate in markets. I don't think I have as targeted examples as you do because I'm very back office. Also, I've come from like auto insurance, where a lot of those are more noticeable to individual consumers, because we're not all getting business insurance in these places, but we are all probably getting auto insurance. I can give kind of a few examples in general practices from my experience. So they are kind of generally responsive to your question and different approaches we might take. I would say one, insurance is highly regulated. So every action, every underwriting decision needs to be
approved by a regulator before you go live, usually. Some of the exceptions in commercial insurance, there has to be some recognition that regulation can't stand in the way of a well-functioning market. So one way the regulation allows for a well-functioning market while still keeping control is by allowing a buffer around what you'd said you'd do. So you can file a price point with regulators and they say, yeah, that's cool. And you can operate within plus or minus 25% of that. That gives you some buffer to operate if market conditions are shifting, as long as you do the corresponding documentation that the regulation also requires. If the regulation is absolutely prohibitive of what you're trying to do in the marketplace, there's a few examples of what you can do. One would be after many of you may recall the wildfires and tragic wildfires in California in early 2025. And there was a lot of news coverage at the time about insurance companies just exiting the market. And what they were basically saying there is we can't operate under those conditions. And that actually resulted in some movement in the public policy conversation around how that market was managed. The other option to essentially improvise outside your boundaries as established by the regulation and legislation would be to operate in what's called a non-standard or ENS market. So essentially, if the regulated marketplace is not effectively servicing the customer's need, then the customers have the option to go outside the regulated marketplace, as long as they can prove that the marketplace isn't serving their needs, and interact with the insurers serving that marketplace in a way that's not regulated.
So that's another way you can operate in difficult markets. But either of those options I mentioned, buffering, moving the regulatory conversation, operating outside the regulated domain, all of those have obstacles and operational burdens associated with them that frequently just make it more preferred to operate within the regulated marketplace.
Career advice and personal growth
All right, fantastic. Sorry, I took us a little off track with the regulatory question. So sorry. We have so many questions left in Slido. We will not get to all of them because we have seven minutes left. So I want to see if we can get at least two more in, and one that I would love to do for Rachel, because Rachel's not here, everybody. Say we miss you, Rachel. Do we have any Rachel's on the call?
Well, not our Rachel. I actually haven't seen any Rachel's on the call. The question is, do you have any career advice for us? Is there something you wish that you had known when you were first entering the industry? Yes. This might be self-evident, but it wasn't self-evident to me until I was in my mid-30s, having my midlife crisis where I was under a mountain backlog at my prior company, and all my stakeholders were mad at me. And I was pulling all-nighters to keep up with my backlog. And I guess what I found when I was going through it was I really found out who my friends were, and it surprised me a little bit because it wasn't necessarily who I thought it would be. It wasn't even necessarily STEM people. I remember one of my coworkers at my prior company who really helped me through it was a lawyer named Bob Becker, who is not on this call, I'm pretty sure. And he basically gave me a lot of advice for how to manage a backlog. And it turns out there's a lot of knowledge and practice and discipline in the legal profession that can actually be applied back to managing a STEM backlog. So I guess where which I'd known that like 20 years ago, but I think the advice I would kind of cultivate that into is if you see somebody struggling, help them. And that doesn't need to be STEM help. Like that can also just be like emotional help, right? And that pays back in a lot of different ways because A, they're going to have your back when you're going through that midlife crisis. Maybe I helped Bob Becker. Maybe he was just a great guy, or maybe I made a good impression on him at some point that he wanted to help me. But B, you establish this more diverse network that brings a broader set of perspectives into your context. So if you provide a lawyer, emotional help, then you don't have a lawyer in your network who broadens your context and gives you more tools in your arsenal to solve problems. So that would be my advice.
if you see somebody struggling, help them. And that doesn't need to be STEM help. Like that can also just be like emotional help, right? And that pays back in a lot of different ways because A, they're going to have your back when you're going through that midlife crisis.
I think that's fantastic advice. Be a good person. Be nice to people.
I love it. It kind of reminds me of Elliot Murphy's advice when he was here with us. He's one of the VPs of engineering at Posit. And his take was like, personal relationships are going to outlive any company, any relationship with any company, like the people really, really matter. And as you were talking about this, I was thinking about things that Isabella and I have hopped on together last minute to get done together, coordinating quickly over Slack or Zoom or something. Yeah, help people out. Okay, we have time for, I think, one more question.
Using AI at work
I'm going to head over to the Slido. And I wanted to get in this question that has risen to the top here. It has the most votes. And it's, how much do you and your team use AI to help you along? It's an anonymous question. A lot. I mean, we're vibe coding everything now. Everybody is, I think. I would say, I don't know how weird we are. My sense from the public reporting is we're not super weird. But at least in my work and with my team, we're still looking for ways to go from kind of POC to production and develop those really high-value use cases at an enterprise level. It is one of those things that I think everybody can pop in the chat and add their perspective on. How are you using AI? I will share that I am using it for non-data science things that are still development that are helping me do my job better. So like last week, I developed a Chrome extension that's helping me do things easier every single day. It kind of feels like I'm cheating now. But I don't know how to develop a Chrome extension, but Cloud Code sure does, right? And so I was able to iterate, package it up, give it to my coworkers as well. Like, hey, do you want to use this extension yourself to help you? That's amazing. It doesn't have anything to do with data science or analysis, but the possibilities are kind of endless.
Libby, I'll give a kind of a quick weird case I have that might be helpful to people. So we have Copilot and all the foundational LLMs in our org. I like to use the LLMs to psychoanalyze myself. So I think it's very important to me as I interact with my stakeholders that I have a consistent philosophical core that I'm articulating and presenting to them. So what I like to do is to expose all my transcripts and communications to Copilot and then evaluate kind of the ethos behind how I'm interacting with them and use it to understand kind of my own philosophical core and evolve that. So I think it's a good introspection tool and thought partner in a way that folks might overlook if they're purely focused on transactional use cases that deploy to kind of their customers' use cases. You can be a customer as well, and it can help you on a personal growth level.
I think it's a good introspection tool and thought partner in a way that folks might overlook if they're purely focused on transactional use cases that deploy to kind of their customers' use cases. You can be a customer as well, and it can help you on a personal growth level.
Absolutely. All right. Well, I will end us, if you can, lightning round this one last one. It's an anonymous one that says, how does your team span expertise in such varied fields? For example, you talked about cybersecurity versus climate change and all kinds of things.
I think working in a diversified org helps a lot. My team is all actuaries who are basically insurance statisticians. So we're not diverse at all, right? If you're in a smaller company, I've never worked at a small company, so I don't have a lot of advice, but I guess my advice is work for a big company where there's a lot of subject matter experts and utilize them.
Fantastic. Yeah. And meet people, make friends, right? And talk to a lot of different people about a lot of different things. I'm very lucky that I get to do that. Well, this was so much fun, Jim. I think that you being so wacky and fun and like loosey goosey made this just the most fun ever. Everybody asked fantastic questions. Thank you for indulging me in my very specific property and casualty regulatory question that nobody else cared about. Thank you to everybody who asked a question today, who answered our polls. I put the results of the polls in the chat. I hope that you saw them. If you'd like to save the chat, three dots at the top right corner of your chat box will let you save the chat. I am so excited to say that we have a fantastic guest next week as well. If you would like to join us, it's Nikolai Breikov. He is the manager of advanced analytics and outcomes at the Children's Healthcare of Atlanta. I'm very excited for you all to meet him. So come join us next week. And then next week at the Data Science Lab, which is on Tuesday, we have Joey Marshall, who is our own Data Science Hangout crew member. He is often walking on his treadmill with us alongside Nellie. And he is going to be using Cloud Code Live to work through a Tidy Tuesday data set. And the way that Joey works is fascinating to me. He doesn't type anything. He uses dictation. It is like next wave, so fun. And Joey is also just a ton of fun and entertaining. And he's got a great like podcast voice. So come join us on Tuesdays at the Data Science Lab, pos.it slash dslab. Take care, everybody. I cannot wait to see you at Conf, whether you are in person or virtual. Don't forget that registration is open. And I will see you on Tuesday or next Thursday. Bye.