image: thumbnail.jpg
Data Science Hangout | Yu Cao, Exeter Finance | Impacting business with data science
Speaker bio: Data science professional, EMBA student, amateur biologist, rock'n'roll zealot. Not only viewing business problems from the perspective of data science, but also from the perspective of the business.
Yu shared his thoughts on creativity as a data scientist, working with the business, growing in your career, and much more.
A few snippets from our conversation on building relationships with the business. (15:32)
It all started with networking. I work with a very analytical team, so a lot of the people there just want to code and build a fantastic model, not spend time networking with people from other business lines.
But actually, this is not the right attitude.
If you want to collaborate with other people, first you need to get to know them. You need to understand their business – and you need to become their friends.
Usually I reach out to them and do a self-introduction to let them know what I do and what my team can do. Let's go out for lunch, let's grab a cup of coffee and talk for 15 minutes to see what kind of business difficulties you are having now.
If you are a sales team, are you worrying about how to increase the volume of sales? If you are a loan origination team, are you worried about how to increase the capture rate?
Let's talk about that. Then I will see, how can I connect my data set knowledge with the problem you are facing now? I can see whether I can build a model to help you solve that problem you have.
Usually, the collaboration starts with this.
_____________
If you want to build a data science team from scratch (25:51) :
1. Figure out what your team can do for the business. If data science has little to do for the business, it will create very little value there and it doesn’t make sense for you to build a team. You probably need to go to a new company.
2. If you determine that data science can provide value to the business, you need to do some public relations. You need to let everybody, all the leaders in the company, know and understand this.
3. Reach out to the leaders in your organization and ask them to provide feedback for your work. Tell them “My goal is to build a data science team and become a data science leader in this company. What do I need to do to make this happen?” Everything starts from the people, the buy-in from other leaders.
4. If you are a data scientist wanting to move into leadership, you need to show ambition to leadership. The job market for data science is hard. This is the right time for you to show the company that you want to invest in yourself - that you don’t want to just code and build models, but also increase your influence, learn more about the whole business, and contribute to the entire organization.
► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software
Twitter: https://twitter.com/posit_pbc
To join future data science hangouts, add to your calendar here: pos.it/dsh (All are welcome! We'd love to see you!)
Transcript#
This transcript was generated automatically and may contain errors.
Hi everybody, happy Thursday. Welcome to the Data Science Hangout. Hope everybody is having a great week. If it is your first time joining us here today, hello. Very nice to meet you. I'm Rachel. This is an open space for the whole data science community to chat about data science leadership, questions you're facing, getting to hear about what's going on in the world of data science at different companies across different industries. And so every week we feature a different data science leader as my co-host here to help lead our discussion and answer questions from you all. So together, we're all dedicated to making this a welcoming environment for everybody. And I love when we can hear from everyone no matter your level of experience or area of work. It is totally okay to just listen in if you want or participate in the chat.
There's always three ways you can ask questions or provide your own perspective on certain topics too. So one, you can always jump in by raising your hand on Zoom. You can put questions into the Zoom chat. You can always put a little star next to it if you want me to read it instead. And then third, we also have a Slido link where you can ask questions anonymously. And I'm sure Hannah is sharing that in the chat here right now. Just to note, we do share the recordings of each session. So you can always go find them up on the POSIT YouTube or also the data science Hangout site. And one last thing, we do have a LinkedIn group for the Hangout too. So if it helps you with connecting with people or finding each other, we'll share that in the chat as well. But thank you so much, Yu, for joining us today. My co-host joining us is Yu Cao, VP of Data Science at Exeter Finance.
Yay. And Yu, I'd love to kick things off with having you introduce yourself a little bit about your role and maybe because rock and roll was included in your bio, I'd like to know who your favorite band is too.
Okay, sure, sure. First, I want to thank Rachel and her team for organizing the hosting and also giving me the opportunity to host this session. As a data scientist, I always believe that it's super important for people to stay connected to the whole community. And that's why today I want to have an opportunity to talk to all of you guys here. And then let me do a little self-introduction myself. I'm Yu Cao. I'm from China.
Sorry, one second. There's a little echo. We'll mute. There we go.
Okay. So I'm from China. So you can share that. You can tell that English is not my native language. So if there is something I didn't speak clearly, please feel free to let me know. Probably Rachel can also help me out. Okay, so right now, I'm the VP of data science at Exeter Finance, which is a subprime auto lender. I'm also an EMBA student at Cornell University. And so today, these are the two topics I really want to share with you guys about. We can talk about how we use data science in the finance industry, how we use data science to manage the portfolios, to manage the default risk. And also, I want to share with you guys why do I believe that to pursue an EMBA degree is super important for the core development, for the development of leadership for data science people.
But before we go into that topic, I can talk a little bit about myself. My background is biology. Actually, I have my undergraduate degree in biotechnology, and I have my master's in biochemistry in the United States. And after that, I decided, okay, I want to pivot to another industry, which is operations research and data science. So I got my second master's degree in operations research in 2010. And after that, I work as an analyst. I work in the analytical industry. I started as an analyst and transitioned to data scientist and manager of data science, senior manager of data science, and eventually VP of data science now.
And probably a little bit fun for myself, because when I see Rachel's self-introduction, so Rachel said, okay, I play guitar. I said, okay, this is just me. I like guitar. I like rock and roll a lot. So I play guitar like an amateur, but I like it a lot. So I maybe started, I started in maybe 2000. I was a high school student in China. And then I happened to saw the famous video of Smell Like Teen Spirits by Nirvana on a Chinese television. And this song resonates with my personality and my character so well. I started from there. I maybe spent like four, five years listening to rock and roll crazily. I will say that. Maybe I spent like five, six hours every day just listening to all this type of music. You know, if you guys maybe remember, Apple used to have a 160 gigabyte iPod. It can hold more than 40,000 songs.
And this is just a product for people like me. So I kind of spent a lot of time on rock and roll music. I listened to the music. I wrote music reveals, album reveals for some online forums. I even published it in some music magazines. It's done for four, five years. And when I came to the United States, so after I graduated from my, with my master's degree, I got an entry level job as an analyst. And during that time, this for me is kind of like a, I will call it, I have two phases in my life. So during the daytime, I work as an analyst. I was very analytical, very rational, very nerdy. But probably after work, after 8 p.m., I got changed, put on my, all my punk rock gear. And then, I don't know why there are other people, by the way, I'm from Dallas, Texas. I don't know why there are other people in the audience from Dallas, Texas too. In Dallas, there is a famous district called Deep Elm in the downtown area. There are a lot of music clubs, pubs, rock and roll shows, all the, all the concerts hold up there. So I spent a lot of nights there.
So probably, especially every weekend. So after 8 p.m., I just go, went there. I joined the concert. I hung out with a lot of rock and roll bands there. I met a lot of my, a lot of bands I idolized, like Mudhoney, Meat Puppies, Sonic Youth, this kind, all these bands. So I just hung out with them and then probably until 1 a.m. and the next day, I just go back to work. And then such life, yeah, such life continued for like five years until like 2050. There is a milestone. I got married. So after that, things changed a lot. So instead of go to having my regular nightlife, like every day or every other day, probably I can go to a concert show maybe every month, that's fine. I remember the first show I tried to brought my wife to the first show. It was a drum show by the Huan Zhongzhai at the American Art Life Center. And after that, she told me, okay, I can't believe that you listen to that kind of noisy music.
So I kind of decided, okay, probably I need to stop going to a concert that often. And then so since then, I can only go to a concert like once a month. And then in 2017, another thing happened. So my wife was pregnant with our first son, William. So I remember, I can still remember, my last concert I worked was in May 2017. That's the last one. And William was born in January 2018.
And then, so you know, it's just very busy, just crazy. The first year was just crazy. And in 2020, William was almost two years old. I thought, okay, he's old enough. He can sleep at night by himself. Probably I can go back to my music life a little bit more. And then you know what happened, right? The pandemic. So everything closed. All these rock and roll clubs, like the famous The Trace, Club Dada, Bob Factory, everything was closed. So I had to, so of course, I had to put a plan on hold. Sadly, right now, a lot of the clubs, they're not very open again due to this shock.
So I have to ask you, do you ever mix music and data science now?
Absolutely, yeah. I can finish that very quickly. So I believe that for me, there are some data science, but I also want to talk to people in my team. So I said, if you want to really do it, want to do it very well, treat it like art, not just science. In order to be a very good data scientist, I believe you need to be creative. And I believe that you need to possess some artistic creativity. And for me, for myself, the creativity comes from the music.
If you want to really do it, want to do it very well, treat it like art, not just science. In order to be a very good data scientist, I believe you need to be creative. And I believe that you need to possess some artistic creativity. And for me, for myself, the creativity comes from the music.
I love it. I absolutely love live music and finding live music. So now I know who to ask for recommendations whenever I go to Texas then. Okay, so obviously we can't put an ending to talk about my music life here. So probably we can start to talk about data science.
We're good at making it a rock and roll hangout too. But I'm really curious, while we wait for questions to come in from everybody, what made you so interested in data science from the beginning?
Well, it just resonates with my nature. So I'm an analytical person. I'm very focused on details and I like to find patterns from data and use it to make predictions. And I like to work with people from the academia, like from universities, from the research team of big tech companies. So everything fits perfectly as a data scientist. So that's why I started my job as a data scientist.
Career progression from analyst to data science leader
The first thing is that because when I work as an analyst, most of my job is kind of ad hoc analysis. So I put data, I build reports, I do some other analysis and present my results for the leadership team. However, I later found that so instead of just doing something ad hoc, I want to do something a kind of more systematic. Again, I'm always, because my secondary degree was in operations research as basically math modeling. So I still want to build models for all the analytical jobs. And because my first job was for a health care company. So that time I want to say, okay, so can we use all the data to predict the behavior of patients? And I start, I built my first predict model during that time. And after that, I believe that this is what I want to do.
So that's why I want to pivot to, and it's a little, it's a small pivoting from analyst to data scientist. And after that, because I changed to, I switched to a financial service company. So I'll come in doing subprime loans, and then which is subprime auto lender. Because if you want to buy a car, you'll go to a dealership. And if you cannot pay cash, you need to apply for a car loan. And because if your credit score is not that well, for our definition, subprime means that people with credit score less than 620. And for that customer segment, there are a lot of risk there. So we need, you need to build a lot of predict models to predict the risk and then price it to manage the price, manage how we service the law, how we originate the law. So this is why our entire subprime lending business is driven by data science.
And because of this, I feel that, okay, I had my knowledge and my desire, my interest plays a super important role for the business. So that's why, okay, this is the industry I like. And I started as a data scientist and gradually I feel, okay, it's not just about modeling. So you can just say they are building model coding every day, but this is the most basic work. So if you want to do it really well, you'll need to understand the business part. You need to understand, okay, so what problem the business is facing and how can you formulate a business problem into a data science problem? And then build a model is the, I would say, the simplest part.
And with this, I kind of, I actually started to, I mean, spend my time to collaborate with all the other teams in the company, try to understand their work, their work, okay. How is the sales, how is the sales team working? How is the marketing team working? How is the local origination teams working? And by gradually, because I have more and more understanding of the business, I kind of gradually move up in the queer hierarchy, I'll call it. And after I become a senior manager of data science, I feel that, okay, so my data science knowledge is not enough. So if you go to a higher level, it's not about coding, it's not about data science. You'll need, suddenly you'll need some hard skills from other areas. For example, you need to understand accounting, you need to understand finance, you need to understand marketing. And for all of these, okay, I think, okay, I need to learn something, probably.
Of course, ideally, you can also outsource these skills, okay, because if you are a leader, you need some accounting, higher accounting. You need some financial knowledge, hire a financial manager or financial analyst. And if you need marketing, hire a marketing manager. But these hard skills, you can outsource. But there are also another part of probably what I call them soft skills. You can only do it by yourself. For example, like, how to build a production, productive team. Like how to, for example, how to do business negotiations. All these skills, you cannot outsource them to other people. And other people help you do it. You need to do it yourself. And that's why I decided, okay, I need to go back to school. I need to pursue an EMBA program. Probably this is the best time for me.
And I started a program last year at Cornell University. And later, if you're interested, I can explain why I think that this program fits a lot of people in the audience for a while because of its format and because of its goal. And so, but right now, Rachel, I can put the mic go back to you. Yeah, absolutely. And I know we shared in the chat, if you want to ask questions.
anonymously, you can use Slido. But feel free to just raise your hand here if you want to jump in the conversation too. But you, I love the way that you explained working with different parts of the business to actually understand the business problems. And I was wondering what, like, what that process looks like for you of building relationships with different people across the business? Is it like joining some of their meetings? Like, how do you go about that?
Actually, it all started with networking. Because I know a lot of, because I work with very analytical, I would say, very nerdy team. So a lot of people, they are, okay, I just want to code. I just want to build a fantastic model. I do not want to spend too much time to network with people from other business lines. But actually, this is probably, I would say, know the right attitude. Because if you want to collaborate with other people, first, you need to get to know them. You need to understand how their business is. And yeah, in other words, probably most essential, you need to become their friends. So usually, I reach out to them and say, okay, do a self introduction. And this is what I do. This is what my team can do. So let us go out for lunch. Let us go out for, let us grab a cup of coffee. Let us talk about for 15 minutes. Let us see what kind of business, what kind of difficulties you are having now in your business. If you are a sales team, are you worrying about how to increase the volume of sales? If you are a loan originations team, are you worried about how to increase the capital rate? And then let us talk about that. Then I will see, okay, how can I connect my data science knowledge with the problem you are facing now? See whether I can build a model to help you solve the problem you have. Usually, the collaboration starts with this.
Sharing models with the business
And when you build a model for them, how do you share that with them?
I would say that probably, again, that is a very important part because our data scientist has to be a very good storyteller. It is easy to build a model today because we have auto ML, we have wire-array automatic pipelines. Basically, we can talk about that later. So we design our model in a pipeline, like a streamline or something like that. So basically, even if you have no knowledge of data science, you will just change some hyperparameters and then you will click a button, of course in RStudio, and after two hours or two days, that is on the side of your data set, you can get a bunch of models. And a lot of time, mostly, this model performs well.
But this is only the beginning. And then the difficult part is that you need to sell these models, like your own product. You need to let the business owners, stakeholders, buy this model. You can build a very good model mirroring dealer risk. However, you need to be able to let your dealer sales managers understand this model. And they should be able to explain this to the dealers. I cannot reach out to every dealer and say, OK, this is a large GBM model, that builds mirroring risk. They won't buy it. No, we don't want to hear about it at all.
So the model needs to be transparent, needs to be very explainable. And to do that, usually, my approach starts from the easiest one. So usually, if we build a new model, for example, either to help for the sales team or mirror the risk of dealers, usually, we start with a simple logistic regression model. Because, frankly, everybody understands a little bit about logistic regression, or pretends that they understand logistic regression. So usually, if you present a logistic regression model, there won't be a lot of pushback. You'll say, OK, I understand that. Just tell me about all the variables. And then if the directions of the variables make sense, we'll buy it. And we can explain that to the other users. And after that, when you build some relationship or trust with your customers, all the users of these models, then later, you can gradually push things forward.
So the model needs to be transparent, needs to be very explainable.
I'll say, OK, so from the next step, we know some machine learning algorithms. The models we build using these algorithms perform better. They are more predictive. And they are explainable. I can try to show that, show you how to explain that. And they are 100% compliant, because we are in a highly regulated industry. So everything we build, we are usually audited by people from the government annually, every year. They will say, OK, they will check all the models we build, say whether this is discriminatory, whether all the models we use are compliant, these kind of things. And we are sure that, OK, the model is fully compliant, is fully transparent, and is accepted by, I won't say accepted by the entire industry, but is accepted by a lot of players in the industry. So because we are not the biggest company in the industry, we are probably the third or fourth largest subprime lenders. So our company, our leadership is a little bit conservative. There are cultures. So what we always say, OK, we have a model. This is the algorithm we just figured out. They will try to say, are we the first one using this in the industry? If the answer is yes, they'll probably say, oh, probably we should wait. And if we say, OK, no, probably Capital One is using it right now. Or maybe Santander, maybe American's price is using this right now. They will say, OK, let's go ahead. I see there's a lot of love for logistic regression in the chat there. But Alan, I see you just asked a question. Do you want to jump in?
Transitioning from individual contributor to VP
Sure. Yes. I'm really curious about how your organization has treated your shifts and roles from individual contributor to a VP of data science. And curious if what's that meant for you in terms of figuring out, what is my mission at each stage? And whether you had to create that for yourself as you became VP, or if that was already a well-established role that you could or if that was already a well-established role that you could slot into and know what success looks like there. How did you make those transitions? You talked a little bit about leadership and about the business, learning the business stuff. But I'm wondering, did the organization help you? Or were you on your own to go, oh, I'm a VP now. What does that mean?
OK. Thank you very much, Alan. First, one more question. And to explain that, first of all, I will say that my leadership, my company, they are super supportive for my growth, for my transition. And they provide a lot of help, either help from the perspective of leadership to help me to get exposure, to see their leadership, and help me learn a lot of things, how to lead a team, how to work with other teams. And they even provide some financial help for my MBA study.
And on the other side, our company is kind of, again, it's not a very big company. So a lot of time, you'll need to, as a VP, I'm the first VP of, I'm the first person to hold the title VP of data science in the company. So basically, I need to build a lot of procedures, processes, all by myself. It's kind of, now I have a team of data scientists, and I need to define my own function. Okay, so now you are VP of data science now. So it doesn't make sense, you're still sitting there building models every day, you need to do something else, because you should delegate these jobs to your repos. Yes. And then I figure out my, the goal I made for myself is that as a leader, as a VP of data science, the most important thing for me is to grow my own team.
So I have some data science repos to me. My goal is that I want to let them know that, okay, so one day you will be in my shoes. I won't be there forever, and one day you'll move up, up, up, and finally you'll be here. And in order to be prepared, I want you to start to, I would say, advise yourself, you need to make clear, clear goals, what you want to achieve in the next three years, five years. No, just from a technical standpoint, no stress, okay, I want to master how to build boosting models, how to kind of propose some innovations to our current model development process, not just that. I want them to make goals on their behaviors, like how to, I mean, how to develop their own leadership skills, how to, I mean, how to create opportunities for themselves to be a leader. And I will be 100% supportive for their goals. I believe this, right now, this is my goal. So I think, for me, probably, I will say that maybe when my boss measure my own performance, I will say that 50% just measure the output of myself and my team, and another 50% I want him to focus on how I advise on the growth of my own team, how everybody in my team, how they grow, how are they, I mean, how are they making progress in their own career path? I believe this is super important for me as a leader right now.
I believe this, right now, this is my goal. So I think, for me, probably, I will say that maybe when my boss measure my own performance, I will say that 50% just measure the output of myself and my team, and another 50% I want him to focus on how I advise on the growth of my own team, how everybody in my team, how they grow, how are they, I mean, how are they making progress in their own career path?
That's great. Thank you. I really, really like hearing both that you have support coming into the role, like good enablement, and also a bunch of room to kind of figure out what it looks like, and being supportive to do all of that stuff on behalf of your team, in terms of their development, you know, like that doesn't happen everywhere, and it's not easy, and so, yeah, I like, I really appreciate hearing your thoughts on that. Thanks. Yeah, again, probably, the company is still small enough, so I kind of have a lot of room. I can make a lot of decisions all by myself, so it means that I can, like, if you can call the shots, I mean, I have, it means that you are afraid to build what you want to do.
Building a data science team from scratch
Cool, thanks. Thank you. I was talking with someone just this week who is starting to, well, wants to build up their data science team within their organization, and wants to, like, move into that VP of data science role that doesn't exist yet today, and I'm wondering if you have any recommendations for, like, where to start with that, because I think there's so many different things for people to think about, like, when you start off at this blank slate where you could have any tools or any infrastructure, it can be kind of overwhelming to know where to start.
If you mean you want to build a data science team from scratch, I'll say that first, you figure out what your team can do for the business. It's not, if data science can have little to do for the business and can create very little values that there, it doesn't make sense for you to build a team there. You need to go to a new company, probably, and if there is a need, so if you figure out, okay, for data science can create this value, this value, this value for the business, then first you'll need to, I will say, do some public relation complaint. You'll need to let everybody, all the leaders in the company understand this. So usually, for example, for my MBA education, they usually ask you to do a 360 feedback, so ask you to reach out to the leaders, your co-workers in the company, and you ask them to provide feedback for yourself, and then probably you also can say, you can also say, okay, my goal is to build a data science team. My goal is to become a data science leader in this company in the next year, so what do I need to do? And this is most important, so everything starts from the people, from Bayi, from other leaders. If you can build this, if you can let other leaders reach agreement that, okay, this guy, the word of this guy makes sense. Probably, if we invest in data science, we can achieve this and this for the business, and it's kind of highly valuable for the business itself, and the company will eventually invest on this.
And then another thing is that to kind of help, if the person just wants to say, okay, I'm a data scientist, I want to grow up to a leadership position, how can I persuade the company, convince the company that I'm the right person? And then also that, besides good results, of course, you also need to make sure that, show them that you can always do a good job whenever they need it, and the other thing is that you need to show the ambition to the leadership. This is my takeaway is that, for the, especially right now, you know that the job market for data science is kind of hard, right? So, okay, this is the right time for you to show the company, I want to invest on my own self, and I'm not just want to be a data scientist doing coding, building models, I want to enlarge my influence, I want to learn more things about the whole business, I want to move up, I want to be a leader of a large organization. And usually, while you do that, the company will realize your ambition, will realize your value, and it will generate good results, good feedbacks from the leaders.
And actually, this is what I did, I remember that when I say, when I reached out to leadership of the company, I said, okay, I want to pursue my EMB degree, and I need your help, it will be a lot of big commitment, I need financial support, and also it will take a lot of my time, my everyday time. And the leaders, after some discussion, they believe that, okay. So, of course, I know that probably in the next two years, you'll be busy, your time will be swim, but I, we feel that it shows that you are, you really understand, you want to understand more about the business, and you really want to do more to contribute to the entire organization. I think that's why they decided, okay, it's a good time to invest on you as an employee.
Staying on top of data science techniques as a leader
Thank you very much, it's really helpful. I see, Catherine, you had asked a question a little bit earlier in the chat, do you want to jump in next? Yeah, I was curious how you stay on top of best practices in machine learning, or new techniques that you can use, data science techniques, as you transition up that ladder, and you start focusing more on big picture things. I know within my own career, I've been finding it more difficult to stay on top of like the new model that's, you know, going to achieve the same thing that I know about, but my knowledge is even six months out of date. So, how do you navigate that so that you stay informed, and can properly manage your team, or understand the models that they're giving back to you?
Okay, thank you, Catherine, that's a perfect question. So, that's, this is probably related to one of my further part of my job. So, I kind of act as the ambassador of the data science team
of Exeter Finance. So, basically, because, you know, I spent time in, you know, I spent a lot of time in universities. I was first in a PhD program, but I dropped out with a master, and then switched to the analytical program. And because of that, I love to work with the academic people. So, probably, I build a lot of connections with people in universities, maybe PhD students, they are postdoc researchers, they are professors, all research scientists from either Microsoft, MITRE, I mean, Facebook, this one. So, usually, every other week or every month, I will kind of go online, and then reveal the latest publication in machine learning, data science. Of course, I will search using some keywords, make sure that they're related to my topic.
And usually, after that, I will, if I find something interesting, I will reach out to the author of the publication, and then I will have a discussion, say, okay, so I read something really interesting. Could you share some codes? Could you, maybe, maybe we have a meeting, we can talk about, if we want to try to implement the algorithm you proposed to our industry, let's see what we can do here, and generate good results. For example, I remember in 20, let's see, yeah, 2020, I don't know whether you guys know, there is a Python package, auto machine learning package, developed by a Microsoft team called Flamo, F-L-A-M-O. It's called, Flamo, what does it mean? I forgot what it means. Fast and lightweight auto ML. At that time, the product was just released, maybe two or three months, so it's completely new. It's kind of, it's kind of 0.1 version, version 0.1. And I happened to find this product, and I found it fits our needs perfectly, and I kind of had a lot of conversations with the Flamo development team, back and forth, back and forth, and they are super nice, they are super responsive, they work super hard, and all of my research scientists in Microsoft. And I go to them, otherwise, okay, because I know that's a good product, but we are a financial service industry, and if we want to use it, we need this feature, this feature, and this feature. And then they said, okay, we can have your idea, and I keep giving them the feedback, okay, so after this feature, after this feature, how it helps our our model development process, or the workflow. And then gradually, finally, their product got better and better and better. Right now, I believe, it's two years from the first version, more than two years, the product is very popular right now, and I kind of become, I will say, very crucial, a critical part, critical component of our model development pipeline right now.
So I will say, that's one part, so that's the first part of my question, so if you want to keep on top, stay informed of all the latest research, all the latest programs in the machine learning industry, just try to read some papers, try to keep in touch with other researchers in the university, because if there are some new discovery, new progress, they are driven by them. And secondly, for my team, there's a, first, I also request them, so every year, every month, or every quarter, you need to spend some time to read these papers, this research progress, and also I give them some freedom, so probably you can use 10%, 15% of your time to do something you believe that will be useful sometime in the future, but not necessarily useful right now. As long as you can justify it, as long as you can tell me why you want to check this algorithm, why you want to attack this algorithm, it's okay. Probably we cannot use it right now. If you want to show me, you want to build a deep neural network model, something like that, it will perform very good, it's great. Right now, our industry cannot use that model because of the, I mean, it's not compliant because of the explainability, but probably in the future, sometimes, you will be able to use it. We will have that preparation for the future, and again, in this whole process, I believe all of my team members, I help them, their own creativity and thinking, because when they do the research, they all think about the business, what kind of model, what kind of algorithm I should try to match with the business, to try to match with what I'm doing right now, and then, so I'm, by doing this, I'm trying to make sure that all people in my team, they are also, they also keep informed, kind of, keep informed of all the latest programs in the production industry.
Networking and staying current
I love how much you point towards networking, when you are both talking about, like, working with the business, but also in keeping up to date, and I see Libby, I think you just called that out in in the chat as well, and I think it's a, it's a skill that we don't often highlight through, like, I don't know, interviews or on a resume, like, your ability to network and get ahead through that. Yeah, and I wanted to add that I like the idea of, you know, I like to keep continuing education, you know, as one of my personal career goals and as the goals for my team, but the way that you were describing it, saying, making that kind of specific, like, I want to research this particular thing or learn about this type of model, can really help, you know, you understand where, where your team is spending their time and can give you some, some insight into those things to stay on top of, so I liked that as an idea. Thank you for sharing. Yeah, thank you.
See, Santiago, you also had some thoughts there to share if you wanted to jump in, too.
I didn't mean to put anybody on the spot, I was just, just reading through the chat there. Oh, no worries, I didn't put my mic in. Yeah, so I, I subscribe to a ton of newsletters, and it clutters your inbox, I have a whole separate email account just to stay on top of these things, but there are a ton of good resources that are free. One will summarize Arc, Viz, or whatever that's, that is, papers, like top 10 papers of the last month or whatever. Most are deep learning related, some are stats. There's a couple of deep learning newsletters that are really good, and then I'm a member of the ASA, and they send out daily webinars, and there's all kinds of good stuff, geospatial, like temporal time series. There's one today on something later, and then through one of those, there's a quick shout out for something that I thought was really cool. Somebody named Cynthia Rudin at Duke and her team developed a series of models, one, I think it's called Ghost, I don't know if I'm saying that right, but it's really cool, interpretable, has high performance.
Cool, that's awesome. I was just trying to type in to find the link to it, if you have it and want to share it in the chat too.
To any of the stuff I mentioned?
Or yeah, the Ghost model you're just mentioning.
Yeah, thank you. Yeah, sure, and her papers are interpretable too, for anyone that's interested in that.
I will shut up now and put some links in.
Thank you. I just wanted to remind everybody as well that you can just raise your hand here if you want to ask some questions, or if you want to put some into the Slido chat, we can reshare that link. I always forget that when you join, if it doesn't show you the messages that were there before in the Zoom chat, but when we were talking about networking and different companies that you reached out to, the past few months we've featured a lot of different pharma customers that have been collaborating together on a few open source projects, and I was curious if there are things like that happening in the finance space? Do you mean like the finance space, network? Do I reach out to data leaders, data science leaders from other companies?
Yeah, or if there's an organization where a lot of finance leaders get together, data science finance leaders.
Oh, right now I would say no, because of the financial service industry, it's a very small world, so people kind of know each other, and they only hire people, they like to hire people who they work with before, so it's kind of, you don't need to network, you don't need to network with other people, you just know everybody working in this small circle. Yeah, but again, networking is, I would say, a super important concept, that is also what I learned from the AIBA program, so to network, it's not just you find some people who can help you, but another equally important opportunity is you can find opportunities for you to help others, so you will grow as people who are being sponsored by other leaders, as well as people who sponsor the growth of other people, so this is, for me, I believe that this idea makes a lot of sense.
to network, it's not just you find some people who can help you, but another equally important opportunity is you can find opportunities for you to help others, so you will grow as people who are being sponsored by other leaders, as well as people who sponsor the growth of other people.
I think for a lot of people, networking can sound scary sometimes, and I'm wondering if you have any any tips for people on how to first reach out to somebody.
I don't know, probably, I think it's part of people's nature, if you are some super extrovert, it's easy for you to talk to anyone, people always talk to other people, and if you are introverted, it doesn't matter, but for me, networking, first, you just need to be genuine, that's for sure, and again, don't kind of go to a networking event with the idea, okay, I need to figure out a new modeling algorithm that I do not know right now, so just be open, just, okay, I just want to try to make some new friends, I think this is the starting point of everything, and then later, gradually, if you become friends with other people, then during the conversation, ideas and opportunities will grow, will rise up naturally.
Being an ambassador for data science
Thank you, so I think this was maybe like 15 minutes or so ago, you mentioned being an ambassador to the data science program or data science community, and I was curious what that meant, and do you have like a data science community within Exeter? Oh, no, but I mean, because data science is, you know, it's a kind of a catchy word, a lot of people are very interested in this, people say, okay, we don't know a lot about data science, but we know it's used by almost every business, so when I work in any group, in any community, I will always try to explain to them what data science is, what I do, for example, again, go back to the EMBA program, the current EMBA program, I'm in the EMBA America's program, it has maybe 170 students in the cohort, but I'm the only data science professional in that cohort, so people are very curious, what do you do, what do you want to pursue in the EMBA program, your job looks like very technical, so what do you want to do, what do you want to join the EMBA program, but that's why I believe I'm the ambassador here, so I always say, okay, because, again, for the EMBA program, the background of your classmates or colleagues can be very diverse, for example, in my program, in the Dallas boardroom, people from Dallas, we have a business owner of a fabrication business, we have a seasoned officer of the special operations of the American army, we have people, we have a head of sales of another financial organization, we have people from the pharma industry, but again, that's why it's super interesting and super cool if you, well, if you complain, okay, this is a data science, this is what I do, this is what I believe data science can do for you, you know, for your job, for your business, and I believe this is a very cool, there's always a very good conversation there, and when you share your experience, your knowledge with people who is holding the similar title to you, but from a completely different field, this is how you open your mind and how you grow up from there.
And again, probably I didn't want to spend about five minutes to talk about the EMBA program, because I believe I fit a lot of people here, because, again, people, if you believe that you want to pursue a high quality, I believe EMBA program, you do not want to quit your job, you do not want to move to another city, to where the school is, the siting of the Cornell EMBA program is where you need, because it's called America's, it has distributed classrooms in maybe 10, about 20 big cities in North America, maybe both the United States and Canada, it's a joint program, sponsored, provided by both Cornell University in the United States and Queen's University in Canada, so every other weekend, you go to the classroom in your city, and you'll meet your customers in person, so you'll have this kind of face-to-face experience, and then the class itself will be taught by a professor in a studio at either Cornell University or Queen's University, and there is a team based, because all the students there are leaders, executives, if you just need to take a class and do some individual homework, it doesn't matter, right, for me, I don't care about grades, I got an A, I got a C, it doesn't matter, right, for me, at least I learned what I learned, so, but if there's a team based, you need to work on a team project, so, you know, okay, I represent team Dallas here, I need to do a good job, I cannot just, I cannot just kind of spend a little bit, a little time, okay, I need to do a good job, as a deciding, make sure that all the students are highly engaged, and make sure that it's competitive, so I believe that, I mean, personally, I believe it's a good shot, maybe I'm embarrassed, but I believe it's a good chance for people here, because for others, it's a big commitment, a big financial commitment, it's a big commitment of time, trust me, so I have other choices, but if you, you know, to other universities, they said that every other weekend, you need to travel there, take two days of classes, and then go back, it's not just travel, because the classes, the borders are, it's very super demanding, you need to spend a lot of time on the homework.
prepare the exam, so probably, if at that time, if you want to go to a big, you know, big program, I thought maybe from a top tier, I would exclude, that's a very good choice, because you can do it from your own city. Thank you, that's great to know, I've been thinking about it a little bit myself, as well, maybe shifting, I keep hearing LinkedIn digging, binging somewhere, and I don't know where it is, trying to shut it off, but maybe shifting gears, just a little bit here, I've recently been talking to a few people, who are talk, maybe moving to the cloud, and I was curious, if that's something that your team is doing, or what your, this is our, this is our new direction, so we use, we just build out, we have our on-prem server, so it's a Linux server, so it's a big server, so it's shared by all the data scientists in my team, and we build models there, and we implemented it in our own IT environment, but starting from last year, we begin to shift into a zero machine learning studio, a zero synapse, so the shifting happens now, so again, after that, so it's kind of, we are still exploring all the functions, all the options we have there, but we believe this is the future.
I realized, Alan, we kind of went on the same wavelength there, at the exact same time, do you want to jump in with your question too? It was, it was really similar, and, and I think you spoke to it a little bit already, maybe the only thing to tack on would be, as you think about moving to cloud thing, do you have a bunch of, like, negotiation that you have to do within the organization for approvals, for security reviews, for, you know, that kind of stuff, sort of just, just curious in general about the level of autonomy that, that you and your team have versus needing to sort of play in the space that the rest of the company kind of is ready to set up for you?
There, there are, but frankly, all the negotiations happens at higher level, so I didn't make the decisions, that happens at a top, top level. Of course, we had a chance to reveal the products, to give them recommendations that, okay, we have this concern on that function, so probably we need to figure that out, that's what we, that's our input to that shift. And, and they know that you're there as stakeholders, so, like, are they asking you, like, hey, we've got an enterprise plan for cloud, whatever, coming along, help us make sure it's good, or do you have to be really assertive and, and, like, remind them that you exist as a function and, and need to be able to input? Oh, I think we go to the assertive way, so we said, oh, we kind of, we make some more clear requirements, okay, so because we are, we, because we are financial industry, we have some more sensitive customer, PII, personal identification, some, some data. Yeah, we said, okay, so kind of data security is our top concern, so we give them a specific list of needs, okay, we need, we need to make sure this, and this, and this, and then kind of, really, kind of, they figure that, okay, we can help you meet all the requirements.
Current challenges and projects
Got it, cool, thank you. Thank you. One anonymous question that's over on Slido is, what is the biggest challenge or project that you and your team are trying to tackle right now?
If you say biggest challenge, we are building, traditionally, we are building a lot of machine learning models, so I won't say any of, some of, some are more difficult than others, but I won't say any specific ones that are especially challenging. One thing we are doing right now is we are trying to build some optimization models, you know, I'm, my major is operations research, so it's not pretty much machine learning, for example, like solving travel, solving model, like traveling salesman, I guess, I'm sure some people in the audience know that, and then this is a new field for my team, and it's, it's kind of, I won't say it's challenging, but it's very interesting.
So, for example, we want to, we have a sales, we have a sales team, they need to visit different dealers, so it's a kind of classic traveling salesman problem, that we have these dealers, we know their, we know their location, so basically, we have the distance between every two points, right, and then we know the starting point, probably the home base of the salesman, sales manager, and the everyday manager can only travel certain amount, certain number of miles, can only visit certain number of dealers, and then for each dealer, we have an expectation, so probably a potential benefit by our visit, and then we need to build an optimization model, too, so, okay, finally, give a list, okay, tell, okay, dealer sales manager, why, okay, on Monday, this is the list of dealers we think you should visit, and we can also give them, give a map, okay, you can start from dealer A, and after A, you jump to dealer C, and then this comes in. This is the project we are working right now, it's kind of, the challenging part is the implementation, and because it can be very complicated, because dealer sales manager, they may have different home base, and maybe the area, the territory he or she manages is so big, and you need to add some other concerns, like how much time he can travel in the morning, how much time he can travel in the afternoon, and if you add all the constraints, the problem can get very complicated, so right now, this is the problem we are trying to tackle, also, it's a challenging one.
Data science as a revenue generator
Thank you. Another question over on Slido is, is your group considered a revenue generator? Have you or your team developed something which made higher-ups take notice? Yeah, I would say absolutely, because actually, I asked the same question when I took the accounting class in the MBA program. I said, okay, so our program, we, do we, shall we be considered as revenue generator? And the professor said, yes, because you generate a lot of, I mean, intelligent assets, so, and especially for the financial service, for the lighting industry, kind of, data science is driven by data science, because you have a lot of, you have a lot of money here, and if you, a lot of people will reach out to you to say, could you borrow me some money, I want to finance my vehicle, and if you cannot use your data science to measure the risk well, to make the rejection, to make the approval or decline decision wisely, then you will lose all your hype very quickly. So, because of this, I believe that, again, I definitely, data science is kind of a core part of revenue generator of our business.
So, because of this, I believe that, again, I definitely, data science is kind of a core part of revenue generator of our business.
Managing risk in the subprime lending industry
Thank you. Okay, I'm not sure how to ask this, this question, but, so, I just watched recently, the big short, and so, I'm just, I'm curious if there's certain things that you have to do as a data science team to make sure that, like, certain areas aren't, like, lending to too many people who might not be able to, to pay back, like, looking at individual areas as compared to, to others?
Could you rephrase the question? So, what's the difference between lending too many people? Well, I guess, yeah, Libby's, Libby is helping me out there, like, managing the risk with subprime loans, like, do you have to look at individual locations versus others, or, like, individual dealers? Got it, got it, yeah, of course, yes. So, I can explain a little bit, it's kind of very interesting, the subprime industry. So, basically, the principle of business that we have a lot of money, we are trying, we are trying, we're trying to lend them out. We'll start from the safest customer segment, and they will keep lending, lending, lending, and then the rest of the portfolio will increase. So, basically, we have a lot of money, we are trying, we are trying to lend, and then the rest of the portfolio will increase, increase, increase, until it reaches the threshold, or reach the level that, okay, the risk is too high, we'll stop.
Yeah, so, and then, again, in this whole process, the risk is just measured by a lot of machine learning models we build. For example, we receive an application from a customer that says, okay, I need to borrow $10,000 to buy a car. The first thing we need to do is, why don't we approve or decline? And then we use bureau data, we use some alternative data, like on payday loans, these kinds of things, to build a model, to make a decision, maybe in 10, 15 seconds. And then say, okay, we approve it, I'll decline this application. If I approve it, then there are some other, other decisions to make. So, if I approve it, what's the price? What's the API ratio charge? What kind of deal we can approve? The customer is, okay, I want to borrow $40,000 for BMW, but we build some models, so it's, sorry, we can only approve, we can only give you maybe $15,000 for Toyota. This is kind of determined by the model. And after that, we'll, there are also another similar series of models, like probably, if we, if we booked a deal, if we booked a loan, I will say that, okay, so, if we have the loan in the portfolio, we'll build a model predict, okay, well, so what's the default risk of this loan during the entire life cycle?
So, we call it post-funding model, something like that. And then we need to, we have models managing, okay, so, if the loan, how, how shall we service this loan? So, if the, if the customer is, is a very risky customer, we shall call them every month to say that, okay, you need to, you need to make the payment, or is it a regular customer, we can just let him, let him or her manage the payment by himself, or herself. So, anyway, there are a bunch of, I would say, we call scorecards, a bunch of credit risk models behind the, behind the loan origination and loan servicing in the subprime industry. And this is what my team do, and this is the product of my team. And this is super interesting, because as a subprime, because the risks are high. If we, for people, a lot of people, most people in this audience, if we belong to the prime or super prime, if the credit score is really high, you don't need, you do not need these models. You just need to, you just need the Excel table, you just, they just need to pull your bureau score, and then there is a table of say, this guy is in this interval, this is the interest rate we will charge. That's it.
Sorry to interrupt. Thank you. I see two more questions that I think we could cover before the, the top of the hour. But Emilio, I see you just asked a question, do you want to jump in? And I can read it. Okay. Oh, yeah, sure. I'm muted.
Impact of COVID-19 and inflation on credit scoring
Hi, how did the COVID experience affected your parameter estimation process or your overall scoring process? I'm sorry, I didn't hear your question. Can you hear me now? I can hear you. Yep. Yep. How did the COVID-19 experience affected your process of estimating parameters and all, the whole overall scoring? Thanks. Oh, I changed, I changed everything, right? The COVID-19, and not COVID-19 itself, just because of the stimulation money. Because stimulation money, all the bureau scores are inflated, right? There is an inflation in the bureau scores, because a lot of credit bureaus, they will stop, I mean, they'll kind of stop taking into account of the default and delinquency of some trade lines of the customers. And then because of stimulation, because of stimulation money, people will make extra payments toward their, their debts. Again, all, all of the, a lot of factors here, but we found that the bureau scores, they are inflated. So a lot of bureau variables, they are inflated. And when today, when the stimulation money went away, it's not a factor anymore.
People's behavior, they are shifting back. And again, because of the COVID-19, the pandemic creates a bubble, and now the bubble burst. And when the people are charging back normal, all the rates, they were holding that bubble, now they are available now. So probably in the past three years, during the pandemic, a lot of loans, they were supposed to go default, but they didn't, because of the, maybe the stimulation money, a lot of other policies. But right now, what is over, suddenly all of them, all of these loans, they began to, they begin to become delinquent, or they begin to default. So, I mean, it's a big, big shift. It creates a lot of challenges for our team. So we are trying to manage these policies right now.
Thank you. I see, Libby, you had a question along similar lines as well. Do you want to jump in? Yeah, I think you, you just kind of answered it though, right? Because my question was, how has inflation in the past year affected things? And you kind of talked about that, like there was this big bubble during the pandemic, and then everything changed right after. And so everything kind of shifted in the other direction.
Again, yeah, I agree. I can give you some more, maybe other answers, I mean, kind of supplemented, because first, we are trying to build a microeconomic model. So using microeconomic factors like inflation, GDP, unemployment rate, this kind of thing, separate it to maybe also then maybe average gas price, these kind of things, to build a model, separate model. It's not very predictive. It's a little bit predictive power on top of everything we have right now. But it's kind of challenging to implement microeconomic model into our decision process. And the other thing is that because of our industry, we are a separate industry, so a lot of our customers, they are contractors. They maybe they work in Target, Walmart. So inflation, I would say, didn't impact them, absolutely. But the impact is not that much. So again, not just inflation, right now the tech layoffs, right, because all the high tech companies, they are starting to lay off people. And so because the impact hasn't reached out to our business right now, just because our customers are different from the people being impacted right now.
Thank you. Okay, one last question that I see I had missed over on Slido. The question was, do you use R, Python, SAS, or Excel, VBA? And in what mix? Are there legacy systems in your industry that you must deal with? When I joined the company, Exeter Finance, the whole modeling process was in SAS. At that time, I know almost nothing
about SAS. So I just fake it. So I'm like, okay, I know, I know SAS. But at that time, I know R a lot. So fortunately, at that time, we were transitioning from SAS to R. And then I started to build our whole pipeline in R. And then right now, I would say that our model pipeline is 10 minutes, I would say it's 95% in R. Again, in the future, R is very nice. I like R a lot. But in the future, from the implementation perspective, Python is more convenient.
And again, you will see a lot of machine learning papers, algorithms. When the author publishes a paper, they just build a Python library and then show it in GitHub. I reach out to them, okay, can you write an R version? They said, no, thanks. So right now, I want our people in my team to be bilingual. So we can code in R, we can code in SAS. We can code in R, we can code in Python. For SAS, we still use it for reporting perspective to generate some reports. But that's it.
For Excel, VBA, we use it a little bit to build some, because for example, if you want to do ad hoc analysis, Excel is very convenient. And sometimes, if you want to build a summary report, kind of model summary, model documentation, we have some VBA in Excel to generate some figures.
Closing remarks
Great. Thank you so much, Yu. I know we have just crossed over the top of the hour here. But before we let you go, I was just curious, what's the best way for people to get in touch with you? Is it LinkedIn?
Yeah, automatically. In my LinkedIn profile, you can also find my cell phone number. So you can reach out to me if you have any questions.
Awesome. I was going to try and go copy it over here very quickly, but somebody might be able to do it faster than me. Oh, here you are. Put your LinkedIn here. There you are. Well, thank you so much, Yu, for joining us today and sharing your experience. It's been great learning from you.
Okay. Yeah. Thank you, Rachel. It's a great event. And thank you for all the people joining this event today. Thank you a lot. It's nice to talk to you.
I'll see everybody next week, hopefully. Have a great rest of the week. Bye.