Resources
Oct 23, 2025 |
55 min
|
213 views
|

Lessons from a Broad & Varied Data Science Career | Arcenis Rojas | Data Science Hangout

ADD THE DATA SCIENCE HANGOUT TO YOUR CALENDAR HERE: https://pos.it/dsh - All are welcome! We'd love to see you!

We were recently joined by Arcenis Rojas, Data Scientist at Indeed, to chat about econometrics, public vs private sector data science, navigating a varied career trajectory, AI integration in the hiring sphere, and making friends at conferences.

In this Hangout, Arcenis talked about how his career journey has been wide as opposed to vertically narrow. He shared that this breadth of experience has given him confidence that he can quickly figure out any dataset. He feels it also taught him how to communicate effectively about data to people at different levels and across various domains. He also shared his tech stack at Indeed, including RStudio, Positron, AWS, Snowflake, Quarto for reporting, Shiny for apps, and Posit Connect for deploying them.

An attendee asked about the impacts of AI on the job search space, and Arcenis shared the AI at Work Report (linked below) from the Indeed Hiring Lab. He says, based on research, generative AI is expected to assist many people but only replace small segments of the workforce in the coming 5-10 years, and that entry-level knowledge work is predicted to be the most highly impacted area.

Resources mentioned in the video and zoom chat:
🔗 Indeed Hiring Lab: AI at Work Report 2025 → https://www.hiringlab.org/2025/09/23/ai-at-work-report-2025-how-genai-is-rewiring-the-dna-of-jobs/
🔗 To Explain or to Predict? (Galit Shmueli, 2010) → https://arxiv.org/abs/1101.0891
🔗 Announcing the 2025 table and plotnine contests → https://posit.co/blog/announcing-the-2025-table-and-plotnine-contests/

If you didn’t join live, one great discussion you missed from the zoom chat was about the wide variety of data types data scientists work with. Attendees shared that their data included genomics, finance/trading, environmental/natural resources, e-commerce products, and medical/clinical data. What kind of data types do you work with?

â–º Subscribe to Our Channel Here: https://bit.ly/2TzgcOu

Follow Us Here:
Website: https://www.posit.co
Hangout: https://pos.it/dsh
LinkedIn: https://www.linkedin.com/company/posit-software
Bluesky: https://bsky.app/profile/posit.co

Thanks for hanging out with us! 💛

Timestamps
00:00 Introduction
06:16 "What do you like to do for fun?"
08:51 "What are the unique aspects of financial and economic data science?"
15:07 "What are econometrics?"
16:02 "Is the difference that hard sciences stats is trying to explain what happened where econometrics might be what might happen in the future?"
19:39 "Suggestions for making data friends and going to a conference alone."
23:26 "Do you see any misconceptions about the job market online, specifically the ATS thing?"
29:52 "How has your varied career trajectory been an advantage or a challenge in data science?"
34:08 "How is the recent hype wave of AI integration manifesting in the hiring sphere?"
40:08 "What are the tools that you use in your job for reporting?"
41:42 "How do you know when it is time to pivot and leave your role because your skills are stagnating?"
45:56 "How would you persuade leadership to use R or Python?"
49:32 "Did you find yourself always trying to use more complex models when simpler ones would serve the audience better?"

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the Posit Data Science Hangout. I'm Libby Heron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

Can't wait to see you there. Let's do it. I am so excited to introduce our featured leader today, Arcenis Rojas. He's a data scientist at Indeed, and he's also a longtime regular at the Data Science Hangout. If you've been here, you've probably seen Arcenis ask a question. Arcenis, would you like to introduce yourself for me? Tell me what you do and something you like to do for fun.

Sure. Thank you, Libby. So real quick, just off the top, I wasn't nervous about this at all until you just now introduced me, so that's kind of fun. Anyway, so yeah, as you mentioned, I'm a data scientist at Indeed with Indeed's hiring lab. And what the hiring lab is, is basically we try to provide unbiased labor market data and analysis to the public at large using the very unique data set that we have at Indeed, which represents both the supply side of labor and the demand side. So that's one of those things. Now, with that said, I am here just representing me, Arcenis, and none of the opinions or thoughts that I share are those of Indeed necessarily, whatever that fun statement is, you've all heard it. So there's that.

Career journey

Now, as for further, a little more introduction. So there are actually two parts of the story, two major chapters of the story in my career. So there's pre-grad school and post-grad school. So for about seven or eight years, I was in finance and did a bunch of other stuff. I actually worked in telecoms for a little while, a lot of fun stuff. After being an equities trader for a while, I actually realized that I liked the research aspect of trading equities more than I actually liked the trading. So that's what prompted me to go to grad school.

While I was in grad school, I got a lot deeper into econometrics. This was for a master's in economics. Got very deep into econometrics, became known as the data guy in my cohort. I was very good at SPSS and then R. Now, funny story, I actually learned Python before I learned R, but kind of got away from Python because of a lot of the things, a lot of the reasons that other folks have talked about, about dependency hell and all that. But now that some of those things have been resolved, I'm kind of going, hey, Python's kind of fun again.

So anyway, that's grad school, did econometrics. And I went to work for the U.S. Bureau of Labor Statistics right after that for the U.S. government and very quickly became an expert on the Consumer Expenditure Surveys Program Public Use Microdata. That's actually the team that I worked for on the Consumer Expenditure Surveys Program. From there, I decided that I wanted to kind of, I was an economist there technically, but I was doing a lot of stuff with R and opening up the possibility of using open source technologies there. This was 2015, 2016. I actually ran the R users group for the entire agency for a couple of years until 2019 when I departed to go into data science a little bit more, kind of more directly.

And I did that as a contractor for the federal government. I did it as a contractor for private industry. I did it as a consultant with Deloitte. And there I was doing, or sorry, consulting for public organizations, government and public service organizations. Then I went on, did it at AWS for a little bit. And, you know, with all that, like I've been at Indeed for a year and a half now, having a ball. So there's all of that.

But through all of those things, one of the, there are two threads in my, through in the second half of my career that I've noticed. One that I've really enjoyed kind of the kind of trying to bridge the gap between descriptive statistics, which is what I fell in love with in econometrics and predictive stuff. So more of the kind of bridging the gap between the frequentist statistics, inferential and like ML, that kind of stuff. I'm trying to bridge that gap. And one thing that I've, that I spent a lot of, a lot of time doing in the first part of this chapter is helping people kind of understand when, which one is useful. There was actually a paper that came out in 2010 by Shmueli. I forget his first name. I'll have to look that up, but. Maybe somebody in the chat can find it.

Yeah. So Shmueli is a S-C-H-M-U-E-L-I. And that paper went into kind of the difference between those two things and all that kind of stuff. So I just kind of fell in love with that and did that for a long time. The other thread is that I love learning new technologies and kind of understanding some of the things that are always coming into our space, into the data science space. So yeah, a little bit of an introduction there. And I think you said something fun. Yep. What do you like to do for fun?

Scuba diving and PositConf

Scuba dive is, is the, the main thing that goes on these days. That's when I think fun, that's what I'm thinking of. As a matter of fact. Yeah. So the day after tiny dev day, which is the, which was the day after PositConf. So this is the Saturday after PositConf. I went diving in the Georgia Aquarium. I went diving in their, I think it's called Voyager, Ocean Voyager. And I got to dive with a whale shark, giant mantas and all that. And it was phenomenal. If you do not have pictures to share with me, I'm going to charge you with a crime. I do have pictures and I can, I can actually put them in the, in Discord. So yeah.

Yeah. That's, that's a, that's a great segue. So here are, we're going to segue really quickly into, are you on the Discord server? If you are not, invites are open right now only for a little while. You can follow this link, POS.IT slash Discord. Change your name to your real name because I see your real name in Zoom. I won't recognize you otherwise. Find your way to the Data Science Hangout channel and tag me so I can give you a little, little tag. And Arcenis, it would be so amazing if you could share some pictures there.

Okay. So what I heard from Arcenis was public and private industry, lots and lots of experience in totally different places. What are some other topics here that we could ask Arcenis about in our Slido questions? So finance, econ. I've actually gotten questions from people in the Data Science Hangout about what contracting is like, like on one-on-one questions. So maybe the difference between contracting consulting versus being a full-time, full-time writer firm, they're, they're very different. Well, Noor has a question that can get us started. Noor, would you like to go ahead and ask yours?

Okay. Put me on blast like that. Okay. I will. Oh God. Where'd it go? Where'd it go? Where'd it go? It was right here. I copy into the chat too. Yes, please. Oh, I clicked the wrong button. I found it. Okay. That's, that's hi. Hi, I'm Noor. So quick question. I noticed that you have a lot of very econ, financial-oriented background in your data science journey. And what do you say the things would make that are unique to financial economic data science compared to other domains of data science? And what are things that are useful to learn if you're someone considering pivoting into that direction?

Economics vs. finance in data science

Sure. So in terms, so what's unique about econ and data science, or about econ and finance. So econ and finance, I actually think of as two very different domains. So for me, econ is the very basic question of economics is why do people make the trade-offs people make the trade-offs that they do? People and groups, why do they make the trade-offs that they do? What are, what are the incentives that make one person or group choose one thing over another? So it's, it's actually about making choices. Finance is more about optimizing for certain outcomes in, you know, for financial outcomes. That's kind of how I see finance, about the bottom line and, you know, your top line, bottom line, that kind of thing. So it's, and there is a lot of overlap. And one of the, one of the challenges I had living in New York was a lot of people, when I lived in New York, I grew up there. People talk to me about economics thinking it was the same as finance. And it's like, not necessarily.

And so what's unique about econ is that it, it is, in my opinion, the, the most statistically rigorous of the social sciences, right? Now, some, some might argue, you know, you might say sociology, this, that, but I can, the, the rigor in economics, the statistical rigor of it is really high, especially when you're talking about things like econometrics. So what makes it unique is a, a deep understanding of econometrics, as well as the models that go into economics. They're not, they're not the same as what you have in the natural sciences or the hard sciences. Some folks will say, you know, econ is a soft science, whereas the natural sciences are hard sciences. It's, it's really understanding probability, probability distributions around things, forecasting, all that, which you do have in other domains, but it's really big in econ overall is, is all about forecasting and doing that kind of thing.

And so what's unique about econ is that it, it is, in my opinion, the, the most statistically rigorous of the social sciences, right?

The other thing that's, I would say, distinct about the social sciences in general is the practitioners of it are, you know, obviously there, there's some differences, but we like flat tables. We like fairly, fairly clear questions and storylines. So everything, things try to, you know, we try to get things down to three dimensions as much as we can, which isn't true in a lot of the other domains I've been. So some of the, so to give someone a to give a little more context, some of the domains that I've done things in the consulting that I did was for the NIH. So I, you know, did a lot of work with clinical data and genetic data, as well as demographics there, aviation data at the FAA. So did all that.

So it's like at the NIH, for example, when I was consulting there, so to, to kind of contrast it with biostats or clinical data, I actually found a lot more, many more incidents of variables, features being multi dimensional, like a single feature having more than one dimension. Think for example, if there is a clinical trial of some kind for a particular drug, that particular drug, you might test it on one individual three times over that trial, right? To look at the outcomes. Well, to answer a particular question, this set of responses that you would get might actually be considered one thing, one categorical thing, right? Because if you limit the responses to let's say three classes of responses, what you're interested in is that set of responses, you're not necessarily, you're not always going to have a numerical value that comes out as a response. What you're going to have is a class of response that the patient had to a particular drug. And you then might have three classes, three responses that you're measuring. And so that set of responses in and of itself is a feature that you'll have over and over.

So that's something that you don't often get a lot of, for example, in economics, like you're not going to see stuff like that a lot. And that actually was kind of an issue that I had to work through myself. But yeah, so the data sometimes are stored differently, and they're understood differently by the practitioners. Me coming from economics, I saw that and I was like, what's going on, break them up. And then I had to have the director of the program of TB Depot explain to me, he's like, no, no, this is one thing. And I was like, oh, that's why that's in a cell. Okay. So- We learn new things all the time. So yeah. Yeah, that was it.

We have a follow on to this one. So let me, let's see, let me let Marlene ask. And Marlene's will have a follow on from Mike. Marlene, are you available? Sorry, Marlene, I saw it was your email, so I couldn't find you at first. There you are.

Hello. Yeah, for those of us who are not worldly, what the hell are econometrics? Because you keep using that term, I don't even know what the heck that is. And if you can explain that all econometrics for baby style from PositConf, I'd really appreciate it. My brain is just not braining today. No worries. No worries. Great question.

So econometrics is a kind of branch of economics that's focused primarily on statistical measurement of things. So if someone goes into an econometrics class, you're basically going into advanced statistics. So this is where you start learning how to do regressions and all the modeling that we do in economics. That's what, when I say econometrics, that's what it means. It's the metrics of economics.

Okay, so follow on from Mike. Mike had asked, is the difference that hard sciences stats is trying to explain what happened where econometrics might be what might happen in the future?

So I wouldn't dispute that characterization, but I think the difference is much bigger, is that the difference that I see is that in the natural sciences, you very often have very precise measurements of things. And this thing did happen, and so I can expect this thing to happen again if I have the same set of conditions. So given the same set of conditions that you have, you should always get the same outcome. That's not true in economics. You may not. And so everything is probabilistic. So in economics, we do a whole lot more like Monte Carlo simulation, that kind of stuff. And if you're not familiar with what Monte Carlo simulation is, basically what it is, is you have, we might call it states. You have a set of things that you set to a certain state to start out. So let's say, for example, what's a good one? A good one might be forecasting. Let's say forecasting sales for a given company. What you might try to do, what you might have as inputs to your model are things like time of year, whether there are any discounts that you're offering, stuff like that. So imagine you have all of these. And then let's say you have a certain amount of capital that you allocate through the sales team to go out and promote. So something you might do is you might draw probability distributions around all of these inputs that you set. And then you create simulations, allowing all of those changes to bounce around in those probabilities. And then you see what your outcomes are. And then in the end, you get a distribution of an outcome, right?

So this is one of the big differences between what I would say, like all statistics in the social sciences and like the natural sciences, that's the primary thing. So in the natural sciences, if I have the same conditions, I should get the same outcome every time, right? And you typically know a whole lot more about the conditions. You can control for them a lot better in the natural sciences, typically, not always. Whereas in the social sciences, there's just no way, you know. Humans are too complex. It's a complex system. Yeah. Any system that you throw a human being in automatically becomes a complex system and becomes incredibly hard.

Making friends at conferences

Yeah. All right. Well, Marlene, thank you so much for asking the question that we all wanted to ask. You got a lot of love in the chat for that. We appreciate you. And I wanted to let Rachel hop in really quickly because I know Rachel had a question for Arsenis. Yeah, I I'm sorry if I'm shifting gears too much from econometrics right now, but I actually had a question more about like networking because Arsenis, I know I got to see you in person at PositConf and you're always like asking great questions here on the Hangout and connecting with people on LinkedIn. And I was just wondering if you had any suggestions for us about making data friends, but also about going to a conference by yourself.

Yeah, so so first the use of the word networking always kind of like puts me on end because it makes me feel like that's people talking about other people as nodes as like it makes it transactional. So I just meant to add that in the question because I know Libby feels that way, too. Yeah. So I just like to go places and make friends. That's it. I just say hi. See how you're doing. Hey, what are you up to? And so one thing one thing that is true about me, I think, is I've always got a question loaded. I'm always wondering what are you saying? What do you mean? And, you know, I'm always processing what people are saying. So taking that curiosity into every every environment makes it pretty easy to just get to know people and, you know, hey, what is it that you do? And, you know, what did you do? What did you learn in the last session that you were in at PositConf, for example? Tell me about that. I was in this session. Which one were you at? All that. So it's just asking questions just and as a teenager, as an adolescent, I was super duper shy. Super. You would a lot of people say they would never believe it now. But it was it's just like I always had this curiosity and I realized, oh, most people do like it when you ask them about something they're doing, like they like to talk about. Yeah. So so be curious about other people and care about them.

So so be curious about other people and care about them.

Yeah. Just say hi and ask them whatever it is that, you know, you have on your mind about what they're doing, you know, within social norms. But of course, just just ask, you know, and every once in a while someone will give you a really kind of nasty, spiky response or whatever. You know, it's not you, it's them. And that's OK. Move on. Yeah. I would say trust the data community to be full of really nice people who if you go up to them and say, hey, I'm new or I'm at a conference by myself, they're going to be nice to you. Yeah. So yeah, about PositConf, I actually found the community that like the people showed up, some of the nicest people in general, like as a as a group. And I was actually intimidated initially about seeing, you know, Hadley and Emil and Max and Julia and like I fanboyed all over Julia. I'm not going to lie. And she was so cool about it. She was like, hey. So we talked about like some of the work that she had done. But I have found what you just said, Libby, about the data community. People do tend to be very cool and very chill and everybody does things that are so different from one another that it's kind of normal to have curiosity and to like learn from one another. So, yeah.

Misconceptions about the job market

I love it. Everybody, if you get a chance today, put the type of data that you work with in the chat, because that has been the biggest interesting thing to me is like we all do data, but we all work with totally different data. Like what are your rows represent in your data? Because often mine are people, but I'm not doing people analytics exactly. I'm not in HR, but the rows are still usually people. OK, let's move on to Zach's question from Slido. Zach has an asterisk, so I'm going to read it for him. And it is based on your time at the BLS. And indeed, do you see any misconceptions about the job market online? And Direction said, especially the ATS thing. Is the ATS thing real? Give us some insider information. OK, so what is the ATS thing?

Ah, so I think that what she's talking about is the conception that when you submit a job application, that it is going through a program that is filing you out if you don't meet certain criteria. And this is not something that I think is across the board everywhere. Everyone's different. Sure. So I can't speak very intelligently to the ATS, because I've never been on that side of things. But it is true that some organizations do use algorithms using OCR and things of that nature to process resumes ahead of time. So that is true. That happens. And I know that it happens because OPM at the federal government does it, right? And if OPM is doing it, other organizations were doing it 10 years before. So yes, it does happen. That's about as much as I can say about that.

Misconceptions about the job market. What I see more than anything is when I look, for example, at news stories from kind of all different sources and all that, what I see is sensationalism a lot of times. And that kind of makes it hard to see the stories that are really underlying things that are actually going on in the data. So misconceptions that I do see tend to be more about the sensationalism than about the data. When people talk about the data that actually exists, I actually don't hear a lot of things that I would characterize as misconceptions. So if you focus on the data, you're generally going to be okay.

Are there any specific misconceptions that you might have had? Well, Zach, I will say, I will pitch that Zach had worked on an article recently. So I will leave it to him to put that in the chat for everybody if they want to see it. But I think the main misconception or the curiosity is just that we keep seeing labor numbers that are like, hey, the job market's not actually bad. But then the individual experience of the job market is that it is terrible, especially for people, white collar jobs and tech. So I think that maybe that's part of it. So yeah, so I guess I can talk about that a little bit. I don't think anybody in this Hangout will be surprised that if you don't dig a little more deeply into the numbers, you can very easily be fooled, right? Right. So I'm not going to talk about the current environment. I will take it back a bunch of years to the Reagan administration, just to not make the environment too hot. So during the Reagan administration, something that happened, and I was just a little kid, so I'm not talking like I, but I've looked at the numbers. And one thing that happened is that there were lots of, there were increases in the job market happening all the time. They were going up, it was a strong job market, et cetera, et cetera. When you dig into the numbers, what you see is there were two, I'm going to, sectors is the wrong word, but two groups that were increasing a lot while other segments of the job market were going down. The groups that were increasing, that were seeing increases in job openings were low-skill employment and contract positions in the government, right? So contracting got really big under Reagan. That was one of the things under that administration.

So those were the groups that increased. So if I take that nugget and bring it forward to today, what I would say is look a little more deeply into the numbers. They're out there. They're there. So if you're just looking at top-line statistics, which is what most people will do, top-line numbers, yeah, you'll be fooled. You will certainly be fooled.

So if you're just looking at top-line statistics, which is what most people will do, top-line numbers, yeah, you'll be fooled. You will certainly be fooled.

Oh, Russ just said what I was going to say, look for the Simpsons paradox or the Simpson paradox, where you've got an overall trend that looks like it goes this way, but then when you actually visualize those groups inside of that trend, they're all going the other way, possibly. Or maybe you just have a highly influential bunch that is causing an overall trend to go up, and maybe two or three sectors are moving one way when all the other ones, by and large, are going the other way. Yeah. I would say look at it in a sector-specific kind of way. So look at specific sectors. For sure. Don't look at the top line. Go look at Zach's article, because he does that. He kind of looks at like, okay, here's the overall, but what about these sectors? Thank you, Zach, for that great question.

Career trajectory and breadth of experience

All right, we have less than half an hour left, and we have a ton of questions. So, Arsenis, let's see if we can get a bunch in and do short answers with a lot of questions. Lightning round. Okay. Let me take a look at my Slido here. Something just popped up on my screen, everybody, and it covered everything. Give me two seconds while I get back to Slido. Okay. Suraj had a question. Suraj, are you available to ask that one live? Yeah, just turning on my microphone. There he is. Hey. Hi, Arsenis. Just a question about, it's slightly career-related, but I find your career trajectory really fascinating. There's a lot of unique experiences, different domains, different disciplines. I'm just wondering how you feel or whether you feel this gives you sort of like an advantage and how this has helped you in your ongoing data science work. I know it does,

but I'm always struggling to explain that to people, how those experiences across different domains and disciplines can put you at an advantage in the data science work. And on the reverse side, do you ever feel like this has somehow been a challenge in going through these different experiences and moving, like going on to the next thing? Just a simple example. I find it hard to explain to people, like even my parents, what exactly I do. And at the end of a long spiel, they'll be like, so what do you do? And I just say, computer stuff.

Great question. Thank you. So I'll answer both. I'll try and be brief here. Advantages are, there's one big one, which is that I am now really confident that there is no data set I can't figure out pretty quickly or tech stack that I can't figure out. So it gives me that, right?

The other advantage is that it has taught me. I've learned how to talk to people at different levels of organizations, as well as in different domains. So the way I can communicate about data has changed quite a bit. So I would say those are the two really big advantages.

Disadvantages, there are some big ones there as well. One, my career trajectory doesn't show progression in the way that a lot of people like to see it on a resume. Oh, you started out as an individual contributor, then you became a team lead, then you became a manager. No. I don't have, I've led multiple teams and that's on my resume and stuff. And I've done things that have bigger and bigger impact in different situations.

But what I've, my progress has been more wide than it has been kind of vertical, right?

But what I've, my progress has been more wide than it has been kind of vertical, right?

So that can be a disadvantage depending on who you're talking to and what you're trying to do, what you're trying to accomplish. Another disadvantage of it is that, like you just kind of alluded to, it gets really, it gets challenging talking about what it is that you do or what it is that you're good at to anybody that's not in the field.

So there's, all they see is you go from this thing to that thing, to that thing, to that, you just seem like whatever. But there is, like in me, I actually have found a very consistent set of interests and things that I'm good at and things that I enjoy doing. And they are, to explain them, it would get, it could get a little bit technical very quickly, right? So I kind of, like you, I stay away from getting into too much detail about what I do.

But I do tell people that the stuff that I do is, you know, I'll say things like, you know, I build models to predict things or sometimes to explain things. That's one of the things that I enjoy most. Also, I talk a little bit about building dashboards. So I built, you know, most people understand the word app. So, you know, like I'll build internal apps for people to kind of get data insights very quickly. So, you know, depending on what it is that I'm working on, that's how I'll describe it. I keep it very high level. Like you just alluded to computer stuff. Yeah. Something like that. But the disadvantages, yeah, those are the disadvantages and big advantages I see. Thank you, Arsenis. Thank you, Suresh, for asking that question.

AI's impact on the hiring sphere

And it is, how is the recent hype wave of AI integration manifesting in Indeed's operations, if at all? And you do not have to speak specifically to Indeed, but just in your experience, what you're seeing in relation to improving a user's experience of finding a job in the age of like ghost jobs and spam postings. So I will reformat this question to say, how do you see AI affecting the hiring sphere and people analytics sphere?

So I'm going to talk about this in two senses. One, very high level, like you just asked Libby, but then I will address some of the stuff at Indeed. So very high level. I actually helped with a big research project called Gen AI at Work. How we expect it to affect different things. And what we did was we took a little over 5,000 skills and we looked at how generative AI would, how well it can do those skills. And then we took different job titles, you might say, and we basically said, okay, of all these job titles that we have, how many of those skills can AI replace or assist with? And so the main author on this, her name is Anina Herring. She actually created an index for this. So that exists. And the short answer to this part of the question is, AI is going to assist a lot of people in their work in the coming 5, 10 years, but it will be able to replace very small segments only of the workforce.

Now, some of the things that are going to be most highly impacted, entry level tech jobs or anything that's entry level knowledge work. Entry level knowledge work is what generative AI is meant to do, right? If you think about what those things are, very general questions.

Okay. So now with Indeed itself, we're actually implementing it in a lot of ways internally. So we have an agentic AI job coaching experience. So if you go on Indeed and you use it in any way, you'll actually see on the website now, there's an offer. If you use it as a job seeker, you'll see that there's this agentic or AI job seeker agent or something like that, that actually helps you go through the process and identify job postings that are better for you. So that's one way. I actually just built a chat bot that analyzes data internally for our team, which could bring the cost of doing that down quite a bit. So yeah, Indeed is using it quite a lot.

And ghost jobs, I want to address that because I'm actually about to start a big research project on that. Well, I don't know how big, but a research project on that. And so there is evidence in the data that there seems to be ghost jobs, at least in our data. I don't know anywhere else, but we're not, I can't say for sure yet there are ghost jobs. Can we define ghost jobs for anybody who isn't aware of what they might be?

Sure. So a ghost job is a case where an employer or somebody puts out a job announcement, but they don't actually intend to hire for it, right? So this is a very kind of brief and kind of rough description of what it is. So yeah, so we're looking at, it appears that we do have some ways of finding out if that's going on in our data. I don't know yet cause I haven't gotten into it, but so there's not a lot I can say yet. But yeah, it seems that ghost jobs are actually happening. Now there are trends that you can actually see on our website. They're actually available and this is not, there's nothing confidential that I'm giving away by saying this and it's that job postings overall have been decreasing overall job postings, but the number of postings that are somewhat new, it appears that that has not been decreasing. And so, you know, there's some trends that some folks saw at Indeed that...

Very interesting. Yeah. Mike says, why? I'm window shopping for people to see who will apply. Like, what is the point of that? I think the point of ghost jobs is to make it look like you're hiring when you're not, so that you look as a publicly traded company, like you are healthy, right? And it's like a market strategy, which I think few people win from that except for big corporations.

Okay, let's move on because we have only 15 minutes left, but I did put a Slido poll up about whether or not you've ever gotten a job on the hidden market before. That's a job that was not posted. And so far, we only have 13 votes, but the majority of those 13 say yes, they have. And I will say many of the jobs I've gotten have not been posted. So this is a great question, Zach, that Zach had posed in the chat.

Tool stack at Indeed

All right, let's head back over to the questions in Slido. And thank you, anonymous asker, whoever that was for asking that. And I wanted to sort of follow on with, let me see. I wanted to ask a little bit more. And Diana had asked actually about the data architecture behind a big company such as Indeed. One of my questions was, you mentioned doing reporting. We didn't ask you about your tool stack. So what are the tools that you use in your job to do something like this big report?

So I'm going to, the core stuff that I use is R, Positron now, I've kind of, I think I've made the switch fully over now from RStudio. So R, Positron, AWS. So on AWS, I use Athena a lot for our databases, as well as Snowflake. So that's kind of the core of everything. Okay. Yeah, there's some other things around it like Asana and all this, but yeah, that's the core data science stuff. Okay, cool. So is reporting like Quarto reporting or is it like inward? Yeah, sorry. So yeah, two things. So reporting is going to be primarily Quarto docs. We do a lot of those internally. But I also build some Shiny applications that are, and we have those on Posit Connect for the most part. So yeah.

Okay. Gosh, I wish I knew more about AWS. Lulu is saying like AWS and Snowflake, interesting. Thank you, Diana, for asking that question, because it was exactly what I wanted to ask as well.

Knowing when to pivot roles

And then there was something that, Arsenis, we talked about in our prep call, we talked privately about this, but I know that everybody in our community will benefit from talking about this. So I would love to ask, and that is, how do you know when it is time to pivot and leave your role because your skills are stagnating? You have worked at a bunch of different places. So have I. Arsenis and I might be called job hoppers, I guess, by some people, maybe. I don't think job hopping is bad. I follow my energy and I follow my interests and what is good for me. So I'm going to pose this question to Arsenis, but I am also going to pose it to everybody in Slido. I put a little poll up there that only has two answers to it. But Arsenis, what are your thoughts on this?

So real quick, I just put a link in the chat for the report that I was talking about, about how we expect Gen AI will affect work. Okay. So if I understood the question correctly, it's like, how do you decide when it's time to go? How do you prioritize yourself? So you're in a role, your skills are stagnating, your company's not going to change. You've already tried. What do you do? And for the poll, for context, I put the answers as you look for a new role at a new company, or you stay at your job and you try to learn outside of work hours, which for me is a no-go.

I've done both. Okay. So the first thing, the first thing that you have to say is, okay, so the first thing, the first thing that you have to start with is, what actually do you bring? What do you enjoy? To have a job, a happy career, I actually see it as the triangulation of three things. Something you're good at, that you enjoy doing, that you can sell.

Now with being in tech, tech is always changing, but the triangulation of those three things in yourself might not change. So you have to know yourself. What is it that you actually like doing, that you're good at, that you can bring to the market, to the labor market? That's step one. And if you find that the role around you is moving in a direction that doesn't correspond well with that, then you have to start asking questions like, do I want to stay in this?

Something you're good at, that you enjoy doing, that you can sell.

So for me, I always start with explaining to the leadership structure around me, hey, this role is moving in a direction that doesn't correspond well with what I'm doing, or what I enjoy doing. Here are some things that I have done in the role, and I try to always have some accomplishments kind of on the list. Here are some things that I have done that correspond really well with what I enjoy doing. No leader, look, no company, no management team, they never want to have employees that hate what they do. Nobody wants that, right? So a good manager, a good leader will try to, will take the information you're giving them on board. But remember, they also have to think about the company, the organization, and if it has to go in the direction that's different from what you like, that's just what's got to happen. And so you might work out a solution. My preference would be to stay at the organization when I can, but that's kind of how I go through the decision-making process. If it's time to look

elsewhere, it's time to do that. But I just can't see making my life miserable, you know what I mean? Just, it's something that nobody wants. If you're a miserable person, you're not giving your company good work anyway, is my argument here. If you need one for the company side.

Mike says he's changed his role four or five times within Pfizer. I think that's a great point to make that if you have a really big company to move around within, that gives you the opportunity to do that. That's a great thing. There was a question that came from a community member just in the past couple of weeks before PositConf, which was, how would you persuade a colleague or a team or leadership to use R or Python when they do not? I'm going to put this in Slido for anybody who wants to answer it. Kind of a long text answer that I can give back to our community member who asks this, but this is a fantastic place to say, you're not always going to change your company. Like sometimes it's just going to be like, nope, it's not. You are the one who has to be in control and move on.

Persuading others to use R or Python

All right, let's go. Oh, would you like to follow up? We only have eight minutes left by the way, everybody. I can do this quickly if you want, because I've done it a bunch of times. So with individuals, if the individual is somebody who is kind of open-minded and is interested in learning stuff, you can actually show them how easy it is to use R or Python by just typing one plus one in the console, right? That usually gets somebody like, oh, what else can I do?

With organizations, it depends a lot on the organization and how kind of tech forward they are and all that. So at the BLS, one of the first things that I was challenged to do by the manager of one of our tech teams, he had me write a very short report on the differences between like SAS, R, SPSS, and some of these things, like the negatives and positives of all of them. So that was one thing I did. She started to get convinced. At the BLS, the effort was a lot bigger because I actually had to convince many, many individuals and I had to get the people that were already on board to start talking about it more. So it was more about like getting community around it. Like, yeah, we're all doing it. And once there was community around it, you know, the leadership couldn't but pay attention. Like they kind of had to. And then when you told them, hey, it'll save you a bunch of money as well. Then they were like, well, maybe we should look.

And once there was community around it, you know, the leadership couldn't but pay attention. Like they kind of had to. And then when you told them, hey, it'll save you a bunch of money as well. Then they were like, well, maybe we should look.

Yeah. And there's a great point from Renato in the chat here that says, I'm an independent consultant. So I learn new things like, say, Quarto on my own time and just try to introduce them in my next gig if they're a good fit for it. Not everybody is at an organization. Some people are independent. Like I have been in the past. And I think that that's a really great thing. Like pick a personal project, go for it, and then feel more confident about suggesting that people use that in your next assignment. You can kind of do that inside of a job too.

AI's impact on entry-level careers

All right. What can we do with our last five minutes? Thank you so much, Arsenis, for giving us your experience on that one. We had one that was about the impact of AI and on entry level careers and people who are pivoting into data science from other domains. I feel like we answered that one a little bit. Is there anything that you want to expand on that one? No, not so much. I think we've covered that a little bit. Okay. Yeah, we covered that a bit.

Renato's question: complexity vs. simplicity

All right. We had a question that I will post to the chat, which was, any advice on balancing learning new skills with pivoting to a new sector or learning new skills when pivoting into a new sector? I think that when you're looking for a full time job and trying to upscale into something, that can be hard. I will leave that one for the chat. Let's ask Renato's question for our last question of the day. Renato, are you available to ask yours live?

Yes. This will be probably our last one. Yes. All right. Yeah. There's this thing where people say that to a hammer, everything is a nail. Once you learn more complex things, when you went from descriptive statistics to predictive statistics, did you find yourself in a situation where you're always trying to use more complex things to solve a problem, but the audience would be better served with something that was less complex than that? Maybe descriptive statistics was good enough for a problem, but you tried to push predictive ones or something like that.

Yeah. I understand your question, I think, really well. Here's how I'll answer it. I'm going to talk about my experience at Deloitte. I had two things pulling at me. I had the folks in my Deloitte structure trying to get me to use more machine learning models, but then the director of the program, the TB Depot program, he wanted just inferential statistics like regular... We were using logistic regression and all that. He was happy with that.

The way that I actually ended up bridging the gap was I looked at... I just talked to the director for a while and I said, okay, what is it you're actually trying to do with this thing? What I learned was they were actually serving two audiences and he wasn't aware of it. One audience was he wanted to serve researchers who would need more inferential statistics, you want confidence intervals, you want all that, but he was also trying to serve clinicians. A clinician does not care what the confidence interval is around an estimate. They want to know, is my patient going to die? That's their concern. For that, predictive stuff was better.

What I did was I ended up using a lot of stuff like SHAP and LIME and all that to help the director go, okay, yeah, we can use ML, using explainable ML. This is 2019. I helped with that, but I ended up doing both on this project. The way that I did it was I helped the director see, hey, you actually have two audiences. First, you have to understand the problem. It's always listen first, figure out what the real question is, and that's how you can bring the two sides together. Did that answer your question? Renato? I hope.

It's always listen first, figure out what the real question is, and that's how you can bring the two sides together.

Renato might not be able to unmute to say thank you, but I think that, yes, and thank you, Renato, for the great question. We only have two minutes left. So, right now, I will say, don't forget that you can save your chat. Top right corner, there are three dots that you can click to save the chat, but also, if you have not joined the Posit data science community discord, I am sticking the link there in the chat. That link to POS.IT is only going to be available for the next couple of hours, and then it's going to go away. Make sure you come to the data science hangout channel and tag me and say hello so I can give you a badge. Arsenis is there. We are going to hope that Arsenis adds pictures from his dive at the Georgia Aquarium.

I will. Yes. All right. Everybody, join me in saying thank you to Arsenis. Arsenis, thank you for hanging out with us today. Thank you all for having me. Thank you so much, Arsenis, and Libby, do you mind if I say one second about the workflow demo coming up this month?

Absolutely. Sometimes, in some of the great feedback we get in the Zoom survey, a few people will ask for more of like a demo-type space as well, so I just want to make sure I always let people know that once a month, we have these workflow demos, and we typically will walk through a different end-to-end workflow demo using POSIT tools, but we've been doing a Positron focus for the last month and the next two months, and so on October 29th, we have one on AI powered data science with Positron, and so we'll dive deeper into DataBot and Positron Assistant. So I just shared the Zoom registration link if anybody wants to check it out. There's also the recording from last month's too, but so nice to see everybody.

It's nice to see everybody too, and I will say for anybody who is just now hopping off, if you have the urge to do fun technical things with your friends and learn about Positron, come find me on the Discord server because I am going to be doing some like live let's hang out and just look at our settings and Positron stuff soon, and I would love to invite you, but I need to know that you're interested in that so I can get that your way. All right, everybody, I will see you on the Discord server. Come hang out, and also keep your eye on the Discord because we might be doing some things after hours at the hangout soon, and I will announce those there. All right, bye. Have a fantastic week. We're going to see you next week with Elliot Murphy, VP of Engineering at Posit. You're not going to want to miss that one. Bye.