Resources
Oct 1, 2024 |
59 min
|
343 views
|

Empowering individuals through AI tools, technologies, and custom apps | Chris Engelhardt @ Gen Re

“Empathy, I think, is a hallmark of a well functioning team.” Chris Engelhardt might lead data teams and AI initiatives, but he’s also got a PhD in Social Psychology and a keen interest in helping people work more happily. This episode of the Hangout went to amazing and unexpected places as we talked about AI use cases, getting buy-in from leadership, and building stakeholder trust alongside building psychological safety in your teams, code review anxiety, and team wellbeing. If you’re curious about enterprise level AI use cases, or how Posit integrates with Databricks, there’s lots to learn from Chris! We hope you’ll join us live soon!

Timestamps
02:35 About Chris Engelhardt Sr. Data and AI Operations Manager at Gen Re
05:55 How do you get people engaged in a vision for AI?
08:10 What are some use cases for AI?
11:40 How do you manage international data science teams?
13:15 What one AI skill data analysts should learn?
14:50 What people skills should data analysts and data scientists learn?
16:25 How do you get buy-in from leadership and data scientists for AI initiatives?
18:25 What is AI? Where do I start? What are pitfalls of AI to avoid?
22:40 Integrating Posit with Databricks, Unity Catalog, and pins
25:55 How do you handle it when two different teams manage Posit versus Databricks?
27:30 What if AI tools provide inaccurate results? How do you regain trust with stakeholders?
29:50 How do you protect your own and your data team’s wellbeing at work?
34:20 How has AI changed hiring and job descriptions? Has AI changed data roles?
36:30 Code review anxiety and psychological safety around code critiques and pull requests
40:00 Helping your team feel safe and willing to talk about things and collaborate
41:40 How do you encourage psychological safety in a cross-cultural international team?
47:15 How do you use Posit pins with Databricks?
49:50 Career advice for working in data - you can’t control everything, but you control your effort
51:15 Don’t underestimate the value of connecting with others
53:33 Does using pins with Databricks Unity Catalog present a challenge with data governance?
56:45 How do you encourage learning and upskilling within your team?

Resources mentioned in this episode:

Code Review Anxiety Toolkit: https://developer-success-lab.gitbook.io/code-review-anxiety-workbook-1/part-three-mini-code-review-anxiety-toolkit/introduction

Matthew Montero from Gen Re’s episode: https://posit.co/data-science-hangout/56-matthew-montero/

Schedule time to chat about Databricks and Posit here: https://posit.co/schedule-a-call/?booking_calendar__c=Databricks

Posit Team Workflow Demos: https://pos.it/team-demo

The Psychological Safety Handbook Daren mentioned: https://handbook.gitlab.com/handbook/leadership/emotional-intelligence/psychological-safety/

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Welcome back to the Data Science Hangout, everybody. I'm Rachel Dempsey. I lead Customer Marketing here at Posit, and I'm one of the co-hosts of the Hangout. I would love to have Libby introduce herself as well.

Hello, everybody. I'm Libby. I am a co-host here with Rachel, and I'm also a Posit Academy mentor for R in Python.

We're so happy to have you joining us today. If this is your first time joining us, the Hangout is our open space to hear what's going on in the world of data across different industries, chat a bit about data science leadership, and connect with others who are facing similar things as you.

And so we get together here every Thursday at the same time, same place. So if you're watching this as a recording in the future, and you want to join us live, there will be details to add it to your own calendar below.

We're all dedicated to keeping this the friendly and welcoming space that you all have made it. So thank you for that. And we love hearing from you, no matter your years of experience, titles, industry, or languages that you work in.

And I like to add this now because I know people really enjoy connecting with other Hangout attendees in the chat. So I want to encourage you to briefly introduce yourself, say hi, maybe include your role, where you're based, something you do for fun, or even share your LinkedIn in the chat to connect with others.

And you can use that chat to also share open roles if your team's firing for roles as well. I know I have a few that I'll have to go grab to put there in the chat in a bit.

And there are three ways to jump in and ask questions today, or to just share your own perspective or your own experience with a topic that we're talking about. You can raise your hand on Zoom, you can always click that little button, we can call on you to jump in live and ask your question. You can put your question in the Zoom chat. And if you can't unmute, maybe you're in a loud coffee shop or something, just put a little asterisk next to your question and Rachel or I will read it for you.

And we also have a Slido link that is going to be in the chat. Isabella just put it in there. If you want to ask a question anonymously, feel free to do that.

And we are so excited to be joined by our other co-host for the day, Chris Engelhardt, who is the Senior Data and AI Operations Manager at Gen Re.

And Chris, I'd love to have you get us started by telling us a little bit about your role, a little bit about Gen Re, but also something you like to do for fun.

Sure. Well, thank you, Rachel. And thanks Libby for hosting and for inviting me to speak today. I could not be more excited to be here at the Posit Data Science Hangout.

As Rachel mentioned, I'm Chris Engelhardt. I'm a Senior Data and AI Operations Manager here at Gen Re.

I work in global IT, which is the first time I've ever worked in kind of an IT environment. I've always kind of been more on the business side, but now have the opportunity to focus more in an IT area where I lead a team of engineers, data scientists, full stack developers, architects, and analysts to deliver on a variety of data science and AI projects and initiatives.

And thankfully, I also have the opportunity to oversee and govern the Posit professional stack. So I oversee Posit Workbench, Posit Connect, and Posit Package Manager as part of our global deployment pattern.

To give a little bit of sense of my background as well, I have a PhD in Social and Personality Psychology, where I studied the correlates and consequences of violent game exposure with my good buddy who's on the call here today as well, Joe Hilgard.

And one of the coolest topics that I studied was whether adults with autism spectrum disorder are differentially affected by violent game content compared with typically developing individuals.

And the take home message from that work is that they're not, nor are they affected by violent game content compared with non-violent game content.

Following academia, I worked as a lead data scientist at Carfax, where I managed and governed, once again, the Posit products, as well as the infrastructure to deliver on one of their flagship products, history-based value. So you may have seen some of those commercials on Carfax, where the Carfox boots a bumper off a car and he says, why should you expect to pay more for this vehicle? You can see it has a rich accident history.

So I was involved in the infrastructure and support of that product as well.

I've also worked at Farmers Insurance, where I wrote the R at Farmers book, got R formally recognized and supported by security, which if you've ever had the fortunate opportunity to do that at your company, you know that can be quite a tall task because it's easier for security to say no than it is for security to say yes. And I also managed their telematics program there, where we looked at predicting losses as a function of actual driving behaviors like distracted driving, for example, whether or not people are texting or swiping while they're driving.

And Rachel, for fun, I enjoy spending time with my family and I enjoy hanging out with our two boys, one of whom is 10, the other is four.

And they are very active in baseball and soccer and I have the amazing opportunity to be their dad and to help them be quality people in this world who are empathetic and care about people and hopefully will make some positive contributions in the world one day.

And that's a little bit about me. And again, really excited to meet with you all today and talk about data science and AI or anything else that might be on your mind.

Developing AI vision at Gen Re

I love that. Thank you so much, Chris, for taking the time to join us today, but also thank you for all that you've done for the community as well and all the different organizations you've worked with that you've brought Posit to as well.

I was thinking to get us started to learn a little bit more about your role today, it might be helpful if you could tell us a little bit about how you're developing a vision for AI initiatives at Gen Re and also how you're starting to get people engaged in that.

Yeah, that's a good question, Rachel. So we've had a lot of focus on vision and kind of how we think about deploying and managing AI applications across our organization. And, you know, one of the ways that we think about that is to incorporate kind of two facets within that. One is obviously with a deep and concerted focus on safety.

To really make sure that, for example, we're using AI in a way that doesn't lead to unintended consequences. And we also ensure that there is always human agency in the loop. So while we might look to use AI to facilitate a decision point, what we're not here to do is to use AI to make a decision.

And so that's a little bit about the vision and how we think about AI more generally here at Gen Re.

Yeah. So what we do is through kind of the role in global IT is we also have a deep focus on enablement and self-service data science and AI capabilities. So within our team, we do a lot of work around standing up custom AI services, in particular in the Microsoft Azure cloud.

And what we do is we kind of make those services available through a variety of means, including through Posit Workbench. So we have two packages that are internal to Gen Re written in R and Python to enable access to a variety of those AI services. We're really looking to kind of enable business with modern AI tools and technologies to help them get started on how do we start to think about, you know, reimagining business workflows with AI at the center of that.

And it's obviously a journey. And the way that we've started to kind of think about that is with some very tractable, you know, kind of challenges that we think and have found that generative AI seems to deliver on.

A production AI use case

Thanks, Chris. So I know in the Hangout, there's been a lot of questions around AI, but I think it's rare that we get to actually hear about the actual like use case and a few examples of how it's being used today. Would you be able to share a specific example with us?

Sure. Absolutely. So the one that we have is the one that we kind of started with. It's now running in production and very happy to talk about that today.

So again, we started with kind of a very tractable problem that really look to use generative AI tools and technologies where they excel, which is mostly around kind of generating content, but also on its, you know, understanding of semantics and language.

And the use case that we worked on and the challenge that we were looking to solve was that for our security compliance team, they receive a lot of questionnaires from external clients, mostly around our security posture. So, you know, things around encryption, etc. And what they do is they receive those incoming questions and then they'll respond to them by hand each time, or historically, that's what they have done.

And so kind of over time, they've kind of accrued, you know, the set of question and answer pairs. And so what we did was we took that history of legally approved question and answer pairs and we put them in what's called a knowledge base. It's really just kind of a list of questions and answers matched to one another.

And what we did from there was we took each of those question and answer pairs and we embedded them. So we used a large language model to essentially convert that text to a vector of floating point numbers. So we're essentially just kind of representing text with numbers.

So we did that for all of the question and answer pairs. And so then when the security compliance team gets a new question that comes in the door, what we do is we embed that question as well with the same embedding model. And then we see which vector or vectors that one is most similar to. And so what we do is we see, okay, you know, we see that this new question is most similar to these 10 other, you know, question answer pairs that we've seen in the past.

And what we do from there is we send that information off to a GPT model, along with some prompting, and have the GPT model reason about which one it thinks is kind of the best answer. And so this is a very tractable problem. And I think a very good one to kind of get started. It was relatively simple, you know, from kind of a technology perspective. And we were also quite confident that we could deliver on it, given our understanding of what, you know, some of these AI tools and technologies could do.

And I think it's great to kind of get started in that way, in part, because you want to show kind of early success. And that's what we were able to do. Since then, we had really good customer feedback on kind of that product delivery, and has also opened up the doors for us through other teams across the Gen Re firm. So, you know, all of this information is publicly available. So I'm able to talk about that particular use case today. But yeah, that was kind of the first kind of foot in the door, Rachel, for us.

And I think it's great to kind of get started in that way, in part, because you want to show kind of early success. And that's what we were able to do.

Managing international teams

Awesome. Thank you. Well, speaking of teams across Gen Re, we have a question from Nikita. Nikita, would you like to ask it?

It's a little bit hard to hear you. So I'll repeat it as well. But it might be. There you go. Try again.

Yeah, I have a mic. Oh, nice microphone. Okay.

How do you manage international teams working on data science projects?

That's a great question, Nikita. And it's challenging, right, because we have, you know, people spanning multiple countries, multiple time zones.

We have one person who is in San Diego, California. And we have other people who are located in the UK, East US, Hungary, India.

We are geospatially distributed, which, you know, does kind of force us to have a very concerted effort around how do we effectively work together asynchronously.

And so one of the ways that I do that is I try to kind of stack my morning meetings with folks who are in India and Europe, as an example, and try to save kind of the afternoons for folks who may be more US-based.

But it is definitely a challenge to do that because, you know, we try to be respectful of, you know, everyone's time and kind of where they're physically located and try not to put too many demands on them from that perspective, because everybody needs to rest and sleep, right.

And, you know, I think we've been pretty effective at that because we're very intentional with, you know, when we have meetings, when we get all the developers together at a reasonable time, that is a good fit for, you know, all parties involved. Perhaps not an ideal fit, but I think it's, you know, kind of making accommodations from time to time to facilitate that global workforce.

Thank you.

Key skills for working with AI

I see Neil has a question in the chat, and there's a little asterisk next to it, so I'll read it. But it's, if there was one AI-related skill you would recommend analysts learn, what would it be?

One AI-related skill analysts learn.

I guess it would be to learn more about the capabilities and limitations. Because I think what often happens is, you know, in business, people see kind of new shiny tools and technologies, and they immediately pivot to this is exactly what we need, we need AI here yesterday.

And I think in some cases, you know, companies may not have data in a position where it is, you know, ideal to kind of, you know, send off to a GPT model, for example. There might be other kind of pre-processing steps that may need to happen.

But I think in general, it's learning more about kind of the limitations and also where, you know, like a generative AI model excels. So it's, you know, mostly around generating content and can help you facilitate having a conversation with a body of knowledge is another way that I tend to think about generative AI models. And, you know, seeing what kind of problem spaces these models, you know, excel in.

And I think it's really just around practicing with them and getting an understanding of what they can do and what they can't do to help inform, you know, okay, now that I have an understanding, what business challenge can I maybe apply this to? That's how I think about it.

Thank you. I see Russ in the chat, you have a question that builds on Neil's question, if you want to go next.

Yeah, thanks. I think Neil's question is really good. My interest is on the other side, what would be a really good like people skill communication skill thing that you would recommend people learn?

Empathy. I think is a hallmark of kind of a well functioning team. And I think being able to kind of put yourself in the shoes of somebody else, and understand how they might be thinking how they might be feeling or why they're behaving in certain ways to not make certain attributions about surface level behaviors.

For example, if somebody is running late for a meeting, where there could be dozens of explanations for that, right, we could make some internal attribution about that person, well, this person is habitually late, I can't depend on them, they're unreliable, they're not conscientious, whereas, you know, maybe something external happened that isn't something internal about that person. And so I think it's empathy, I would say. And, you know, I think that that is a critical skill for teamwork in general. And I think that is something that AI may not ever be able to replace. And I think there's something kind of uniquely human about that. And so I think in terms of, you know, the human experience and to pick kind of one kind of communication dimension to focus on for me, it'd be empathy.

And I think that is something that AI may not ever be able to replace. And I think there's something kind of uniquely human about that.

Sounds like Russ agrees. He said he's professional counsel in a prior career, and I agree, too. I have a question for you, Chris. Sure. How do you build a vision for your company around these AI initiatives to get buy-in from both the leadership and from the technical teams that will be tackling these?

Yeah, good question. So I think that what we try to do is to kind of show early success, you know, with them. And that helps get kind of buy-in around, okay, maybe we're going to start with kind of standing up some initial tools and technologies that are just out there for people to use, right? So, for example, we have here Azure OpenAI, which we make available through AI Studio, Azure AI Studio. And what people are able to do with that is to go in and to experiment and to see kind of what the capabilities are and, again, learn about, you know, how do I think about prompting? How do I think about system prompts? You know, how do I think about customizing, you know, some of the parameters that can kind of control the outputs from some of these models?

So I think it's largely about showing initial success and then kind of taking that on the road. But I think critically important to that is having executive sponsorship for it because it has to come, in my mind, in large part from senior leadership. So someone who kind of recognizes the value of, you know, kind of what generative AI models can do. And then also discussing more broadly across the organization on, you know, some of the use cases that they perceive they have where AI could be relevant. And in some cases they are, but in some cases they're not. And what we would try to do is to kind of peel off the ones that we think, you know, would have, you know, kind of an impact on revenue or a large impact on efficiency and think about prioritization of those AI projects in that way.

Audience questions on AI use cases

Thank you. Ailee, I see you have a question in the chat. Do you want to jump in here? Sure. Yeah. So, Chris, I'm sort of after picking your brain a little bit because I've recognized that AI is being increasingly used. But I struggle to understand exactly what AI is and also to think about how it might be implemented in sort of my field and others field. And so I was so I work in infectious disease modeling. And so we're all familiar with everything that happened during COVID. And I just kind of think about the future and the next pandemic and sort of want to brainstorm a little bit about how we could use AI to make the responses better or just improve our modeling, improve our responses. But it's a bit overwhelming to figure out how to start because there's all these different models out there. There's chat GPT. There's other versions that are better for certain things. And so I guess my question to you is just where's a good place to start and what are important pitfalls to avoid?

Yeah, good question, Kylie. I think that definitions of AI vary widely. Some people would even go so far as to say that linear regression is a form of AI. I don't know how I feel about that quite yet. But I wonder, Kylie, if we're kind of in the train of kind of more traditional statistical approaches because a lot of the kind of GPT models, again, kind of what they specialize in is in generating content. So you think about things like drafting an email or drafting a paper. That's not to say that they can't help in other ways, too, because we're also using it to kind of pluck out certain pieces of information from a body of text. So there are kind of various domains where they can be applied.

I think that, you know, at least in my experience, I've seen those models applied less to kind of the kind of challenges that you're talking about with kind of COVID response, but to the extent that it could apply to kind of generating content, you know, for people to kind of understand, okay, you know, how do I draft this message for particular audiences? Or kind of given these, you know, variables or characteristics that I want to highlight in a message, you know, I can include that as part of my, you know, drafting of content through a GPT model, let's say. So I think there's, you know, different use cases depending on the type of AI that we're talking about here. The ones that we tend to focus, you know, most here at Genry is on kind of the large language models and generative AI.

Okay. And then just a quick follow up. So if they're really good at generating content, do you think they could then be applied to sort of analyzing large bodies of text, such as tweets, scoots, whatever, to see what, to kind of get an idea of what people's sort of perception is. So could it sort of flip around and rather than generating the content to share, could it take the content and then say, okay, this is how people are feeling about X, Y, or Z?

Yeah, I think that would be a reasonable use case for tools like this. And so not only can it kind of expand, right, in terms of generating content, but it can also summarize content as well. And it's very, very good at that as well, arbitrary content. And I think where, you know, I would maybe get started with that is kind of, you know, seeing what set of instructions I can kind of pass along to the GPT model as well through prompt engineering. You know, people sometimes refer to that as, but it's really just kind of a set of instructions that you pair with the content that you're sending to those models to kind of achieve the result that you're looking to achieve. Which in this case, it may be, you know, some type of sentiment analysis. It sounds like you're hinting at here perhaps, but, you know, it can be, you know, summarizing that content to kind of give, you know, its assessment of whatever dimension that you might be curious about. Great. Thanks. Sure. Good questions.

Integrating Posit and Databricks

Chris, I know I got to hear a bunch of little snippets of different use cases on our prep call ahead of the Hangout. And one thing that we talked about was a lot of the work the team has done lately integrating Posit and Databricks. And I was wondering if you could share a little bit about that work with us. Sure. Absolutely, Rachel. So today what we have are Posit products and workloads that are running on a single virtual machine. So just kind of a single instance, a single machine, single server. And these are running all over the world. And by and large, that's where the bulk of our work gets done around kind of these single virtual machine servers.

However, sometimes people do need to work with data that are stored in other locations. For example, we have a lot of data stored in our data lakes, again, that are geospatially distributed. And we support working with those data in the data lake through two different options. The first is that people can directly read from the data lake and write to the data lake directly from Posit Workbench, as an example. But they can also pull data, irrespective of whether they're aggregated, from the data lake, which of course gives them the option to push compute to big data tools and technologies. I don't want to get into necessarily the semantics of what big data is, so I just kind of refer to them loosely as, you know, big data tools and technologies.

One of the more recent developments, of course, that I'm particularly excited about is Posit's integration with Databricks' Unity Catalog. If you haven't heard of Unity Catalog, some of you on the call may not have, it's effectively a data governance layer for granular control and managing data assets that exist within a Databricks environment. And a few of our teams have now proved that capability, where we've actually used Posit Workbench to connect to Unity Catalog and be able to query and work with data and views that sit on top of our data lake. And Rachel, we're doing a lot more on operational reporting today on this. And so if the data are, you know, kind of smallish, then what we do is we, you know, have Workbench connect to Unity Catalog directly. We just kind of specify a catalog, a schema, and a table. And then we can, you know, from that perspective, have the opportunity to kind of write dplyr code against that, and that dplyr code can be converted to Spark SQL or other flavors of SQL. And we can pull those data back to Posit, so we can do compute locally on the server. And then we can, you know, in some cases, what we're doing today is we're then writing that output to PINs. I know there might be some PINs aficionados on the call today, which ultimately, in turn, serves some of our Power BI reporting that we do.

If the data are larger, however, sometimes what we'll do is, you know, kind of lean almost entirely on kind of that Databricks specific workflow. So we'll have kind of the Posit Workbench kickoff, you know, jobs that run on the Databricks clusters. So a lot of what we're doing kind of on the Posit and Databricks front is around self-service reporting. And I really look forward to, you know, kind of the possibilities that stem through that, perhaps offloading arbitrary R workloads to Databricks. So, you know, the way that we're starting to think about this is how do we kind of enable, you know, business using, you know, premier data science tools and technologies with data that we always, as kind of a matter of habit, store in our data lake or central data repositories.

Thanks, Chris. Something that has come up for me in talking with teams is, regardless of what the tools are, but I guess in this case, Databricks and Posit, sometimes they're managed by different teams and it's hard to bring both sides together. And I was wondering if you have any kind of recommendations for some of us who may be trying to do that. So when you say bring the teams together, Rachel, do you mean business teams and IT teams? So I guess in this specific example, if there's one team who's managing the Databricks implementation, but a different team is managing Posit.

Yeah, I think we're in maybe a fortuitous position because the team who manages our data lake and the team who manages the data science tools and technologies, the Posit stack in particular, we all exist within the same IT tower. So we all roll up to the same manager, which is great. So we have routine touch points. We meet once a week to discuss anything that may exist at the intersection of management activities, but in particular. And in fact, just this last week, we had a discussion on Posit Workbench and the data lake. And so kind of from then we met offline, we clarified some approaches, and it was very easy to get alignment. I think it may not be that way at every company, though, which could be a challenge. But I think at least in terms of how IT is structured here at Genry, it is amenable to kind of facilitating that communication. Thank you. Libby, I'll pass it over to you for the next one.

Handling inaccurate results and stakeholder trust

Yeah, I see Michelle had a great question and it has asterisks. So I'm going to go ahead and read it. And it's, have you had tools that were deployed into production and ended up providing inaccurate or bad results? And if so, how did you regain trust in the tool with your stakeholders?

Inaccurate or bad results? I mean, I think that, sure, absolutely. I mean, we, you know, and thinking back to kind of my days at Carfax, right? So we would, you know, try to predict the price of a used vehicle given its vehicle attributes. And we never were able to perfectly predict, you know, what that, you know, car would go for ultimately. And so it's really just kind of projection. So there's an inherent amount of error, you know, kind of within that process, although we give kind of our best guesses. So I think the way to kind of do that is put some metrics around, you know, how much error there is and why that's expected. And to try to manage that effectively. So to try to help, you know, have people manage expectations around, okay, you know, here's why we're maybe seeing, you know, some of these errors and some of our predictions.

Here at GenRe, you know, I think that in some cases, we do see errors as well, because we're, you know, not always going to, you know, suggest the right answer to a net new incoming question for our security compliance team. But they're also able to see that, although it's not 100% accurate, we are doing pretty well. We're doing, you know, 90% accuracy, which in the grand scheme of things, if you can populate, you know, 90% of responses accurately, that's going to save them a lot of time. From having to kind of go through and identifying those answers on their own. And of course, you know, they also have to vet kind of the veracity of those responses. So there's human agency within that loop. So again, although we're kind of automatically, you know, suggesting options and facilitating a decision point, a human still has the ultimate oversight on kind of what goes out the door, so to speak, in an email. So I think it's, you know, although we're going to see errors all the time, or we're going to see outright fabrications. But I think it's, you know, mostly around end user education and helping people manage expectations and showing them kind of how the positives kind of outweigh the negatives in terms of output.

We're doing, you know, 90% accuracy, which in the grand scheme of things, if you can populate, you know, 90% of responses accurately, that's going to save them a lot of time. So there's human agency within that loop. So again, although we're kind of automatically, you know, suggesting options and facilitating a decision point, a human still has the ultimate oversight on kind of what goes out the door, so to speak, in an email.

Thanks, Chris. I saw there was a question on Slido, and I realized I accidentally went to last week's Slido. But for this week, the question was, how do you protect your own and your team's well-being at work? That is a great question. So let me think how I want to kind of respond to that.

So I'm a social psychologist, and I've long been interested in why people do what they do when they're around other people. That's what I studied. And I try to take an evidence-based approach to that to inform what I think and what I do.

And even as recently as this week, you know, we had an all-hands kind of team meeting where I've kind of called out a lot of topics related to what's referred to as psychological safety. And a lot of my kind of previous management ethos was guided by self-determination theory. So this was, you know, some early kind of motivational theories kind of back in the 90s, where they had kind of a deep focus on, you know, facilitating basic human needs that we all share. You know, a need for autonomy or having a sense of control, a need for competency, you know, a need to feel like you are able to execute on something effectively, and a need for relatedness. You know, the ability to connect, you know, with one another in meaningful ways.

And we had a lot of deliberate kind of implementations around that, right? So I try to give people high-level challenges and be a collaborator rather than dictate to them what they should do or how to solve a solution. Because ultimately, you know, they're super smart and very good at what they do as well. You know, I give positive feedback on their skills and abilities. And we also have, you know, dedicated coffee chats where once a month, we all get together for a half hour. And my only rule is we don't talk about work. We could talk about anything else but work. And so, you know, it's really with an eye toward kind of helping people, you know, feel comfortable with one another.

And, you know, again, more recently, I've had a deeper focus on these ideas of psychological safety in the team. And the general kind of thrust behind this is that there's a shared belief held by members of a team that the team is safe for interpersonal risk-taking. So in that context, when I'm talking about interpersonal risks, what I'm referring to here is the ability to speak up, to offer ideas, to admit mistakes, ask for help, provide candid feedback, share worries and concerns. Without there being any fear of consequence for that.

And this is something that is supported by decades of research, actually. So if you go back to like even the mid-60s, there was some, you know, early pioneering work at kind of psychological safety at an individual level. But more recently in kind of the late 90s, Amy Edmondson had kind of a series of papers on psychological safety at a team level. And as it turns out, it's predictive of a lot of things that you probably would care about, right? It's basically correlated with getting stuff done. So it's very highly correlated with engagement and task performance, information sharing, creativity, the desire for people to learn, how happy they are, you know, showing up to work.

And, you know, being in kind of the position that I'm in, I take it as a very serious responsibility to kind of protect my team in kind of that way and to make sure they have that space where they can be themselves. Where we can celebrate individual differences and, you know, show up to work and know it's okay to not be okay. Like these are things that I care very deeply about. And, you know, it permeates, you know, a lot of the messaging that I have within the team.

Where we can celebrate individual differences and, you know, show up to work and know it's okay to not be okay. Like these are things that I care very deeply about.

So I think that's a great question. And that's some of the ways that I try to approach it. And we were excited for this journey. I just presented on some of these ideas to the team. And so we're going to now start to kind of think through for our team specifically, you know, how do we implement it? I think we've already started to do that, but how do we implement it? And then more importantly, how do we sustain it?

Fantastic. I was going to my next question was going to be like, how does your background in psychology kind of affect the way that you lead? But I think that you fully answered that question with your answer. So I would love to actually throw to another one of my favorite social science people, Alan, to ask your question.

AI's impact on team roles

Thanks, Libby. Hi, Chris. You're related to the sort of team development and team interaction angle from the sort of people's roles perspective. I'm curious if as you've gotten into more of the large language model, the Gen AI stuff, has that resulted in distinct changes in people's roles? How is the balance of sort of pre-LLM work changed, if at all? Are you doing a lot of that same stuff or has that drawn down as LLM related stuff has come up? And sort of overall, what kind of impact does that have on the way you design the team or describe your roles or let people specialize, that sort of thing?

Yeah, I would say, Alan, you know, kind of more similar than dissimilar. You know, I think that, you know, the idea of a, you know, kind of solution architect, full stack engineer, data engineer, like these, these roles have been around for a while, as has like MLOps, you know, for example, as a practice. And it's kind of a natural extension in some ways of some of those roles, but applied to, you know, kind of this domain specific area of, you know, generative AI. And, you know, there's still a deep focus, you know, at least on our team on, you know, how do we get all of these modular independent pieces to communicate with one another? You know, how are we going to stitch together this solution? You know, how are we going to think intelligently about, you know, how do I, you know, implement, you know, the right chunking strategy or splitting up my document? How do I put in the right tags? How do I add the right metadata to make searching, you know, more efficient?

So these are, these are things that we, you know, think about, you know, quite a bit, and we do a lot of experimentation around it. And I think it's kind of a natural extension in some ways of what, you know, a more traditional data scientist, you know, might do, but just apply to this other domain, which is ripe with uncertainty and ambiguity, which makes it both fun and nerve wracking, you know, at times, Alan, because we don't know exactly what's going to happen. And, but it's pretty cool to kind of operate in that space, to be able to lead, you know, kind of a modernization within the company, generally, and that's quite an exciting opportunity that we have.

That's great. Thank you. I appreciate hearing that.

Code review anxiety and psychological safety

I want to go back to the idea or the topic of psychological safety, because I see, Darren, you shared a link in the chat that's getting a lot of love. And I just want to make sure that's part of the recording as well. Darren, do you want to talk about that briefly?

Yeah, so I'm glad to see that there's a lot of appreciation for this topic. And Chris, thanks for bringing it up. So, this was on my mind a few months ago, and I brought it up with my team as it relates to pull requests, because, you know, everyone feels a little bit differently about how their code is critiqued and the feedback they receive. And it's not just how it's been received, but it's also how it's been given. So understanding how people receive feedback and how people appreciate it in different ways is really important to understand. And I'll have Rachel or Olivia reshare that link if it gets buried in the chat, but there's like a triangle of psychological safety when it comes to anxiety with code review and how you respond to it and how the result of how you behave as a result of feedback presents itself in different ways for different people, depending on how they respond.

And so creating resources for people or providing resources for people to understand how to work through that and work through it in a positive way, I think it's really, really important for building a really functional team.

Yeah, I agree, Darren. And I think that's a great point. You know, I think there's an inherent tradeoff between what is best for the individual and what is best for the team, in some cases, not always, but in some cases, right? So, like, if I'm out there constantly admitting mistakes, I'm out there, you know, constantly saying, hey, I screwed that up, and actually did this in front of my team, you know, when I talked about this the other day, I went through like three mistakes I've made very recently, right? And it talked about the consequences of those. And so that's hard to do, Darren, right? Like, it's hard for people to admit mistakes, because the interpersonal risk there, right, is that people start to view you as incompetent, or they view you as less capable in some ways.

But at the same time, I think it establishes trust, too, because we all know that we make mistakes. You know, maybe we don't celebrate them as publicly all the time, but we're all fundamentally human, and we all fundamentally make mistakes. And I don't think that anyone truly shows up with the intent to sabotage, you know, a work team, necessarily. But it's kind of this idea, Darren, where, you know, if we can get kind of increasingly comfortable with being uncomfortable, you know, kind of in those moments, you know, I think it's, again, I'm not saying it's easy to do or easy to sustain, I think it's a real challenge, which is why I'm excited to take this on with our team.

But I think that, you know, coming from a leadership perspective, you know, someone can help kind of set that context and can reinforce, you know, certain behaviors of what they want to see, and have broader team discussions around it. Like, hey, how are we going to deal with, you know, how are we going to think about, you know, how someone might react to feedback on a pull request? You know, how can we be more empathetic to, like, how might someone receive this message, given whatever context or history there might be in the team? So I think those are all important considerations. And, you know, certainly something to be thoughtful about, because there's, you know, definitely a lot of contextual factors at play there.

Yeah, I would just add to, you know, the environment that you're in with your team, Chris, like, you want to make sure that it's, before you even have this discussion of anxiety related to pull requests, that the team feels comfortable talking about those things. And to do that, you have to talk about things outside of work. And so we do something similar, where we have a weekly catch up meeting with my team. And the first 15 minutes is just like an open question, like, here's a question of the week. And the last week that we did, it was like, what TV show are you watching recently? Or like, what's your favorite holiday? Or, you know, that kind of thing, just to kind of set the tone of having some personal relationships that are built as a foundation before you can have discussions related to anxiety, which is a very personal thing.

It is, it is. And, you know, let's face it, we spend a tremendous amount of time with our teammates, more time than we probably spend with anybody else. And so, you know, to the extent that I can help control, you know, how people feel coming into work, and them finding work meaningful, you know, showing up with this expectation that I can be me. Right, and I don't need to be, you know, I can show up with my concerns, my worries, my insecurities, my anxieties, I can show up with all of that. And I know that I'll still be accepted on the team. So that's what I'm, you know, working toward. And it's a hard challenge, Darren, I would say, you know, and because, you know, again, I think there's an inherent conflict there between, you know, maybe what's best for us individually, and you know, what ultimately leads to kind of better team outcomes.

Cross-cultural considerations

Thank you. Moro, I see you have a follow up along question here. And I'd love to have you jump in here.

Thank you. Thanks, Rachel. Yeah, I'm excited about this topic. I think I'm a struggler in a way. And I've seen other strugglers. So again, I can tell when I see a struggler in this in this area, it looks like a great idea. I would like love to read more. But I've seen people including myself wanting to promote that psychological safety, particularly across cultures, it's very hard to even predict how the person is going to take what you say. So I wonder, you know, what advice could you be, could you give that is kind of general enough that, you know, international team, someone who wants to promote can get there?

Yeah, no, that's a good call. And again, I think it's, you know, really important to be knowledgeable and insightful about respecting individual differences, you know, because we're all from different backgrounds. But that's what's unique about us. And it makes, you know, everyone like so special and can bring these unique contributions to the team. I know that Hofstede's had kind of a lot of work around, you know, kind of individual differences in culture. Since then, you know, I haven't really kept up maybe as much as I should, with some of the kind of cross cultural, you know, psychological implications there. But, you know, more around, I think just being kind of knowledgeable about individual differences in the team, kind of what their values, you know, are, you know, how they think about things like what, what, you know, maybe promotes their anxieties or makes them feel uncomfortable.

And, you know, definitely, you know, and because we have that on our team as well. And, you know, we, you know, people who are from India originally, and again, someone's in Hungary, in Budapest, and, you know, we're all over the world. And we all come from different cultures and backgrounds and, you know, schools of thought, and, you know, certainly something to be mindful about, I would say, you know, how do we kind of take that into account when thinking through this at a team level?

You know, it's not a, not a one size fits all, Mauro, I don't, you know, intend to, you know, kind of paint that picture here. But it but it's also interesting, because there, there was this study at Google, you know, because, you know, Amy Edmondson's her early work back in the late 90s, this was on kind of a, like an office, you know, manufacturing store where she did this initial study and saw kind of the merits of psychological safety. But, you know, Google also has poured millions of dollars into understanding kind of what makes teams effective. And in 2012, they had a study called Project Aristotle, where they looked at, you know, over 150 teams, they, you know, they looked at, you know, a multitude of factors to see what could, you know, create the perfect team.

And they, you know, the conventional wisdom was all around individual differences, right? So, you know, does it matter that, you know, I pair people who are extroverted, or people who have the same academic pedigree, or people who, you know, enjoy hanging out on the weekends? It doesn't matter how long they've been at the company, you know, so all these individual differences, and kind of what they found through that is that, you know, who they are does not matter. And in terms of those individual differences, it's in what they found was that the single best predictor of team performance, believe it or not, more than a third of the time, they're not going to be able to predict a team performance. Believe it or not, Mauro was psychological safety, that was the most important factor for team outcomes. And, you know, it just blows me away. You know, because it's, it's kind of an interesting idea, though, because in some small way, like it reminds me of, you know, just being kind of validated for who you are. And, you know, having that kind of, you know, lead to better team outcomes, you know, and it's kind of the freedom to be you. And that's, it's not that way everywhere, right?

Believe it or not, Mauro was psychological safety, that was the most important factor for team outcomes. And, you know, it just blows me away. You know, because it's, it's kind of an interesting idea, though, because in some small way, like it reminds me of, you know, just being kind of validated for who you are. And, you know, having that kind of, you know, lead to better team outcomes, you know, and it's kind of the freedom to be you.

So, and, but yeah, so, you know, I kind of try to paint the picture of like, here's why I think it's important. And then I support it with evidence. And, and, you know, it spans both non technical teams, technical teams, there's a lot of research out there, and agile teams with these ideas have been applied. And so psychological safety or perceptions of that actually predicts, you know, team level outcomes, like speaking up and software quality initiatives, above and beyond, statistically, all of the other variables that you're going to see.

All of the other variables that you probably think would matter, like, you know, history with agile years of experience, you know, all of these. So it's, it's a construct that I'm becoming increasingly more interested in and speaking more about, especially at the intersection of, you know, kind of software development and AI and AI safety.

Thanks a lot, Chris. I think one aspect that is super important to kind of realize is the value that, you know, organizations like Google are put into it, because I sometimes I feel that particularly in a fast moving environment of say, a consultancy, where you need to accomplish something within an hour, you know, people kind of really stress about, you know, feeding everything in that one hour without realizing that maybe someone is not catching up to, you know, the speed that the things are moving and inducing, not the psychological safety that would actually create the most profitable outcome out of that hour. Thank you very, very much for your thoughts.

Yeah, thank you, Mario. Good question. I love to see the discussion, like the discussion that we run into with Hangouts. I had no idea we were going to go that way.

Actually, I kind of want to like, we have about 12 minutes left. Take a technical turn. Jordan, if you're still here, I saw that you had a great question. And I have the same question. So would you like to ask that?

Yeah, was it? I'm gonna take a guess. Pins and Databricks? Yes. Yeah. Yes. Yeah. Yeah. Because our, my company uses Databricks. I use pins a lot, but I don't use it with Databricks at all. So I'm kind of curious, like what, how, how you're using pins, either within Databricks, or how you're using it with it before I spend like a week trying to build something new?

Pins and Databricks integration

Yeah. Yeah. Good question, Jordan. Hopefully I can, I can save you a little time. So, so what we do, at least in some situations is, you know, so our data exists, once again, in our Azure data lake. And we can connect to that, you know, so to speak, via Posit Workbench, and you know, some of the packages that Posit develops, like sparklyr, PySparklyr, etc.

And, at least in the use case that I'm aware of, you know, they would connect to it, and then just straight up read, you know, those data back to Workbench, because these data are trivially small. So from there, all the compute on the server would happen locally. And from there, they would then publish out some end result to pins.

And then from, from pins, once those data are in pins, then we have kind of a reporting structure that would sit on top of that. So there's not kind of this direct connection between Databricks and pins, although, you know, that could be on the roadmap for Posit. But that's, you know, that's my understanding of kind of what's potential today, what's potentially doable today, and what we're doing today.

Yeah, because I mean, it sounds like then it's kind of using the same sort of storage for any of the other data, but just using in the sort of pins method. So I know that there's a lot of stuff to play around with in Databricks, including like file store and volumes and stuff that are still writable, but probably more embedded in Databricks right now.

Yeah, my guess is that they're just pinning it back to the data lake. That would be my intuition here, Jordan. But, you know, that's, you know, kind of what I would I would think would be reasonable for them to do.

Cool. Well, that's I mean, that's very exciting. They're, they're two really great tools that, you know, when they work well together, they work well together.

Yeah, thanks, Jordan. Thanks. Hopefully it saves you a week.

If anyone listening in has like specific questions about how the tools work together, I just didn't want to miss the opportunity to put this link in the chat in case it's something that we can't cover in an hour long hangout, feel free to use the calendar there to book some more time to chat with our team.

Career advice

Okay, I can't believe how quickly our time goes here. But a question I always want to make sure I have the chance to ask is, Chris, is there a piece of career advice that you've either given someone on your team or that you've received over the course of your career that you'd like to share with us?

That's a good question.

I think one of the things and I think this kind of gets back to, you know, the psychological safety and basic, you know, social psychological needs that we all share is to control what we can control.

And because there's a lot of things in this world that we can't control, right? We can't control getting COVID. We can't control, you know, the weather outside. But what I can control is my effort. And I can control the choices that I make and I can control my behaviors. And I can also control showing up for my team every day. And these are all things that mean a lot to me.

And so I guess, you know, among the best career advice, this goes back to kind of when I, you know, played baseball in college was, you know, control, you can control and you can always control effort. You know, even if you can't control like what direction a ball bounces, or, you know, the call an umpire makes can't control that, no matter how much you might argue or wish that the ground was, you know, lying differently. But you can always control effort.

And to have focus on things where you have autonomy, you know, I think leads to kind of better outcomes.

I know you asked for one, Rachel, I guess the other one that I would say is to not undervalue kind of the meaning of connectedness in relationships. Because you never know, like who you're going to cross paths with, or who somebody knows, maybe they can help you in the future, maybe you can help them in the future. And one of the ways this has come to fruition for me is with my current role.

So I was, you know, friends with Nick Rorbaugh, who's at Posit. You know, we had been LinkedIn kind of buddies for a while. And he shared my manager, Matthew Montero's post for the current position. And I read the position. And I was like, Oh, my gosh, this sounds like me. Like, this is a perfect fit for me. And what I want to do, this is a chance to return to doing more with R and doing more with the Posit stack and getting back more into data science.

And I could not have been more excited for it. So I reached out to him on LinkedIn. I said, Hey, Matthew, you know, this, this is like a role, I think I'd be a good fit for it. And here's three reasons why, you know, can we hop on a 15 minute call.

And I think a lot of that started my role here and kind of growing, you know, kind of within the team and, you know, working more in the AI space, like it all kind of started with a connection that I formed many years ago.

And, and I think that, you know, kind of having a focus on, you know, kind of that aspect of as well is also something that I would recommend, like establish those personal connections and reach out to people and support one another.

Because that's, you know, to me, fundamentally human.

Because that's, you know, to me, fundamentally human.

And, you know, I guess, you know, again, controlling what you can control and connecting with people.

Thank you so much, Chris. I was going to try and grab Matthew's hangout recording to put into the chat as well.

Yeah, totally. You had a good one too.

So I'm glad I got to get to ask the career question. I see there's a few different Databricks questions that I think that are popping up in the chat now as well. So we can go back to some of those. But Travis, I think I saw some questions you were asking, and I want to make sure we cover that.

I was only dovetailing most recently on another question. Okay.

Maybe it was Alan's question then.

Alan's, thank you. Yeah, yeah. It was back to the pins, plus or minus Unity Catalog and or just regular file store in Databricks. Like, I'm struggling to identify the use case. It seems cool, but I, but do you lose the user level of governance and access that you get from Databricks out of the box?

Yeah, not necessarily on the on the pin side a whole lot, Travis, but it's really on kind of the data access part of it.

So when we're connecting from Posit Workbench, for example, we connect directly with Unity Catalog and the data engineering team kind of manages the kind of row level column level management and governance kind of of those environments or the views that people have access to.

So when you, you know, you connect, you immediately see on kind of your connections pane, like what's available to you only. And so that's the way that they've kind of managed and governed, you know, that process. The pins part to me, as best I know, is independent of that.

Unless they were to further engage with the engineering team and, you know, have that formally recognized kind of within the data lake with a view or some other means.

Well, thank you. And I see Tariq just helpfully said he took a note to. Sounds like stay tuned, everyone.

No, no, I'm not promising anything. It is. It is something that I've been intrigued by the question of, like, what is what is the intersection of pins with, like, really large data stores? And how do you think about that? And what kind of pass through? And we didn't when we were talking about building pins in the first place, one of the big concerns people had was. The question is like, hey, you can't replace a database with pins. Right? And that's correct. Right? So, is there is there a middle ground there? That makes sense or isn't there? I don't know. I don't but I think it's worthy of at least having a conversation about it.

Nice. Thanks. Thanks.

Okay, while we're talking about this, I can't not do my own advertising for every every month. So, the last Wednesday of every month, we do a different workflow demo and this month's workflow demo is going to have the connect team joining us to talk about using Posit Connect and Databricks Unity Catalog together so that you can inherit data level permissions.

So, if you have an app where only one set of users should see specific data based on what's in your Unity Catalog, you can do that now. So, I just wanted to make sure I call that out. I shared the link in the chat to.

Hopefully, I got got that quick summary right to tree.

Alan, I think you had one other question on it as well. And I know we have three minutes here. So, you want to go next.

I think I don't think I had an independent one. Mine was the, the one that the traps and I and.

Okay, we're good.

So, I think I'm good. Yeah. Okay.

It is determined with needs more follow up. I think.

Libby, is there a question that you see from earlier? You want to bring in?

I was just wondering if we had enough time. We have 2 minutes left. I think a good 1 would be from Slido. How do you encourage learning within your teams? And are there initiatives that you can share about?

I think just by virtue of the nature of the work that we're doing, there is a lot of required learning that has to happen. So, I think, just by the nature of the work itself, people want to show up, want to learn, want to experiment. And in addition to that, we also seek out training opportunities. So, we, for example, have people attend Posit conference.

We have people attend Databricks conferences or other conferences through Microsoft. So, there's kind of a variety of both, you know, kind of asynchronous and in person training opportunities that we support.

And part of that is through kind of allocated money that we have in budget as well. So, we kind of set aside money formally for training opportunities for folks on the team.

Thank you. So, Chris, I know you're also involved with a few regular meetups as well. And I wanted to give you the opportunity to share a bit about those too.

Oh, which meetups did you have in mind, Rachel?

In your bio when you mentioned regular meetups for our programming or AI development. So, I just wanted to give you a chance to share those too.

Oh, yeah. Happy to chat about those. Those are internal meetups, Rachel. So, we have a, what we call a regular meetup. It's a portmanteau of regular and R. So, we do this bimonthly, I believe. And then we have an upcoming AI town hall and workshop in Cologne, Germany coming up.

Unfortunately, just internal, but we'll have a chance to kind of celebrate, you know, all the amazing work that, you know, folks are doing with AI tools and technologies across the business and kind of give them a chance to socialize that with other executive leaders.

And so, it's really fun because I think there's a groundswell of excitement around learning, you know, especially in this space of AI that we now find ourselves in today.

Well, thank you for organizing those groups internally. I hadn't realized it wasn't an external one. Thanks for sharing that.

Well, thank you so much, Chris, for joining us today. It's been great getting to learn from your experience and to see all the questions that everyone has here today.

If this was your first Data Science Hangout, we'd love to have you join us again. You can also find the link to the recordings of the Hangout on the Posit site, but also on the Posit YouTube, too.